2026-02-24T07:16:42.016 INFO:root:teuthology version: 1.2.4.dev9+gfa17720d0 2026-02-24T07:16:42.032 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com 2026-02-24T07:16:42.111 INFO:teuthology.run:Config: archive_path: /home/teuthworker/mnt/teuthology/shraddhaag-2026-02-24_06:51:01-crimson-rados-main-distro-debug-trial/67566 branch: main description: crimson-rados/thrash/{0-crimson_install 0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{default} clusters/{crimson-fixed-2} crimson-supported-all-distro/centos_latest crimson_qa_overrides deploy/cephadm objectstore/seastore/rbm$/{crimson_seastore_rbm} thrashers/{pggrow} thrashosds-health workloads/radosbench} email: null first_in_suite: false flavor: debug job_id: '67566' kernel: branch: distro kdb: true sha1: distro last_in_suite: false machine_type: trial name: shraddhaag-2026-02-24_06:51:01-crimson-rados-main-distro-debug-trial no_nested_subset: false os_type: centos os_version: 9.stream overrides: admin_socket: branch: main ceph: conf: client.0: debug ms: 1 debug objecter: 20 debug rados: 20 global: crimson_allow_pg_split: false enable experimental unrecoverable data corrupting features: crimson ms client mode: crc ms cluster mode: crc ms mon client mode: crc ms mon cluster mode: crc ms mon service mode: crc ms service mode: crc osd_max_pg_log_entries: 2 osd_min_pg_log_entries: 1 osd_pg_log_trim_min: 0 osd_pool_default_min_size: 2 osd_pool_default_size: 3 mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon min osdmap epochs: 50 mon osdmap full prune interval: 2 mon osdmap full prune min: 15 mon osdmap full prune txsize: 2 osd pool default crimson: true paxos service trim min: 10 osd: crimson cpu num: 4 crimson osd obc lru size: 10 debug ms: 20 debug osd: 20 debug seastore: 20 debug seastore backref: 20 debug seastore cache: 20 debug seastore cleaner: 20 debug seastore device: 20 debug seastore epm: 20 debug seastore fixedkv tree: 20 debug seastore journal: 20 debug seastore lba: 20 debug seastore odata: 20 debug seastore omap: 20 debug seastore onode: 20 debug seastore t: 20 debug seastore tm: 20 osd blocked scrub grace period: 3600 osd debug reject backfill probability: 0.3 osd delete sleep: 1 osd max backfills: 3 osd max markdown count: 1000 osd objectstore: seastore osd scrub max interval: 120 osd scrub min interval: 60 osd shutdown pgref assert: true osd snap trim sleep: 2 seastore cachepin size pershard: 64M seastore main device type: RANDOM_BLOCK_SSD seastore max concurrent transactions: 8 crimson_compat: true flavor: debug log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(POOL_APP_NOT_ENABLED\) - but it is still running - objects unfound and apparently lost - overall HEALTH_ - \(OSDMAP_FLAGS\) - \(OSD_ - \(PG_ - \(POOL_ - \(CACHE_POOL_ - \(SMALLER_PGP_NUM\) - \(OBJECT_ - SLOW_OPS - \(REQUEST_SLOW\) - \(TOO_FEW_PGS\) - slow request - timeout on replica - late reservation from - MON_DOWN - OSDMAP_FLAGS - OSD_DOWN - PG_DEGRADED - PG_AVAILABILITY - POOL_APP_NOT_ENABLED - mons down - mon down - out of quorum - noscrub - nodeep-scrub - Degraded data redundancy - is down - osds down - pg .*? is .*?degraded.*?, acting - pg .*? is stuck - pg degraded - PG_BACKFILL_FULL - Low space hindering backfill .*? backfill_toofull - OSD_ROOT_DOWN - pgs degraded - pgs undersized - is active.*backfill_toofull.* sha1: ff63f3d997e38f69382fd23ac0206f07f7b05017 ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} install: ceph: flavor: debug sha1: ff63f3d997e38f69382fd23ac0206f07f7b05017 thrashosds: min_in: 4 workunit: branch: wip-shraddhaag-enable-cephadm-tests env: CRIMSON_COMPAT: '1' sha1: a78040ddef3ba5a489697ca2fe95df59a7d8bd08 owner: scheduled_shraddhaag@soko04.front.sepia.ceph.com priority: 49 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.x - osd.0 - osd.1 - client.0 - node-exporter.a - - osd.2 - osd.3 - client.1 - prometheus.a - node-exporter.b seed: 5980 sha1: ff63f3d997e38f69382fd23ac0206f07f7b05017 sleep_before_teardown: 0 suite: crimson-rados suite_branch: wip-shraddhaag-enable-cephadm-tests suite_path: /home/teuthworker/src/github.com_shraddhaag_ceph_a78040ddef3ba5a489697ca2fe95df59a7d8bd08/qa suite_relpath: qa suite_repo: https://github.com/shraddhaag/ceph.git suite_sha1: a78040ddef3ba5a489697ca2fe95df59a7d8bd08 targets: trial035.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBMkLVh8AgGhmKw2Stkw+3USojs6eyeGByb7aC0fpUEe2qEmfwW0m439JcFuxDwWDJjR6Njh7/H/92ZlwQtfruVk= trial091.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBNNb+/I90EX06LOzj/V4rr7HQbhVvVenL2CUvAq4KXlBq1T/7o8q9ci0MXO7mQbl1E63zI2Wj3XyRTH/o0ChZjU= tasks: - install: extra_packages: deb: - ceph-osd-crimson - ceph-osd-crimson-dbg rpm: - ceph-osd-crimson - ceph-osd-crimson-debuginfo - pexec: all: - sudo update-alternatives --set ceph-osd /usr/bin/ceph-osd-crimson - cephadm: conf: mgr: debug mgr: 20 debug ms: 1 debug osd: 10 osd_type: crimson - cephadm.shell: mon.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - thrashosds: ceph_objectstore_tool: false chance_down: 0 chance_inject_pause_short: 0 chance_pgnum_grow: 2 chance_pgpnum_fix: 1 chance_reset_purged_snaps_last: 0 chance_thrash_cluster_full: 0 chance_thrash_pg_upmap: 0 dump_ops_enable: false min_in: 2 noscrub_toggle_delay: 0 reweight_osd: 0 sighup_delay: 0 thrash_primary_affinity: false timeout: 2400 - full_sequential: - radosbench: clients: - client.0 time: 90 - radosbench: clients: - client.0 time: 90 - radosbench: clients: - client.0 time: 90 - radosbench: clients: - client.0 time: 90 - radosbench: clients: - client.0 time: 90 teuthology: fragments_dropped: [] meta: {} postmerge: [] teuthology_branch: main teuthology_sha1: fa17720d0088c3ac28e473468bfc79eeaff5cd38 timestamp: 2026-02-24_06:51:01 tube: trial user: shraddhaag verbose: false verify_ceph_hash: false worker_log: /home/teuthworker/mnt/teuthology/worker_logs/dispatcher.trial.904699 2026-02-24T07:16:42.111 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_shraddhaag_ceph_a78040ddef3ba5a489697ca2fe95df59a7d8bd08/qa; will attempt to use it 2026-02-24T07:16:42.112 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_shraddhaag_ceph_a78040ddef3ba5a489697ca2fe95df59a7d8bd08/qa/tasks 2026-02-24T07:16:42.112 INFO:teuthology.run_tasks:Running task internal.save_config... 2026-02-24T07:16:42.113 INFO:teuthology.task.internal:Saving configuration 2026-02-24T07:16:42.121 INFO:teuthology.run_tasks:Running task internal.check_lock... 2026-02-24T07:16:42.133 INFO:teuthology.task.internal.check_lock:Checking locks... 2026-02-24T07:16:42.191 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'trial035.front.sepia.ceph.com', 'description': '/home/teuthworker/mnt/teuthology/shraddhaag-2026-02-24_06:51:01-crimson-rados-main-distro-debug-trial/67566', 'up': True, 'machine_type': 'trial', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2026-02-24 07:06:35.131449', 'locked_by': 'scheduled_shraddhaag@soko04.front.sepia.ceph.com', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBMkLVh8AgGhmKw2Stkw+3USojs6eyeGByb7aC0fpUEe2qEmfwW0m439JcFuxDwWDJjR6Njh7/H/92ZlwQtfruVk='} 2026-02-24T07:16:42.249 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'trial091.front.sepia.ceph.com', 'description': '/home/teuthworker/mnt/teuthology/shraddhaag-2026-02-24_06:51:01-crimson-rados-main-distro-debug-trial/67566', 'up': True, 'machine_type': 'trial', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2026-02-24 07:06:35.130880', 'locked_by': 'scheduled_shraddhaag@soko04.front.sepia.ceph.com', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBNNb+/I90EX06LOzj/V4rr7HQbhVvVenL2CUvAq4KXlBq1T/7o8q9ci0MXO7mQbl1E63zI2Wj3XyRTH/o0ChZjU='} 2026-02-24T07:16:42.249 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2026-02-24T07:16:42.260 INFO:teuthology.task.internal:roles: ubuntu@trial035.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'osd.0', 'osd.1', 'client.0', 'node-exporter.a'] 2026-02-24T07:16:42.260 INFO:teuthology.task.internal:roles: ubuntu@trial091.front.sepia.ceph.com - ['osd.2', 'osd.3', 'client.1', 'prometheus.a', 'node-exporter.b'] 2026-02-24T07:16:42.260 INFO:teuthology.run_tasks:Running task console_log... 2026-02-24T07:16:42.394 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x70f938c1c0e0>, signals=[15]) 2026-02-24T07:16:42.394 INFO:teuthology.run_tasks:Running task internal.connect... 2026-02-24T07:16:42.405 INFO:teuthology.task.internal:Opening connections... 2026-02-24T07:16:42.405 DEBUG:teuthology.task.internal:connecting to ubuntu@trial035.front.sepia.ceph.com 2026-02-24T07:16:42.406 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial035.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2026-02-24T07:16:42.467 DEBUG:teuthology.task.internal:connecting to ubuntu@trial091.front.sepia.ceph.com 2026-02-24T07:16:42.467 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial091.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2026-02-24T07:16:42.530 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2026-02-24T07:16:42.550 DEBUG:teuthology.orchestra.run.trial035:> uname -m 2026-02-24T07:16:42.571 INFO:teuthology.orchestra.run.trial035.stdout:x86_64 2026-02-24T07:16:42.571 DEBUG:teuthology.orchestra.run.trial035:> cat /etc/os-release 2026-02-24T07:16:42.632 INFO:teuthology.orchestra.run.trial035.stdout:NAME="CentOS Stream" 2026-02-24T07:16:42.632 INFO:teuthology.orchestra.run.trial035.stdout:VERSION="9" 2026-02-24T07:16:42.632 INFO:teuthology.orchestra.run.trial035.stdout:ID="centos" 2026-02-24T07:16:42.632 INFO:teuthology.orchestra.run.trial035.stdout:ID_LIKE="rhel fedora" 2026-02-24T07:16:42.632 INFO:teuthology.orchestra.run.trial035.stdout:VERSION_ID="9" 2026-02-24T07:16:42.632 INFO:teuthology.orchestra.run.trial035.stdout:PLATFORM_ID="platform:el9" 2026-02-24T07:16:42.632 INFO:teuthology.orchestra.run.trial035.stdout:PRETTY_NAME="CentOS Stream 9" 2026-02-24T07:16:42.632 INFO:teuthology.orchestra.run.trial035.stdout:ANSI_COLOR="0;31" 2026-02-24T07:16:42.632 INFO:teuthology.orchestra.run.trial035.stdout:LOGO="fedora-logo-icon" 2026-02-24T07:16:42.632 INFO:teuthology.orchestra.run.trial035.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2026-02-24T07:16:42.632 INFO:teuthology.orchestra.run.trial035.stdout:HOME_URL="https://centos.org/" 2026-02-24T07:16:42.632 INFO:teuthology.orchestra.run.trial035.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2026-02-24T07:16:42.632 INFO:teuthology.orchestra.run.trial035.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2026-02-24T07:16:42.632 INFO:teuthology.orchestra.run.trial035.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2026-02-24T07:16:42.633 INFO:teuthology.lock.ops:Updating trial035.front.sepia.ceph.com on lock server 2026-02-24T07:16:42.702 DEBUG:teuthology.orchestra.run.trial091:> uname -m 2026-02-24T07:16:42.718 INFO:teuthology.orchestra.run.trial091.stdout:x86_64 2026-02-24T07:16:42.719 DEBUG:teuthology.orchestra.run.trial091:> cat /etc/os-release 2026-02-24T07:16:42.774 INFO:teuthology.orchestra.run.trial091.stdout:NAME="CentOS Stream" 2026-02-24T07:16:42.775 INFO:teuthology.orchestra.run.trial091.stdout:VERSION="9" 2026-02-24T07:16:42.775 INFO:teuthology.orchestra.run.trial091.stdout:ID="centos" 2026-02-24T07:16:42.775 INFO:teuthology.orchestra.run.trial091.stdout:ID_LIKE="rhel fedora" 2026-02-24T07:16:42.775 INFO:teuthology.orchestra.run.trial091.stdout:VERSION_ID="9" 2026-02-24T07:16:42.775 INFO:teuthology.orchestra.run.trial091.stdout:PLATFORM_ID="platform:el9" 2026-02-24T07:16:42.775 INFO:teuthology.orchestra.run.trial091.stdout:PRETTY_NAME="CentOS Stream 9" 2026-02-24T07:16:42.775 INFO:teuthology.orchestra.run.trial091.stdout:ANSI_COLOR="0;31" 2026-02-24T07:16:42.775 INFO:teuthology.orchestra.run.trial091.stdout:LOGO="fedora-logo-icon" 2026-02-24T07:16:42.775 INFO:teuthology.orchestra.run.trial091.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2026-02-24T07:16:42.775 INFO:teuthology.orchestra.run.trial091.stdout:HOME_URL="https://centos.org/" 2026-02-24T07:16:42.776 INFO:teuthology.orchestra.run.trial091.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2026-02-24T07:16:42.776 INFO:teuthology.orchestra.run.trial091.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2026-02-24T07:16:42.776 INFO:teuthology.orchestra.run.trial091.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2026-02-24T07:16:42.776 INFO:teuthology.lock.ops:Updating trial091.front.sepia.ceph.com on lock server 2026-02-24T07:16:42.855 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2026-02-24T07:16:42.868 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2026-02-24T07:16:42.881 INFO:teuthology.task.internal:Checking for old test directory... 2026-02-24T07:16:42.881 DEBUG:teuthology.orchestra.run.trial035:> test '!' -e /home/ubuntu/cephtest 2026-02-24T07:16:42.883 DEBUG:teuthology.orchestra.run.trial091:> test '!' -e /home/ubuntu/cephtest 2026-02-24T07:16:42.899 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2026-02-24T07:16:42.910 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2026-02-24T07:16:42.911 DEBUG:teuthology.orchestra.run.trial035:> test -z $(ls -A /var/lib/ceph) 2026-02-24T07:16:42.940 DEBUG:teuthology.orchestra.run.trial091:> test -z $(ls -A /var/lib/ceph) 2026-02-24T07:16:42.957 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2026-02-24T07:16:43.119 INFO:teuthology.run_tasks:Running task kernel... 2026-02-24T07:16:43.139 INFO:teuthology.task.kernel:normalize config orig: {'branch': 'distro', 'kdb': True, 'sha1': 'distro'} 2026-02-24T07:16:43.139 INFO:teuthology.task.kernel:config {'mon.a': {'branch': 'distro', 'kdb': True, 'sha1': 'distro'}, 'osd.2': {'branch': 'distro', 'kdb': True, 'sha1': 'distro'}}, timeout 300 2026-02-24T07:16:43.139 DEBUG:teuthology.orchestra.run.trial035:> test -f /run/.containerenv -o -f /.dockerenv 2026-02-24T07:16:43.139 DEBUG:teuthology.orchestra.run.trial091:> test -f /run/.containerenv -o -f /.dockerenv 2026-02-24T07:16:43.155 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-24T07:16:43.156 DEBUG:teuthology.orchestra.run.trial091:> uname -r 2026-02-24T07:16:43.156 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-24T07:16:43.156 DEBUG:teuthology.orchestra.run.trial035:> uname -r 2026-02-24T07:16:43.212 INFO:teuthology.orchestra.run.trial035.stdout:5.14.0-681.el9.x86_64 2026-02-24T07:16:43.212 INFO:teuthology.task.kernel:Running kernel on trial035: 5.14.0-681.el9.x86_64 2026-02-24T07:16:43.212 DEBUG:teuthology.orchestra.run.trial035:> sudo yum install -y kernel 2026-02-24T07:16:43.213 INFO:teuthology.orchestra.run.trial091.stdout:5.14.0-681.el9.x86_64 2026-02-24T07:16:43.213 INFO:teuthology.task.kernel:Running kernel on trial091: 5.14.0-681.el9.x86_64 2026-02-24T07:16:43.213 DEBUG:teuthology.orchestra.run.trial091:> sudo yum install -y kernel 2026-02-24T07:16:45.374 INFO:teuthology.orchestra.run.trial091.stdout:CentOS Stream 9 - BaseOS 5.0 MB/s | 8.9 MB 00:01 2026-02-24T07:16:47.502 INFO:teuthology.orchestra.run.trial091.stdout:CentOS Stream 9 - AppStream 17 MB/s | 27 MB 00:01 2026-02-24T07:16:52.182 INFO:teuthology.orchestra.run.trial035.stdout:CentOS Stream 9 - BaseOS 1.0 MB/s | 8.9 MB 00:08 2026-02-24T07:16:54.770 INFO:teuthology.orchestra.run.trial035.stdout:CentOS Stream 9 - AppStream 13 MB/s | 27 MB 00:02 2026-02-24T07:16:59.966 INFO:teuthology.orchestra.run.trial035.stdout:CentOS Stream 9 - CRB 2.8 MB/s | 8.0 MB 00:02 2026-02-24T07:17:00.222 INFO:teuthology.orchestra.run.trial091.stdout:CentOS Stream 9 - CRB 790 kB/s | 8.0 MB 00:10 2026-02-24T07:17:00.924 INFO:teuthology.orchestra.run.trial035.stdout:CentOS Stream 9 - Extras packages 96 kB/s | 20 kB 00:00 2026-02-24T07:17:01.425 INFO:teuthology.orchestra.run.trial091.stdout:CentOS Stream 9 - Extras packages 45 kB/s | 20 kB 00:00 2026-02-24T07:17:01.813 INFO:teuthology.orchestra.run.trial035.stdout:Extra Packages for Enterprise Linux 25 MB/s | 20 MB 00:00 2026-02-24T07:17:05.039 INFO:teuthology.orchestra.run.trial035.stdout:lab-extras 2.5 MB/s | 50 kB 00:00 2026-02-24T07:17:06.064 INFO:teuthology.orchestra.run.trial035.stdout:Package kernel-5.14.0-645.el9.x86_64 is already installed. 2026-02-24T07:17:06.064 INFO:teuthology.orchestra.run.trial035.stdout:Package kernel-5.14.0-681.el9.x86_64 is already installed. 2026-02-24T07:17:06.084 INFO:teuthology.orchestra.run.trial035.stdout:Dependencies resolved. 2026-02-24T07:17:06.085 INFO:teuthology.orchestra.run.trial035.stdout:================================================================================ 2026-02-24T07:17:06.086 INFO:teuthology.orchestra.run.trial035.stdout: Package Architecture Version Repository Size 2026-02-24T07:17:06.086 INFO:teuthology.orchestra.run.trial035.stdout:================================================================================ 2026-02-24T07:17:06.086 INFO:teuthology.orchestra.run.trial035.stdout:Installing: 2026-02-24T07:17:06.086 INFO:teuthology.orchestra.run.trial035.stdout: kernel x86_64 5.14.0-686.el9 baseos 901 k 2026-02-24T07:17:06.086 INFO:teuthology.orchestra.run.trial035.stdout:Installing dependencies: 2026-02-24T07:17:06.086 INFO:teuthology.orchestra.run.trial035.stdout: kernel-core x86_64 5.14.0-686.el9 baseos 17 M 2026-02-24T07:17:06.086 INFO:teuthology.orchestra.run.trial035.stdout: kernel-modules x86_64 5.14.0-686.el9 baseos 40 M 2026-02-24T07:17:06.086 INFO:teuthology.orchestra.run.trial035.stdout: kernel-modules-core x86_64 5.14.0-686.el9 baseos 31 M 2026-02-24T07:17:06.086 INFO:teuthology.orchestra.run.trial035.stdout: 2026-02-24T07:17:06.087 INFO:teuthology.orchestra.run.trial035.stdout:Transaction Summary 2026-02-24T07:17:06.087 INFO:teuthology.orchestra.run.trial035.stdout:================================================================================ 2026-02-24T07:17:06.087 INFO:teuthology.orchestra.run.trial035.stdout:Install 4 Packages 2026-02-24T07:17:06.087 INFO:teuthology.orchestra.run.trial035.stdout: 2026-02-24T07:17:06.087 INFO:teuthology.orchestra.run.trial035.stdout:Total download size: 89 M 2026-02-24T07:17:06.087 INFO:teuthology.orchestra.run.trial035.stdout:Installed size: 135 M 2026-02-24T07:17:06.087 INFO:teuthology.orchestra.run.trial035.stdout:Downloading Packages: 2026-02-24T07:17:07.052 INFO:teuthology.orchestra.run.trial035.stdout:(1/4): kernel-5.14.0-686.el9.x86_64.rpm 992 kB/s | 901 kB 00:00 2026-02-24T07:17:07.519 INFO:teuthology.orchestra.run.trial035.stdout:(2/4): kernel-core-5.14.0-686.el9.x86_64.rpm 12 MB/s | 17 MB 00:01 2026-02-24T07:17:08.043 INFO:teuthology.orchestra.run.trial035.stdout:(3/4): kernel-modules-5.14.0-686.el9.x86_64.rpm 21 MB/s | 40 MB 00:01 2026-02-24T07:17:08.783 INFO:teuthology.orchestra.run.trial035.stdout:(4/4): kernel-modules-core-5.14.0-686.el9.x86_6 18 MB/s | 31 MB 00:01 2026-02-24T07:17:08.783 INFO:teuthology.orchestra.run.trial035.stdout:-------------------------------------------------------------------------------- 2026-02-24T07:17:08.783 INFO:teuthology.orchestra.run.trial035.stdout:Total 33 MB/s | 89 MB 00:02 2026-02-24T07:17:08.977 INFO:teuthology.orchestra.run.trial035.stdout:Running transaction check 2026-02-24T07:17:09.181 INFO:teuthology.orchestra.run.trial035.stdout:Transaction check succeeded. 2026-02-24T07:17:09.182 INFO:teuthology.orchestra.run.trial035.stdout:Running transaction test 2026-02-24T07:17:09.567 INFO:teuthology.orchestra.run.trial035.stdout:Transaction test succeeded. 2026-02-24T07:17:09.568 INFO:teuthology.orchestra.run.trial035.stdout:Running transaction 2026-02-24T07:17:10.688 INFO:teuthology.orchestra.run.trial035.stdout: Preparing : 1/1 2026-02-24T07:17:10.979 INFO:teuthology.orchestra.run.trial035.stdout: Installing : kernel-modules-core-5.14.0-686.el9.x86_64 1/4 2026-02-24T07:17:11.001 INFO:teuthology.orchestra.run.trial035.stdout: Installing : kernel-core-5.14.0-686.el9.x86_64 2/4 2026-02-24T07:17:13.558 INFO:teuthology.orchestra.run.trial035.stdout: Running scriptlet: kernel-core-5.14.0-686.el9.x86_64 2/4 2026-02-24T07:17:13.592 INFO:teuthology.orchestra.run.trial035.stdout: Installing : kernel-modules-5.14.0-686.el9.x86_64 3/4 2026-02-24T07:17:16.630 INFO:teuthology.orchestra.run.trial035.stdout: Running scriptlet: kernel-modules-5.14.0-686.el9.x86_64 3/4 2026-02-24T07:17:16.637 INFO:teuthology.orchestra.run.trial035.stdout: Installing : kernel-5.14.0-686.el9.x86_64 4/4 2026-02-24T07:17:19.687 INFO:teuthology.orchestra.run.trial035.stdout: Running scriptlet: kernel-modules-core-5.14.0-686.el9.x86_64 4/4 2026-02-24T07:17:37.102 INFO:teuthology.orchestra.run.trial035.stdout: Running scriptlet: kernel-core-5.14.0-686.el9.x86_64 4/4 2026-02-24T07:17:37.126 INFO:teuthology.orchestra.run.trial035.stdout: Running scriptlet: kernel-modules-5.14.0-686.el9.x86_64 4/4 2026-02-24T07:17:37.314 INFO:teuthology.orchestra.run.trial035.stdout: Running scriptlet: kernel-5.14.0-686.el9.x86_64 4/4 2026-02-24T07:17:37.314 INFO:teuthology.orchestra.run.trial035.stdout: Verifying : kernel-5.14.0-686.el9.x86_64 1/4 2026-02-24T07:17:37.314 INFO:teuthology.orchestra.run.trial035.stdout: Verifying : kernel-core-5.14.0-686.el9.x86_64 2/4 2026-02-24T07:17:37.314 INFO:teuthology.orchestra.run.trial035.stdout: Verifying : kernel-modules-5.14.0-686.el9.x86_64 3/4 2026-02-24T07:17:37.372 INFO:teuthology.orchestra.run.trial035.stdout: Verifying : kernel-modules-core-5.14.0-686.el9.x86_64 4/4 2026-02-24T07:17:37.373 INFO:teuthology.orchestra.run.trial035.stdout: 2026-02-24T07:17:37.373 INFO:teuthology.orchestra.run.trial035.stdout:Installed: 2026-02-24T07:17:37.373 INFO:teuthology.orchestra.run.trial035.stdout: kernel-5.14.0-686.el9.x86_64 2026-02-24T07:17:37.373 INFO:teuthology.orchestra.run.trial035.stdout: kernel-core-5.14.0-686.el9.x86_64 2026-02-24T07:17:37.373 INFO:teuthology.orchestra.run.trial035.stdout: kernel-modules-5.14.0-686.el9.x86_64 2026-02-24T07:17:37.373 INFO:teuthology.orchestra.run.trial035.stdout: kernel-modules-core-5.14.0-686.el9.x86_64 2026-02-24T07:17:37.373 INFO:teuthology.orchestra.run.trial035.stdout: 2026-02-24T07:17:37.373 INFO:teuthology.orchestra.run.trial035.stdout:Complete! 2026-02-24T07:17:37.436 DEBUG:teuthology.orchestra.run.trial035:> rpm -q kernel | sort -rV | head -n 1 2026-02-24T07:17:37.467 INFO:teuthology.orchestra.run.trial035.stdout:kernel-5.14.0-686.el9.x86_64 2026-02-24T07:17:37.467 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-686.el9.x86_64 2026-02-24T07:17:37.467 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-681.el9.x86_64 Expected: 5.14.0-686.el9.x86_64 2026-02-24T07:17:37.467 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2026-02-24T07:17:37.467 DEBUG:teuthology.task.kernel:src is distro, skipping download 2026-02-24T07:17:37.468 INFO:teuthology.task.kernel:Installing distro kernel on mon.a... 2026-02-24T07:17:37.468 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@trial035.front.sepia.ceph.com, path=None, version=distro) 2026-02-24T07:17:37.468 DEBUG:teuthology.orchestra.run.trial035:> rpm -q kernel | sort -rV | head -n 1 2026-02-24T07:17:37.533 INFO:teuthology.orchestra.run.trial035.stdout:kernel-5.14.0-686.el9.x86_64 2026-02-24T07:17:37.533 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-686.el9.x86_64 2026-02-24T07:17:37.534 DEBUG:teuthology.orchestra.run.trial035:> sudo rpm -qi grub2-tools 2026-02-24T07:17:37.602 INFO:teuthology.orchestra.run.trial035.stdout:Name : grub2-tools 2026-02-24T07:17:37.602 INFO:teuthology.orchestra.run.trial035.stdout:Epoch : 1 2026-02-24T07:17:37.602 INFO:teuthology.orchestra.run.trial035.stdout:Version : 2.06 2026-02-24T07:17:37.602 INFO:teuthology.orchestra.run.trial035.stdout:Release : 120.el9 2026-02-24T07:17:37.602 INFO:teuthology.orchestra.run.trial035.stdout:Architecture: x86_64 2026-02-24T07:17:37.602 INFO:teuthology.orchestra.run.trial035.stdout:Install Date: Wed 18 Feb 2026 05:18:42 PM UTC 2026-02-24T07:17:37.603 INFO:teuthology.orchestra.run.trial035.stdout:Group : Unspecified 2026-02-24T07:17:37.603 INFO:teuthology.orchestra.run.trial035.stdout:Size : 9108611 2026-02-24T07:17:37.603 INFO:teuthology.orchestra.run.trial035.stdout:License : GPLv3+ 2026-02-24T07:17:37.603 INFO:teuthology.orchestra.run.trial035.stdout:Signature : RSA/SHA256, Thu 04 Dec 2025 09:42:59 AM UTC, Key ID 05b555b38483c65d 2026-02-24T07:17:37.603 INFO:teuthology.orchestra.run.trial035.stdout:Source RPM : grub2-2.06-120.el9.src.rpm 2026-02-24T07:17:37.603 INFO:teuthology.orchestra.run.trial035.stdout:Build Date : Wed 03 Dec 2025 12:10:38 PM UTC 2026-02-24T07:17:37.603 INFO:teuthology.orchestra.run.trial035.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2026-02-24T07:17:37.603 INFO:teuthology.orchestra.run.trial035.stdout:Packager : builder@centos.org 2026-02-24T07:17:37.603 INFO:teuthology.orchestra.run.trial035.stdout:Vendor : CentOS 2026-02-24T07:17:37.603 INFO:teuthology.orchestra.run.trial035.stdout:URL : http://www.gnu.org/software/grub/ 2026-02-24T07:17:37.603 INFO:teuthology.orchestra.run.trial035.stdout:Summary : Support tools for GRUB. 2026-02-24T07:17:37.603 INFO:teuthology.orchestra.run.trial035.stdout:Description : 2026-02-24T07:17:37.604 INFO:teuthology.orchestra.run.trial035.stdout: 2026-02-24T07:17:37.604 INFO:teuthology.orchestra.run.trial035.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2026-02-24T07:17:37.604 INFO:teuthology.orchestra.run.trial035.stdout:customizable bootloader with modular architecture. It supports a rich 2026-02-24T07:17:37.604 INFO:teuthology.orchestra.run.trial035.stdout:variety of kernel formats, file systems, computer architectures and 2026-02-24T07:17:37.604 INFO:teuthology.orchestra.run.trial035.stdout:hardware devices. 2026-02-24T07:17:37.604 INFO:teuthology.orchestra.run.trial035.stdout: 2026-02-24T07:17:37.604 INFO:teuthology.orchestra.run.trial035.stdout:This subpackage provides tools for support of all platforms. 2026-02-24T07:17:37.605 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2026-02-24T07:17:37.605 INFO:teuthology.task.kernel:Updating grub on trial035 to boot 5.14.0-686.el9.x86_64 2026-02-24T07:17:37.606 DEBUG:teuthology.orchestra.run.trial035:> sudo find /boot/loader/entries -maxdepth 1 -name "*.conf" -type f -print -quit 2026-02-24T07:17:37.669 INFO:teuthology.orchestra.run.trial035.stdout:/boot/loader/entries/b0373be789df49cdafedf423703d2566-5.14.0-681.el9.x86_64.conf 2026-02-24T07:17:37.671 DEBUG:teuthology.orchestra.run.trial035:> sudo command -v grubby && echo yes || echo no 2026-02-24T07:17:37.733 INFO:teuthology.orchestra.run.trial035.stdout:/sbin/grubby 2026-02-24T07:17:37.733 INFO:teuthology.orchestra.run.trial035.stdout:yes 2026-02-24T07:17:37.734 DEBUG:teuthology.orchestra.run.trial035:> sudo find /boot/loader/entries -maxdepth 1 -name "*.conf" -type f | grep -F "5.14.0-686.el9.x86_64" | head -n 1 2026-02-24T07:17:37.798 INFO:teuthology.orchestra.run.trial035.stdout:/boot/loader/entries/b0373be789df49cdafedf423703d2566-5.14.0-686.el9.x86_64.conf 2026-02-24T07:17:37.799 DEBUG:teuthology.orchestra.run.trial035:> sudo grub2-set-default b0373be789df49cdafedf423703d2566-5.14.0-686.el9.x86_64 2026-02-24T07:17:37.871 DEBUG:teuthology.orchestra.run.trial035:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2026-02-24T07:17:38.075 INFO:teuthology.orchestra.run.trial035.stderr:Generating grub configuration file ... 2026-02-24T07:17:38.141 INFO:teuthology.orchestra.run.trial035.stderr:Found linux image: /boot/vmlinuz-5.14.0-686.el9.x86_64 2026-02-24T07:17:38.148 INFO:teuthology.orchestra.run.trial035.stderr:Found initrd image: /boot/initramfs-5.14.0-686.el9.x86_64.img 2026-02-24T07:17:38.398 INFO:teuthology.orchestra.run.trial035.stderr:Found linux image: /boot/vmlinuz-5.14.0-681.el9.x86_64 2026-02-24T07:17:38.404 INFO:teuthology.orchestra.run.trial035.stderr:Found initrd image: /boot/initramfs-5.14.0-681.el9.x86_64.img 2026-02-24T07:17:38.439 INFO:teuthology.orchestra.run.trial035.stderr:Found linux image: /boot/vmlinuz-5.14.0-645.el9.x86_64 2026-02-24T07:17:38.445 INFO:teuthology.orchestra.run.trial035.stderr:Found initrd image: /boot/initramfs-5.14.0-645.el9.x86_64.img 2026-02-24T07:17:38.477 INFO:teuthology.orchestra.run.trial035.stderr:Found linux image: /boot/vmlinuz-0-rescue-b0373be789df49cdafedf423703d2566 2026-02-24T07:17:38.483 INFO:teuthology.orchestra.run.trial035.stderr:Found initrd image: /boot/initramfs-0-rescue-b0373be789df49cdafedf423703d2566.img 2026-02-24T07:17:38.507 INFO:teuthology.orchestra.run.trial035.stderr:Found linux image: /boot/vmlinuz-0-rescue-2734456005d743dca7f3dc77c35a2d0d 2026-02-24T07:17:38.513 INFO:teuthology.orchestra.run.trial035.stderr:Found initrd image: /boot/initramfs-0-rescue-2734456005d743dca7f3dc77c35a2d0d.img 2026-02-24T07:17:38.599 INFO:teuthology.orchestra.run.trial035.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2026-02-24T07:17:38.607 INFO:teuthology.orchestra.run.trial035.stderr:done 2026-02-24T07:17:38.609 DEBUG:teuthology.orchestra.run.trial035:> test -d /sys/firmware/efi 2026-02-24T07:17:38.625 DEBUG:teuthology.orchestra.run.trial035:> sudo find /boot/efi/EFI -mindepth 1 -maxdepth 1 -type d || true 2026-02-24T07:17:38.691 INFO:teuthology.orchestra.run.trial035.stdout:/boot/efi/EFI/BOOT 2026-02-24T07:17:38.691 INFO:teuthology.orchestra.run.trial035.stdout:/boot/efi/EFI/centos 2026-02-24T07:17:38.693 DEBUG:teuthology.orchestra.run.trial035:> sudo test -f /boot/efi/EFI/BOOT/grub.cfg && echo yes || echo no 2026-02-24T07:17:38.758 INFO:teuthology.orchestra.run.trial035.stdout:no 2026-02-24T07:17:38.758 DEBUG:teuthology.orchestra.run.trial035:> sudo test -f /boot/efi/EFI/centos/grub.cfg && echo yes || echo no 2026-02-24T07:17:38.823 INFO:teuthology.orchestra.run.trial035.stdout:yes 2026-02-24T07:17:38.824 DEBUG:teuthology.orchestra.run.trial035:> sudo cp -f /boot/grub2/grub.cfg /boot/efi/EFI/centos/grub.cfg 2026-02-24T07:17:38.889 INFO:teuthology.orchestra.remote:Triggering safe hard reboot via SysRq on trial035 2026-02-24T07:17:38.889 DEBUG:teuthology.orchestra.run.trial035:> sudo bash -c ' 2026-02-24T07:17:38.890 DEBUG:teuthology.orchestra.run.trial035:> echo pci > /sys/kernel/reboot/type && echo 1 > /sys/kernel/reboot/force && sync && echo s > /proc/sysrq-trigger && echo u > /proc/sysrq-trigger && sleep 2 && echo b > /proc/sysrq-trigger 2026-02-24T07:17:38.890 DEBUG:teuthology.orchestra.run.trial035:> ' 2026-02-24T07:18:08.932 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2026-02-24T07:18:08.933 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@trial035.front.sepia.ceph.com' 2026-02-24T07:18:08.934 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial035.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2026-02-24T07:19:08.937 DEBUG:teuthology.orchestra.remote:timed out 2026-02-24T07:19:17.938 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@trial035.front.sepia.ceph.com' 2026-02-24T07:19:17.940 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial035.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2026-02-24T07:19:18.074 DEBUG:teuthology.orchestra.run.trial035:> true 2026-02-24T07:19:18.239 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@trial035.front.sepia.ceph.com' 2026-02-24T07:19:18.239 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "5.14.0-686.el9.x86_64"... 2026-02-24T07:19:18.239 DEBUG:teuthology.orchestra.run.trial035:> uname -r 2026-02-24T07:19:18.295 INFO:teuthology.orchestra.run.trial035.stdout:5.14.0-686.el9.x86_64 2026-02-24T07:19:18.296 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-686.el9.x86_64 vs 5.14.0-686.el9.x86_64 2026-02-24T07:19:18.296 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2026-02-24T07:19:18.296 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2026-02-24T07:19:19.297 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2026-02-24T07:19:19.298 DEBUG:teuthology.orchestra.run.trial035:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2026-02-24T07:19:19.329 INFO:teuthology.orchestra.run.trial035.stdout:ttyS1 2026-02-24T07:19:19.351 DEBUG:teuthology.parallel:result is None 2026-02-24T07:23:27.387 INFO:teuthology.orchestra.run.trial091.stdout:Extra Packages for Enterprise Linux 54 kB/s | 20 MB 06:25 2026-02-24T07:23:30.579 INFO:teuthology.orchestra.run.trial091.stdout:lab-extras 1.7 MB/s | 50 kB 00:00 2026-02-24T07:23:31.603 INFO:teuthology.orchestra.run.trial091.stdout:Package kernel-5.14.0-645.el9.x86_64 is already installed. 2026-02-24T07:23:31.604 INFO:teuthology.orchestra.run.trial091.stdout:Package kernel-5.14.0-681.el9.x86_64 is already installed. 2026-02-24T07:23:31.623 INFO:teuthology.orchestra.run.trial091.stdout:Dependencies resolved. 2026-02-24T07:23:31.624 INFO:teuthology.orchestra.run.trial091.stdout:================================================================================ 2026-02-24T07:23:31.625 INFO:teuthology.orchestra.run.trial091.stdout: Package Architecture Version Repository Size 2026-02-24T07:23:31.625 INFO:teuthology.orchestra.run.trial091.stdout:================================================================================ 2026-02-24T07:23:31.625 INFO:teuthology.orchestra.run.trial091.stdout:Installing: 2026-02-24T07:23:31.625 INFO:teuthology.orchestra.run.trial091.stdout: kernel x86_64 5.14.0-686.el9 baseos 901 k 2026-02-24T07:23:31.625 INFO:teuthology.orchestra.run.trial091.stdout:Installing dependencies: 2026-02-24T07:23:31.625 INFO:teuthology.orchestra.run.trial091.stdout: kernel-core x86_64 5.14.0-686.el9 baseos 17 M 2026-02-24T07:23:31.625 INFO:teuthology.orchestra.run.trial091.stdout: kernel-modules x86_64 5.14.0-686.el9 baseos 40 M 2026-02-24T07:23:31.625 INFO:teuthology.orchestra.run.trial091.stdout: kernel-modules-core x86_64 5.14.0-686.el9 baseos 31 M 2026-02-24T07:23:31.626 INFO:teuthology.orchestra.run.trial091.stdout: 2026-02-24T07:23:31.626 INFO:teuthology.orchestra.run.trial091.stdout:Transaction Summary 2026-02-24T07:23:31.626 INFO:teuthology.orchestra.run.trial091.stdout:================================================================================ 2026-02-24T07:23:31.626 INFO:teuthology.orchestra.run.trial091.stdout:Install 4 Packages 2026-02-24T07:23:31.626 INFO:teuthology.orchestra.run.trial091.stdout: 2026-02-24T07:23:31.626 INFO:teuthology.orchestra.run.trial091.stdout:Total download size: 89 M 2026-02-24T07:23:31.626 INFO:teuthology.orchestra.run.trial091.stdout:Installed size: 135 M 2026-02-24T07:23:31.626 INFO:teuthology.orchestra.run.trial091.stdout:Downloading Packages: 2026-02-24T07:23:31.918 INFO:teuthology.orchestra.run.trial091.stdout:(1/4): kernel-5.14.0-686.el9.x86_64.rpm 4.3 MB/s | 901 kB 00:00 2026-02-24T07:23:32.085 INFO:teuthology.orchestra.run.trial091.stdout:(2/4): kernel-core-5.14.0-686.el9.x86_64.rpm 46 MB/s | 17 MB 00:00 2026-02-24T07:23:32.255 INFO:teuthology.orchestra.run.trial091.stdout:(3/4): kernel-modules-core-5.14.0-686.el9.x86_6 92 MB/s | 31 MB 00:00 2026-02-24T07:23:32.313 INFO:teuthology.orchestra.run.trial091.stdout:(4/4): kernel-modules-5.14.0-686.el9.x86_64.rpm 66 MB/s | 40 MB 00:00 2026-02-24T07:23:32.313 INFO:teuthology.orchestra.run.trial091.stdout:-------------------------------------------------------------------------------- 2026-02-24T07:23:32.314 INFO:teuthology.orchestra.run.trial091.stdout:Total 129 MB/s | 89 MB 00:00 2026-02-24T07:23:32.504 INFO:teuthology.orchestra.run.trial091.stdout:Running transaction check 2026-02-24T07:23:32.704 INFO:teuthology.orchestra.run.trial091.stdout:Transaction check succeeded. 2026-02-24T07:23:32.704 INFO:teuthology.orchestra.run.trial091.stdout:Running transaction test 2026-02-24T07:23:33.094 INFO:teuthology.orchestra.run.trial091.stdout:Transaction test succeeded. 2026-02-24T07:23:33.094 INFO:teuthology.orchestra.run.trial091.stdout:Running transaction 2026-02-24T07:23:34.200 INFO:teuthology.orchestra.run.trial091.stdout: Preparing : 1/1 2026-02-24T07:23:34.487 INFO:teuthology.orchestra.run.trial091.stdout: Installing : kernel-modules-core-5.14.0-686.el9.x86_64 1/4 2026-02-24T07:23:34.509 INFO:teuthology.orchestra.run.trial091.stdout: Installing : kernel-core-5.14.0-686.el9.x86_64 2/4 2026-02-24T07:23:37.086 INFO:teuthology.orchestra.run.trial091.stdout: Running scriptlet: kernel-core-5.14.0-686.el9.x86_64 2/4 2026-02-24T07:23:37.119 INFO:teuthology.orchestra.run.trial091.stdout: Installing : kernel-modules-5.14.0-686.el9.x86_64 3/4 2026-02-24T07:23:40.158 INFO:teuthology.orchestra.run.trial091.stdout: Running scriptlet: kernel-modules-5.14.0-686.el9.x86_64 3/4 2026-02-24T07:23:40.163 INFO:teuthology.orchestra.run.trial091.stdout: Installing : kernel-5.14.0-686.el9.x86_64 4/4 2026-02-24T07:23:43.212 INFO:teuthology.orchestra.run.trial091.stdout: Running scriptlet: kernel-modules-core-5.14.0-686.el9.x86_64 4/4 2026-02-24T07:24:00.752 INFO:teuthology.orchestra.run.trial091.stdout: Running scriptlet: kernel-core-5.14.0-686.el9.x86_64 4/4 2026-02-24T07:24:00.774 INFO:teuthology.orchestra.run.trial091.stdout: Running scriptlet: kernel-modules-5.14.0-686.el9.x86_64 4/4 2026-02-24T07:24:00.967 INFO:teuthology.orchestra.run.trial091.stdout: Running scriptlet: kernel-5.14.0-686.el9.x86_64 4/4 2026-02-24T07:24:00.967 INFO:teuthology.orchestra.run.trial091.stdout: Verifying : kernel-5.14.0-686.el9.x86_64 1/4 2026-02-24T07:24:00.967 INFO:teuthology.orchestra.run.trial091.stdout: Verifying : kernel-core-5.14.0-686.el9.x86_64 2/4 2026-02-24T07:24:00.967 INFO:teuthology.orchestra.run.trial091.stdout: Verifying : kernel-modules-5.14.0-686.el9.x86_64 3/4 2026-02-24T07:24:01.024 INFO:teuthology.orchestra.run.trial091.stdout: Verifying : kernel-modules-core-5.14.0-686.el9.x86_64 4/4 2026-02-24T07:24:01.024 INFO:teuthology.orchestra.run.trial091.stdout: 2026-02-24T07:24:01.025 INFO:teuthology.orchestra.run.trial091.stdout:Installed: 2026-02-24T07:24:01.025 INFO:teuthology.orchestra.run.trial091.stdout: kernel-5.14.0-686.el9.x86_64 2026-02-24T07:24:01.025 INFO:teuthology.orchestra.run.trial091.stdout: kernel-core-5.14.0-686.el9.x86_64 2026-02-24T07:24:01.025 INFO:teuthology.orchestra.run.trial091.stdout: kernel-modules-5.14.0-686.el9.x86_64 2026-02-24T07:24:01.025 INFO:teuthology.orchestra.run.trial091.stdout: kernel-modules-core-5.14.0-686.el9.x86_64 2026-02-24T07:24:01.025 INFO:teuthology.orchestra.run.trial091.stdout: 2026-02-24T07:24:01.025 INFO:teuthology.orchestra.run.trial091.stdout:Complete! 2026-02-24T07:24:01.087 DEBUG:teuthology.orchestra.run.trial091:> rpm -q kernel | sort -rV | head -n 1 2026-02-24T07:24:01.116 INFO:teuthology.orchestra.run.trial091.stdout:kernel-5.14.0-686.el9.x86_64 2026-02-24T07:24:01.117 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-686.el9.x86_64 2026-02-24T07:24:01.117 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-681.el9.x86_64 Expected: 5.14.0-686.el9.x86_64 2026-02-24T07:24:01.117 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2026-02-24T07:24:01.118 DEBUG:teuthology.task.kernel:src is distro, skipping download 2026-02-24T07:24:01.118 INFO:teuthology.task.kernel:Installing distro kernel on osd.2... 2026-02-24T07:24:01.118 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@trial091.front.sepia.ceph.com, path=None, version=distro) 2026-02-24T07:24:01.118 DEBUG:teuthology.orchestra.run.trial091:> rpm -q kernel | sort -rV | head -n 1 2026-02-24T07:24:01.182 INFO:teuthology.orchestra.run.trial091.stdout:kernel-5.14.0-686.el9.x86_64 2026-02-24T07:24:01.183 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-686.el9.x86_64 2026-02-24T07:24:01.183 DEBUG:teuthology.orchestra.run.trial091:> sudo rpm -qi grub2-tools 2026-02-24T07:24:01.251 INFO:teuthology.orchestra.run.trial091.stdout:Name : grub2-tools 2026-02-24T07:24:01.252 INFO:teuthology.orchestra.run.trial091.stdout:Epoch : 1 2026-02-24T07:24:01.252 INFO:teuthology.orchestra.run.trial091.stdout:Version : 2.06 2026-02-24T07:24:01.252 INFO:teuthology.orchestra.run.trial091.stdout:Release : 120.el9 2026-02-24T07:24:01.252 INFO:teuthology.orchestra.run.trial091.stdout:Architecture: x86_64 2026-02-24T07:24:01.252 INFO:teuthology.orchestra.run.trial091.stdout:Install Date: Wed 18 Feb 2026 05:18:42 PM UTC 2026-02-24T07:24:01.252 INFO:teuthology.orchestra.run.trial091.stdout:Group : Unspecified 2026-02-24T07:24:01.252 INFO:teuthology.orchestra.run.trial091.stdout:Size : 9108611 2026-02-24T07:24:01.252 INFO:teuthology.orchestra.run.trial091.stdout:License : GPLv3+ 2026-02-24T07:24:01.252 INFO:teuthology.orchestra.run.trial091.stdout:Signature : RSA/SHA256, Thu 04 Dec 2025 09:42:59 AM UTC, Key ID 05b555b38483c65d 2026-02-24T07:24:01.253 INFO:teuthology.orchestra.run.trial091.stdout:Source RPM : grub2-2.06-120.el9.src.rpm 2026-02-24T07:24:01.253 INFO:teuthology.orchestra.run.trial091.stdout:Build Date : Wed 03 Dec 2025 12:10:38 PM UTC 2026-02-24T07:24:01.253 INFO:teuthology.orchestra.run.trial091.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2026-02-24T07:24:01.253 INFO:teuthology.orchestra.run.trial091.stdout:Packager : builder@centos.org 2026-02-24T07:24:01.253 INFO:teuthology.orchestra.run.trial091.stdout:Vendor : CentOS 2026-02-24T07:24:01.253 INFO:teuthology.orchestra.run.trial091.stdout:URL : http://www.gnu.org/software/grub/ 2026-02-24T07:24:01.253 INFO:teuthology.orchestra.run.trial091.stdout:Summary : Support tools for GRUB. 2026-02-24T07:24:01.253 INFO:teuthology.orchestra.run.trial091.stdout:Description : 2026-02-24T07:24:01.253 INFO:teuthology.orchestra.run.trial091.stdout: 2026-02-24T07:24:01.253 INFO:teuthology.orchestra.run.trial091.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2026-02-24T07:24:01.253 INFO:teuthology.orchestra.run.trial091.stdout:customizable bootloader with modular architecture. It supports a rich 2026-02-24T07:24:01.253 INFO:teuthology.orchestra.run.trial091.stdout:variety of kernel formats, file systems, computer architectures and 2026-02-24T07:24:01.254 INFO:teuthology.orchestra.run.trial091.stdout:hardware devices. 2026-02-24T07:24:01.254 INFO:teuthology.orchestra.run.trial091.stdout: 2026-02-24T07:24:01.254 INFO:teuthology.orchestra.run.trial091.stdout:This subpackage provides tools for support of all platforms. 2026-02-24T07:24:01.255 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2026-02-24T07:24:01.255 INFO:teuthology.task.kernel:Updating grub on trial091 to boot 5.14.0-686.el9.x86_64 2026-02-24T07:24:01.255 DEBUG:teuthology.orchestra.run.trial091:> sudo find /boot/loader/entries -maxdepth 1 -name "*.conf" -type f -print -quit 2026-02-24T07:24:01.320 INFO:teuthology.orchestra.run.trial091.stdout:/boot/loader/entries/b0373be789df49cdafedf423703d2566-5.14.0-681.el9.x86_64.conf 2026-02-24T07:24:01.321 DEBUG:teuthology.orchestra.run.trial091:> sudo command -v grubby && echo yes || echo no 2026-02-24T07:24:01.385 INFO:teuthology.orchestra.run.trial091.stdout:/sbin/grubby 2026-02-24T07:24:01.386 INFO:teuthology.orchestra.run.trial091.stdout:yes 2026-02-24T07:24:01.387 DEBUG:teuthology.orchestra.run.trial091:> sudo find /boot/loader/entries -maxdepth 1 -name "*.conf" -type f | grep -F "5.14.0-686.el9.x86_64" | head -n 1 2026-02-24T07:24:01.451 INFO:teuthology.orchestra.run.trial091.stdout:/boot/loader/entries/b0373be789df49cdafedf423703d2566-5.14.0-686.el9.x86_64.conf 2026-02-24T07:24:01.453 DEBUG:teuthology.orchestra.run.trial091:> sudo grub2-set-default b0373be789df49cdafedf423703d2566-5.14.0-686.el9.x86_64 2026-02-24T07:24:01.525 DEBUG:teuthology.orchestra.run.trial091:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2026-02-24T07:24:01.732 INFO:teuthology.orchestra.run.trial091.stderr:Generating grub configuration file ... 2026-02-24T07:24:01.801 INFO:teuthology.orchestra.run.trial091.stderr:Found linux image: /boot/vmlinuz-5.14.0-686.el9.x86_64 2026-02-24T07:24:01.807 INFO:teuthology.orchestra.run.trial091.stderr:Found initrd image: /boot/initramfs-5.14.0-686.el9.x86_64.img 2026-02-24T07:24:02.060 INFO:teuthology.orchestra.run.trial091.stderr:Found linux image: /boot/vmlinuz-5.14.0-681.el9.x86_64 2026-02-24T07:24:02.068 INFO:teuthology.orchestra.run.trial091.stderr:Found initrd image: /boot/initramfs-5.14.0-681.el9.x86_64.img 2026-02-24T07:24:02.102 INFO:teuthology.orchestra.run.trial091.stderr:Found linux image: /boot/vmlinuz-5.14.0-645.el9.x86_64 2026-02-24T07:24:02.108 INFO:teuthology.orchestra.run.trial091.stderr:Found initrd image: /boot/initramfs-5.14.0-645.el9.x86_64.img 2026-02-24T07:24:02.140 INFO:teuthology.orchestra.run.trial091.stderr:Found linux image: /boot/vmlinuz-0-rescue-b0373be789df49cdafedf423703d2566 2026-02-24T07:24:02.146 INFO:teuthology.orchestra.run.trial091.stderr:Found initrd image: /boot/initramfs-0-rescue-b0373be789df49cdafedf423703d2566.img 2026-02-24T07:24:02.170 INFO:teuthology.orchestra.run.trial091.stderr:Found linux image: /boot/vmlinuz-0-rescue-2734456005d743dca7f3dc77c35a2d0d 2026-02-24T07:24:02.177 INFO:teuthology.orchestra.run.trial091.stderr:Found initrd image: /boot/initramfs-0-rescue-2734456005d743dca7f3dc77c35a2d0d.img 2026-02-24T07:24:02.263 INFO:teuthology.orchestra.run.trial091.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2026-02-24T07:24:02.271 INFO:teuthology.orchestra.run.trial091.stderr:done 2026-02-24T07:24:02.273 DEBUG:teuthology.orchestra.run.trial091:> test -d /sys/firmware/efi 2026-02-24T07:24:02.289 DEBUG:teuthology.orchestra.run.trial091:> sudo find /boot/efi/EFI -mindepth 1 -maxdepth 1 -type d || true 2026-02-24T07:24:02.354 INFO:teuthology.orchestra.run.trial091.stdout:/boot/efi/EFI/BOOT 2026-02-24T07:24:02.354 INFO:teuthology.orchestra.run.trial091.stdout:/boot/efi/EFI/centos 2026-02-24T07:24:02.356 DEBUG:teuthology.orchestra.run.trial091:> sudo test -f /boot/efi/EFI/BOOT/grub.cfg && echo yes || echo no 2026-02-24T07:24:02.421 INFO:teuthology.orchestra.run.trial091.stdout:no 2026-02-24T07:24:02.422 DEBUG:teuthology.orchestra.run.trial091:> sudo test -f /boot/efi/EFI/centos/grub.cfg && echo yes || echo no 2026-02-24T07:24:02.485 INFO:teuthology.orchestra.run.trial091.stdout:yes 2026-02-24T07:24:02.485 DEBUG:teuthology.orchestra.run.trial091:> sudo cp -f /boot/grub2/grub.cfg /boot/efi/EFI/centos/grub.cfg 2026-02-24T07:24:02.548 INFO:teuthology.orchestra.remote:Triggering safe hard reboot via SysRq on trial091 2026-02-24T07:24:02.549 DEBUG:teuthology.orchestra.run.trial091:> sudo bash -c ' 2026-02-24T07:24:02.549 DEBUG:teuthology.orchestra.run.trial091:> echo pci > /sys/kernel/reboot/type && echo 1 > /sys/kernel/reboot/force && sync && echo s > /proc/sysrq-trigger && echo u > /proc/sysrq-trigger && sleep 2 && echo b > /proc/sysrq-trigger 2026-02-24T07:24:02.549 DEBUG:teuthology.orchestra.run.trial091:> ' 2026-02-24T07:24:32.591 INFO:teuthology.task.kernel:Checking client osd.2 for new kernel version... 2026-02-24T07:24:32.592 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@trial091.front.sepia.ceph.com' 2026-02-24T07:24:32.593 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial091.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2026-02-24T07:25:32.595 DEBUG:teuthology.orchestra.remote:timed out 2026-02-24T07:25:41.596 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@trial091.front.sepia.ceph.com' 2026-02-24T07:25:41.596 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial091.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2026-02-24T07:25:41.731 DEBUG:teuthology.orchestra.run.trial091:> true 2026-02-24T07:25:41.879 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@trial091.front.sepia.ceph.com' 2026-02-24T07:25:41.879 INFO:teuthology.task.kernel:Checking kernel version of osd.2, want "5.14.0-686.el9.x86_64"... 2026-02-24T07:25:41.879 DEBUG:teuthology.orchestra.run.trial091:> uname -r 2026-02-24T07:25:41.935 INFO:teuthology.orchestra.run.trial091.stdout:5.14.0-686.el9.x86_64 2026-02-24T07:25:41.935 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-686.el9.x86_64 vs 5.14.0-686.el9.x86_64 2026-02-24T07:25:41.936 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2026-02-24T07:25:41.936 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2026-02-24T07:25:42.937 INFO:teuthology.task.kernel:Enabling kdb on osd.2... 2026-02-24T07:25:42.937 DEBUG:teuthology.orchestra.run.trial091:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2026-02-24T07:25:42.967 INFO:teuthology.orchestra.run.trial091.stdout:ttyS1 2026-02-24T07:25:42.990 DEBUG:teuthology.parallel:result is None 2026-02-24T07:25:42.990 INFO:teuthology.run_tasks:Running task internal.base... 2026-02-24T07:25:43.000 INFO:teuthology.task.internal:Creating test directory... 2026-02-24T07:25:43.000 DEBUG:teuthology.orchestra.run.trial035:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2026-02-24T07:25:43.013 DEBUG:teuthology.orchestra.run.trial091:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2026-02-24T07:25:43.045 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2026-02-24T07:25:43.056 INFO:teuthology.run_tasks:Running task internal.archive... 2026-02-24T07:25:43.069 INFO:teuthology.task.internal:Creating archive directory... 2026-02-24T07:25:43.069 DEBUG:teuthology.orchestra.run.trial035:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2026-02-24T07:25:43.070 DEBUG:teuthology.orchestra.run.trial091:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2026-02-24T07:25:43.108 INFO:teuthology.run_tasks:Running task internal.coredump... 2026-02-24T07:25:43.119 INFO:teuthology.task.internal:Enabling coredump saving... 2026-02-24T07:25:43.119 DEBUG:teuthology.orchestra.run.trial035:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2026-02-24T07:25:43.127 DEBUG:teuthology.orchestra.run.trial091:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2026-02-24T07:25:43.149 INFO:teuthology.orchestra.run.trial035.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-24T07:25:43.158 INFO:teuthology.orchestra.run.trial035.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-24T07:25:43.166 INFO:teuthology.orchestra.run.trial091.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-24T07:25:43.174 INFO:teuthology.orchestra.run.trial091.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-24T07:25:43.176 INFO:teuthology.run_tasks:Running task internal.sudo... 2026-02-24T07:25:43.190 INFO:teuthology.task.internal:Configuring sudo... 2026-02-24T07:25:43.190 DEBUG:teuthology.orchestra.run.trial035:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2026-02-24T07:25:43.202 DEBUG:teuthology.orchestra.run.trial091:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2026-02-24T07:25:43.239 INFO:teuthology.run_tasks:Running task internal.syslog... 2026-02-24T07:25:43.251 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2026-02-24T07:25:43.251 DEBUG:teuthology.orchestra.run.trial035:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2026-02-24T07:25:43.267 DEBUG:teuthology.orchestra.run.trial091:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2026-02-24T07:25:43.294 DEBUG:teuthology.orchestra.run.trial035:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2026-02-24T07:25:43.441 DEBUG:teuthology.orchestra.run.trial035:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2026-02-24T07:25:43.466 DEBUG:teuthology.orchestra.run.trial035:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2026-02-24T07:25:43.612 DEBUG:teuthology.orchestra.run.trial035:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2026-02-24T07:25:43.636 DEBUG:teuthology.orchestra.run.trial035:> set -ex 2026-02-24T07:25:43.636 DEBUG:teuthology.orchestra.run.trial035:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2026-02-24T07:25:43.700 DEBUG:teuthology.orchestra.run.trial091:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2026-02-24T07:25:43.815 DEBUG:teuthology.orchestra.run.trial091:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2026-02-24T07:25:43.840 DEBUG:teuthology.orchestra.run.trial091:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2026-02-24T07:25:43.981 DEBUG:teuthology.orchestra.run.trial091:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2026-02-24T07:25:44.006 DEBUG:teuthology.orchestra.run.trial091:> set -ex 2026-02-24T07:25:44.006 DEBUG:teuthology.orchestra.run.trial091:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2026-02-24T07:25:44.069 DEBUG:teuthology.orchestra.run.trial035:> sudo service rsyslog restart 2026-02-24T07:25:44.071 DEBUG:teuthology.orchestra.run.trial091:> sudo service rsyslog restart 2026-02-24T07:25:44.098 INFO:teuthology.orchestra.run.trial035.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2026-02-24T07:25:44.139 INFO:teuthology.orchestra.run.trial091.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2026-02-24T07:25:44.667 INFO:teuthology.run_tasks:Running task internal.timer... 2026-02-24T07:25:44.681 INFO:teuthology.task.internal:Starting timer... 2026-02-24T07:25:44.681 INFO:teuthology.run_tasks:Running task pcp... 2026-02-24T07:25:44.704 INFO:teuthology.run_tasks:Running task selinux... 2026-02-24T07:25:44.720 DEBUG:teuthology.orchestra.run.trial035:> sudo service auditd rotate 2026-02-24T07:25:44.750 INFO:teuthology.orchestra.run.trial035.stdout:Rotating logs: 2026-02-24T07:25:44.752 DEBUG:teuthology.orchestra.run.trial091:> sudo service auditd rotate 2026-02-24T07:25:44.781 INFO:teuthology.orchestra.run.trial091.stdout:Rotating logs: 2026-02-24T07:25:44.782 DEBUG:teuthology.task.selinux:Getting current SELinux state 2026-02-24T07:25:44.782 DEBUG:teuthology.orchestra.run.trial035:> /usr/sbin/getenforce 2026-02-24T07:25:44.808 INFO:teuthology.orchestra.run.trial035.stdout:Permissive 2026-02-24T07:25:44.808 DEBUG:teuthology.orchestra.run.trial091:> /usr/sbin/getenforce 2026-02-24T07:25:44.838 INFO:teuthology.orchestra.run.trial091.stdout:Permissive 2026-02-24T07:25:44.838 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@trial035.front.sepia.ceph.com': 'permissive', 'ubuntu@trial091.front.sepia.ceph.com': 'permissive'} 2026-02-24T07:25:44.838 DEBUG:teuthology.orchestra.run.trial035:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 2026-02-24T07:25:44.872 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-24T07:25:44.873 DEBUG:teuthology.orchestra.run.trial091:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 2026-02-24T07:25:44.902 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-24T07:25:44.903 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2026-02-24T07:25:44.903 DEBUG:teuthology.orchestra.run.trial035:> sudo /usr/sbin/setenforce permissive 2026-02-24T07:25:44.938 DEBUG:teuthology.orchestra.run.trial091:> sudo /usr/sbin/setenforce permissive 2026-02-24T07:25:44.966 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2026-02-24T07:25:44.981 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2026-02-24T07:25:44.992 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2026-02-24T07:25:44.994 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit trial035.front.sepia.ceph.com,trial091.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2026-02-24T07:29:03.989 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@trial035.front.sepia.ceph.com'), Remote(name='ubuntu@trial091.front.sepia.ceph.com')] 2026-02-24T07:29:03.991 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@trial035.front.sepia.ceph.com' 2026-02-24T07:29:03.992 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial035.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2026-02-24T07:29:04.143 DEBUG:teuthology.orchestra.run.trial035:> true 2026-02-24T07:29:04.222 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@trial035.front.sepia.ceph.com' 2026-02-24T07:29:04.222 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@trial091.front.sepia.ceph.com' 2026-02-24T07:29:04.223 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial091.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2026-02-24T07:29:04.328 DEBUG:teuthology.orchestra.run.trial091:> true 2026-02-24T07:29:04.403 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@trial091.front.sepia.ceph.com' 2026-02-24T07:29:04.403 INFO:teuthology.run_tasks:Running task clock... 2026-02-24T07:29:04.418 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2026-02-24T07:29:04.418 INFO:teuthology.orchestra.run:Running command with timeout 360 2026-02-24T07:29:04.419 DEBUG:teuthology.orchestra.run.trial035:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2026-02-24T07:29:04.421 INFO:teuthology.orchestra.run:Running command with timeout 360 2026-02-24T07:29:04.421 DEBUG:teuthology.orchestra.run.trial091:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2026-02-24T07:29:04.448 INFO:teuthology.orchestra.run.trial035.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2026-02-24T07:29:04.460 INFO:teuthology.orchestra.run.trial035.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2026-02-24T07:29:04.470 INFO:teuthology.orchestra.run.trial091.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2026-02-24T07:29:04.481 INFO:teuthology.orchestra.run.trial091.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2026-02-24T07:29:04.482 INFO:teuthology.orchestra.run.trial035.stderr:sudo: ntpd: command not found 2026-02-24T07:29:04.491 INFO:teuthology.orchestra.run.trial035.stdout:506 Cannot talk to daemon 2026-02-24T07:29:04.500 INFO:teuthology.orchestra.run.trial091.stderr:sudo: ntpd: command not found 2026-02-24T07:29:04.504 INFO:teuthology.orchestra.run.trial035.stderr:Failed to start ntp.service: Unit ntp.service not found. 2026-02-24T07:29:04.511 INFO:teuthology.orchestra.run.trial091.stdout:506 Cannot talk to daemon 2026-02-24T07:29:04.516 INFO:teuthology.orchestra.run.trial035.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2026-02-24T07:29:04.522 INFO:teuthology.orchestra.run.trial091.stderr:Failed to start ntp.service: Unit ntp.service not found. 2026-02-24T07:29:04.534 INFO:teuthology.orchestra.run.trial091.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2026-02-24T07:29:04.563 INFO:teuthology.orchestra.run.trial035.stderr:bash: line 1: ntpq: command not found 2026-02-24T07:29:04.565 INFO:teuthology.orchestra.run.trial035.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2026-02-24T07:29:04.565 INFO:teuthology.orchestra.run.trial035.stdout:=============================================================================== 2026-02-24T07:29:04.577 INFO:teuthology.orchestra.run.trial091.stderr:bash: line 1: ntpq: command not found 2026-02-24T07:29:04.586 INFO:teuthology.orchestra.run.trial091.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2026-02-24T07:29:04.586 INFO:teuthology.orchestra.run.trial091.stdout:=============================================================================== 2026-02-24T07:29:04.586 INFO:teuthology.orchestra.run.trial091.stdout:^? 172-104-209-204.ip.linod> 0 6 0 - +0ns[ +0ns] +/- 0ns 2026-02-24T07:29:04.586 INFO:teuthology.orchestra.run.trial091.stdout:^? ntp.maxhost.io 0 6 0 - +0ns[ +0ns] +/- 0ns 2026-02-24T07:29:04.587 INFO:teuthology.run_tasks:Running task install... 2026-02-24T07:29:04.601 DEBUG:teuthology.task.install:project ceph 2026-02-24T07:29:04.601 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'debug', 'sha1': 'ff63f3d997e38f69382fd23ac0206f07f7b05017'}} 2026-02-24T07:29:04.601 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['ceph-osd-crimson', 'ceph-osd-crimson-dbg'], 'rpm': ['ceph-osd-crimson', 'ceph-osd-crimson-debuginfo']}, 'flavor': 'debug', 'sha1': 'ff63f3d997e38f69382fd23ac0206f07f7b05017'} 2026-02-24T07:29:04.601 INFO:teuthology.task.install:Using flavor: debug 2026-02-24T07:29:04.611 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-osd', 'ceph-osd-classic', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-osd', 'ceph-osd-classic', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2026-02-24T07:29:04.611 INFO:teuthology.task.install:extra packages: {'deb': ['ceph-osd-crimson', 'ceph-osd-crimson-dbg'], 'rpm': ['ceph-osd-crimson', 'ceph-osd-crimson-debuginfo']} 2026-02-24T07:29:04.612 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['ceph-osd-crimson', 'ceph-osd-crimson-dbg'], 'rpm': ['ceph-osd-crimson', 'ceph-osd-crimson-debuginfo']}, 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'debug', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'ff63f3d997e38f69382fd23ac0206f07f7b05017', 'tag': None, 'wait_for_package': False} 2026-02-24T07:29:04.613 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=debug&distros=centos%2F9%2Fx86_64&sha1=ff63f3d997e38f69382fd23ac0206f07f7b05017 2026-02-24T07:29:04.615 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['ceph-osd-crimson', 'ceph-osd-crimson-dbg'], 'rpm': ['ceph-osd-crimson', 'ceph-osd-crimson-debuginfo']}, 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'debug', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'ff63f3d997e38f69382fd23ac0206f07f7b05017', 'tag': None, 'wait_for_package': False} 2026-02-24T07:29:04.615 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=debug&distros=centos%2F9%2Fx86_64&sha1=ff63f3d997e38f69382fd23ac0206f07f7b05017 2026-02-24T07:29:04.810 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/wip-shraddhaag-enable-cephadm-tests/ff63f3d997e38f69382fd23ac0206f07f7b05017/centos/9/flavors/debug/ 2026-02-24T07:29:04.810 INFO:teuthology.task.install.rpm:Package version is 20.3.0-5440.gff63f3d9 2026-02-24T07:29:04.821 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/wip-shraddhaag-enable-cephadm-tests/ff63f3d997e38f69382fd23ac0206f07f7b05017/centos/9/flavors/debug/ 2026-02-24T07:29:04.822 INFO:teuthology.task.install.rpm:Package version is 20.3.0-5440.gff63f3d9 2026-02-24T07:29:04.964 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/wip-shraddhaag-enable-cephadm-tests/ff63f3d997e38f69382fd23ac0206f07f7b05017/centos/9/flavors/debug/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-shraddhaag-enable-cephadm-tests/ff63f3d997e38f69382fd23ac0206f07f7b05017/centos/9/flavors/debug/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-shraddhaag-enable-cephadm-tests/ff63f3d997e38f69382fd23ac0206f07f7b05017/centos/9/flavors/debug/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2026-02-24T07:29:04.964 DEBUG:teuthology.orchestra.run.trial035:> set -ex 2026-02-24T07:29:04.964 DEBUG:teuthology.orchestra.run.trial035:> sudo dd of=/etc/yum.repos.d/ceph.repo 2026-02-24T07:29:04.971 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/wip-shraddhaag-enable-cephadm-tests/ff63f3d997e38f69382fd23ac0206f07f7b05017/centos/9/flavors/debug/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-shraddhaag-enable-cephadm-tests/ff63f3d997e38f69382fd23ac0206f07f7b05017/centos/9/flavors/debug/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-shraddhaag-enable-cephadm-tests/ff63f3d997e38f69382fd23ac0206f07f7b05017/centos/9/flavors/debug/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2026-02-24T07:29:04.971 DEBUG:teuthology.orchestra.run.trial091:> set -ex 2026-02-24T07:29:04.971 DEBUG:teuthology.orchestra.run.trial091:> sudo dd of=/etc/yum.repos.d/ceph.repo 2026-02-24T07:29:04.990 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-osd, ceph-osd-classic, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, ceph-osd-crimson, ceph-osd-crimson-debuginfo on remote rpm x86_64 2026-02-24T07:29:04.990 DEBUG:teuthology.orchestra.run.trial035:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/ff63f3d997e38f69382fd23ac0206f07f7b05017/;g' /etc/yum.repos.d/ceph.repo ; fi 2026-02-24T07:29:04.996 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-osd, ceph-osd-classic, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, ceph-osd-crimson, ceph-osd-crimson-debuginfo on remote rpm x86_64 2026-02-24T07:29:04.996 DEBUG:teuthology.orchestra.run.trial091:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/ff63f3d997e38f69382fd23ac0206f07f7b05017/;g' /etc/yum.repos.d/ceph.repo ; fi 2026-02-24T07:29:05.055 DEBUG:teuthology.orchestra.run.trial035:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2026-02-24T07:29:05.061 DEBUG:teuthology.orchestra.run.trial091:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2026-02-24T07:29:05.128 DEBUG:teuthology.orchestra.run.trial035:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2026-02-24T07:29:05.134 DEBUG:teuthology.orchestra.run.trial091:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2026-02-24T07:29:05.192 INFO:teuthology.orchestra.run.trial035.stdout:check_obsoletes = 1 2026-02-24T07:29:05.193 DEBUG:teuthology.orchestra.run.trial035:> sudo yum clean all 2026-02-24T07:29:05.197 INFO:teuthology.orchestra.run.trial091.stdout:check_obsoletes = 1 2026-02-24T07:29:05.199 DEBUG:teuthology.orchestra.run.trial091:> sudo yum clean all 2026-02-24T07:29:05.342 INFO:teuthology.orchestra.run.trial035.stdout:41 files removed 2026-02-24T07:29:05.346 INFO:teuthology.orchestra.run.trial091.stdout:41 files removed 2026-02-24T07:29:05.357 DEBUG:teuthology.orchestra.run.trial035:> sudo yum -y install ceph-radosgw 2026-02-24T07:29:05.361 DEBUG:teuthology.orchestra.run.trial091:> sudo yum -y install ceph-radosgw 2026-02-24T07:29:05.651 INFO:teuthology.orchestra.run.trial091.stdout:ceph packages for x86_64 660 kB/s | 94 kB 00:00 2026-02-24T07:29:05.653 INFO:teuthology.orchestra.run.trial035.stdout:ceph packages for x86_64 634 kB/s | 94 kB 00:00 2026-02-24T07:29:05.792 INFO:teuthology.orchestra.run.trial091.stdout:ceph noarch packages 217 kB/s | 25 kB 00:00 2026-02-24T07:29:05.799 INFO:teuthology.orchestra.run.trial035.stdout:ceph noarch packages 209 kB/s | 25 kB 00:00 2026-02-24T07:29:05.922 INFO:teuthology.orchestra.run.trial091.stdout:ceph source packages 20 kB/s | 2.1 kB 00:00 2026-02-24T07:29:05.929 INFO:teuthology.orchestra.run.trial035.stdout:ceph source packages 20 kB/s | 2.1 kB 00:00 2026-02-24T07:29:06.368 INFO:teuthology.orchestra.run.trial035.stdout:CentOS Stream 9 - BaseOS 21 MB/s | 8.9 MB 00:00 2026-02-24T07:29:07.733 INFO:teuthology.orchestra.run.trial091.stdout:CentOS Stream 9 - BaseOS 5.0 MB/s | 8.9 MB 00:01 2026-02-24T07:29:08.569 INFO:teuthology.orchestra.run.trial091.stdout:CentOS Stream 9 - AppStream 89 MB/s | 27 MB 00:00 2026-02-24T07:29:09.544 INFO:teuthology.orchestra.run.trial035.stdout:CentOS Stream 9 - AppStream 10 MB/s | 27 MB 00:02 2026-02-24T07:29:12.514 INFO:teuthology.orchestra.run.trial091.stdout:CentOS Stream 9 - CRB 5.1 MB/s | 8.0 MB 00:01 2026-02-24T07:29:13.276 INFO:teuthology.orchestra.run.trial035.stdout:CentOS Stream 9 - CRB 5.9 MB/s | 8.0 MB 00:01 2026-02-24T07:29:14.086 INFO:teuthology.orchestra.run.trial091.stdout:CentOS Stream 9 - Extras packages 24 kB/s | 20 kB 00:00 2026-02-24T07:29:14.243 INFO:teuthology.orchestra.run.trial035.stdout:CentOS Stream 9 - Extras packages 91 kB/s | 20 kB 00:00 2026-02-24T07:29:18.735 INFO:teuthology.orchestra.run.trial035.stdout:Extra Packages for Enterprise Linux 4.6 MB/s | 20 MB 00:04 2026-02-24T07:29:21.921 INFO:teuthology.orchestra.run.trial035.stdout:lab-extras 1.4 MB/s | 50 kB 00:00 2026-02-24T07:29:23.140 INFO:teuthology.orchestra.run.trial035.stdout:Dependencies resolved. 2026-02-24T07:29:23.141 INFO:teuthology.orchestra.run.trial035.stdout:================================================================================ 2026-02-24T07:29:23.141 INFO:teuthology.orchestra.run.trial035.stdout: Package Arch Version Repository Size 2026-02-24T07:29:23.141 INFO:teuthology.orchestra.run.trial035.stdout:================================================================================ 2026-02-24T07:29:23.141 INFO:teuthology.orchestra.run.trial035.stdout:Installing: 2026-02-24T07:29:23.141 INFO:teuthology.orchestra.run.trial035.stdout: ceph-radosgw x86_64 2:20.3.0-5440.gff63f3d9.el9 ceph 11 M 2026-02-24T07:29:23.141 INFO:teuthology.orchestra.run.trial035.stdout:Upgrading: 2026-02-24T07:29:23.141 INFO:teuthology.orchestra.run.trial035.stdout: librados2 x86_64 2:20.3.0-5440.gff63f3d9.el9 ceph 3.3 M 2026-02-24T07:29:23.141 INFO:teuthology.orchestra.run.trial035.stdout: librbd1 x86_64 2:20.3.0-5440.gff63f3d9.el9 ceph 2.7 M 2026-02-24T07:29:23.142 INFO:teuthology.orchestra.run.trial035.stdout:Installing dependencies: 2026-02-24T07:29:23.142 INFO:teuthology.orchestra.run.trial035.stdout: boost-program-options x86_64 1.75.0-13.el9 appstream 104 k 2026-02-24T07:29:23.142 INFO:teuthology.orchestra.run.trial035.stdout: ceph-base x86_64 2:20.3.0-5440.gff63f3d9.el9 ceph 5.1 M 2026-02-24T07:29:23.142 INFO:teuthology.orchestra.run.trial035.stdout: ceph-common x86_64 2:20.3.0-5440.gff63f3d9.el9 ceph 21 M 2026-02-24T07:29:23.142 INFO:teuthology.orchestra.run.trial035.stdout: ceph-selinux x86_64 2:20.3.0-5440.gff63f3d9.el9 ceph 25 k 2026-02-24T07:29:23.142 INFO:teuthology.orchestra.run.trial035.stdout: gperftools-libs x86_64 2.9.1-3.el9 epel 308 k 2026-02-24T07:29:23.142 INFO:teuthology.orchestra.run.trial035.stdout: libarrow x86_64 9.0.0-15.el9 epel 4.4 M 2026-02-24T07:29:23.142 INFO:teuthology.orchestra.run.trial035.stdout: libarrow-doc noarch 9.0.0-15.el9 epel 25 k 2026-02-24T07:29:23.142 INFO:teuthology.orchestra.run.trial035.stdout: libcephfs2 x86_64 2:20.3.0-5440.gff63f3d9.el9 ceph 859 k 2026-02-24T07:29:23.142 INFO:teuthology.orchestra.run.trial035.stdout: libnbd x86_64 1.20.3-4.el9 appstream 164 k 2026-02-24T07:29:23.142 INFO:teuthology.orchestra.run.trial035.stdout: liboath x86_64 2.6.12-1.el9 epel 49 k 2026-02-24T07:29:23.142 INFO:teuthology.orchestra.run.trial035.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2026-02-24T07:29:23.142 INFO:teuthology.orchestra.run.trial035.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2026-02-24T07:29:23.142 INFO:teuthology.orchestra.run.trial035.stdout: libradosstriper1 x86_64 2:20.3.0-5440.gff63f3d9.el9 ceph 263 k 2026-02-24T07:29:23.142 INFO:teuthology.orchestra.run.trial035.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2026-02-24T07:29:23.142 INFO:teuthology.orchestra.run.trial035.stdout: librgw2 x86_64 2:20.3.0-5440.gff63f3d9.el9 ceph 5.6 M 2026-02-24T07:29:23.142 INFO:teuthology.orchestra.run.trial035.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2026-02-24T07:29:23.142 INFO:teuthology.orchestra.run.trial035.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2026-02-24T07:29:23.142 INFO:teuthology.orchestra.run.trial035.stdout: mailcap noarch 2.1.49-5.el9 baseos 33 k 2026-02-24T07:29:23.142 INFO:teuthology.orchestra.run.trial035.stdout: parquet-libs x86_64 9.0.0-15.el9 epel 838 k 2026-02-24T07:29:23.142 INFO:teuthology.orchestra.run.trial035.stdout: pciutils x86_64 3.7.0-7.el9 baseos 93 k 2026-02-24T07:29:23.142 INFO:teuthology.orchestra.run.trial035.stdout: python3-ceph-argparse x86_64 2:20.3.0-5440.gff63f3d9.el9 ceph 45 k 2026-02-24T07:29:23.142 INFO:teuthology.orchestra.run.trial035.stdout: python3-ceph-common x86_64 2:20.3.0-5440.gff63f3d9.el9 ceph 186 k 2026-02-24T07:29:23.142 INFO:teuthology.orchestra.run.trial035.stdout: python3-cephfs x86_64 2:20.3.0-5440.gff63f3d9.el9 ceph 204 k 2026-02-24T07:29:23.142 INFO:teuthology.orchestra.run.trial035.stdout: python3-rados x86_64 2:20.3.0-5440.gff63f3d9.el9 ceph 324 k 2026-02-24T07:29:23.142 INFO:teuthology.orchestra.run.trial035.stdout: python3-rbd x86_64 2:20.3.0-5440.gff63f3d9.el9 ceph 306 k 2026-02-24T07:29:23.142 INFO:teuthology.orchestra.run.trial035.stdout: python3-rgw x86_64 2:20.3.0-5440.gff63f3d9.el9 ceph 100 k 2026-02-24T07:29:23.142 INFO:teuthology.orchestra.run.trial035.stdout: qatlib x86_64 25.08.0-2.el9 appstream 240 k 2026-02-24T07:29:23.142 INFO:teuthology.orchestra.run.trial035.stdout: qatzip-libs x86_64 1.3.1-1.el9 appstream 66 k 2026-02-24T07:29:23.142 INFO:teuthology.orchestra.run.trial035.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2026-02-24T07:29:23.142 INFO:teuthology.orchestra.run.trial035.stdout: thrift x86_64 0.15.0-4.el9 epel 1.6 M 2026-02-24T07:29:23.142 INFO:teuthology.orchestra.run.trial035.stdout:Installing weak dependencies: 2026-02-24T07:29:23.142 INFO:teuthology.orchestra.run.trial035.stdout: libcephfs-daemon x86_64 2:20.3.0-5440.gff63f3d9.el9 ceph 39 k 2026-02-24T07:29:23.142 INFO:teuthology.orchestra.run.trial035.stdout: libcephfs-proxy2 x86_64 2:20.3.0-5440.gff63f3d9.el9 ceph 26 k 2026-02-24T07:29:23.142 INFO:teuthology.orchestra.run.trial035.stdout: nvme-cli x86_64 2.16-1.el9 baseos 1.2 M 2026-02-24T07:29:23.142 INFO:teuthology.orchestra.run.trial035.stdout: qatlib-service x86_64 25.08.0-2.el9 appstream 37 k 2026-02-24T07:29:23.143 INFO:teuthology.orchestra.run.trial035.stdout: 2026-02-24T07:29:23.143 INFO:teuthology.orchestra.run.trial035.stdout:Transaction Summary 2026-02-24T07:29:23.143 INFO:teuthology.orchestra.run.trial035.stdout:================================================================================ 2026-02-24T07:29:23.143 INFO:teuthology.orchestra.run.trial035.stdout:Install 35 Packages 2026-02-24T07:29:23.143 INFO:teuthology.orchestra.run.trial035.stdout:Upgrade 2 Packages 2026-02-24T07:29:23.143 INFO:teuthology.orchestra.run.trial035.stdout: 2026-02-24T07:29:23.143 INFO:teuthology.orchestra.run.trial035.stdout:Total download size: 61 M 2026-02-24T07:29:23.143 INFO:teuthology.orchestra.run.trial035.stdout:Downloading Packages: 2026-02-24T07:29:24.200 INFO:teuthology.orchestra.run.trial035.stdout:(1/37): ceph-base-20.3.0-5440.gff63f3d9.el9.x86 14 MB/s | 5.1 MB 00:00 2026-02-24T07:29:24.243 INFO:teuthology.orchestra.run.trial035.stdout:(2/37): ceph-selinux-20.3.0-5440.gff63f3d9.el9. 589 kB/s | 25 kB 00:00 2026-02-24T07:29:24.288 INFO:teuthology.orchestra.run.trial035.stdout:(3/37): libcephfs-daemon-20.3.0-5440.gff63f3d9. 858 kB/s | 39 kB 00:00 2026-02-24T07:29:24.335 INFO:teuthology.orchestra.run.trial035.stdout:(4/37): ceph-radosgw-20.3.0-5440.gff63f3d9.el9. 22 MB/s | 11 MB 00:00 2026-02-24T07:29:24.335 INFO:teuthology.orchestra.run.trial035.stdout:(5/37): libcephfs-proxy2-20.3.0-5440.gff63f3d9. 555 kB/s | 26 kB 00:00 2026-02-24T07:29:24.371 INFO:teuthology.orchestra.run.trial035.stdout:(6/37): libcephfs2-20.3.0-5440.gff63f3d9.el9.x8 23 MB/s | 859 kB 00:00 2026-02-24T07:29:24.376 INFO:teuthology.orchestra.run.trial035.stdout:(7/37): libradosstriper1-20.3.0-5440.gff63f3d9. 6.3 MB/s | 263 kB 00:00 2026-02-24T07:29:24.426 INFO:teuthology.orchestra.run.trial035.stdout:(8/37): python3-ceph-argparse-20.3.0-5440.gff63 898 kB/s | 45 kB 00:00 2026-02-24T07:29:24.482 INFO:teuthology.orchestra.run.trial035.stdout:(9/37): python3-ceph-common-20.3.0-5440.gff63f3 3.3 MB/s | 186 kB 00:00 2026-02-24T07:29:24.541 INFO:teuthology.orchestra.run.trial035.stdout:(10/37): python3-cephfs-20.3.0-5440.gff63f3d9.e 3.4 MB/s | 204 kB 00:00 2026-02-24T07:29:24.573 INFO:teuthology.orchestra.run.trial035.stdout:(11/37): librgw2-20.3.0-5440.gff63f3d9.el9.x86_ 28 MB/s | 5.6 MB 00:00 2026-02-24T07:29:24.580 INFO:teuthology.orchestra.run.trial035.stdout:(12/37): python3-rados-20.3.0-5440.gff63f3d9.el 8.2 MB/s | 324 kB 00:00 2026-02-24T07:29:24.609 INFO:teuthology.orchestra.run.trial035.stdout:(13/37): python3-rbd-20.3.0-5440.gff63f3d9.el9. 8.3 MB/s | 306 kB 00:00 2026-02-24T07:29:24.613 INFO:teuthology.orchestra.run.trial035.stdout:(14/37): python3-rgw-20.3.0-5440.gff63f3d9.el9. 3.0 MB/s | 100 kB 00:00 2026-02-24T07:29:24.719 INFO:teuthology.orchestra.run.trial035.stdout:(15/37): ceph-common-20.3.0-5440.gff63f3d9.el9. 24 MB/s | 21 MB 00:00 2026-02-24T07:29:25.172 INFO:teuthology.orchestra.run.trial035.stdout:(16/37): mailcap-2.1.49-5.el9.noarch.rpm 59 kB/s | 33 kB 00:00 2026-02-24T07:29:25.381 INFO:teuthology.orchestra.run.trial035.stdout:(17/37): pciutils-3.7.0-7.el9.x86_64.rpm 141 kB/s | 93 kB 00:00 2026-02-24T07:29:25.584 INFO:teuthology.orchestra.run.trial035.stdout:(18/37): nvme-cli-2.16-1.el9.x86_64.rpm 1.2 MB/s | 1.2 MB 00:00 2026-02-24T07:29:25.803 INFO:teuthology.orchestra.run.trial035.stdout:(19/37): boost-program-options-1.75.0-13.el9.x8 165 kB/s | 104 kB 00:00 2026-02-24T07:29:25.902 INFO:teuthology.orchestra.run.trial035.stdout:(20/37): librabbitmq-0.11.0-7.el9.x86_64.rpm 457 kB/s | 45 kB 00:00 2026-02-24T07:29:26.106 INFO:teuthology.orchestra.run.trial035.stdout:(21/37): libnbd-1.20.3-4.el9.x86_64.rpm 226 kB/s | 164 kB 00:00 2026-02-24T07:29:26.237 INFO:teuthology.orchestra.run.trial035.stdout:(22/37): libpmemobj-1.12.1-1.el9.x86_64.rpm 245 kB/s | 160 kB 00:00 2026-02-24T07:29:26.517 INFO:teuthology.orchestra.run.trial035.stdout:(23/37): lttng-ust-2.12.0-6.el9.x86_64.rpm 713 kB/s | 292 kB 00:00 2026-02-24T07:29:26.571 INFO:teuthology.orchestra.run.trial035.stdout:(24/37): qatlib-25.08.0-2.el9.x86_64.rpm 717 kB/s | 240 kB 00:00 2026-02-24T07:29:26.615 INFO:teuthology.orchestra.run.trial035.stdout:(25/37): qatlib-service-25.08.0-2.el9.x86_64.rp 377 kB/s | 37 kB 00:00 2026-02-24T07:29:26.671 INFO:teuthology.orchestra.run.trial035.stdout:(26/37): qatzip-libs-1.3.1-1.el9.x86_64.rpm 670 kB/s | 66 kB 00:00 2026-02-24T07:29:26.709 INFO:teuthology.orchestra.run.trial035.stdout:(27/37): librdkafka-1.6.1-102.el9.x86_64.rpm 821 kB/s | 662 kB 00:00 2026-02-24T07:29:26.847 INFO:teuthology.orchestra.run.trial035.stdout:(28/37): libarrow-doc-9.0.0-15.el9.noarch.rpm 180 kB/s | 25 kB 00:00 2026-02-24T07:29:26.903 INFO:teuthology.orchestra.run.trial035.stdout:(29/37): gperftools-libs-2.9.1-3.el9.x86_64.rpm 1.0 MB/s | 308 kB 00:00 2026-02-24T07:29:26.939 INFO:teuthology.orchestra.run.trial035.stdout:(30/37): liboath-2.6.12-1.el9.x86_64.rpm 533 kB/s | 49 kB 00:00 2026-02-24T07:29:26.952 INFO:teuthology.orchestra.run.trial035.stdout:(31/37): libunwind-1.6.2-1.el9.x86_64.rpm 1.4 MB/s | 67 kB 00:00 2026-02-24T07:29:27.002 INFO:teuthology.orchestra.run.trial035.stdout:(32/37): re2-20211101-20.el9.x86_64.rpm 3.7 MB/s | 191 kB 00:00 2026-02-24T07:29:27.187 INFO:teuthology.orchestra.run.trial035.stdout:(33/37): parquet-libs-9.0.0-15.el9.x86_64.rpm 3.3 MB/s | 838 kB 00:00 2026-02-24T07:29:27.202 INFO:teuthology.orchestra.run.trial035.stdout:(34/37): libarrow-9.0.0-15.el9.x86_64.rpm 8.3 MB/s | 4.4 MB 00:00 2026-02-24T07:29:27.215 INFO:teuthology.orchestra.run.trial035.stdout:(35/37): thrift-0.15.0-4.el9.x86_64.rpm 7.4 MB/s | 1.6 MB 00:00 2026-02-24T07:29:27.408 INFO:teuthology.orchestra.run.trial035.stdout:(36/37): librbd1-20.3.0-5440.gff63f3d9.el9.x86_ 13 MB/s | 2.7 MB 00:00 2026-02-24T07:29:27.420 INFO:teuthology.orchestra.run.trial035.stdout:(37/37): librados2-20.3.0-5440.gff63f3d9.el9.x8 14 MB/s | 3.3 MB 00:00 2026-02-24T07:29:27.423 INFO:teuthology.orchestra.run.trial035.stdout:-------------------------------------------------------------------------------- 2026-02-24T07:29:27.423 INFO:teuthology.orchestra.run.trial035.stdout:Total 14 MB/s | 61 MB 00:04 2026-02-24T07:29:27.519 INFO:teuthology.orchestra.run.trial035.stdout:Running transaction check 2026-02-24T07:29:27.537 INFO:teuthology.orchestra.run.trial035.stdout:Transaction check succeeded. 2026-02-24T07:29:27.537 INFO:teuthology.orchestra.run.trial035.stdout:Running transaction test 2026-02-24T07:29:27.783 INFO:teuthology.orchestra.run.trial035.stdout:Transaction test succeeded. 2026-02-24T07:29:27.783 INFO:teuthology.orchestra.run.trial035.stdout:Running transaction 2026-02-24T07:29:28.077 INFO:teuthology.orchestra.run.trial035.stdout: Preparing : 1/1 2026-02-24T07:29:28.215 INFO:teuthology.orchestra.run.trial035.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 1/39 2026-02-24T07:29:28.220 INFO:teuthology.orchestra.run.trial035.stdout: Upgrading : librados2-2:20.3.0-5440.gff63f3d9.el9.x86_64 2/39 2026-02-24T07:29:28.268 INFO:teuthology.orchestra.run.trial035.stdout: Running scriptlet: librados2-2:20.3.0-5440.gff63f3d9.el9.x86_64 2/39 2026-02-24T07:29:28.269 INFO:teuthology.orchestra.run.trial035.stdout: Installing : libcephfs2-2:20.3.0-5440.gff63f3d9.el9.x86_64 3/39 2026-02-24T07:29:28.280 INFO:teuthology.orchestra.run.trial035.stdout: Running scriptlet: libcephfs2-2:20.3.0-5440.gff63f3d9.el9.x86_64 3/39 2026-02-24T07:29:28.282 INFO:teuthology.orchestra.run.trial035.stdout: Installing : libcephfs-daemon-2:20.3.0-5440.gff63f3d9.el9.x86_6 4/39 2026-02-24T07:29:28.283 INFO:teuthology.orchestra.run.trial035.stdout: Installing : libcephfs-proxy2-2:20.3.0-5440.gff63f3d9.el9.x86_6 5/39 2026-02-24T07:29:28.311 INFO:teuthology.orchestra.run.trial035.stdout: Running scriptlet: libcephfs-proxy2-2:20.3.0-5440.gff63f3d9.el9.x86_6 5/39 2026-02-24T07:29:28.313 INFO:teuthology.orchestra.run.trial035.stdout: Installing : python3-rados-2:20.3.0-5440.gff63f3d9.el9.x86_64 6/39 2026-02-24T07:29:28.322 INFO:teuthology.orchestra.run.trial035.stdout: Installing : liboath-2.6.12-1.el9.x86_64 7/39 2026-02-24T07:29:28.325 INFO:teuthology.orchestra.run.trial035.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 8/39 2026-02-24T07:29:28.328 INFO:teuthology.orchestra.run.trial035.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 9/39 2026-02-24T07:29:28.332 INFO:teuthology.orchestra.run.trial035.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 10/39 2026-02-24T07:29:28.343 INFO:teuthology.orchestra.run.trial035.stdout: Installing : python3-ceph-argparse-2:20.3.0-5440.gff63f3d9.el9. 11/39 2026-02-24T07:29:28.356 INFO:teuthology.orchestra.run.trial035.stdout: Installing : python3-cephfs-2:20.3.0-5440.gff63f3d9.el9.x86_64 12/39 2026-02-24T07:29:28.356 INFO:teuthology.orchestra.run.trial035.stdout: Installing : libradosstriper1-2:20.3.0-5440.gff63f3d9.el9.x86_6 13/39 2026-02-24T07:29:28.380 INFO:teuthology.orchestra.run.trial035.stdout: Running scriptlet: libradosstriper1-2:20.3.0-5440.gff63f3d9.el9.x86_6 13/39 2026-02-24T07:29:28.384 INFO:teuthology.orchestra.run.trial035.stdout: Installing : thrift-0.15.0-4.el9.x86_64 14/39 2026-02-24T07:29:28.415 INFO:teuthology.orchestra.run.trial035.stdout: Installing : re2-1:20211101-20.el9.x86_64 15/39 2026-02-24T07:29:28.418 INFO:teuthology.orchestra.run.trial035.stdout: Installing : libarrow-9.0.0-15.el9.x86_64 16/39 2026-02-24T07:29:28.424 INFO:teuthology.orchestra.run.trial035.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 17/39 2026-02-24T07:29:28.426 INFO:teuthology.orchestra.run.trial035.stdout: Installing : gperftools-libs-2.9.1-3.el9.x86_64 18/39 2026-02-24T07:29:28.431 INFO:teuthology.orchestra.run.trial035.stdout: Installing : libarrow-doc-9.0.0-15.el9.noarch 19/39 2026-02-24T07:29:28.544 INFO:teuthology.orchestra.run.trial035.stdout: Installing : libnbd-1.20.3-4.el9.x86_64 20/39 2026-02-24T07:29:28.545 INFO:teuthology.orchestra.run.trial035.stdout: Upgrading : librbd1-2:20.3.0-5440.gff63f3d9.el9.x86_64 21/39 2026-02-24T07:29:28.571 INFO:teuthology.orchestra.run.trial035.stdout: Running scriptlet: librbd1-2:20.3.0-5440.gff63f3d9.el9.x86_64 21/39 2026-02-24T07:29:28.573 INFO:teuthology.orchestra.run.trial035.stdout: Installing : python3-rbd-2:20.3.0-5440.gff63f3d9.el9.x86_64 22/39 2026-02-24T07:29:28.580 INFO:teuthology.orchestra.run.trial035.stdout: Installing : boost-program-options-1.75.0-13.el9.x86_64 23/39 2026-02-24T07:29:28.827 INFO:teuthology.orchestra.run.trial035.stdout: Installing : parquet-libs-9.0.0-15.el9.x86_64 24/39 2026-02-24T07:29:28.828 INFO:teuthology.orchestra.run.trial035.stdout: Installing : librgw2-2:20.3.0-5440.gff63f3d9.el9.x86_64 25/39 2026-02-24T07:29:28.843 INFO:teuthology.orchestra.run.trial035.stdout: Running scriptlet: librgw2-2:20.3.0-5440.gff63f3d9.el9.x86_64 25/39 2026-02-24T07:29:28.847 INFO:teuthology.orchestra.run.trial035.stdout: Installing : python3-rgw-2:20.3.0-5440.gff63f3d9.el9.x86_64 26/39 2026-02-24T07:29:28.852 INFO:teuthology.orchestra.run.trial035.stdout: Installing : pciutils-3.7.0-7.el9.x86_64 27/39 2026-02-24T07:29:28.854 INFO:teuthology.orchestra.run.trial035.stdout: Installing : qatlib-25.08.0-2.el9.x86_64 28/39 2026-02-24T07:29:28.855 INFO:teuthology.orchestra.run.trial035.stdout: Installing : qatlib-service-25.08.0-2.el9.x86_64 29/39 2026-02-24T07:29:28.873 INFO:teuthology.orchestra.run.trial035.stdout: Running scriptlet: qatlib-service-25.08.0-2.el9.x86_64 29/39 2026-02-24T07:29:28.990 INFO:teuthology.orchestra.run.trial035.stdout: Installing : qatzip-libs-1.3.1-1.el9.x86_64 30/39 2026-02-24T07:29:28.992 INFO:teuthology.orchestra.run.trial035.stdout: Installing : nvme-cli-2.16-1.el9.x86_64 31/39 2026-02-24T07:29:29.266 INFO:teuthology.orchestra.run.trial035.stdout: Running scriptlet: nvme-cli-2.16-1.el9.x86_64 31/39 2026-02-24T07:29:29.266 INFO:teuthology.orchestra.run.trial035.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmefc-boot-connections.service → /usr/lib/systemd/system/nvmefc-boot-connections.service. 2026-02-24T07:29:29.266 INFO:teuthology.orchestra.run.trial035.stdout: 2026-02-24T07:29:29.312 INFO:teuthology.orchestra.run.trial035.stdout: Installing : mailcap-2.1.49-5.el9.noarch 32/39 2026-02-24T07:29:29.314 INFO:teuthology.orchestra.run.trial035.stdout: Installing : python3-ceph-common-2:20.3.0-5440.gff63f3d9.el9.x8 33/39 2026-02-24T07:29:30.252 INFO:teuthology.orchestra.run.trial035.stdout: Running scriptlet: ceph-common-2:20.3.0-5440.gff63f3d9.el9.x86_64 34/39 2026-02-24T07:29:30.254 INFO:teuthology.orchestra.run.trial035.stdout: Installing : ceph-common-2:20.3.0-5440.gff63f3d9.el9.x86_64 34/39 2026-02-24T07:29:30.514 INFO:teuthology.orchestra.run.trial035.stdout: Running scriptlet: ceph-common-2:20.3.0-5440.gff63f3d9.el9.x86_64 34/39 2026-02-24T07:29:30.518 INFO:teuthology.orchestra.run.trial035.stdout: Installing : ceph-base-2:20.3.0-5440.gff63f3d9.el9.x86_64 35/39 2026-02-24T07:29:30.558 INFO:teuthology.orchestra.run.trial035.stdout: Running scriptlet: ceph-base-2:20.3.0-5440.gff63f3d9.el9.x86_64 35/39 2026-02-24T07:29:30.558 INFO:teuthology.orchestra.run.trial035.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2026-02-24T07:29:30.558 INFO:teuthology.orchestra.run.trial035.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2026-02-24T07:29:30.558 INFO:teuthology.orchestra.run.trial035.stdout: 2026-02-24T07:29:30.561 INFO:teuthology.orchestra.run.trial035.stdout: Installing : ceph-selinux-2:20.3.0-5440.gff63f3d9.el9.x86_64 36/39 2026-02-24T07:29:35.098 INFO:teuthology.orchestra.run.trial035.stdout: Running scriptlet: ceph-selinux-2:20.3.0-5440.gff63f3d9.el9.x86_64 36/39 2026-02-24T07:29:35.098 INFO:teuthology.orchestra.run.trial035.stdout:skipping the directory /sys 2026-02-24T07:29:35.099 INFO:teuthology.orchestra.run.trial035.stdout:skipping the directory /proc 2026-02-24T07:29:35.099 INFO:teuthology.orchestra.run.trial035.stdout:skipping the directory /mnt 2026-02-24T07:29:35.099 INFO:teuthology.orchestra.run.trial035.stdout:skipping the directory /var/tmp 2026-02-24T07:29:35.099 INFO:teuthology.orchestra.run.trial035.stdout:skipping the directory /home 2026-02-24T07:29:35.099 INFO:teuthology.orchestra.run.trial035.stdout:skipping the directory /root 2026-02-24T07:29:35.099 INFO:teuthology.orchestra.run.trial035.stdout:skipping the directory /tmp 2026-02-24T07:29:35.099 INFO:teuthology.orchestra.run.trial035.stdout: 2026-02-24T07:29:35.591 INFO:teuthology.orchestra.run.trial035.stdout: Installing : ceph-radosgw-2:20.3.0-5440.gff63f3d9.el9.x86_64 37/39 2026-02-24T07:29:35.614 INFO:teuthology.orchestra.run.trial035.stdout: Running scriptlet: ceph-radosgw-2:20.3.0-5440.gff63f3d9.el9.x86_64 37/39 2026-02-24T07:29:35.614 INFO:teuthology.orchestra.run.trial035.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-24T07:29:35.614 INFO:teuthology.orchestra.run.trial035.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2026-02-24T07:29:35.614 INFO:teuthology.orchestra.run.trial035.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2026-02-24T07:29:35.615 INFO:teuthology.orchestra.run.trial035.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2026-02-24T07:29:35.615 INFO:teuthology.orchestra.run.trial035.stdout: 2026-02-24T07:29:35.616 INFO:teuthology.orchestra.run.trial035.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 38/39 2026-02-24T07:29:35.627 INFO:teuthology.orchestra.run.trial035.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 38/39 2026-02-24T07:29:35.627 INFO:teuthology.orchestra.run.trial035.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 39/39 2026-02-24T07:29:35.681 INFO:teuthology.orchestra.run.trial035.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 39/39 2026-02-24T07:29:35.681 INFO:teuthology.orchestra.run.trial035.stdout:Creating group 'qat' with GID 992. 2026-02-24T07:29:35.681 INFO:teuthology.orchestra.run.trial035.stdout: 2026-02-24T07:29:35.971 INFO:teuthology.orchestra.run.trial035.stdout: Verifying : ceph-base-2:20.3.0-5440.gff63f3d9.el9.x86_64 1/39 2026-02-24T07:29:35.971 INFO:teuthology.orchestra.run.trial035.stdout: Verifying : ceph-common-2:20.3.0-5440.gff63f3d9.el9.x86_64 2/39 2026-02-24T07:29:35.971 INFO:teuthology.orchestra.run.trial035.stdout: Verifying : ceph-radosgw-2:20.3.0-5440.gff63f3d9.el9.x86_64 3/39 2026-02-24T07:29:35.971 INFO:teuthology.orchestra.run.trial035.stdout: Verifying : ceph-selinux-2:20.3.0-5440.gff63f3d9.el9.x86_64 4/39 2026-02-24T07:29:35.971 INFO:teuthology.orchestra.run.trial035.stdout: Verifying : libcephfs-daemon-2:20.3.0-5440.gff63f3d9.el9.x86_6 5/39 2026-02-24T07:29:35.971 INFO:teuthology.orchestra.run.trial035.stdout: Verifying : libcephfs-proxy2-2:20.3.0-5440.gff63f3d9.el9.x86_6 6/39 2026-02-24T07:29:35.971 INFO:teuthology.orchestra.run.trial035.stdout: Verifying : libcephfs2-2:20.3.0-5440.gff63f3d9.el9.x86_64 7/39 2026-02-24T07:29:35.971 INFO:teuthology.orchestra.run.trial035.stdout: Verifying : libradosstriper1-2:20.3.0-5440.gff63f3d9.el9.x86_6 8/39 2026-02-24T07:29:35.972 INFO:teuthology.orchestra.run.trial035.stdout: Verifying : librgw2-2:20.3.0-5440.gff63f3d9.el9.x86_64 9/39 2026-02-24T07:29:35.972 INFO:teuthology.orchestra.run.trial035.stdout: Verifying : python3-ceph-argparse-2:20.3.0-5440.gff63f3d9.el9. 10/39 2026-02-24T07:29:35.972 INFO:teuthology.orchestra.run.trial035.stdout: Verifying : python3-ceph-common-2:20.3.0-5440.gff63f3d9.el9.x8 11/39 2026-02-24T07:29:35.972 INFO:teuthology.orchestra.run.trial035.stdout: Verifying : python3-cephfs-2:20.3.0-5440.gff63f3d9.el9.x86_64 12/39 2026-02-24T07:29:35.972 INFO:teuthology.orchestra.run.trial035.stdout: Verifying : python3-rados-2:20.3.0-5440.gff63f3d9.el9.x86_64 13/39 2026-02-24T07:29:35.972 INFO:teuthology.orchestra.run.trial035.stdout: Verifying : python3-rbd-2:20.3.0-5440.gff63f3d9.el9.x86_64 14/39 2026-02-24T07:29:35.972 INFO:teuthology.orchestra.run.trial035.stdout: Verifying : python3-rgw-2:20.3.0-5440.gff63f3d9.el9.x86_64 15/39 2026-02-24T07:29:35.972 INFO:teuthology.orchestra.run.trial035.stdout: Verifying : mailcap-2.1.49-5.el9.noarch 16/39 2026-02-24T07:29:35.972 INFO:teuthology.orchestra.run.trial035.stdout: Verifying : nvme-cli-2.16-1.el9.x86_64 17/39 2026-02-24T07:29:35.972 INFO:teuthology.orchestra.run.trial035.stdout: Verifying : pciutils-3.7.0-7.el9.x86_64 18/39 2026-02-24T07:29:35.973 INFO:teuthology.orchestra.run.trial035.stdout: Verifying : boost-program-options-1.75.0-13.el9.x86_64 19/39 2026-02-24T07:29:35.973 INFO:teuthology.orchestra.run.trial035.stdout: Verifying : libnbd-1.20.3-4.el9.x86_64 20/39 2026-02-24T07:29:35.973 INFO:teuthology.orchestra.run.trial035.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 21/39 2026-02-24T07:29:35.973 INFO:teuthology.orchestra.run.trial035.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 22/39 2026-02-24T07:29:35.973 INFO:teuthology.orchestra.run.trial035.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 23/39 2026-02-24T07:29:35.973 INFO:teuthology.orchestra.run.trial035.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 24/39 2026-02-24T07:29:35.973 INFO:teuthology.orchestra.run.trial035.stdout: Verifying : qatlib-25.08.0-2.el9.x86_64 25/39 2026-02-24T07:29:35.973 INFO:teuthology.orchestra.run.trial035.stdout: Verifying : qatlib-service-25.08.0-2.el9.x86_64 26/39 2026-02-24T07:29:35.973 INFO:teuthology.orchestra.run.trial035.stdout: Verifying : qatzip-libs-1.3.1-1.el9.x86_64 27/39 2026-02-24T07:29:35.973 INFO:teuthology.orchestra.run.trial035.stdout: Verifying : gperftools-libs-2.9.1-3.el9.x86_64 28/39 2026-02-24T07:29:35.973 INFO:teuthology.orchestra.run.trial035.stdout: Verifying : libarrow-9.0.0-15.el9.x86_64 29/39 2026-02-24T07:29:35.973 INFO:teuthology.orchestra.run.trial035.stdout: Verifying : libarrow-doc-9.0.0-15.el9.noarch 30/39 2026-02-24T07:29:35.974 INFO:teuthology.orchestra.run.trial035.stdout: Verifying : liboath-2.6.12-1.el9.x86_64 31/39 2026-02-24T07:29:35.974 INFO:teuthology.orchestra.run.trial035.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 32/39 2026-02-24T07:29:35.974 INFO:teuthology.orchestra.run.trial035.stdout: Verifying : parquet-libs-9.0.0-15.el9.x86_64 33/39 2026-02-24T07:29:35.974 INFO:teuthology.orchestra.run.trial035.stdout: Verifying : re2-1:20211101-20.el9.x86_64 34/39 2026-02-24T07:29:35.974 INFO:teuthology.orchestra.run.trial035.stdout: Verifying : thrift-0.15.0-4.el9.x86_64 35/39 2026-02-24T07:29:35.974 INFO:teuthology.orchestra.run.trial035.stdout: Verifying : librados2-2:20.3.0-5440.gff63f3d9.el9.x86_64 36/39 2026-02-24T07:29:35.974 INFO:teuthology.orchestra.run.trial035.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 37/39 2026-02-24T07:29:35.974 INFO:teuthology.orchestra.run.trial035.stdout: Verifying : librbd1-2:20.3.0-5440.gff63f3d9.el9.x86_64 38/39 2026-02-24T07:29:36.043 INFO:teuthology.orchestra.run.trial035.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 39/39 2026-02-24T07:29:36.043 INFO:teuthology.orchestra.run.trial035.stdout: 2026-02-24T07:29:36.043 INFO:teuthology.orchestra.run.trial035.stdout:Upgraded: 2026-02-24T07:29:36.043 INFO:teuthology.orchestra.run.trial035.stdout: librados2-2:20.3.0-5440.gff63f3d9.el9.x86_64 2026-02-24T07:29:36.043 INFO:teuthology.orchestra.run.trial035.stdout: librbd1-2:20.3.0-5440.gff63f3d9.el9.x86_64 2026-02-24T07:29:36.043 INFO:teuthology.orchestra.run.trial035.stdout:Installed: 2026-02-24T07:29:36.043 INFO:teuthology.orchestra.run.trial035.stdout: boost-program-options-1.75.0-13.el9.x86_64 2026-02-24T07:29:36.043 INFO:teuthology.orchestra.run.trial035.stdout: ceph-base-2:20.3.0-5440.gff63f3d9.el9.x86_64 2026-02-24T07:29:36.043 INFO:teuthology.orchestra.run.trial035.stdout: ceph-common-2:20.3.0-5440.gff63f3d9.el9.x86_64 2026-02-24T07:29:36.043 INFO:teuthology.orchestra.run.trial035.stdout: ceph-radosgw-2:20.3.0-5440.gff63f3d9.el9.x86_64 2026-02-24T07:29:36.043 INFO:teuthology.orchestra.run.trial035.stdout: ceph-selinux-2:20.3.0-5440.gff63f3d9.el9.x86_64 2026-02-24T07:29:36.043 INFO:teuthology.orchestra.run.trial035.stdout: gperftools-libs-2.9.1-3.el9.x86_64 2026-02-24T07:29:36.043 INFO:teuthology.orchestra.run.trial035.stdout: libarrow-9.0.0-15.el9.x86_64 2026-02-24T07:29:36.044 INFO:teuthology.orchestra.run.trial035.stdout: libarrow-doc-9.0.0-15.el9.noarch 2026-02-24T07:29:36.044 INFO:teuthology.orchestra.run.trial035.stdout: libcephfs-daemon-2:20.3.0-5440.gff63f3d9.el9.x86_64 2026-02-24T07:29:36.044 INFO:teuthology.orchestra.run.trial035.stdout: libcephfs-proxy2-2:20.3.0-5440.gff63f3d9.el9.x86_64 2026-02-24T07:29:36.044 INFO:teuthology.orchestra.run.trial035.stdout: libcephfs2-2:20.3.0-5440.gff63f3d9.el9.x86_64 2026-02-24T07:29:36.044 INFO:teuthology.orchestra.run.trial035.stdout: libnbd-1.20.3-4.el9.x86_64 2026-02-24T07:29:36.044 INFO:teuthology.orchestra.run.trial035.stdout: liboath-2.6.12-1.el9.x86_64 2026-02-24T07:29:36.044 INFO:teuthology.orchestra.run.trial035.stdout: libpmemobj-1.12.1-1.el9.x86_64 2026-02-24T07:29:36.044 INFO:teuthology.orchestra.run.trial035.stdout: librabbitmq-0.11.0-7.el9.x86_64 2026-02-24T07:29:36.044 INFO:teuthology.orchestra.run.trial035.stdout: libradosstriper1-2:20.3.0-5440.gff63f3d9.el9.x86_64 2026-02-24T07:29:36.044 INFO:teuthology.orchestra.run.trial035.stdout: librdkafka-1.6.1-102.el9.x86_64 2026-02-24T07:29:36.044 INFO:teuthology.orchestra.run.trial035.stdout: librgw2-2:20.3.0-5440.gff63f3d9.el9.x86_64 2026-02-24T07:29:36.044 INFO:teuthology.orchestra.run.trial035.stdout: libunwind-1.6.2-1.el9.x86_64 2026-02-24T07:29:36.044 INFO:teuthology.orchestra.run.trial035.stdout: lttng-ust-2.12.0-6.el9.x86_64 2026-02-24T07:29:36.044 INFO:teuthology.orchestra.run.trial035.stdout: mailcap-2.1.49-5.el9.noarch 2026-02-24T07:29:36.044 INFO:teuthology.orchestra.run.trial035.stdout: nvme-cli-2.16-1.el9.x86_64 2026-02-24T07:29:36.044 INFO:teuthology.orchestra.run.trial035.stdout: parquet-libs-9.0.0-15.el9.x86_64 2026-02-24T07:29:36.044 INFO:teuthology.orchestra.run.trial035.stdout: pciutils-3.7.0-7.el9.x86_64 2026-02-24T07:29:36.044 INFO:teuthology.orchestra.run.trial035.stdout: python3-ceph-argparse-2:20.3.0-5440.gff63f3d9.el9.x86_64 2026-02-24T07:29:36.044 INFO:teuthology.orchestra.run.trial035.stdout: python3-ceph-common-2:20.3.0-5440.gff63f3d9.el9.x86_64 2026-02-24T07:29:36.044 INFO:teuthology.orchestra.run.trial035.stdout: python3-cephfs-2:20.3.0-5440.gff63f3d9.el9.x86_64 2026-02-24T07:29:36.044 INFO:teuthology.orchestra.run.trial035.stdout: python3-rados-2:20.3.0-5440.gff63f3d9.el9.x86_64 2026-02-24T07:29:36.044 INFO:teuthology.orchestra.run.trial035.stdout: python3-rbd-2:20.3.0-5440.gff63f3d9.el9.x86_64 2026-02-24T07:29:36.044 INFO:teuthology.orchestra.run.trial035.stdout: python3-rgw-2:20.3.0-5440.gff63f3d9.el9.x86_64 2026-02-24T07:29:36.044 INFO:teuthology.orchestra.run.trial035.stdout: qatlib-25.08.0-2.el9.x86_64 2026-02-24T07:29:36.044 INFO:teuthology.orchestra.run.trial035.stdout: qatlib-service-25.08.0-2.el9.x86_64 2026-02-24T07:29:36.044 INFO:teuthology.orchestra.run.trial035.stdout: qatzip-libs-1.3.1-1.el9.x86_64 2026-02-24T07:29:36.044 INFO:teuthology.orchestra.run.trial035.stdout: re2-1:20211101-20.el9.x86_64 2026-02-24T07:29:36.044 INFO:teuthology.orchestra.run.trial035.stdout: thrift-0.15.0-4.el9.x86_64 2026-02-24T07:29:36.044 INFO:teuthology.orchestra.run.trial035.stdout: 2026-02-24T07:29:36.044 INFO:teuthology.orchestra.run.trial035.stdout:Complete! 2026-02-24T07:29:36.085 DEBUG:teuthology.orchestra.run.trial035:> sudo yum -y install ceph-test 2026-02-24T07:29:36.405 INFO:teuthology.orchestra.run.trial035.stdout:Last metadata expiration check: 0:00:15 ago on Tue 24 Feb 2026 07:29:21 AM UTC. 2026-02-24T07:29:36.488 INFO:teuthology.orchestra.run.trial035.stdout:Dependencies resolved. 2026-02-24T07:29:36.489 INFO:teuthology.orchestra.run.trial035.stdout:================================================================================ 2026-02-24T07:29:36.489 INFO:teuthology.orchestra.run.trial035.stdout: Package Arch Version Repository Size 2026-02-24T07:29:36.489 INFO:teuthology.orchestra.run.trial035.stdout:================================================================================ 2026-02-24T07:29:36.489 INFO:teuthology.orchestra.run.trial035.stdout:Installing: 2026-02-24T07:29:36.490 INFO:teuthology.orchestra.run.trial035.stdout: ceph-test x86_64 2:20.3.0-5440.gff63f3d9.el9 ceph 126 M 2026-02-24T07:29:36.490 INFO:teuthology.orchestra.run.trial035.stdout:Installing dependencies: 2026-02-24T07:29:36.490 INFO:teuthology.orchestra.run.trial035.stdout: hwloc-libs x86_64 2.4.1-6.el9 baseos 2.1 M 2026-02-24T07:29:36.490 INFO:teuthology.orchestra.run.trial035.stdout: libasan8 x86_64 15.2.1-7.el9 appstream 525 k 2026-02-24T07:29:36.490 INFO:teuthology.orchestra.run.trial035.stdout: libatomic x86_64 11.5.0-14.el9 baseos 23 k 2026-02-24T07:29:36.490 INFO:teuthology.orchestra.run.trial035.stdout: libcephsqlite x86_64 2:20.3.0-5440.gff63f3d9.el9 ceph 153 k 2026-02-24T07:29:36.490 INFO:teuthology.orchestra.run.trial035.stdout: libubsan x86_64 11.5.0-14.el9 appstream 176 k 2026-02-24T07:29:36.490 INFO:teuthology.orchestra.run.trial035.stdout: libxslt x86_64 1.1.34-12.el9 appstream 233 k 2026-02-24T07:29:36.490 INFO:teuthology.orchestra.run.trial035.stdout: lksctp-tools x86_64 1.0.19-2.el9 baseos 94 k 2026-02-24T07:29:36.490 INFO:teuthology.orchestra.run.trial035.stdout: protobuf x86_64 3.14.0-17.el9 appstream 1.0 M 2026-02-24T07:29:36.490 INFO:teuthology.orchestra.run.trial035.stdout: socat x86_64 1.7.4.1-8.el9 appstream 303 k 2026-02-24T07:29:36.491 INFO:teuthology.orchestra.run.trial035.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2026-02-24T07:29:36.491 INFO:teuthology.orchestra.run.trial035.stdout: yaml-cpp x86_64 0.6.3-5.el9 epel 122 k 2026-02-24T07:29:36.491 INFO:teuthology.orchestra.run.trial035.stdout: 2026-02-24T07:29:36.491 INFO:teuthology.orchestra.run.trial035.stdout:Transaction Summary 2026-02-24T07:29:36.491 INFO:teuthology.orchestra.run.trial035.stdout:================================================================================ 2026-02-24T07:29:36.491 INFO:teuthology.orchestra.run.trial035.stdout:Install 12 Packages 2026-02-24T07:29:36.491 INFO:teuthology.orchestra.run.trial035.stdout: 2026-02-24T07:29:36.491 INFO:teuthology.orchestra.run.trial035.stdout:Total download size: 131 M 2026-02-24T07:29:36.491 INFO:teuthology.orchestra.run.trial035.stdout:Installed size: 977 M 2026-02-24T07:29:36.491 INFO:teuthology.orchestra.run.trial035.stdout:Downloading Packages: 2026-02-24T07:29:37.041 INFO:teuthology.orchestra.run.trial035.stdout:(1/12): libcephsqlite-20.3.0-5440.gff63f3d9.el9 1.7 MB/s | 153 kB 00:00 2026-02-24T07:29:37.124 INFO:teuthology.orchestra.run.trial035.stdout:(2/12): libatomic-11.5.0-14.el9.x86_64.rpm 275 kB/s | 23 kB 00:00 2026-02-24T07:29:37.170 INFO:teuthology.orchestra.run.trial035.stdout:(3/12): hwloc-libs-2.4.1-6.el9.x86_64.rpm 9.6 MB/s | 2.1 MB 00:00 2026-02-24T07:29:37.176 INFO:teuthology.orchestra.run.trial035.stdout:(4/12): lksctp-tools-1.0.19-2.el9.x86_64.rpm 1.8 MB/s | 94 kB 00:00 2026-02-24T07:29:37.634 INFO:teuthology.orchestra.run.trial035.stdout:(5/12): libubsan-11.5.0-14.el9.x86_64.rpm 384 kB/s | 176 kB 00:00 2026-02-24T07:29:37.756 INFO:teuthology.orchestra.run.trial035.stdout:(6/12): libasan8-15.2.1-7.el9.x86_64.rpm 895 kB/s | 525 kB 00:00 2026-02-24T07:29:37.801 INFO:teuthology.orchestra.run.trial035.stdout:(7/12): libxslt-1.1.34-12.el9.x86_64.rpm 1.4 MB/s | 233 kB 00:00 2026-02-24T07:29:37.921 INFO:teuthology.orchestra.run.trial035.stdout:(8/12): socat-1.7.4.1-8.el9.x86_64.rpm 2.5 MB/s | 303 kB 00:00 2026-02-24T07:29:37.961 INFO:teuthology.orchestra.run.trial035.stdout:(9/12): protobuf-3.14.0-17.el9.x86_64.rpm 4.9 MB/s | 1.0 MB 00:00 2026-02-24T07:29:38.007 INFO:teuthology.orchestra.run.trial035.stdout:(10/12): xmlstarlet-1.6.1-20.el9.x86_64.rpm 743 kB/s | 64 kB 00:00 2026-02-24T07:29:39.454 INFO:teuthology.orchestra.run.trial035.stdout:(11/12): ceph-test-20.3.0-5440.gff63f3d9.el9.x8 50 MB/s | 126 MB 00:02 2026-02-24T07:29:40.735 INFO:teuthology.orchestra.run.trial035.stdout:(12/12): yaml-cpp-0.6.3-5.el9.x86_64.rpm 44 kB/s | 122 kB 00:02 2026-02-24T07:29:40.739 INFO:teuthology.orchestra.run.trial035.stdout:-------------------------------------------------------------------------------- 2026-02-24T07:29:40.740 INFO:teuthology.orchestra.run.trial035.stdout:Total 31 MB/s | 131 MB 00:04 2026-02-24T07:29:40.824 INFO:teuthology.orchestra.run.trial035.stdout:Running transaction check 2026-02-24T07:29:40.836 INFO:teuthology.orchestra.run.trial035.stdout:Transaction check succeeded. 2026-02-24T07:29:40.836 INFO:teuthology.orchestra.run.trial035.stdout:Running transaction test 2026-02-24T07:29:41.258 INFO:teuthology.orchestra.run.trial035.stdout:Transaction test succeeded. 2026-02-24T07:29:41.258 INFO:teuthology.orchestra.run.trial035.stdout:Running transaction 2026-02-24T07:29:41.710 INFO:teuthology.orchestra.run.trial035.stdout: Preparing : 1/1 2026-02-24T07:29:41.718 INFO:teuthology.orchestra.run.trial035.stdout: Installing : yaml-cpp-0.6.3-5.el9.x86_64 1/12 2026-02-24T07:29:41.727 INFO:teuthology.orchestra.run.trial035.stdout: Installing : socat-1.7.4.1-8.el9.x86_64 2/12 2026-02-24T07:29:41.732 INFO:teuthology.orchestra.run.trial035.stdout: Installing : protobuf-3.14.0-17.el9.x86_64 3/12 2026-02-24T07:29:41.736 INFO:teuthology.orchestra.run.trial035.stdout: Installing : libxslt-1.1.34-12.el9.x86_64 4/12 2026-02-24T07:29:41.740 INFO:teuthology.orchestra.run.trial035.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 5/12 2026-02-24T07:29:41.744 INFO:teuthology.orchestra.run.trial035.stdout: Installing : libubsan-11.5.0-14.el9.x86_64 6/12 2026-02-24T07:29:41.745 INFO:teuthology.orchestra.run.trial035.stdout: Installing : libasan8-15.2.1-7.el9.x86_64 7/12 2026-02-24T07:29:41.761 INFO:teuthology.orchestra.run.trial035.stdout: Running scriptlet: libasan8-15.2.1-7.el9.x86_64 7/12 2026-02-24T07:29:41.763 INFO:teuthology.orchestra.run.trial035.stdout: Installing : lksctp-tools-1.0.19-2.el9.x86_64 8/12 2026-02-24T07:29:41.773 INFO:teuthology.orchestra.run.trial035.stdout: Installing : libatomic-11.5.0-14.el9.x86_64 9/12 2026-02-24T07:29:41.781 INFO:teuthology.orchestra.run.trial035.stdout: Installing : hwloc-libs-2.4.1-6.el9.x86_64 10/12 2026-02-24T07:29:41.782 INFO:teuthology.orchestra.run.trial035.stdout: Installing : libcephsqlite-2:20.3.0-5440.gff63f3d9.el9.x86_64 11/12 2026-02-24T07:29:48.658 INFO:teuthology.orchestra.run.trial035.stdout: Running scriptlet: libcephsqlite-2:20.3.0-5440.gff63f3d9.el9.x86_64 11/12 2026-02-24T07:29:48.677 INFO:teuthology.orchestra.run.trial035.stdout: Installing : ceph-test-2:20.3.0-5440.gff63f3d9.el9.x86_64 12/12 2026-02-24T07:29:49.477 INFO:teuthology.orchestra.run.trial035.stdout: Running scriptlet: ceph-test-2:20.3.0-5440.gff63f3d9.el9.x86_64 12/12 2026-02-24T07:29:49.477 INFO:teuthology.orchestra.run.trial035.stdout: Verifying : ceph-test-2:20.3.0-5440.gff63f3d9.el9.x86_64 1/12 2026-02-24T07:29:49.477 INFO:teuthology.orchestra.run.trial035.stdout: Verifying : libcephsqlite-2:20.3.0-5440.gff63f3d9.el9.x86_64 2/12 2026-02-24T07:29:49.477 INFO:teuthology.orchestra.run.trial035.stdout: Verifying : hwloc-libs-2.4.1-6.el9.x86_64 3/12 2026-02-24T07:29:49.477 INFO:teuthology.orchestra.run.trial035.stdout: Verifying : libatomic-11.5.0-14.el9.x86_64 4/12 2026-02-24T07:29:49.477 INFO:teuthology.orchestra.run.trial035.stdout: Verifying : lksctp-tools-1.0.19-2.el9.x86_64 5/12 2026-02-24T07:29:49.478 INFO:teuthology.orchestra.run.trial035.stdout: Verifying : libasan8-15.2.1-7.el9.x86_64 6/12 2026-02-24T07:29:49.478 INFO:teuthology.orchestra.run.trial035.stdout: Verifying : libubsan-11.5.0-14.el9.x86_64 7/12 2026-02-24T07:29:49.478 INFO:teuthology.orchestra.run.trial035.stdout: Verifying : libxslt-1.1.34-12.el9.x86_64 8/12 2026-02-24T07:29:49.478 INFO:teuthology.orchestra.run.trial035.stdout: Verifying : protobuf-3.14.0-17.el9.x86_64 9/12 2026-02-24T07:29:49.478 INFO:teuthology.orchestra.run.trial035.stdout: Verifying : socat-1.7.4.1-8.el9.x86_64 10/12 2026-02-24T07:29:49.478 INFO:teuthology.orchestra.run.trial035.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 11/12 2026-02-24T07:29:49.535 INFO:teuthology.orchestra.run.trial035.stdout: Verifying : yaml-cpp-0.6.3-5.el9.x86_64 12/12 2026-02-24T07:29:49.535 INFO:teuthology.orchestra.run.trial035.stdout: 2026-02-24T07:29:49.535 INFO:teuthology.orchestra.run.trial035.stdout:Installed: 2026-02-24T07:29:49.535 INFO:teuthology.orchestra.run.trial035.stdout: ceph-test-2:20.3.0-5440.gff63f3d9.el9.x86_64 2026-02-24T07:29:49.536 INFO:teuthology.orchestra.run.trial035.stdout: hwloc-libs-2.4.1-6.el9.x86_64 2026-02-24T07:29:49.536 INFO:teuthology.orchestra.run.trial035.stdout: libasan8-15.2.1-7.el9.x86_64 2026-02-24T07:29:49.536 INFO:teuthology.orchestra.run.trial035.stdout: libatomic-11.5.0-14.el9.x86_64 2026-02-24T07:29:49.536 INFO:teuthology.orchestra.run.trial035.stdout: libcephsqlite-2:20.3.0-5440.gff63f3d9.el9.x86_64 2026-02-24T07:29:49.536 INFO:teuthology.orchestra.run.trial035.stdout: libubsan-11.5.0-14.el9.x86_64 2026-02-24T07:29:49.536 INFO:teuthology.orchestra.run.trial035.stdout: libxslt-1.1.34-12.el9.x86_64 2026-02-24T07:29:49.536 INFO:teuthology.orchestra.run.trial035.stdout: lksctp-tools-1.0.19-2.el9.x86_64 2026-02-24T07:29:49.536 INFO:teuthology.orchestra.run.trial035.stdout: protobuf-3.14.0-17.el9.x86_64 2026-02-24T07:29:49.536 INFO:teuthology.orchestra.run.trial035.stdout: socat-1.7.4.1-8.el9.x86_64 2026-02-24T07:29:49.536 INFO:teuthology.orchestra.run.trial035.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2026-02-24T07:29:49.536 INFO:teuthology.orchestra.run.trial035.stdout: yaml-cpp-0.6.3-5.el9.x86_64 2026-02-24T07:29:49.536 INFO:teuthology.orchestra.run.trial035.stdout: 2026-02-24T07:29:49.537 INFO:teuthology.orchestra.run.trial035.stdout:Complete! 2026-02-24T07:29:49.577 DEBUG:teuthology.orchestra.run.trial035:> sudo yum -y install ceph 2026-02-24T07:29:49.890 INFO:teuthology.orchestra.run.trial035.stdout:Last metadata expiration check: 0:00:28 ago on Tue 24 Feb 2026 07:29:21 AM UTC. 2026-02-24T07:29:49.978 INFO:teuthology.orchestra.run.trial035.stdout:Dependencies resolved. 2026-02-24T07:29:49.980 INFO:teuthology.orchestra.run.trial035.stdout:======================================================================================= 2026-02-24T07:29:49.981 INFO:teuthology.orchestra.run.trial035.stdout: Package Arch Version Repository Size 2026-02-24T07:29:49.981 INFO:teuthology.orchestra.run.trial035.stdout:======================================================================================= 2026-02-24T07:29:49.981 INFO:teuthology.orchestra.run.trial035.stdout:Installing: 2026-02-24T07:29:49.981 INFO:teuthology.orchestra.run.trial035.stdout: ceph x86_64 2:20.3.0-5440.gff63f3d9.el9 ceph 6.5 k 2026-02-24T07:29:49.981 INFO:teuthology.orchestra.run.trial035.stdout:Installing dependencies: 2026-02-24T07:29:49.981 INFO:teuthology.orchestra.run.trial035.stdout: abseil-cpp x86_64 20211102.0-4.el9 epel 551 k 2026-02-24T07:29:49.981 INFO:teuthology.orchestra.run.trial035.stdout: ceph-grafana-dashboards noarch 2:20.3.0-5440.gff63f3d9.el9 ceph-noarch 57 k 2026-02-24T07:29:49.981 INFO:teuthology.orchestra.run.trial035.stdout: ceph-mds x86_64 2:20.3.0-5440.gff63f3d9.el9 ceph 2.0 M 2026-02-24T07:29:49.981 INFO:teuthology.orchestra.run.trial035.stdout: ceph-mgr x86_64 2:20.3.0-5440.gff63f3d9.el9 ceph 937 k 2026-02-24T07:29:49.981 INFO:teuthology.orchestra.run.trial035.stdout: ceph-mgr-modules-core noarch 2:20.3.0-5440.gff63f3d9.el9 ceph-noarch 298 k 2026-02-24T07:29:49.982 INFO:teuthology.orchestra.run.trial035.stdout: ceph-mon x86_64 2:20.3.0-5440.gff63f3d9.el9 ceph 4.9 M 2026-02-24T07:29:49.982 INFO:teuthology.orchestra.run.trial035.stdout: ceph-osd x86_64 2:20.3.0-5440.gff63f3d9.el9 ceph 163 k 2026-02-24T07:29:49.982 INFO:teuthology.orchestra.run.trial035.stdout: ceph-osd-classic x86_64 2:20.3.0-5440.gff63f3d9.el9 ceph 15 M 2026-02-24T07:29:49.982 INFO:teuthology.orchestra.run.trial035.stdout: ceph-osd-crimson x86_64 2:20.3.0-5440.gff63f3d9.el9 ceph 87 M 2026-02-24T07:29:49.982 INFO:teuthology.orchestra.run.trial035.stdout: ceph-prometheus-alerts noarch 2:20.3.0-5440.gff63f3d9.el9 ceph-noarch 17 k 2026-02-24T07:29:49.982 INFO:teuthology.orchestra.run.trial035.stdout: cephadm noarch 2:20.3.0-5440.gff63f3d9.el9 ceph-noarch 1.0 M 2026-02-24T07:29:49.982 INFO:teuthology.orchestra.run.trial035.stdout: cryptsetup x86_64 2.8.1-3.el9 baseos 351 k 2026-02-24T07:29:49.982 INFO:teuthology.orchestra.run.trial035.stdout: flexiblas x86_64 3.0.4-9.el9 appstream 30 k 2026-02-24T07:29:49.982 INFO:teuthology.orchestra.run.trial035.stdout: flexiblas-netlib x86_64 3.0.4-9.el9 appstream 3.0 M 2026-02-24T07:29:49.982 INFO:teuthology.orchestra.run.trial035.stdout: flexiblas-openblas-openmp x86_64 3.0.4-9.el9 appstream 15 k 2026-02-24T07:29:49.982 INFO:teuthology.orchestra.run.trial035.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2026-02-24T07:29:49.982 INFO:teuthology.orchestra.run.trial035.stdout: ledmon-libs x86_64 1.1.0-3.el9 baseos 40 k 2026-02-24T07:29:49.983 INFO:teuthology.orchestra.run.trial035.stdout: libconfig x86_64 1.7.2-9.el9 baseos 72 k 2026-02-24T07:29:49.983 INFO:teuthology.orchestra.run.trial035.stdout: libgfortran x86_64 11.5.0-14.el9 baseos 794 k 2026-02-24T07:29:49.983 INFO:teuthology.orchestra.run.trial035.stdout: libquadmath x86_64 11.5.0-14.el9 baseos 184 k 2026-02-24T07:29:49.983 INFO:teuthology.orchestra.run.trial035.stdout: libstoragemgmt x86_64 1.10.1-1.el9 appstream 246 k 2026-02-24T07:29:49.983 INFO:teuthology.orchestra.run.trial035.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2026-02-24T07:29:49.983 INFO:teuthology.orchestra.run.trial035.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2026-02-24T07:29:49.983 INFO:teuthology.orchestra.run.trial035.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2026-02-24T07:29:49.983 INFO:teuthology.orchestra.run.trial035.stdout: openblas x86_64 0.3.29-1.el9 appstream 42 k 2026-02-24T07:29:49.983 INFO:teuthology.orchestra.run.trial035.stdout: openblas-openmp x86_64 0.3.29-1.el9 appstream 5.3 M 2026-02-24T07:29:49.983 INFO:teuthology.orchestra.run.trial035.stdout: protobuf-compiler x86_64 3.14.0-17.el9 crb 862 k 2026-02-24T07:29:49.983 INFO:teuthology.orchestra.run.trial035.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2026-02-24T07:29:49.983 INFO:teuthology.orchestra.run.trial035.stdout: python3-autocommand noarch 2.2.2-8.el9 epel 29 k 2026-02-24T07:29:49.983 INFO:teuthology.orchestra.run.trial035.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2026-02-24T07:29:49.984 INFO:teuthology.orchestra.run.trial035.stdout: python3-backports-tarfile noarch 1.2.0-1.el9 epel 60 k 2026-02-24T07:29:49.984 INFO:teuthology.orchestra.run.trial035.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2026-02-24T07:29:49.984 INFO:teuthology.orchestra.run.trial035.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2026-02-24T07:29:49.984 INFO:teuthology.orchestra.run.trial035.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2026-02-24T07:29:49.984 INFO:teuthology.orchestra.run.trial035.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2026-02-24T07:29:49.984 INFO:teuthology.orchestra.run.trial035.stdout: python3-cheroot noarch 10.0.1-4.el9 epel 173 k 2026-02-24T07:29:49.984 INFO:teuthology.orchestra.run.trial035.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2026-02-24T07:29:49.984 INFO:teuthology.orchestra.run.trial035.stdout: python3-cryptography x86_64 36.0.1-5.el9 baseos 1.2 M 2026-02-24T07:29:49.984 INFO:teuthology.orchestra.run.trial035.stdout: python3-devel x86_64 3.9.25-3.el9 appstream 244 k 2026-02-24T07:29:49.984 INFO:teuthology.orchestra.run.trial035.stdout: python3-google-auth noarch 1:2.45.0-1.el9 epel 254 k 2026-02-24T07:29:49.984 INFO:teuthology.orchestra.run.trial035.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2026-02-24T07:29:49.984 INFO:teuthology.orchestra.run.trial035.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2026-02-24T07:29:49.984 INFO:teuthology.orchestra.run.trial035.stdout: python3-isodate noarch 0.6.1-3.el9 epel 56 k 2026-02-24T07:29:49.985 INFO:teuthology.orchestra.run.trial035.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2026-02-24T07:29:49.985 INFO:teuthology.orchestra.run.trial035.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2026-02-24T07:29:49.985 INFO:teuthology.orchestra.run.trial035.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2026-02-24T07:29:49.985 INFO:teuthology.orchestra.run.trial035.stdout: python3-jaraco-context noarch 6.0.1-3.el9 epel 20 k 2026-02-24T07:29:49.985 INFO:teuthology.orchestra.run.trial035.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2026-02-24T07:29:49.985 INFO:teuthology.orchestra.run.trial035.stdout: python3-jaraco-text noarch 4.0.0-2.el9 epel 26 k 2026-02-24T07:29:49.985 INFO:teuthology.orchestra.run.trial035.stdout: python3-jinja2 noarch 2.11.3-8.el9 appstream 249 k 2026-02-24T07:29:49.985 INFO:teuthology.orchestra.run.trial035.stdout: python3-jmespath noarch 1.0.1-1.el9 appstream 48 k 2026-02-24T07:29:49.985 INFO:teuthology.orchestra.run.trial035.stdout: python3-kubernetes noarch 1:26.1.0-3.el9 epel 1.0 M 2026-02-24T07:29:49.985 INFO:teuthology.orchestra.run.trial035.stdout: python3-libstoragemgmt x86_64 1.10.1-1.el9 appstream 177 k 2026-02-24T07:29:49.985 INFO:teuthology.orchestra.run.trial035.stdout: python3-lxml x86_64 4.6.5-3.el9 appstream 1.2 M 2026-02-24T07:29:49.985 INFO:teuthology.orchestra.run.trial035.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2026-02-24T07:29:49.985 INFO:teuthology.orchestra.run.trial035.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2026-02-24T07:29:49.986 INFO:teuthology.orchestra.run.trial035.stdout: python3-msgpack x86_64 1.0.3-2.el9 epel 86 k 2026-02-24T07:29:49.986 INFO:teuthology.orchestra.run.trial035.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2026-02-24T07:29:49.986 INFO:teuthology.orchestra.run.trial035.stdout: python3-numpy x86_64 1:1.23.5-2.el9 appstream 6.1 M 2026-02-24T07:29:49.986 INFO:teuthology.orchestra.run.trial035.stdout: python3-numpy-f2py x86_64 1:1.23.5-2.el9 appstream 442 k 2026-02-24T07:29:49.986 INFO:teuthology.orchestra.run.trial035.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2026-02-24T07:29:49.986 INFO:teuthology.orchestra.run.trial035.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2026-02-24T07:29:49.986 INFO:teuthology.orchestra.run.trial035.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2026-02-24T07:29:49.986 INFO:teuthology.orchestra.run.trial035.stdout: python3-protobuf noarch 3.14.0-17.el9 appstream 267 k 2026-02-24T07:29:49.986 INFO:teuthology.orchestra.run.trial035.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2026-02-24T07:29:49.986 INFO:teuthology.orchestra.run.trial035.stdout: python3-pyasn1 noarch 0.4.8-7.el9 appstream 157 k 2026-02-24T07:29:49.986 INFO:teuthology.orchestra.run.trial035.stdout: python3-pyasn1-modules noarch 0.4.8-7.el9 appstream 277 k 2026-02-24T07:29:49.986 INFO:teuthology.orchestra.run.trial035.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2026-02-24T07:29:49.987 INFO:teuthology.orchestra.run.trial035.stdout: python3-pyparsing noarch 2.4.7-9.el9 baseos 150 k 2026-02-24T07:29:49.987 INFO:teuthology.orchestra.run.trial035.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2026-02-24T07:29:49.987 INFO:teuthology.orchestra.run.trial035.stdout: python3-requests noarch 2.25.1-10.el9 baseos 126 k 2026-02-24T07:29:49.987 INFO:teuthology.orchestra.run.trial035.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2026-02-24T07:29:49.987 INFO:teuthology.orchestra.run.trial035.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2026-02-24T07:29:49.987 INFO:teuthology.orchestra.run.trial035.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2026-02-24T07:29:49.987 INFO:teuthology.orchestra.run.trial035.stdout: python3-scipy x86_64 1.9.3-2.el9 appstream 19 M 2026-02-24T07:29:49.987 INFO:teuthology.orchestra.run.trial035.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2026-02-24T07:29:49.987 INFO:teuthology.orchestra.run.trial035.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2026-02-24T07:29:49.987 INFO:teuthology.orchestra.run.trial035.stdout: python3-typing-extensions noarch 4.15.0-1.el9 epel 86 k 2026-02-24T07:29:49.987 INFO:teuthology.orchestra.run.trial035.stdout: python3-urllib3 noarch 1.26.5-7.el9 baseos 218 k 2026-02-24T07:29:49.987 INFO:teuthology.orchestra.run.trial035.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2026-02-24T07:29:49.987 INFO:teuthology.orchestra.run.trial035.stdout: python3-xmlsec x86_64 1.3.13-1.el9 epel 48 k 2026-02-24T07:29:49.987 INFO:teuthology.orchestra.run.trial035.stdout: python3-xmltodict noarch 0.12.0-15.el9 epel 22 k 2026-02-24T07:29:49.988 INFO:teuthology.orchestra.run.trial035.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2026-02-24T07:29:49.988 INFO:teuthology.orchestra.run.trial035.stdout: unzip x86_64 6.0-59.el9 baseos 182 k 2026-02-24T07:29:49.988 INFO:teuthology.orchestra.run.trial035.stdout: xmlsec1 x86_64 1.2.29-13.el9 appstream 189 k 2026-02-24T07:29:49.988 INFO:teuthology.orchestra.run.trial035.stdout: xmlsec1-openssl x86_64 1.2.29-13.el9 appstream 90 k 2026-02-24T07:29:49.988 INFO:teuthology.orchestra.run.trial035.stdout: zip x86_64 3.0-35.el9 baseos 266 k 2026-02-24T07:29:49.988 INFO:teuthology.orchestra.run.trial035.stdout:Installing weak dependencies: 2026-02-24T07:29:49.988 INFO:teuthology.orchestra.run.trial035.stdout: ceph-mgr-cephadm noarch 2:20.3.0-5440.gff63f3d9.el9 ceph-noarch 184 k 2026-02-24T07:29:49.988 INFO:teuthology.orchestra.run.trial035.stdout: ceph-mgr-dashboard noarch 2:20.3.0-5440.gff63f3d9.el9 ceph-noarch 51 M 2026-02-24T07:29:49.988 INFO:teuthology.orchestra.run.trial035.stdout: ceph-mgr-diskprediction-local noarch 2:20.3.0-5440.gff63f3d9.el9 ceph-noarch 7.4 M 2026-02-24T07:29:49.988 INFO:teuthology.orchestra.run.trial035.stdout: ceph-mgr-k8sevents noarch 2:20.3.0-5440.gff63f3d9.el9 ceph-noarch 22 k 2026-02-24T07:29:49.988 INFO:teuthology.orchestra.run.trial035.stdout: ceph-mgr-rook noarch 2:20.3.0-5440.gff63f3d9.el9 ceph-noarch 50 k 2026-02-24T07:29:49.988 INFO:teuthology.orchestra.run.trial035.stdout: ceph-volume noarch 2:20.3.0-5440.gff63f3d9.el9 ceph-noarch 301 k 2026-02-24T07:29:49.989 INFO:teuthology.orchestra.run.trial035.stdout: python3-influxdb noarch 5.3.1-1.el9 epel 139 k 2026-02-24T07:29:49.989 INFO:teuthology.orchestra.run.trial035.stdout: python3-saml noarch 1.16.0-1.el9 epel 125 k 2026-02-24T07:29:49.989 INFO:teuthology.orchestra.run.trial035.stdout: 2026-02-24T07:29:49.989 INFO:teuthology.orchestra.run.trial035.stdout:Transaction Summary 2026-02-24T07:29:49.989 INFO:teuthology.orchestra.run.trial035.stdout:======================================================================================= 2026-02-24T07:29:49.989 INFO:teuthology.orchestra.run.trial035.stdout:Install 96 Packages 2026-02-24T07:29:49.989 INFO:teuthology.orchestra.run.trial035.stdout: 2026-02-24T07:29:49.990 INFO:teuthology.orchestra.run.trial035.stdout:Total download size: 225 M 2026-02-24T07:29:49.990 INFO:teuthology.orchestra.run.trial035.stdout:Installed size: 1.5 G 2026-02-24T07:29:49.990 INFO:teuthology.orchestra.run.trial035.stdout:Downloading Packages: 2026-02-24T07:29:50.872 INFO:teuthology.orchestra.run.trial035.stdout:(1/96): ceph-20.3.0-5440.gff63f3d9.el9.x86_64.r 109 kB/s | 6.5 kB 00:00 2026-02-24T07:29:50.960 INFO:teuthology.orchestra.run.trial035.stdout:(2/96): ceph-mgr-20.3.0-5440.gff63f3d9.el9.x86_ 6.2 MB/s | 937 kB 00:00 2026-02-24T07:29:50.981 INFO:teuthology.orchestra.run.trial035.stdout:(3/96): ceph-mds-20.3.0-5440.gff63f3d9.el9.x86_ 12 MB/s | 2.0 MB 00:00 2026-02-24T07:29:50.984 INFO:teuthology.orchestra.run.trial035.stdout:(4/96): ceph-osd-20.3.0-5440.gff63f3d9.el9.x86_ 6.5 MB/s | 163 kB 00:00 2026-02-24T07:29:51.114 INFO:teuthology.orchestra.run.trial035.stdout:(5/96): ceph-mon-20.3.0-5440.gff63f3d9.el9.x86_ 20 MB/s | 4.9 MB 00:00 2026-02-24T07:29:51.155 INFO:teuthology.orchestra.run.trial035.stdout:(6/96): ceph-grafana-dashboards-20.3.0-5440.gff 1.4 MB/s | 57 kB 00:00 2026-02-24T07:29:51.201 INFO:teuthology.orchestra.run.trial035.stdout:(7/96): ceph-mgr-cephadm-20.3.0-5440.gff63f3d9. 4.0 MB/s | 184 kB 00:00 2026-02-24T07:29:51.600 INFO:teuthology.orchestra.run.trial035.stdout:(8/96): ceph-osd-classic-20.3.0-5440.gff63f3d9. 24 MB/s | 15 MB 00:00 2026-02-24T07:29:51.953 INFO:teuthology.orchestra.run.trial035.stdout:(9/96): ceph-mgr-diskprediction-local-20.3.0-54 21 MB/s | 7.4 MB 00:00 2026-02-24T07:29:51.984 INFO:teuthology.orchestra.run.trial035.stdout:(10/96): ceph-mgr-k8sevents-20.3.0-5440.gff63f3 705 kB/s | 22 kB 00:00 2026-02-24T07:29:52.024 INFO:teuthology.orchestra.run.trial035.stdout:(11/96): ceph-mgr-modules-core-20.3.0-5440.gff6 7.4 MB/s | 298 kB 00:00 2026-02-24T07:29:52.058 INFO:teuthology.orchestra.run.trial035.stdout:(12/96): ceph-mgr-rook-20.3.0-5440.gff63f3d9.el 1.4 MB/s | 50 kB 00:00 2026-02-24T07:29:52.093 INFO:teuthology.orchestra.run.trial035.stdout:(13/96): ceph-prometheus-alerts-20.3.0-5440.gff 502 kB/s | 17 kB 00:00 2026-02-24T07:29:52.133 INFO:teuthology.orchestra.run.trial035.stdout:(14/96): ceph-volume-20.3.0-5440.gff63f3d9.el9. 7.4 MB/s | 301 kB 00:00 2026-02-24T07:29:52.187 INFO:teuthology.orchestra.run.trial035.stdout:(15/96): cephadm-20.3.0-5440.gff63f3d9.el9.noar 19 MB/s | 1.0 MB 00:00 2026-02-24T07:29:53.027 INFO:teuthology.orchestra.run.trial035.stdout:(16/96): cryptsetup-2.8.1-3.el9.x86_64.rpm 418 kB/s | 351 kB 00:00 2026-02-24T07:29:53.187 INFO:teuthology.orchestra.run.trial035.stdout:(17/96): ceph-mgr-dashboard-20.3.0-5440.gff63f3 26 MB/s | 51 MB 00:01 2026-02-24T07:29:53.187 INFO:teuthology.orchestra.run.trial035.stdout:(18/96): ledmon-libs-1.1.0-3.el9.x86_64.rpm 253 kB/s | 40 kB 00:00 2026-02-24T07:29:53.364 INFO:teuthology.orchestra.run.trial035.stdout:(19/96): libconfig-1.7.2-9.el9.x86_64.rpm 406 kB/s | 72 kB 00:00 2026-02-24T07:29:53.620 INFO:teuthology.orchestra.run.trial035.stdout:(20/96): libquadmath-11.5.0-14.el9.x86_64.rpm 721 kB/s | 184 kB 00:00 2026-02-24T07:29:53.958 INFO:teuthology.orchestra.run.trial035.stdout:(21/96): ceph-osd-crimson-20.3.0-5440.gff63f3d9 29 MB/s | 87 MB 00:02 2026-02-24T07:29:53.996 INFO:teuthology.orchestra.run.trial035.stdout:(22/96): python3-cffi-1.14.5-5.el9.x86_64.rpm 674 kB/s | 253 kB 00:00 2026-02-24T07:29:54.174 INFO:teuthology.orchestra.run.trial035.stdout:(23/96): python3-ply-3.11-14.el9.noarch.rpm 599 kB/s | 106 kB 00:00 2026-02-24T07:29:54.352 INFO:teuthology.orchestra.run.trial035.stdout:(24/96): python3-pycparser-2.20-6.el9.noarch.rp 759 kB/s | 135 kB 00:00 2026-02-24T07:29:54.492 INFO:teuthology.orchestra.run.trial035.stdout:(25/96): libgfortran-11.5.0-14.el9.x86_64.rpm 609 kB/s | 794 kB 00:01 2026-02-24T07:29:54.608 INFO:teuthology.orchestra.run.trial035.stdout:(26/96): python3-pyparsing-2.4.7-9.el9.noarch.r 587 kB/s | 150 kB 00:00 2026-02-24T07:29:54.671 INFO:teuthology.orchestra.run.trial035.stdout:(27/96): python3-requests-2.25.1-10.el9.noarch. 707 kB/s | 126 kB 00:00 2026-02-24T07:29:54.928 INFO:teuthology.orchestra.run.trial035.stdout:(28/96): unzip-6.0-59.el9.x86_64.rpm 709 kB/s | 182 kB 00:00 2026-02-24T07:29:54.943 INFO:teuthology.orchestra.run.trial035.stdout:(29/96): python3-urllib3-1.26.5-7.el9.noarch.rp 651 kB/s | 218 kB 00:00 2026-02-24T07:29:54.989 INFO:teuthology.orchestra.run.trial035.stdout:(30/96): flexiblas-3.0.4-9.el9.x86_64.rpm 638 kB/s | 30 kB 00:00 2026-02-24T07:29:55.122 INFO:teuthology.orchestra.run.trial035.stdout:(31/96): flexiblas-netlib-3.0.4-9.el9.x86_64.rp 23 MB/s | 3.0 MB 00:00 2026-02-24T07:29:55.143 INFO:teuthology.orchestra.run.trial035.stdout:(32/96): flexiblas-openblas-openmp-3.0.4-9.el9. 683 kB/s | 15 kB 00:00 2026-02-24T07:29:55.167 INFO:teuthology.orchestra.run.trial035.stdout:(33/96): libstoragemgmt-1.10.1-1.el9.x86_64.rpm 10 MB/s | 246 kB 00:00 2026-02-24T07:29:55.191 INFO:teuthology.orchestra.run.trial035.stdout:(34/96): lua-5.4.4-4.el9.x86_64.rpm 7.8 MB/s | 188 kB 00:00 2026-02-24T07:29:55.213 INFO:teuthology.orchestra.run.trial035.stdout:(35/96): openblas-0.3.29-1.el9.x86_64.rpm 1.9 MB/s | 42 kB 00:00 2026-02-24T07:29:55.262 INFO:teuthology.orchestra.run.trial035.stdout:(36/96): zip-3.0-35.el9.x86_64.rpm 794 kB/s | 266 kB 00:00 2026-02-24T07:29:55.326 INFO:teuthology.orchestra.run.trial035.stdout:(37/96): openblas-openmp-0.3.29-1.el9.x86_64.rp 47 MB/s | 5.3 MB 00:00 2026-02-24T07:29:55.351 INFO:teuthology.orchestra.run.trial035.stdout:(38/96): python3-devel-3.9.25-3.el9.x86_64.rpm 9.5 MB/s | 244 kB 00:00 2026-02-24T07:29:55.375 INFO:teuthology.orchestra.run.trial035.stdout:(39/96): python3-jinja2-2.11.3-8.el9.noarch.rpm 10 MB/s | 249 kB 00:00 2026-02-24T07:29:55.398 INFO:teuthology.orchestra.run.trial035.stdout:(40/96): python3-jmespath-1.0.1-1.el9.noarch.rp 2.1 MB/s | 48 kB 00:00 2026-02-24T07:29:55.421 INFO:teuthology.orchestra.run.trial035.stdout:(41/96): python3-libstoragemgmt-1.10.1-1.el9.x8 7.4 MB/s | 177 kB 00:00 2026-02-24T07:29:55.468 INFO:teuthology.orchestra.run.trial035.stdout:(42/96): python3-lxml-4.6.5-3.el9.x86_64.rpm 26 MB/s | 1.2 MB 00:00 2026-02-24T07:29:55.491 INFO:teuthology.orchestra.run.trial035.stdout:(43/96): python3-markupsafe-1.1.1-12.el9.x86_64 1.5 MB/s | 35 kB 00:00 2026-02-24T07:29:55.498 INFO:teuthology.orchestra.run.trial035.stdout:(44/96): python3-babel-2.9.1-2.el9.noarch.rpm 25 MB/s | 6.0 MB 00:00 2026-02-24T07:29:55.523 INFO:teuthology.orchestra.run.trial035.stdout:(45/96): python3-numpy-f2py-1.23.5-2.el9.x86_64 17 MB/s | 442 kB 00:00 2026-02-24T07:29:55.546 INFO:teuthology.orchestra.run.trial035.stdout:(46/96): python3-packaging-20.9-5.el9.noarch.rp 3.3 MB/s | 77 kB 00:00 2026-02-24T07:29:55.569 INFO:teuthology.orchestra.run.trial035.stdout:(47/96): python3-protobuf-3.14.0-17.el9.noarch. 11 MB/s | 267 kB 00:00 2026-02-24T07:29:55.593 INFO:teuthology.orchestra.run.trial035.stdout:(48/96): python3-pyasn1-0.4.8-7.el9.noarch.rpm 6.4 MB/s | 157 kB 00:00 2026-02-24T07:29:55.617 INFO:teuthology.orchestra.run.trial035.stdout:(49/96): python3-pyasn1-modules-0.4.8-7.el9.noa 11 MB/s | 277 kB 00:00 2026-02-24T07:29:55.625 INFO:teuthology.orchestra.run.trial035.stdout:(50/96): python3-numpy-1.23.5-2.el9.x86_64.rpm 46 MB/s | 6.1 MB 00:00 2026-02-24T07:29:55.640 INFO:teuthology.orchestra.run.trial035.stdout:(51/96): python3-requests-oauthlib-1.3.0-12.el9 2.3 MB/s | 54 kB 00:00 2026-02-24T07:29:55.662 INFO:teuthology.orchestra.run.trial035.stdout:(52/96): python3-toml-0.10.2-6.el9.noarch.rpm 1.8 MB/s | 42 kB 00:00 2026-02-24T07:29:55.684 INFO:teuthology.orchestra.run.trial035.stdout:(53/96): xmlsec1-1.2.29-13.el9.x86_64.rpm 8.3 MB/s | 189 kB 00:00 2026-02-24T07:29:55.707 INFO:teuthology.orchestra.run.trial035.stdout:(54/96): xmlsec1-openssl-1.2.29-13.el9.x86_64.r 3.9 MB/s | 90 kB 00:00 2026-02-24T07:29:55.735 INFO:teuthology.orchestra.run.trial035.stdout:(55/96): lua-devel-5.4.4-4.el9.x86_64.rpm 796 kB/s | 22 kB 00:00 2026-02-24T07:29:55.774 INFO:teuthology.orchestra.run.trial035.stdout:(56/96): python3-cryptography-36.0.1-5.el9.x86_ 702 kB/s | 1.2 MB 00:01 2026-02-24T07:29:55.778 INFO:teuthology.orchestra.run.trial035.stdout:(57/96): protobuf-compiler-3.14.0-17.el9.x86_64 19 MB/s | 862 kB 00:00 2026-02-24T07:29:56.008 INFO:teuthology.orchestra.run.trial035.stdout:(58/96): python3-scipy-1.9.3-2.el9.x86_64.rpm 50 MB/s | 19 MB 00:00 2026-02-24T07:29:57.889 INFO:teuthology.orchestra.run.trial035.stdout:(59/96): grpc-data-1.46.7-10.el9.noarch.rpm 9.2 kB/s | 19 kB 00:02 2026-02-24T07:29:59.444 INFO:teuthology.orchestra.run.trial035.stdout:(60/96): abseil-cpp-20211102.0-4.el9.x86_64.rpm 150 kB/s | 551 kB 00:03 2026-02-24T07:29:59.833 INFO:teuthology.orchestra.run.trial035.stdout:(61/96): luarocks-3.9.2-5.el9.noarch.rpm 40 kB/s | 151 kB 00:03 2026-02-24T07:29:59.937 INFO:teuthology.orchestra.run.trial035.stdout:(62/96): python3-asyncssh-2.13.2-5.el9.noarch.r 268 kB/s | 548 kB 00:02 2026-02-24T07:30:00.444 INFO:teuthology.orchestra.run.trial035.stdout:(63/96): python3-backports-tarfile-1.2.0-1.el9. 98 kB/s | 60 kB 00:00 2026-02-24T07:30:00.940 INFO:teuthology.orchestra.run.trial035.stdout:(64/96): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 43 kB/s | 43 kB 00:01 2026-02-24T07:30:01.061 INFO:teuthology.orchestra.run.trial035.stdout:(65/96): python3-cachetools-4.2.4-1.el9.noarch. 52 kB/s | 32 kB 00:00 2026-02-24T07:30:01.382 INFO:teuthology.orchestra.run.trial035.stdout:(66/96): python3-certifi-2023.05.07-4.el9.noarc 32 kB/s | 14 kB 00:00 2026-02-24T07:30:02.615 INFO:teuthology.orchestra.run.trial035.stdout:(67/96): python3-autocommand-2.2.2-8.el9.noarch 9.3 kB/s | 29 kB 00:03 2026-02-24T07:30:02.951 INFO:teuthology.orchestra.run.trial035.stdout:(68/96): python3-cherrypy-18.6.1-2.el9.noarch.r 228 kB/s | 358 kB 00:01 2026-02-24T07:30:03.346 INFO:teuthology.orchestra.run.trial035.stdout:(69/96): python3-cheroot-10.0.1-4.el9.noarch.rp 76 kB/s | 173 kB 00:02 2026-02-24T07:30:04.168 INFO:teuthology.orchestra.run.trial035.stdout:(70/96): python3-google-auth-2.45.0-1.el9.noarc 164 kB/s | 254 kB 00:01 2026-02-24T07:30:05.479 INFO:teuthology.orchestra.run.trial035.stdout:(71/96): python3-influxdb-5.3.1-1.el9.noarch.rp 106 kB/s | 139 kB 00:01 2026-02-24T07:30:05.641 INFO:teuthology.orchestra.run.trial035.stdout:(72/96): python3-grpcio-tools-1.46.7-10.el9.x86 63 kB/s | 144 kB 00:02 2026-02-24T07:30:06.199 INFO:teuthology.orchestra.run.trial035.stdout:(73/96): python3-isodate-0.6.1-3.el9.noarch.rpm 78 kB/s | 56 kB 00:00 2026-02-24T07:30:06.837 INFO:teuthology.orchestra.run.trial035.stdout:(74/96): python3-jaraco-classes-3.2.1-5.el9.noa 28 kB/s | 18 kB 00:00 2026-02-24T07:30:06.892 INFO:teuthology.orchestra.run.trial035.stdout:(75/96): python3-jaraco-8.2.1-3.el9.noarch.rpm 8.5 kB/s | 11 kB 00:01 2026-02-24T07:30:07.200 INFO:teuthology.orchestra.run.trial035.stdout:(76/96): python3-jaraco-collections-3.0.0-8.el9 64 kB/s | 23 kB 00:00 2026-02-24T07:30:07.385 INFO:teuthology.orchestra.run.trial035.stdout:(77/96): python3-jaraco-context-6.0.1-3.el9.noa 40 kB/s | 20 kB 00:00 2026-02-24T07:30:08.428 INFO:teuthology.orchestra.run.trial035.stdout:(78/96): python3-jaraco-text-4.0.0-2.el9.noarch 25 kB/s | 26 kB 00:01 2026-02-24T07:30:08.951 INFO:teuthology.orchestra.run.trial035.stdout:(79/96): python3-jaraco-functools-3.5.0-2.el9.n 11 kB/s | 19 kB 00:01 2026-02-24T07:30:10.028 INFO:teuthology.orchestra.run.trial035.stdout:(80/96): python3-more-itertools-8.12.0-2.el9.no 73 kB/s | 79 kB 00:01 2026-02-24T07:30:11.446 INFO:teuthology.orchestra.run.trial035.stdout:(81/96): python3-msgpack-1.0.3-2.el9.x86_64.rpm 61 kB/s | 86 kB 00:01 2026-02-24T07:30:12.365 INFO:teuthology.orchestra.run.trial035.stdout:(82/96): python3-natsort-7.1.1-5.el9.noarch.rpm 63 kB/s | 58 kB 00:00 2026-02-24T07:30:12.987 INFO:teuthology.orchestra.run.trial035.stdout:(83/96): python3-portend-3.1.0-2.el9.noarch.rpm 26 kB/s | 16 kB 00:00 2026-02-24T07:30:13.735 INFO:teuthology.orchestra.run.trial035.stdout:(84/96): python3-grpcio-1.46.7-10.el9.x86_64.rp 193 kB/s | 2.0 MB 00:10 2026-02-24T07:30:14.423 INFO:teuthology.orchestra.run.trial035.stdout:(85/96): python3-pyOpenSSL-21.0.0-1.el9.noarch. 63 kB/s | 90 kB 00:01 2026-02-24T07:30:16.230 INFO:teuthology.orchestra.run.trial035.stdout:(86/96): python3-repoze-lru-0.7-16.el9.noarch.r 12 kB/s | 31 kB 00:02 2026-02-24T07:30:17.019 INFO:teuthology.orchestra.run.trial035.stdout:(87/96): python3-routes-2.5.1-5.el9.noarch.rpm 73 kB/s | 188 kB 00:02 2026-02-24T07:30:20.621 INFO:teuthology.orchestra.run.trial035.stdout:(88/96): python3-rsa-4.9-2.el9.noarch.rpm 13 kB/s | 59 kB 00:04 2026-02-24T07:30:20.621 INFO:teuthology.orchestra.run.trial035.stdout:(89/96): python3-saml-1.16.0-1.el9.noarch.rpm 35 kB/s | 125 kB 00:03 2026-02-24T07:30:21.293 INFO:teuthology.orchestra.run.trial035.stdout:(90/96): python3-tempora-5.0.0-2.el9.noarch.rpm 53 kB/s | 36 kB 00:00 2026-02-24T07:30:21.792 INFO:teuthology.orchestra.run.trial035.stdout:(91/96): python3-typing-extensions-4.15.0-1.el9 74 kB/s | 86 kB 00:01 2026-02-24T07:30:22.363 INFO:teuthology.orchestra.run.trial035.stdout:(92/96): python3-websocket-client-1.2.3-2.el9.n 84 kB/s | 90 kB 00:01 2026-02-24T07:30:23.493 INFO:teuthology.orchestra.run.trial035.stdout:(93/96): python3-xmltodict-0.12.0-15.el9.noarch 20 kB/s | 22 kB 00:01 2026-02-24T07:30:23.705 INFO:teuthology.orchestra.run.trial035.stdout:(94/96): python3-xmlsec-1.3.13-1.el9.x86_64.rpm 25 kB/s | 48 kB 00:01 2026-02-24T07:30:24.485 INFO:teuthology.orchestra.run.trial035.stdout:(95/96): python3-zc-lockfile-2.0-10.el9.noarch. 20 kB/s | 20 kB 00:00 2026-02-24T07:30:27.278 INFO:teuthology.orchestra.run.trial035.stdout:(96/96): python3-kubernetes-26.1.0-3.el9.noarch 55 kB/s | 1.0 MB 00:18 2026-02-24T07:30:27.280 INFO:teuthology.orchestra.run.trial035.stdout:-------------------------------------------------------------------------------- 2026-02-24T07:30:27.280 INFO:teuthology.orchestra.run.trial035.stdout:Total 6.0 MB/s | 225 MB 00:37 2026-02-24T07:30:27.666 INFO:teuthology.orchestra.run.trial035.stdout:Running transaction check 2026-02-24T07:30:27.705 INFO:teuthology.orchestra.run.trial035.stdout:Transaction check succeeded. 2026-02-24T07:30:27.705 INFO:teuthology.orchestra.run.trial035.stdout:Running transaction test 2026-02-24T07:30:28.500 INFO:teuthology.orchestra.run.trial035.stdout:Transaction test succeeded. 2026-02-24T07:30:28.500 INFO:teuthology.orchestra.run.trial035.stdout:Running transaction 2026-02-24T07:30:29.322 INFO:teuthology.orchestra.run.trial035.stdout: Preparing : 1/1 2026-02-24T07:30:29.324 INFO:teuthology.orchestra.run.trial035.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 1/96 2026-02-24T07:30:29.328 INFO:teuthology.orchestra.run.trial035.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 2/96 2026-02-24T07:30:29.350 INFO:teuthology.orchestra.run.trial035.stdout: Installing : xmlsec1-1.2.29-13.el9.x86_64 3/96 2026-02-24T07:30:29.383 INFO:teuthology.orchestra.run.trial035.stdout: Installing : python3-pyasn1-0.4.8-7.el9.noarch 4/96 2026-02-24T07:30:29.390 INFO:teuthology.orchestra.run.trial035.stdout: Installing : python3-lxml-4.6.5-3.el9.x86_64 5/96 2026-02-24T07:30:29.393 INFO:teuthology.orchestra.run.trial035.stdout: Installing : lua-5.4.4-4.el9.x86_64 6/96 2026-02-24T07:30:29.397 INFO:teuthology.orchestra.run.trial035.stdout: Installing : flexiblas-3.0.4-9.el9.x86_64 7/96 2026-02-24T07:30:29.420 INFO:teuthology.orchestra.run.trial035.stdout: Installing : unzip-6.0-59.el9.x86_64 8/96 2026-02-24T07:30:29.433 INFO:teuthology.orchestra.run.trial035.stdout: Installing : python3-urllib3-1.26.5-7.el9.noarch 9/96 2026-02-24T07:30:29.436 INFO:teuthology.orchestra.run.trial035.stdout: Installing : python3-requests-2.25.1-10.el9.noarch 10/96 2026-02-24T07:30:29.442 INFO:teuthology.orchestra.run.trial035.stdout: Installing : libquadmath-11.5.0-14.el9.x86_64 11/96 2026-02-24T07:30:29.444 INFO:teuthology.orchestra.run.trial035.stdout: Installing : libgfortran-11.5.0-14.el9.x86_64 12/96 2026-02-24T07:30:29.455 INFO:teuthology.orchestra.run.trial035.stdout: Installing : ledmon-libs-1.1.0-3.el9.x86_64 13/96 2026-02-24T07:30:29.461 INFO:teuthology.orchestra.run.trial035.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 14/96 2026-02-24T07:30:29.485 INFO:teuthology.orchestra.run.trial035.stdout: Installing : zip-3.0-35.el9.x86_64 15/96 2026-02-24T07:30:29.488 INFO:teuthology.orchestra.run.trial035.stdout: Installing : luarocks-3.9.2-5.el9.noarch 16/96 2026-02-24T07:30:29.539 INFO:teuthology.orchestra.run.trial035.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 17/96 2026-02-24T07:30:29.552 INFO:teuthology.orchestra.run.trial035.stdout: Installing : python3-pyasn1-modules-0.4.8-7.el9.noarch 18/96 2026-02-24T07:30:29.554 INFO:teuthology.orchestra.run.trial035.stdout: Installing : python3-rsa-4.9-2.el9.noarch 19/96 2026-02-24T07:30:29.559 INFO:teuthology.orchestra.run.trial035.stdout: Installing : xmlsec1-openssl-1.2.29-13.el9.x86_64 20/96 2026-02-24T07:30:29.564 INFO:teuthology.orchestra.run.trial035.stdout: Installing : python3-xmlsec-1.3.13-1.el9.x86_64 21/96 2026-02-24T07:30:29.569 INFO:teuthology.orchestra.run.trial035.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 22/96 2026-02-24T07:30:29.572 INFO:teuthology.orchestra.run.trial035.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 23/96 2026-02-24T07:30:29.587 INFO:teuthology.orchestra.run.trial035.stdout: Installing : python3-xmltodict-0.12.0-15.el9.noarch 24/96 2026-02-24T07:30:29.591 INFO:teuthology.orchestra.run.trial035.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 25/96 2026-02-24T07:30:29.597 INFO:teuthology.orchestra.run.trial035.stdout: Installing : python3-typing-extensions-4.15.0-1.el9.noarch 26/96 2026-02-24T07:30:29.608 INFO:teuthology.orchestra.run.trial035.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 27/96 2026-02-24T07:30:29.617 INFO:teuthology.orchestra.run.trial035.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 28/96 2026-02-24T07:30:29.623 INFO:teuthology.orchestra.run.trial035.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 29/96 2026-02-24T07:30:29.643 INFO:teuthology.orchestra.run.trial035.stdout: Installing : python3-msgpack-1.0.3-2.el9.x86_64 30/96 2026-02-24T07:30:29.654 INFO:teuthology.orchestra.run.trial035.stdout: Installing : python3-influxdb-5.3.1-1.el9.noarch 31/96 2026-02-24T07:30:29.670 INFO:teuthology.orchestra.run.trial035.stdout: Installing : python3-isodate-0.6.1-3.el9.noarch 32/96 2026-02-24T07:30:29.674 INFO:teuthology.orchestra.run.trial035.stdout: Installing : python3-saml-1.16.0-1.el9.noarch 33/96 2026-02-24T07:30:29.682 INFO:teuthology.orchestra.run.trial035.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 34/96 2026-02-24T07:30:29.722 INFO:teuthology.orchestra.run.trial035.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 35/96 2026-02-24T07:30:30.049 INFO:teuthology.orchestra.run.trial035.stdout: Installing : python3-google-auth-1:2.45.0-1.el9.noarch 36/96 2026-02-24T07:30:30.059 INFO:teuthology.orchestra.run.trial035.stdout: Installing : python3-kubernetes-1:26.1.0-3.el9.noarch 37/96 2026-02-24T07:30:30.063 INFO:teuthology.orchestra.run.trial035.stdout: Installing : python3-backports-tarfile-1.2.0-1.el9.noarch 38/96 2026-02-24T07:30:30.069 INFO:teuthology.orchestra.run.trial035.stdout: Installing : python3-jaraco-context-6.0.1-3.el9.noarch 39/96 2026-02-24T07:30:30.071 INFO:teuthology.orchestra.run.trial035.stdout: Installing : python3-autocommand-2.2.2-8.el9.noarch 40/96 2026-02-24T07:30:30.099 INFO:teuthology.orchestra.run.trial035.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 41/96 2026-02-24T07:30:30.142 INFO:teuthology.orchestra.run.trial035.stdout: Installing : abseil-cpp-20211102.0-4.el9.x86_64 42/96 2026-02-24T07:30:30.151 INFO:teuthology.orchestra.run.trial035.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 43/96 2026-02-24T07:30:30.156 INFO:teuthology.orchestra.run.trial035.stdout: Installing : protobuf-compiler-3.14.0-17.el9.x86_64 44/96 2026-02-24T07:30:30.160 INFO:teuthology.orchestra.run.trial035.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 45/96 2026-02-24T07:30:30.166 INFO:teuthology.orchestra.run.trial035.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 46/96 2026-02-24T07:30:30.169 INFO:teuthology.orchestra.run.trial035.stdout: Installing : python3-jaraco-text-4.0.0-2.el9.noarch 47/96 2026-02-24T07:30:30.176 INFO:teuthology.orchestra.run.trial035.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 48/96 2026-02-24T07:30:30.180 INFO:teuthology.orchestra.run.trial035.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 49/96 2026-02-24T07:30:30.208 INFO:teuthology.orchestra.run.trial035.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 50/96 2026-02-24T07:30:30.218 INFO:teuthology.orchestra.run.trial035.stdout: Installing : python3-protobuf-3.14.0-17.el9.noarch 51/96 2026-02-24T07:30:30.224 INFO:teuthology.orchestra.run.trial035.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 52/96 2026-02-24T07:30:30.231 INFO:teuthology.orchestra.run.trial035.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 53/96 2026-02-24T07:30:30.265 INFO:teuthology.orchestra.run.trial035.stdout: Installing : python3-jmespath-1.0.1-1.el9.noarch 54/96 2026-02-24T07:30:30.492 INFO:teuthology.orchestra.run.trial035.stdout: Installing : python3-devel-3.9.25-3.el9.x86_64 55/96 2026-02-24T07:30:30.516 INFO:teuthology.orchestra.run.trial035.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 56/96 2026-02-24T07:30:30.520 INFO:teuthology.orchestra.run.trial035.stdout: Installing : python3-jinja2-2.11.3-8.el9.noarch 57/96 2026-02-24T07:30:30.579 INFO:teuthology.orchestra.run.trial035.stdout: Installing : openblas-0.3.29-1.el9.x86_64 58/96 2026-02-24T07:30:30.581 INFO:teuthology.orchestra.run.trial035.stdout: Installing : openblas-openmp-0.3.29-1.el9.x86_64 59/96 2026-02-24T07:30:30.602 INFO:teuthology.orchestra.run.trial035.stdout: Installing : flexiblas-openblas-openmp-3.0.4-9.el9.x86_64 60/96 2026-02-24T07:30:30.930 INFO:teuthology.orchestra.run.trial035.stdout: Installing : flexiblas-netlib-3.0.4-9.el9.x86_64 61/96 2026-02-24T07:30:31.003 INFO:teuthology.orchestra.run.trial035.stdout: Installing : python3-numpy-1:1.23.5-2.el9.x86_64 62/96 2026-02-24T07:30:31.678 INFO:teuthology.orchestra.run.trial035.stdout: Installing : python3-numpy-f2py-1:1.23.5-2.el9.x86_64 63/96 2026-02-24T07:30:31.696 INFO:teuthology.orchestra.run.trial035.stdout: Installing : python3-scipy-1.9.3-2.el9.x86_64 64/96 2026-02-24T07:30:31.706 INFO:teuthology.orchestra.run.trial035.stdout: Installing : python3-pyparsing-2.4.7-9.el9.noarch 65/96 2026-02-24T07:30:31.713 INFO:teuthology.orchestra.run.trial035.stdout: Installing : python3-packaging-20.9-5.el9.noarch 66/96 2026-02-24T07:30:31.726 INFO:teuthology.orchestra.run.trial035.stdout: Installing : python3-ply-3.11-14.el9.noarch 67/96 2026-02-24T07:30:31.742 INFO:teuthology.orchestra.run.trial035.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 68/96 2026-02-24T07:30:31.818 INFO:teuthology.orchestra.run.trial035.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 69/96 2026-02-24T07:30:31.827 INFO:teuthology.orchestra.run.trial035.stdout: Installing : python3-cryptography-36.0.1-5.el9.x86_64 70/96 2026-02-24T07:30:31.850 INFO:teuthology.orchestra.run.trial035.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 71/96 2026-02-24T07:30:31.880 INFO:teuthology.orchestra.run.trial035.stdout: Installing : python3-cheroot-10.0.1-4.el9.noarch 72/96 2026-02-24T07:30:31.931 INFO:teuthology.orchestra.run.trial035.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 73/96 2026-02-24T07:30:31.937 INFO:teuthology.orchestra.run.trial035.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 74/96 2026-02-24T07:30:31.941 INFO:teuthology.orchestra.run.trial035.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 75/96 2026-02-24T07:30:31.942 INFO:teuthology.orchestra.run.trial035.stdout: Installing : libconfig-1.7.2-9.el9.x86_64 76/96 2026-02-24T07:30:31.960 INFO:teuthology.orchestra.run.trial035.stdout: Running scriptlet: libstoragemgmt-1.10.1-1.el9.x86_64 77/96 2026-02-24T07:30:31.960 INFO:teuthology.orchestra.run.trial035.stdout:Creating group 'libstoragemgmt' with GID 991. 2026-02-24T07:30:31.960 INFO:teuthology.orchestra.run.trial035.stdout:Creating user 'libstoragemgmt' (daemon account for libstoragemgmt) with UID 991 and GID 991. 2026-02-24T07:30:31.960 INFO:teuthology.orchestra.run.trial035.stdout: 2026-02-24T07:30:31.967 INFO:teuthology.orchestra.run.trial035.stdout: Installing : libstoragemgmt-1.10.1-1.el9.x86_64 77/96 2026-02-24T07:30:31.992 INFO:teuthology.orchestra.run.trial035.stdout: Running scriptlet: libstoragemgmt-1.10.1-1.el9.x86_64 77/96 2026-02-24T07:30:31.992 INFO:teuthology.orchestra.run.trial035.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/libstoragemgmt.service → /usr/lib/systemd/system/libstoragemgmt.service. 2026-02-24T07:30:31.992 INFO:teuthology.orchestra.run.trial035.stdout: 2026-02-24T07:30:32.026 INFO:teuthology.orchestra.run.trial035.stdout: Installing : python3-libstoragemgmt-1.10.1-1.el9.x86_64 78/96 2026-02-24T07:30:32.614 INFO:teuthology.orchestra.run.trial035.stdout: Installing : cryptsetup-2.8.1-3.el9.x86_64 79/96 2026-02-24T07:30:37.965 INFO:teuthology.orchestra.run.trial035.stdout: Installing : ceph-osd-classic-2:20.3.0-5440.gff63f3d9.el9.x86_6 80/96 2026-02-24T07:30:38.029 INFO:teuthology.orchestra.run.trial035.stdout: Installing : ceph-osd-crimson-2:20.3.0-5440.gff63f3d9.el9.x86_6 81/96 2026-02-24T07:30:38.031 INFO:teuthology.orchestra.run.trial035.stdout: Installing : ceph-volume-2:20.3.0-5440.gff63f3d9.el9.noarch 82/96 2026-02-24T07:30:38.045 INFO:teuthology.orchestra.run.trial035.stdout: Running scriptlet: ceph-volume-2:20.3.0-5440.gff63f3d9.el9.noarch 82/96 2026-02-24T07:30:38.045 INFO:teuthology.orchestra.run.trial035.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-24T07:30:38.045 INFO:teuthology.orchestra.run.trial035.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2026-02-24T07:30:38.045 INFO:teuthology.orchestra.run.trial035.stdout: 2026-02-24T07:30:38.055 INFO:teuthology.orchestra.run.trial035.stdout: Installing : ceph-osd-2:20.3.0-5440.gff63f3d9.el9.x86_64 83/96 2026-02-24T07:30:38.079 INFO:teuthology.orchestra.run.trial035.stdout: Running scriptlet: ceph-osd-2:20.3.0-5440.gff63f3d9.el9.x86_64 83/96 2026-02-24T07:30:38.079 INFO:teuthology.orchestra.run.trial035.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-24T07:30:38.079 INFO:teuthology.orchestra.run.trial035.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2026-02-24T07:30:38.080 INFO:teuthology.orchestra.run.trial035.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2026-02-24T07:30:38.080 INFO:teuthology.orchestra.run.trial035.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2026-02-24T07:30:38.080 INFO:teuthology.orchestra.run.trial035.stdout: 2026-02-24T07:30:38.133 INFO:teuthology.orchestra.run.trial035.stdout: Running scriptlet: cephadm-2:20.3.0-5440.gff63f3d9.el9.noarch 84/96 2026-02-24T07:30:38.135 INFO:teuthology.orchestra.run.trial035.stdout: Installing : cephadm-2:20.3.0-5440.gff63f3d9.el9.noarch 84/96 2026-02-24T07:30:38.144 INFO:teuthology.orchestra.run.trial035.stdout: Installing : ceph-prometheus-alerts-2:20.3.0-5440.gff63f3d9.el9 85/96 2026-02-24T07:30:38.168 INFO:teuthology.orchestra.run.trial035.stdout: Installing : ceph-grafana-dashboards-2:20.3.0-5440.gff63f3d9.el 86/96 2026-02-24T07:30:38.169 INFO:teuthology.orchestra.run.trial035.stdout: Installing : ceph-mgr-cephadm-2:20.3.0-5440.gff63f3d9.el9.noarc 87/96 2026-02-24T07:30:41.071 INFO:teuthology.orchestra.run.trial035.stdout: Running scriptlet: ceph-mgr-cephadm-2:20.3.0-5440.gff63f3d9.el9.noarc 87/96 2026-02-24T07:30:41.078 INFO:teuthology.orchestra.run.trial035.stdout: Installing : ceph-mgr-dashboard-2:20.3.0-5440.gff63f3d9.el9.noa 88/96 2026-02-24T07:30:41.530 INFO:teuthology.orchestra.run.trial035.stdout: Running scriptlet: ceph-mgr-dashboard-2:20.3.0-5440.gff63f3d9.el9.noa 88/96 2026-02-24T07:30:41.530 INFO:teuthology.orchestra.run.trial035.stdout: Installing : ceph-mgr-diskprediction-local-2:20.3.0-5440.gff63f 89/96 2026-02-24T07:30:41.543 INFO:teuthology.orchestra.run.trial035.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:20.3.0-5440.gff63f 89/96 2026-02-24T07:30:41.543 INFO:teuthology.orchestra.run.trial035.stdout: Installing : ceph-mgr-k8sevents-2:20.3.0-5440.gff63f3d9.el9.noa 90/96 2026-02-24T07:30:41.605 INFO:teuthology.orchestra.run.trial035.stdout: Running scriptlet: ceph-mgr-k8sevents-2:20.3.0-5440.gff63f3d9.el9.noa 90/96 2026-02-24T07:30:41.645 INFO:teuthology.orchestra.run.trial035.stdout: Installing : ceph-mgr-modules-core-2:20.3.0-5440.gff63f3d9.el9. 91/96 2026-02-24T07:30:41.646 INFO:teuthology.orchestra.run.trial035.stdout: Installing : ceph-mgr-2:20.3.0-5440.gff63f3d9.el9.x86_64 92/96 2026-02-24T07:30:41.667 INFO:teuthology.orchestra.run.trial035.stdout: Running scriptlet: ceph-mgr-2:20.3.0-5440.gff63f3d9.el9.x86_64 92/96 2026-02-24T07:30:41.667 INFO:teuthology.orchestra.run.trial035.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-24T07:30:41.667 INFO:teuthology.orchestra.run.trial035.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2026-02-24T07:30:41.667 INFO:teuthology.orchestra.run.trial035.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2026-02-24T07:30:41.668 INFO:teuthology.orchestra.run.trial035.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2026-02-24T07:30:41.668 INFO:teuthology.orchestra.run.trial035.stdout: 2026-02-24T07:30:41.679 INFO:teuthology.orchestra.run.trial035.stdout: Installing : ceph-mgr-rook-2:20.3.0-5440.gff63f3d9.el9.noarch 93/96 2026-02-24T07:30:41.890 INFO:teuthology.orchestra.run.trial035.stdout: Running scriptlet: ceph-mgr-rook-2:20.3.0-5440.gff63f3d9.el9.noarch 93/96 2026-02-24T07:30:41.891 INFO:teuthology.orchestra.run.trial035.stdout: Installing : ceph-mon-2:20.3.0-5440.gff63f3d9.el9.x86_64 94/96 2026-02-24T07:30:41.912 INFO:teuthology.orchestra.run.trial035.stdout: Running scriptlet: ceph-mon-2:20.3.0-5440.gff63f3d9.el9.x86_64 94/96 2026-02-24T07:30:41.912 INFO:teuthology.orchestra.run.trial035.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-24T07:30:41.913 INFO:teuthology.orchestra.run.trial035.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2026-02-24T07:30:41.913 INFO:teuthology.orchestra.run.trial035.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2026-02-24T07:30:41.913 INFO:teuthology.orchestra.run.trial035.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2026-02-24T07:30:41.913 INFO:teuthology.orchestra.run.trial035.stdout: 2026-02-24T07:30:41.994 INFO:teuthology.orchestra.run.trial035.stdout: Installing : ceph-mds-2:20.3.0-5440.gff63f3d9.el9.x86_64 95/96 2026-02-24T07:30:42.014 INFO:teuthology.orchestra.run.trial035.stdout: Running scriptlet: ceph-mds-2:20.3.0-5440.gff63f3d9.el9.x86_64 95/96 2026-02-24T07:30:42.014 INFO:teuthology.orchestra.run.trial035.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-24T07:30:42.014 INFO:teuthology.orchestra.run.trial035.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2026-02-24T07:30:42.014 INFO:teuthology.orchestra.run.trial035.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2026-02-24T07:30:42.014 INFO:teuthology.orchestra.run.trial035.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2026-02-24T07:30:42.014 INFO:teuthology.orchestra.run.trial035.stdout: 2026-02-24T07:30:42.015 INFO:teuthology.orchestra.run.trial035.stdout: Installing : ceph-2:20.3.0-5440.gff63f3d9.el9.x86_64 96/96 2026-02-24T07:30:42.021 INFO:teuthology.orchestra.run.trial035.stdout: Running scriptlet: ceph-osd-classic-2:20.3.0-5440.gff63f3d9.el9.x86_6 96/96 2026-02-24T07:30:42.046 INFO:teuthology.orchestra.run.trial035.stdout: Running scriptlet: ceph-osd-crimson-2:20.3.0-5440.gff63f3d9.el9.x86_6 96/96 2026-02-24T07:30:43.475 INFO:teuthology.orchestra.run.trial035.stdout: Running scriptlet: ceph-2:20.3.0-5440.gff63f3d9.el9.x86_64 96/96 2026-02-24T07:30:43.475 INFO:teuthology.orchestra.run.trial035.stdout: Verifying : ceph-2:20.3.0-5440.gff63f3d9.el9.x86_64 1/96 2026-02-24T07:30:43.476 INFO:teuthology.orchestra.run.trial035.stdout: Verifying : ceph-mds-2:20.3.0-5440.gff63f3d9.el9.x86_64 2/96 2026-02-24T07:30:43.476 INFO:teuthology.orchestra.run.trial035.stdout: Verifying : ceph-mgr-2:20.3.0-5440.gff63f3d9.el9.x86_64 3/96 2026-02-24T07:30:43.476 INFO:teuthology.orchestra.run.trial035.stdout: Verifying : ceph-mon-2:20.3.0-5440.gff63f3d9.el9.x86_64 4/96 2026-02-24T07:30:43.476 INFO:teuthology.orchestra.run.trial035.stdout: Verifying : ceph-osd-2:20.3.0-5440.gff63f3d9.el9.x86_64 5/96 2026-02-24T07:30:43.476 INFO:teuthology.orchestra.run.trial035.stdout: Verifying : ceph-osd-classic-2:20.3.0-5440.gff63f3d9.el9.x86_6 6/96 2026-02-24T07:30:43.476 INFO:teuthology.orchestra.run.trial035.stdout: Verifying : ceph-osd-crimson-2:20.3.0-5440.gff63f3d9.el9.x86_6 7/96 2026-02-24T07:30:43.476 INFO:teuthology.orchestra.run.trial035.stdout: Verifying : ceph-grafana-dashboards-2:20.3.0-5440.gff63f3d9.el 8/96 2026-02-24T07:30:43.476 INFO:teuthology.orchestra.run.trial035.stdout: Verifying : ceph-mgr-cephadm-2:20.3.0-5440.gff63f3d9.el9.noarc 9/96 2026-02-24T07:30:43.476 INFO:teuthology.orchestra.run.trial035.stdout: Verifying : ceph-mgr-dashboard-2:20.3.0-5440.gff63f3d9.el9.noa 10/96 2026-02-24T07:30:43.476 INFO:teuthology.orchestra.run.trial035.stdout: Verifying : ceph-mgr-diskprediction-local-2:20.3.0-5440.gff63f 11/96 2026-02-24T07:30:43.476 INFO:teuthology.orchestra.run.trial035.stdout: Verifying : ceph-mgr-k8sevents-2:20.3.0-5440.gff63f3d9.el9.noa 12/96 2026-02-24T07:30:43.477 INFO:teuthology.orchestra.run.trial035.stdout: Verifying : ceph-mgr-modules-core-2:20.3.0-5440.gff63f3d9.el9. 13/96 2026-02-24T07:30:43.477 INFO:teuthology.orchestra.run.trial035.stdout: Verifying : ceph-mgr-rook-2:20.3.0-5440.gff63f3d9.el9.noarch 14/96 2026-02-24T07:30:43.477 INFO:teuthology.orchestra.run.trial035.stdout: Verifying : ceph-prometheus-alerts-2:20.3.0-5440.gff63f3d9.el9 15/96 2026-02-24T07:30:43.477 INFO:teuthology.orchestra.run.trial035.stdout: Verifying : ceph-volume-2:20.3.0-5440.gff63f3d9.el9.noarch 16/96 2026-02-24T07:30:43.477 INFO:teuthology.orchestra.run.trial035.stdout: Verifying : cephadm-2:20.3.0-5440.gff63f3d9.el9.noarch 17/96 2026-02-24T07:30:43.477 INFO:teuthology.orchestra.run.trial035.stdout: Verifying : cryptsetup-2.8.1-3.el9.x86_64 18/96 2026-02-24T07:30:43.477 INFO:teuthology.orchestra.run.trial035.stdout: Verifying : ledmon-libs-1.1.0-3.el9.x86_64 19/96 2026-02-24T07:30:43.477 INFO:teuthology.orchestra.run.trial035.stdout: Verifying : libconfig-1.7.2-9.el9.x86_64 20/96 2026-02-24T07:30:43.477 INFO:teuthology.orchestra.run.trial035.stdout: Verifying : libgfortran-11.5.0-14.el9.x86_64 21/96 2026-02-24T07:30:43.477 INFO:teuthology.orchestra.run.trial035.stdout: Verifying : libquadmath-11.5.0-14.el9.x86_64 22/96 2026-02-24T07:30:43.477 INFO:teuthology.orchestra.run.trial035.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 23/96 2026-02-24T07:30:43.478 INFO:teuthology.orchestra.run.trial035.stdout: Verifying : python3-cryptography-36.0.1-5.el9.x86_64 24/96 2026-02-24T07:30:43.478 INFO:teuthology.orchestra.run.trial035.stdout: Verifying : python3-ply-3.11-14.el9.noarch 25/96 2026-02-24T07:30:43.478 INFO:teuthology.orchestra.run.trial035.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 26/96 2026-02-24T07:30:43.478 INFO:teuthology.orchestra.run.trial035.stdout: Verifying : python3-pyparsing-2.4.7-9.el9.noarch 27/96 2026-02-24T07:30:43.478 INFO:teuthology.orchestra.run.trial035.stdout: Verifying : python3-requests-2.25.1-10.el9.noarch 28/96 2026-02-24T07:30:43.478 INFO:teuthology.orchestra.run.trial035.stdout: Verifying : python3-urllib3-1.26.5-7.el9.noarch 29/96 2026-02-24T07:30:43.478 INFO:teuthology.orchestra.run.trial035.stdout: Verifying : unzip-6.0-59.el9.x86_64 30/96 2026-02-24T07:30:43.478 INFO:teuthology.orchestra.run.trial035.stdout: Verifying : zip-3.0-35.el9.x86_64 31/96 2026-02-24T07:30:43.478 INFO:teuthology.orchestra.run.trial035.stdout: Verifying : flexiblas-3.0.4-9.el9.x86_64 32/96 2026-02-24T07:30:43.478 INFO:teuthology.orchestra.run.trial035.stdout: Verifying : flexiblas-netlib-3.0.4-9.el9.x86_64 33/96 2026-02-24T07:30:43.478 INFO:teuthology.orchestra.run.trial035.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-9.el9.x86_64 34/96 2026-02-24T07:30:43.478 INFO:teuthology.orchestra.run.trial035.stdout: Verifying : libstoragemgmt-1.10.1-1.el9.x86_64 35/96 2026-02-24T07:30:43.478 INFO:teuthology.orchestra.run.trial035.stdout: Verifying : lua-5.4.4-4.el9.x86_64 36/96 2026-02-24T07:30:43.479 INFO:teuthology.orchestra.run.trial035.stdout: Verifying : openblas-0.3.29-1.el9.x86_64 37/96 2026-02-24T07:30:43.479 INFO:teuthology.orchestra.run.trial035.stdout: Verifying : openblas-openmp-0.3.29-1.el9.x86_64 38/96 2026-02-24T07:30:43.479 INFO:teuthology.orchestra.run.trial035.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 39/96 2026-02-24T07:30:43.479 INFO:teuthology.orchestra.run.trial035.stdout: Verifying : python3-devel-3.9.25-3.el9.x86_64 40/96 2026-02-24T07:30:43.479 INFO:teuthology.orchestra.run.trial035.stdout: Verifying : python3-jinja2-2.11.3-8.el9.noarch 41/96 2026-02-24T07:30:43.479 INFO:teuthology.orchestra.run.trial035.stdout: Verifying : python3-jmespath-1.0.1-1.el9.noarch 42/96 2026-02-24T07:30:43.479 INFO:teuthology.orchestra.run.trial035.stdout: Verifying : python3-libstoragemgmt-1.10.1-1.el9.x86_64 43/96 2026-02-24T07:30:43.479 INFO:teuthology.orchestra.run.trial035.stdout: Verifying : python3-lxml-4.6.5-3.el9.x86_64 44/96 2026-02-24T07:30:43.479 INFO:teuthology.orchestra.run.trial035.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 45/96 2026-02-24T07:30:43.479 INFO:teuthology.orchestra.run.trial035.stdout: Verifying : python3-numpy-1:1.23.5-2.el9.x86_64 46/96 2026-02-24T07:30:43.479 INFO:teuthology.orchestra.run.trial035.stdout: Verifying : python3-numpy-f2py-1:1.23.5-2.el9.x86_64 47/96 2026-02-24T07:30:43.479 INFO:teuthology.orchestra.run.trial035.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 48/96 2026-02-24T07:30:43.479 INFO:teuthology.orchestra.run.trial035.stdout: Verifying : python3-protobuf-3.14.0-17.el9.noarch 49/96 2026-02-24T07:30:43.480 INFO:teuthology.orchestra.run.trial035.stdout: Verifying : python3-pyasn1-0.4.8-7.el9.noarch 50/96 2026-02-24T07:30:43.480 INFO:teuthology.orchestra.run.trial035.stdout: Verifying : python3-pyasn1-modules-0.4.8-7.el9.noarch 51/96 2026-02-24T07:30:43.480 INFO:teuthology.orchestra.run.trial035.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 52/96 2026-02-24T07:30:43.480 INFO:teuthology.orchestra.run.trial035.stdout: Verifying : python3-scipy-1.9.3-2.el9.x86_64 53/96 2026-02-24T07:30:43.480 INFO:teuthology.orchestra.run.trial035.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 54/96 2026-02-24T07:30:43.480 INFO:teuthology.orchestra.run.trial035.stdout: Verifying : xmlsec1-1.2.29-13.el9.x86_64 55/96 2026-02-24T07:30:43.480 INFO:teuthology.orchestra.run.trial035.stdout: Verifying : xmlsec1-openssl-1.2.29-13.el9.x86_64 56/96 2026-02-24T07:30:43.480 INFO:teuthology.orchestra.run.trial035.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 57/96 2026-02-24T07:30:43.480 INFO:teuthology.orchestra.run.trial035.stdout: Verifying : protobuf-compiler-3.14.0-17.el9.x86_64 58/96 2026-02-24T07:30:43.480 INFO:teuthology.orchestra.run.trial035.stdout: Verifying : abseil-cpp-20211102.0-4.el9.x86_64 59/96 2026-02-24T07:30:43.480 INFO:teuthology.orchestra.run.trial035.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 60/96 2026-02-24T07:30:43.480 INFO:teuthology.orchestra.run.trial035.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 61/96 2026-02-24T07:30:43.480 INFO:teuthology.orchestra.run.trial035.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 62/96 2026-02-24T07:30:43.481 INFO:teuthology.orchestra.run.trial035.stdout: Verifying : python3-autocommand-2.2.2-8.el9.noarch 63/96 2026-02-24T07:30:43.481 INFO:teuthology.orchestra.run.trial035.stdout: Verifying : python3-backports-tarfile-1.2.0-1.el9.noarch 64/96 2026-02-24T07:30:43.481 INFO:teuthology.orchestra.run.trial035.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 65/96 2026-02-24T07:30:43.481 INFO:teuthology.orchestra.run.trial035.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 66/96 2026-02-24T07:30:43.481 INFO:teuthology.orchestra.run.trial035.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 67/96 2026-02-24T07:30:43.481 INFO:teuthology.orchestra.run.trial035.stdout: Verifying : python3-cheroot-10.0.1-4.el9.noarch 68/96 2026-02-24T07:30:43.481 INFO:teuthology.orchestra.run.trial035.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 69/96 2026-02-24T07:30:43.481 INFO:teuthology.orchestra.run.trial035.stdout: Verifying : python3-google-auth-1:2.45.0-1.el9.noarch 70/96 2026-02-24T07:30:43.481 INFO:teuthology.orchestra.run.trial035.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 71/96 2026-02-24T07:30:43.481 INFO:teuthology.orchestra.run.trial035.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 72/96 2026-02-24T07:30:43.481 INFO:teuthology.orchestra.run.trial035.stdout: Verifying : python3-influxdb-5.3.1-1.el9.noarch 73/96 2026-02-24T07:30:43.481 INFO:teuthology.orchestra.run.trial035.stdout: Verifying : python3-isodate-0.6.1-3.el9.noarch 74/96 2026-02-24T07:30:43.481 INFO:teuthology.orchestra.run.trial035.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 75/96 2026-02-24T07:30:43.482 INFO:teuthology.orchestra.run.trial035.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 76/96 2026-02-24T07:30:43.482 INFO:teuthology.orchestra.run.trial035.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 77/96 2026-02-24T07:30:43.482 INFO:teuthology.orchestra.run.trial035.stdout: Verifying : python3-jaraco-context-6.0.1-3.el9.noarch 78/96 2026-02-24T07:30:43.482 INFO:teuthology.orchestra.run.trial035.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 79/96 2026-02-24T07:30:43.482 INFO:teuthology.orchestra.run.trial035.stdout: Verifying : python3-jaraco-text-4.0.0-2.el9.noarch 80/96 2026-02-24T07:30:43.482 INFO:teuthology.orchestra.run.trial035.stdout: Verifying : python3-kubernetes-1:26.1.0-3.el9.noarch 81/96 2026-02-24T07:30:43.482 INFO:teuthology.orchestra.run.trial035.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 82/96 2026-02-24T07:30:43.482 INFO:teuthology.orchestra.run.trial035.stdout: Verifying : python3-msgpack-1.0.3-2.el9.x86_64 83/96 2026-02-24T07:30:43.482 INFO:teuthology.orchestra.run.trial035.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 84/96 2026-02-24T07:30:43.482 INFO:teuthology.orchestra.run.trial035.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 85/96 2026-02-24T07:30:43.482 INFO:teuthology.orchestra.run.trial035.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 86/96 2026-02-24T07:30:43.482 INFO:teuthology.orchestra.run.trial035.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 87/96 2026-02-24T07:30:43.482 INFO:teuthology.orchestra.run.trial035.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 88/96 2026-02-24T07:30:43.483 INFO:teuthology.orchestra.run.trial035.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 89/96 2026-02-24T07:30:43.483 INFO:teuthology.orchestra.run.trial035.stdout: Verifying : python3-saml-1.16.0-1.el9.noarch 90/96 2026-02-24T07:30:43.483 INFO:teuthology.orchestra.run.trial035.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 91/96 2026-02-24T07:30:43.483 INFO:teuthology.orchestra.run.trial035.stdout: Verifying : python3-typing-extensions-4.15.0-1.el9.noarch 92/96 2026-02-24T07:30:43.483 INFO:teuthology.orchestra.run.trial035.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 93/96 2026-02-24T07:30:43.483 INFO:teuthology.orchestra.run.trial035.stdout: Verifying : python3-xmlsec-1.3.13-1.el9.x86_64 94/96 2026-02-24T07:30:43.483 INFO:teuthology.orchestra.run.trial035.stdout: Verifying : python3-xmltodict-0.12.0-15.el9.noarch 95/96 2026-02-24T07:30:43.545 INFO:teuthology.orchestra.run.trial035.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 96/96 2026-02-24T07:30:43.545 INFO:teuthology.orchestra.run.trial035.stdout: 2026-02-24T07:30:43.545 INFO:teuthology.orchestra.run.trial035.stdout:Installed: 2026-02-24T07:30:43.545 INFO:teuthology.orchestra.run.trial035.stdout: abseil-cpp-20211102.0-4.el9.x86_64 2026-02-24T07:30:43.545 INFO:teuthology.orchestra.run.trial035.stdout: ceph-2:20.3.0-5440.gff63f3d9.el9.x86_64 2026-02-24T07:30:43.545 INFO:teuthology.orchestra.run.trial035.stdout: ceph-grafana-dashboards-2:20.3.0-5440.gff63f3d9.el9.noarch 2026-02-24T07:30:43.545 INFO:teuthology.orchestra.run.trial035.stdout: ceph-mds-2:20.3.0-5440.gff63f3d9.el9.x86_64 2026-02-24T07:30:43.545 INFO:teuthology.orchestra.run.trial035.stdout: ceph-mgr-2:20.3.0-5440.gff63f3d9.el9.x86_64 2026-02-24T07:30:43.546 INFO:teuthology.orchestra.run.trial035.stdout: ceph-mgr-cephadm-2:20.3.0-5440.gff63f3d9.el9.noarch 2026-02-24T07:30:43.546 INFO:teuthology.orchestra.run.trial035.stdout: ceph-mgr-dashboard-2:20.3.0-5440.gff63f3d9.el9.noarch 2026-02-24T07:30:43.546 INFO:teuthology.orchestra.run.trial035.stdout: ceph-mgr-diskprediction-local-2:20.3.0-5440.gff63f3d9.el9.noarch 2026-02-24T07:30:43.546 INFO:teuthology.orchestra.run.trial035.stdout: ceph-mgr-k8sevents-2:20.3.0-5440.gff63f3d9.el9.noarch 2026-02-24T07:30:43.546 INFO:teuthology.orchestra.run.trial035.stdout: ceph-mgr-modules-core-2:20.3.0-5440.gff63f3d9.el9.noarch 2026-02-24T07:30:43.546 INFO:teuthology.orchestra.run.trial035.stdout: ceph-mgr-rook-2:20.3.0-5440.gff63f3d9.el9.noarch 2026-02-24T07:30:43.546 INFO:teuthology.orchestra.run.trial035.stdout: ceph-mon-2:20.3.0-5440.gff63f3d9.el9.x86_64 2026-02-24T07:30:43.546 INFO:teuthology.orchestra.run.trial035.stdout: ceph-osd-2:20.3.0-5440.gff63f3d9.el9.x86_64 2026-02-24T07:30:43.546 INFO:teuthology.orchestra.run.trial035.stdout: ceph-osd-classic-2:20.3.0-5440.gff63f3d9.el9.x86_64 2026-02-24T07:30:43.546 INFO:teuthology.orchestra.run.trial035.stdout: ceph-osd-crimson-2:20.3.0-5440.gff63f3d9.el9.x86_64 2026-02-24T07:30:43.546 INFO:teuthology.orchestra.run.trial035.stdout: ceph-prometheus-alerts-2:20.3.0-5440.gff63f3d9.el9.noarch 2026-02-24T07:30:43.546 INFO:teuthology.orchestra.run.trial035.stdout: ceph-volume-2:20.3.0-5440.gff63f3d9.el9.noarch 2026-02-24T07:30:43.547 INFO:teuthology.orchestra.run.trial035.stdout: cephadm-2:20.3.0-5440.gff63f3d9.el9.noarch 2026-02-24T07:30:43.547 INFO:teuthology.orchestra.run.trial035.stdout: cryptsetup-2.8.1-3.el9.x86_64 2026-02-24T07:30:43.547 INFO:teuthology.orchestra.run.trial035.stdout: flexiblas-3.0.4-9.el9.x86_64 2026-02-24T07:30:43.547 INFO:teuthology.orchestra.run.trial035.stdout: flexiblas-netlib-3.0.4-9.el9.x86_64 2026-02-24T07:30:43.547 INFO:teuthology.orchestra.run.trial035.stdout: flexiblas-openblas-openmp-3.0.4-9.el9.x86_64 2026-02-24T07:30:43.547 INFO:teuthology.orchestra.run.trial035.stdout: grpc-data-1.46.7-10.el9.noarch 2026-02-24T07:30:43.547 INFO:teuthology.orchestra.run.trial035.stdout: ledmon-libs-1.1.0-3.el9.x86_64 2026-02-24T07:30:43.547 INFO:teuthology.orchestra.run.trial035.stdout: libconfig-1.7.2-9.el9.x86_64 2026-02-24T07:30:43.547 INFO:teuthology.orchestra.run.trial035.stdout: libgfortran-11.5.0-14.el9.x86_64 2026-02-24T07:30:43.547 INFO:teuthology.orchestra.run.trial035.stdout: libquadmath-11.5.0-14.el9.x86_64 2026-02-24T07:30:43.547 INFO:teuthology.orchestra.run.trial035.stdout: libstoragemgmt-1.10.1-1.el9.x86_64 2026-02-24T07:30:43.547 INFO:teuthology.orchestra.run.trial035.stdout: lua-5.4.4-4.el9.x86_64 2026-02-24T07:30:43.548 INFO:teuthology.orchestra.run.trial035.stdout: lua-devel-5.4.4-4.el9.x86_64 2026-02-24T07:30:43.548 INFO:teuthology.orchestra.run.trial035.stdout: luarocks-3.9.2-5.el9.noarch 2026-02-24T07:30:43.548 INFO:teuthology.orchestra.run.trial035.stdout: openblas-0.3.29-1.el9.x86_64 2026-02-24T07:30:43.548 INFO:teuthology.orchestra.run.trial035.stdout: openblas-openmp-0.3.29-1.el9.x86_64 2026-02-24T07:30:43.548 INFO:teuthology.orchestra.run.trial035.stdout: protobuf-compiler-3.14.0-17.el9.x86_64 2026-02-24T07:30:43.548 INFO:teuthology.orchestra.run.trial035.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2026-02-24T07:30:43.548 INFO:teuthology.orchestra.run.trial035.stdout: python3-autocommand-2.2.2-8.el9.noarch 2026-02-24T07:30:43.548 INFO:teuthology.orchestra.run.trial035.stdout: python3-babel-2.9.1-2.el9.noarch 2026-02-24T07:30:43.548 INFO:teuthology.orchestra.run.trial035.stdout: python3-backports-tarfile-1.2.0-1.el9.noarch 2026-02-24T07:30:43.548 INFO:teuthology.orchestra.run.trial035.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2026-02-24T07:30:43.548 INFO:teuthology.orchestra.run.trial035.stdout: python3-cachetools-4.2.4-1.el9.noarch 2026-02-24T07:30:43.548 INFO:teuthology.orchestra.run.trial035.stdout: python3-certifi-2023.05.07-4.el9.noarch 2026-02-24T07:30:43.549 INFO:teuthology.orchestra.run.trial035.stdout: python3-cffi-1.14.5-5.el9.x86_64 2026-02-24T07:30:43.549 INFO:teuthology.orchestra.run.trial035.stdout: python3-cheroot-10.0.1-4.el9.noarch 2026-02-24T07:30:43.549 INFO:teuthology.orchestra.run.trial035.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2026-02-24T07:30:43.549 INFO:teuthology.orchestra.run.trial035.stdout: python3-cryptography-36.0.1-5.el9.x86_64 2026-02-24T07:30:43.549 INFO:teuthology.orchestra.run.trial035.stdout: python3-devel-3.9.25-3.el9.x86_64 2026-02-24T07:30:43.549 INFO:teuthology.orchestra.run.trial035.stdout: python3-google-auth-1:2.45.0-1.el9.noarch 2026-02-24T07:30:43.549 INFO:teuthology.orchestra.run.trial035.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2026-02-24T07:30:43.549 INFO:teuthology.orchestra.run.trial035.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2026-02-24T07:30:43.549 INFO:teuthology.orchestra.run.trial035.stdout: python3-influxdb-5.3.1-1.el9.noarch 2026-02-24T07:30:43.549 INFO:teuthology.orchestra.run.trial035.stdout: python3-isodate-0.6.1-3.el9.noarch 2026-02-24T07:30:43.549 INFO:teuthology.orchestra.run.trial035.stdout: python3-jaraco-8.2.1-3.el9.noarch 2026-02-24T07:30:43.549 INFO:teuthology.orchestra.run.trial035.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2026-02-24T07:30:43.550 INFO:teuthology.orchestra.run.trial035.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2026-02-24T07:30:43.550 INFO:teuthology.orchestra.run.trial035.stdout: python3-jaraco-context-6.0.1-3.el9.noarch 2026-02-24T07:30:43.550 INFO:teuthology.orchestra.run.trial035.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2026-02-24T07:30:43.550 INFO:teuthology.orchestra.run.trial035.stdout: python3-jaraco-text-4.0.0-2.el9.noarch 2026-02-24T07:30:43.550 INFO:teuthology.orchestra.run.trial035.stdout: python3-jinja2-2.11.3-8.el9.noarch 2026-02-24T07:30:43.550 INFO:teuthology.orchestra.run.trial035.stdout: python3-jmespath-1.0.1-1.el9.noarch 2026-02-24T07:30:43.550 INFO:teuthology.orchestra.run.trial035.stdout: python3-kubernetes-1:26.1.0-3.el9.noarch 2026-02-24T07:30:43.550 INFO:teuthology.orchestra.run.trial035.stdout: python3-libstoragemgmt-1.10.1-1.el9.x86_64 2026-02-24T07:30:43.550 INFO:teuthology.orchestra.run.trial035.stdout: python3-lxml-4.6.5-3.el9.x86_64 2026-02-24T07:30:43.550 INFO:teuthology.orchestra.run.trial035.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2026-02-24T07:30:43.550 INFO:teuthology.orchestra.run.trial035.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2026-02-24T07:30:43.550 INFO:teuthology.orchestra.run.trial035.stdout: python3-msgpack-1.0.3-2.el9.x86_64 2026-02-24T07:30:43.550 INFO:teuthology.orchestra.run.trial035.stdout: python3-natsort-7.1.1-5.el9.noarch 2026-02-24T07:30:43.551 INFO:teuthology.orchestra.run.trial035.stdout: python3-numpy-1:1.23.5-2.el9.x86_64 2026-02-24T07:30:43.551 INFO:teuthology.orchestra.run.trial035.stdout: python3-numpy-f2py-1:1.23.5-2.el9.x86_64 2026-02-24T07:30:43.551 INFO:teuthology.orchestra.run.trial035.stdout: python3-packaging-20.9-5.el9.noarch 2026-02-24T07:30:43.551 INFO:teuthology.orchestra.run.trial035.stdout: python3-ply-3.11-14.el9.noarch 2026-02-24T07:30:43.551 INFO:teuthology.orchestra.run.trial035.stdout: python3-portend-3.1.0-2.el9.noarch 2026-02-24T07:30:43.551 INFO:teuthology.orchestra.run.trial035.stdout: python3-protobuf-3.14.0-17.el9.noarch 2026-02-24T07:30:43.551 INFO:teuthology.orchestra.run.trial035.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2026-02-24T07:30:43.551 INFO:teuthology.orchestra.run.trial035.stdout: python3-pyasn1-0.4.8-7.el9.noarch 2026-02-24T07:30:43.551 INFO:teuthology.orchestra.run.trial035.stdout: python3-pyasn1-modules-0.4.8-7.el9.noarch 2026-02-24T07:30:43.551 INFO:teuthology.orchestra.run.trial035.stdout: python3-pycparser-2.20-6.el9.noarch 2026-02-24T07:30:43.551 INFO:teuthology.orchestra.run.trial035.stdout: python3-pyparsing-2.4.7-9.el9.noarch 2026-02-24T07:30:43.551 INFO:teuthology.orchestra.run.trial035.stdout: python3-repoze-lru-0.7-16.el9.noarch 2026-02-24T07:30:43.552 INFO:teuthology.orchestra.run.trial035.stdout: python3-requests-2.25.1-10.el9.noarch 2026-02-24T07:30:43.552 INFO:teuthology.orchestra.run.trial035.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2026-02-24T07:30:43.552 INFO:teuthology.orchestra.run.trial035.stdout: python3-routes-2.5.1-5.el9.noarch 2026-02-24T07:30:43.552 INFO:teuthology.orchestra.run.trial035.stdout: python3-rsa-4.9-2.el9.noarch 2026-02-24T07:30:43.552 INFO:teuthology.orchestra.run.trial035.stdout: python3-saml-1.16.0-1.el9.noarch 2026-02-24T07:30:43.552 INFO:teuthology.orchestra.run.trial035.stdout: python3-scipy-1.9.3-2.el9.x86_64 2026-02-24T07:30:43.552 INFO:teuthology.orchestra.run.trial035.stdout: python3-tempora-5.0.0-2.el9.noarch 2026-02-24T07:30:43.552 INFO:teuthology.orchestra.run.trial035.stdout: python3-toml-0.10.2-6.el9.noarch 2026-02-24T07:30:43.552 INFO:teuthology.orchestra.run.trial035.stdout: python3-typing-extensions-4.15.0-1.el9.noarch 2026-02-24T07:30:43.552 INFO:teuthology.orchestra.run.trial035.stdout: python3-urllib3-1.26.5-7.el9.noarch 2026-02-24T07:30:43.552 INFO:teuthology.orchestra.run.trial035.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2026-02-24T07:30:43.552 INFO:teuthology.orchestra.run.trial035.stdout: python3-xmlsec-1.3.13-1.el9.x86_64 2026-02-24T07:30:43.552 INFO:teuthology.orchestra.run.trial035.stdout: python3-xmltodict-0.12.0-15.el9.noarch 2026-02-24T07:30:43.553 INFO:teuthology.orchestra.run.trial035.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2026-02-24T07:30:43.553 INFO:teuthology.orchestra.run.trial035.stdout: unzip-6.0-59.el9.x86_64 2026-02-24T07:30:43.553 INFO:teuthology.orchestra.run.trial035.stdout: xmlsec1-1.2.29-13.el9.x86_64 2026-02-24T07:30:43.553 INFO:teuthology.orchestra.run.trial035.stdout: xmlsec1-openssl-1.2.29-13.el9.x86_64 2026-02-24T07:30:43.553 INFO:teuthology.orchestra.run.trial035.stdout: zip-3.0-35.el9.x86_64 2026-02-24T07:30:43.553 INFO:teuthology.orchestra.run.trial035.stdout: 2026-02-24T07:30:43.553 INFO:teuthology.orchestra.run.trial035.stdout:Complete! 2026-02-24T07:30:43.613 DEBUG:teuthology.orchestra.run.trial035:> sudo yum -y install ceph-base 2026-02-24T07:30:43.929 INFO:teuthology.orchestra.run.trial035.stdout:Last metadata expiration check: 0:01:22 ago on Tue 24 Feb 2026 07:29:21 AM UTC. 2026-02-24T07:30:43.994 INFO:teuthology.orchestra.run.trial035.stdout:Package ceph-base-2:20.3.0-5440.gff63f3d9.el9.x86_64 is already installed. 2026-02-24T07:30:44.012 INFO:teuthology.orchestra.run.trial035.stdout:Dependencies resolved. 2026-02-24T07:30:44.013 INFO:teuthology.orchestra.run.trial035.stdout:Nothing to do. 2026-02-24T07:30:44.013 INFO:teuthology.orchestra.run.trial035.stdout:Complete! 2026-02-24T07:30:44.034 DEBUG:teuthology.orchestra.run.trial035:> sudo yum -y install cephadm 2026-02-24T07:30:44.350 INFO:teuthology.orchestra.run.trial035.stdout:Last metadata expiration check: 0:01:23 ago on Tue 24 Feb 2026 07:29:21 AM UTC. 2026-02-24T07:30:44.415 INFO:teuthology.orchestra.run.trial035.stdout:Package cephadm-2:20.3.0-5440.gff63f3d9.el9.noarch is already installed. 2026-02-24T07:30:44.434 INFO:teuthology.orchestra.run.trial035.stdout:Dependencies resolved. 2026-02-24T07:30:44.435 INFO:teuthology.orchestra.run.trial035.stdout:Nothing to do. 2026-02-24T07:30:44.435 INFO:teuthology.orchestra.run.trial035.stdout:Complete! 2026-02-24T07:30:44.456 DEBUG:teuthology.orchestra.run.trial035:> sudo yum -y install ceph-immutable-object-cache 2026-02-24T07:30:44.776 INFO:teuthology.orchestra.run.trial035.stdout:Last metadata expiration check: 0:01:23 ago on Tue 24 Feb 2026 07:29:21 AM UTC. 2026-02-24T07:30:44.861 INFO:teuthology.orchestra.run.trial035.stdout:Dependencies resolved. 2026-02-24T07:30:44.861 INFO:teuthology.orchestra.run.trial035.stdout:================================================================================ 2026-02-24T07:30:44.861 INFO:teuthology.orchestra.run.trial035.stdout: Package Arch Version Repo Size 2026-02-24T07:30:44.861 INFO:teuthology.orchestra.run.trial035.stdout:================================================================================ 2026-02-24T07:30:44.861 INFO:teuthology.orchestra.run.trial035.stdout:Installing: 2026-02-24T07:30:44.861 INFO:teuthology.orchestra.run.trial035.stdout: ceph-immutable-object-cache x86_64 2:20.3.0-5440.gff63f3d9.el9 ceph 149 k 2026-02-24T07:30:44.861 INFO:teuthology.orchestra.run.trial035.stdout: 2026-02-24T07:30:44.861 INFO:teuthology.orchestra.run.trial035.stdout:Transaction Summary 2026-02-24T07:30:44.861 INFO:teuthology.orchestra.run.trial035.stdout:================================================================================ 2026-02-24T07:30:44.861 INFO:teuthology.orchestra.run.trial035.stdout:Install 1 Package 2026-02-24T07:30:44.861 INFO:teuthology.orchestra.run.trial035.stdout: 2026-02-24T07:30:44.861 INFO:teuthology.orchestra.run.trial035.stdout:Total download size: 149 k 2026-02-24T07:30:44.861 INFO:teuthology.orchestra.run.trial035.stdout:Installed size: 455 k 2026-02-24T07:30:44.861 INFO:teuthology.orchestra.run.trial035.stdout:Downloading Packages: 2026-02-24T07:30:45.050 INFO:teuthology.orchestra.run.trial035.stdout:ceph-immutable-object-cache-20.3.0-5440.gff63f3 792 kB/s | 149 kB 00:00 2026-02-24T07:30:45.051 INFO:teuthology.orchestra.run.trial035.stdout:-------------------------------------------------------------------------------- 2026-02-24T07:30:45.051 INFO:teuthology.orchestra.run.trial035.stdout:Total 787 kB/s | 149 kB 00:00 2026-02-24T07:30:45.051 INFO:teuthology.orchestra.run.trial035.stdout:Running transaction check 2026-02-24T07:30:45.056 INFO:teuthology.orchestra.run.trial035.stdout:Transaction check succeeded. 2026-02-24T07:30:45.056 INFO:teuthology.orchestra.run.trial035.stdout:Running transaction test 2026-02-24T07:30:45.087 INFO:teuthology.orchestra.run.trial035.stdout:Transaction test succeeded. 2026-02-24T07:30:45.087 INFO:teuthology.orchestra.run.trial035.stdout:Running transaction 2026-02-24T07:30:45.144 INFO:teuthology.orchestra.run.trial035.stdout: Preparing : 1/1 2026-02-24T07:30:45.146 INFO:teuthology.orchestra.run.trial035.stdout: Installing : ceph-immutable-object-cache-2:20.3.0-5440.gff63f3d9. 1/1 2026-02-24T07:30:45.169 INFO:teuthology.orchestra.run.trial035.stdout: Running scriptlet: ceph-immutable-object-cache-2:20.3.0-5440.gff63f3d9. 1/1 2026-02-24T07:30:45.169 INFO:teuthology.orchestra.run.trial035.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-24T07:30:45.169 INFO:teuthology.orchestra.run.trial035.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2026-02-24T07:30:45.169 INFO:teuthology.orchestra.run.trial035.stdout: 2026-02-24T07:30:45.487 INFO:teuthology.orchestra.run.trial035.stdout: Verifying : ceph-immutable-object-cache-2:20.3.0-5440.gff63f3d9. 1/1 2026-02-24T07:30:45.487 INFO:teuthology.orchestra.run.trial035.stdout: 2026-02-24T07:30:45.487 INFO:teuthology.orchestra.run.trial035.stdout:Installed: 2026-02-24T07:30:45.487 INFO:teuthology.orchestra.run.trial035.stdout: ceph-immutable-object-cache-2:20.3.0-5440.gff63f3d9.el9.x86_64 2026-02-24T07:30:45.487 INFO:teuthology.orchestra.run.trial035.stdout: 2026-02-24T07:30:45.487 INFO:teuthology.orchestra.run.trial035.stdout:Complete! 2026-02-24T07:30:45.514 DEBUG:teuthology.orchestra.run.trial035:> sudo yum -y install ceph-mgr 2026-02-24T07:30:45.835 INFO:teuthology.orchestra.run.trial035.stdout:Last metadata expiration check: 0:01:24 ago on Tue 24 Feb 2026 07:29:21 AM UTC. 2026-02-24T07:30:45.901 INFO:teuthology.orchestra.run.trial035.stdout:Package ceph-mgr-2:20.3.0-5440.gff63f3d9.el9.x86_64 is already installed. 2026-02-24T07:30:45.920 INFO:teuthology.orchestra.run.trial035.stdout:Dependencies resolved. 2026-02-24T07:30:45.920 INFO:teuthology.orchestra.run.trial035.stdout:Nothing to do. 2026-02-24T07:30:45.920 INFO:teuthology.orchestra.run.trial035.stdout:Complete! 2026-02-24T07:30:45.942 DEBUG:teuthology.orchestra.run.trial035:> sudo yum -y install ceph-mgr-dashboard 2026-02-24T07:30:46.258 INFO:teuthology.orchestra.run.trial035.stdout:Last metadata expiration check: 0:01:25 ago on Tue 24 Feb 2026 07:29:21 AM UTC. 2026-02-24T07:30:46.324 INFO:teuthology.orchestra.run.trial035.stdout:Package ceph-mgr-dashboard-2:20.3.0-5440.gff63f3d9.el9.noarch is already installed. 2026-02-24T07:30:46.342 INFO:teuthology.orchestra.run.trial035.stdout:Dependencies resolved. 2026-02-24T07:30:46.343 INFO:teuthology.orchestra.run.trial035.stdout:Nothing to do. 2026-02-24T07:30:46.343 INFO:teuthology.orchestra.run.trial035.stdout:Complete! 2026-02-24T07:30:46.365 DEBUG:teuthology.orchestra.run.trial035:> sudo yum -y install ceph-mgr-diskprediction-local 2026-02-24T07:30:46.681 INFO:teuthology.orchestra.run.trial035.stdout:Last metadata expiration check: 0:01:25 ago on Tue 24 Feb 2026 07:29:21 AM UTC. 2026-02-24T07:30:46.745 INFO:teuthology.orchestra.run.trial035.stdout:Package ceph-mgr-diskprediction-local-2:20.3.0-5440.gff63f3d9.el9.noarch is already installed. 2026-02-24T07:30:46.764 INFO:teuthology.orchestra.run.trial035.stdout:Dependencies resolved. 2026-02-24T07:30:46.765 INFO:teuthology.orchestra.run.trial035.stdout:Nothing to do. 2026-02-24T07:30:46.765 INFO:teuthology.orchestra.run.trial035.stdout:Complete! 2026-02-24T07:30:46.786 DEBUG:teuthology.orchestra.run.trial035:> sudo yum -y install ceph-mgr-rook 2026-02-24T07:30:47.098 INFO:teuthology.orchestra.run.trial035.stdout:Last metadata expiration check: 0:01:26 ago on Tue 24 Feb 2026 07:29:21 AM UTC. 2026-02-24T07:30:47.165 INFO:teuthology.orchestra.run.trial035.stdout:Package ceph-mgr-rook-2:20.3.0-5440.gff63f3d9.el9.noarch is already installed. 2026-02-24T07:30:47.184 INFO:teuthology.orchestra.run.trial035.stdout:Dependencies resolved. 2026-02-24T07:30:47.184 INFO:teuthology.orchestra.run.trial035.stdout:Nothing to do. 2026-02-24T07:30:47.184 INFO:teuthology.orchestra.run.trial035.stdout:Complete! 2026-02-24T07:30:47.206 DEBUG:teuthology.orchestra.run.trial035:> sudo yum -y install ceph-mgr-cephadm 2026-02-24T07:30:47.523 INFO:teuthology.orchestra.run.trial035.stdout:Last metadata expiration check: 0:01:26 ago on Tue 24 Feb 2026 07:29:21 AM UTC. 2026-02-24T07:30:47.589 INFO:teuthology.orchestra.run.trial035.stdout:Package ceph-mgr-cephadm-2:20.3.0-5440.gff63f3d9.el9.noarch is already installed. 2026-02-24T07:30:47.607 INFO:teuthology.orchestra.run.trial035.stdout:Dependencies resolved. 2026-02-24T07:30:47.608 INFO:teuthology.orchestra.run.trial035.stdout:Nothing to do. 2026-02-24T07:30:47.608 INFO:teuthology.orchestra.run.trial035.stdout:Complete! 2026-02-24T07:30:47.630 DEBUG:teuthology.orchestra.run.trial035:> sudo yum -y install ceph-osd 2026-02-24T07:30:47.941 INFO:teuthology.orchestra.run.trial035.stdout:Last metadata expiration check: 0:01:26 ago on Tue 24 Feb 2026 07:29:21 AM UTC. 2026-02-24T07:30:48.006 INFO:teuthology.orchestra.run.trial035.stdout:Package ceph-osd-2:20.3.0-5440.gff63f3d9.el9.x86_64 is already installed. 2026-02-24T07:30:48.025 INFO:teuthology.orchestra.run.trial035.stdout:Dependencies resolved. 2026-02-24T07:30:48.025 INFO:teuthology.orchestra.run.trial035.stdout:Nothing to do. 2026-02-24T07:30:48.026 INFO:teuthology.orchestra.run.trial035.stdout:Complete! 2026-02-24T07:30:48.047 DEBUG:teuthology.orchestra.run.trial035:> sudo yum -y install ceph-osd-classic 2026-02-24T07:30:48.365 INFO:teuthology.orchestra.run.trial035.stdout:Last metadata expiration check: 0:01:27 ago on Tue 24 Feb 2026 07:29:21 AM UTC. 2026-02-24T07:30:48.432 INFO:teuthology.orchestra.run.trial035.stdout:Package ceph-osd-classic-2:20.3.0-5440.gff63f3d9.el9.x86_64 is already installed. 2026-02-24T07:30:48.451 INFO:teuthology.orchestra.run.trial035.stdout:Dependencies resolved. 2026-02-24T07:30:48.451 INFO:teuthology.orchestra.run.trial035.stdout:Nothing to do. 2026-02-24T07:30:48.451 INFO:teuthology.orchestra.run.trial035.stdout:Complete! 2026-02-24T07:30:48.473 DEBUG:teuthology.orchestra.run.trial035:> sudo yum -y install ceph-fuse 2026-02-24T07:30:48.788 INFO:teuthology.orchestra.run.trial035.stdout:Last metadata expiration check: 0:01:27 ago on Tue 24 Feb 2026 07:29:21 AM UTC. 2026-02-24T07:30:48.873 INFO:teuthology.orchestra.run.trial035.stdout:Dependencies resolved. 2026-02-24T07:30:48.873 INFO:teuthology.orchestra.run.trial035.stdout:================================================================================ 2026-02-24T07:30:48.873 INFO:teuthology.orchestra.run.trial035.stdout: Package Arch Version Repo Size 2026-02-24T07:30:48.873 INFO:teuthology.orchestra.run.trial035.stdout:================================================================================ 2026-02-24T07:30:48.873 INFO:teuthology.orchestra.run.trial035.stdout:Installing: 2026-02-24T07:30:48.874 INFO:teuthology.orchestra.run.trial035.stdout: ceph-fuse x86_64 2:20.3.0-5440.gff63f3d9.el9 ceph 914 k 2026-02-24T07:30:48.874 INFO:teuthology.orchestra.run.trial035.stdout: 2026-02-24T07:30:48.874 INFO:teuthology.orchestra.run.trial035.stdout:Transaction Summary 2026-02-24T07:30:48.874 INFO:teuthology.orchestra.run.trial035.stdout:================================================================================ 2026-02-24T07:30:48.874 INFO:teuthology.orchestra.run.trial035.stdout:Install 1 Package 2026-02-24T07:30:48.874 INFO:teuthology.orchestra.run.trial035.stdout: 2026-02-24T07:30:48.874 INFO:teuthology.orchestra.run.trial035.stdout:Total download size: 914 k 2026-02-24T07:30:48.874 INFO:teuthology.orchestra.run.trial035.stdout:Installed size: 2.9 M 2026-02-24T07:30:48.874 INFO:teuthology.orchestra.run.trial035.stdout:Downloading Packages: 2026-02-24T07:30:48.996 INFO:teuthology.orchestra.run.trial035.stdout:ceph-fuse-20.3.0-5440.gff63f3d9.el9.x86_64.rpm 7.3 MB/s | 914 kB 00:00 2026-02-24T07:30:48.997 INFO:teuthology.orchestra.run.trial035.stdout:-------------------------------------------------------------------------------- 2026-02-24T07:30:48.997 INFO:teuthology.orchestra.run.trial035.stdout:Total 7.2 MB/s | 914 kB 00:00 2026-02-24T07:30:48.997 INFO:teuthology.orchestra.run.trial035.stdout:Running transaction check 2026-02-24T07:30:49.001 INFO:teuthology.orchestra.run.trial035.stdout:Transaction check succeeded. 2026-02-24T07:30:49.002 INFO:teuthology.orchestra.run.trial035.stdout:Running transaction test 2026-02-24T07:30:49.034 INFO:teuthology.orchestra.run.trial035.stdout:Transaction test succeeded. 2026-02-24T07:30:49.035 INFO:teuthology.orchestra.run.trial035.stdout:Running transaction 2026-02-24T07:30:49.126 INFO:teuthology.orchestra.run.trial035.stdout: Preparing : 1/1 2026-02-24T07:30:49.145 INFO:teuthology.orchestra.run.trial035.stdout: Installing : ceph-fuse-2:20.3.0-5440.gff63f3d9.el9.x86_64 1/1 2026-02-24T07:30:49.398 INFO:teuthology.orchestra.run.trial035.stdout: Running scriptlet: ceph-fuse-2:20.3.0-5440.gff63f3d9.el9.x86_64 1/1 2026-02-24T07:30:49.460 INFO:teuthology.orchestra.run.trial035.stdout: Verifying : ceph-fuse-2:20.3.0-5440.gff63f3d9.el9.x86_64 1/1 2026-02-24T07:30:49.460 INFO:teuthology.orchestra.run.trial035.stdout: 2026-02-24T07:30:49.460 INFO:teuthology.orchestra.run.trial035.stdout:Installed: 2026-02-24T07:30:49.460 INFO:teuthology.orchestra.run.trial035.stdout: ceph-fuse-2:20.3.0-5440.gff63f3d9.el9.x86_64 2026-02-24T07:30:49.460 INFO:teuthology.orchestra.run.trial035.stdout: 2026-02-24T07:30:49.460 INFO:teuthology.orchestra.run.trial035.stdout:Complete! 2026-02-24T07:30:49.488 DEBUG:teuthology.orchestra.run.trial035:> sudo yum -y install ceph-volume 2026-02-24T07:30:49.813 INFO:teuthology.orchestra.run.trial035.stdout:Last metadata expiration check: 0:01:28 ago on Tue 24 Feb 2026 07:29:21 AM UTC. 2026-02-24T07:30:49.879 INFO:teuthology.orchestra.run.trial035.stdout:Package ceph-volume-2:20.3.0-5440.gff63f3d9.el9.noarch is already installed. 2026-02-24T07:30:49.898 INFO:teuthology.orchestra.run.trial035.stdout:Dependencies resolved. 2026-02-24T07:30:49.898 INFO:teuthology.orchestra.run.trial035.stdout:Nothing to do. 2026-02-24T07:30:49.899 INFO:teuthology.orchestra.run.trial035.stdout:Complete! 2026-02-24T07:30:49.920 DEBUG:teuthology.orchestra.run.trial035:> sudo yum -y install librados-devel 2026-02-24T07:30:50.242 INFO:teuthology.orchestra.run.trial035.stdout:Last metadata expiration check: 0:01:29 ago on Tue 24 Feb 2026 07:29:21 AM UTC. 2026-02-24T07:30:50.327 INFO:teuthology.orchestra.run.trial035.stdout:Dependencies resolved. 2026-02-24T07:30:50.327 INFO:teuthology.orchestra.run.trial035.stdout:================================================================================ 2026-02-24T07:30:50.327 INFO:teuthology.orchestra.run.trial035.stdout: Package Arch Version Repo Size 2026-02-24T07:30:50.327 INFO:teuthology.orchestra.run.trial035.stdout:================================================================================ 2026-02-24T07:30:50.328 INFO:teuthology.orchestra.run.trial035.stdout:Installing: 2026-02-24T07:30:50.328 INFO:teuthology.orchestra.run.trial035.stdout: librados-devel x86_64 2:20.3.0-5440.gff63f3d9.el9 ceph 124 k 2026-02-24T07:30:50.328 INFO:teuthology.orchestra.run.trial035.stdout: 2026-02-24T07:30:50.328 INFO:teuthology.orchestra.run.trial035.stdout:Transaction Summary 2026-02-24T07:30:50.328 INFO:teuthology.orchestra.run.trial035.stdout:================================================================================ 2026-02-24T07:30:50.328 INFO:teuthology.orchestra.run.trial035.stdout:Install 1 Package 2026-02-24T07:30:50.328 INFO:teuthology.orchestra.run.trial035.stdout: 2026-02-24T07:30:50.328 INFO:teuthology.orchestra.run.trial035.stdout:Total download size: 124 k 2026-02-24T07:30:50.328 INFO:teuthology.orchestra.run.trial035.stdout:Installed size: 434 k 2026-02-24T07:30:50.328 INFO:teuthology.orchestra.run.trial035.stdout:Downloading Packages: 2026-02-24T07:30:50.414 INFO:teuthology.orchestra.run.trial035.stdout:librados-devel-20.3.0-5440.gff63f3d9.el9.x86_64 1.4 MB/s | 124 kB 00:00 2026-02-24T07:30:50.414 INFO:teuthology.orchestra.run.trial035.stdout:-------------------------------------------------------------------------------- 2026-02-24T07:30:50.414 INFO:teuthology.orchestra.run.trial035.stdout:Total 1.4 MB/s | 124 kB 00:00 2026-02-24T07:30:50.414 INFO:teuthology.orchestra.run.trial035.stdout:Running transaction check 2026-02-24T07:30:50.417 INFO:teuthology.orchestra.run.trial035.stdout:Transaction check succeeded. 2026-02-24T07:30:50.417 INFO:teuthology.orchestra.run.trial035.stdout:Running transaction test 2026-02-24T07:30:50.449 INFO:teuthology.orchestra.run.trial035.stdout:Transaction test succeeded. 2026-02-24T07:30:50.449 INFO:teuthology.orchestra.run.trial035.stdout:Running transaction 2026-02-24T07:30:50.504 INFO:teuthology.orchestra.run.trial035.stdout: Preparing : 1/1 2026-02-24T07:30:50.522 INFO:teuthology.orchestra.run.trial035.stdout: Installing : librados-devel-2:20.3.0-5440.gff63f3d9.el9.x86_64 1/1 2026-02-24T07:30:50.606 INFO:teuthology.orchestra.run.trial035.stdout: Running scriptlet: librados-devel-2:20.3.0-5440.gff63f3d9.el9.x86_64 1/1 2026-02-24T07:30:50.668 INFO:teuthology.orchestra.run.trial035.stdout: Verifying : librados-devel-2:20.3.0-5440.gff63f3d9.el9.x86_64 1/1 2026-02-24T07:30:50.668 INFO:teuthology.orchestra.run.trial035.stdout: 2026-02-24T07:30:50.668 INFO:teuthology.orchestra.run.trial035.stdout:Installed: 2026-02-24T07:30:50.668 INFO:teuthology.orchestra.run.trial035.stdout: librados-devel-2:20.3.0-5440.gff63f3d9.el9.x86_64 2026-02-24T07:30:50.668 INFO:teuthology.orchestra.run.trial035.stdout: 2026-02-24T07:30:50.668 INFO:teuthology.orchestra.run.trial035.stdout:Complete! 2026-02-24T07:30:50.695 DEBUG:teuthology.orchestra.run.trial035:> sudo yum -y install libcephfs2 2026-02-24T07:30:51.017 INFO:teuthology.orchestra.run.trial035.stdout:Last metadata expiration check: 0:01:30 ago on Tue 24 Feb 2026 07:29:21 AM UTC. 2026-02-24T07:30:51.083 INFO:teuthology.orchestra.run.trial035.stdout:Package libcephfs2-2:20.3.0-5440.gff63f3d9.el9.x86_64 is already installed. 2026-02-24T07:30:51.102 INFO:teuthology.orchestra.run.trial035.stdout:Dependencies resolved. 2026-02-24T07:30:51.103 INFO:teuthology.orchestra.run.trial035.stdout:Nothing to do. 2026-02-24T07:30:51.103 INFO:teuthology.orchestra.run.trial035.stdout:Complete! 2026-02-24T07:30:51.124 DEBUG:teuthology.orchestra.run.trial035:> sudo yum -y install libcephfs-devel 2026-02-24T07:30:51.441 INFO:teuthology.orchestra.run.trial035.stdout:Last metadata expiration check: 0:01:30 ago on Tue 24 Feb 2026 07:29:21 AM UTC. 2026-02-24T07:30:51.527 INFO:teuthology.orchestra.run.trial035.stdout:Dependencies resolved. 2026-02-24T07:30:51.528 INFO:teuthology.orchestra.run.trial035.stdout:================================================================================ 2026-02-24T07:30:51.528 INFO:teuthology.orchestra.run.trial035.stdout: Package Arch Version Repo Size 2026-02-24T07:30:51.528 INFO:teuthology.orchestra.run.trial035.stdout:================================================================================ 2026-02-24T07:30:51.528 INFO:teuthology.orchestra.run.trial035.stdout:Installing: 2026-02-24T07:30:51.528 INFO:teuthology.orchestra.run.trial035.stdout: libcephfs-devel x86_64 2:20.3.0-5440.gff63f3d9.el9 ceph 37 k 2026-02-24T07:30:51.528 INFO:teuthology.orchestra.run.trial035.stdout: 2026-02-24T07:30:51.529 INFO:teuthology.orchestra.run.trial035.stdout:Transaction Summary 2026-02-24T07:30:51.529 INFO:teuthology.orchestra.run.trial035.stdout:================================================================================ 2026-02-24T07:30:51.529 INFO:teuthology.orchestra.run.trial035.stdout:Install 1 Package 2026-02-24T07:30:51.529 INFO:teuthology.orchestra.run.trial035.stdout: 2026-02-24T07:30:51.529 INFO:teuthology.orchestra.run.trial035.stdout:Total download size: 37 k 2026-02-24T07:30:51.529 INFO:teuthology.orchestra.run.trial035.stdout:Installed size: 171 k 2026-02-24T07:30:51.529 INFO:teuthology.orchestra.run.trial035.stdout:Downloading Packages: 2026-02-24T07:30:51.591 INFO:teuthology.orchestra.run.trial035.stdout:libcephfs-devel-20.3.0-5440.gff63f3d9.el9.x86_6 592 kB/s | 37 kB 00:00 2026-02-24T07:30:51.592 INFO:teuthology.orchestra.run.trial035.stdout:-------------------------------------------------------------------------------- 2026-02-24T07:30:51.592 INFO:teuthology.orchestra.run.trial035.stdout:Total 585 kB/s | 37 kB 00:00 2026-02-24T07:30:51.592 INFO:teuthology.orchestra.run.trial035.stdout:Running transaction check 2026-02-24T07:30:51.593 INFO:teuthology.orchestra.run.trial035.stdout:Transaction check succeeded. 2026-02-24T07:30:51.594 INFO:teuthology.orchestra.run.trial035.stdout:Running transaction test 2026-02-24T07:30:51.596 INFO:teuthology.orchestra.run.trial035.stdout:Transaction test succeeded. 2026-02-24T07:30:51.596 INFO:teuthology.orchestra.run.trial035.stdout:Running transaction 2026-02-24T07:30:51.618 INFO:teuthology.orchestra.run.trial035.stdout: Preparing : 1/1 2026-02-24T07:30:51.635 INFO:teuthology.orchestra.run.trial035.stdout: Installing : libcephfs-devel-2:20.3.0-5440.gff63f3d9.el9.x86_64 1/1 2026-02-24T07:30:51.696 INFO:teuthology.orchestra.run.trial035.stdout: Running scriptlet: libcephfs-devel-2:20.3.0-5440.gff63f3d9.el9.x86_64 1/1 2026-02-24T07:30:51.758 INFO:teuthology.orchestra.run.trial035.stdout: Verifying : libcephfs-devel-2:20.3.0-5440.gff63f3d9.el9.x86_64 1/1 2026-02-24T07:30:51.758 INFO:teuthology.orchestra.run.trial035.stdout: 2026-02-24T07:30:51.758 INFO:teuthology.orchestra.run.trial035.stdout:Installed: 2026-02-24T07:30:51.758 INFO:teuthology.orchestra.run.trial035.stdout: libcephfs-devel-2:20.3.0-5440.gff63f3d9.el9.x86_64 2026-02-24T07:30:51.759 INFO:teuthology.orchestra.run.trial035.stdout: 2026-02-24T07:30:51.759 INFO:teuthology.orchestra.run.trial035.stdout:Complete! 2026-02-24T07:30:51.785 DEBUG:teuthology.orchestra.run.trial035:> sudo yum -y install librados2 2026-02-24T07:30:52.102 INFO:teuthology.orchestra.run.trial035.stdout:Last metadata expiration check: 0:01:31 ago on Tue 24 Feb 2026 07:29:21 AM UTC. 2026-02-24T07:30:52.168 INFO:teuthology.orchestra.run.trial035.stdout:Package librados2-2:20.3.0-5440.gff63f3d9.el9.x86_64 is already installed. 2026-02-24T07:30:52.185 INFO:teuthology.orchestra.run.trial035.stdout:Dependencies resolved. 2026-02-24T07:30:52.186 INFO:teuthology.orchestra.run.trial035.stdout:Nothing to do. 2026-02-24T07:30:52.186 INFO:teuthology.orchestra.run.trial035.stdout:Complete! 2026-02-24T07:30:52.208 DEBUG:teuthology.orchestra.run.trial035:> sudo yum -y install librbd1 2026-02-24T07:30:52.521 INFO:teuthology.orchestra.run.trial035.stdout:Last metadata expiration check: 0:01:31 ago on Tue 24 Feb 2026 07:29:21 AM UTC. 2026-02-24T07:30:52.586 INFO:teuthology.orchestra.run.trial035.stdout:Package librbd1-2:20.3.0-5440.gff63f3d9.el9.x86_64 is already installed. 2026-02-24T07:30:52.604 INFO:teuthology.orchestra.run.trial035.stdout:Dependencies resolved. 2026-02-24T07:30:52.604 INFO:teuthology.orchestra.run.trial035.stdout:Nothing to do. 2026-02-24T07:30:52.605 INFO:teuthology.orchestra.run.trial035.stdout:Complete! 2026-02-24T07:30:52.626 DEBUG:teuthology.orchestra.run.trial035:> sudo yum -y install python3-rados 2026-02-24T07:30:52.940 INFO:teuthology.orchestra.run.trial035.stdout:Last metadata expiration check: 0:01:31 ago on Tue 24 Feb 2026 07:29:21 AM UTC. 2026-02-24T07:30:53.008 INFO:teuthology.orchestra.run.trial035.stdout:Package python3-rados-2:20.3.0-5440.gff63f3d9.el9.x86_64 is already installed. 2026-02-24T07:30:53.026 INFO:teuthology.orchestra.run.trial035.stdout:Dependencies resolved. 2026-02-24T07:30:53.026 INFO:teuthology.orchestra.run.trial035.stdout:Nothing to do. 2026-02-24T07:30:53.027 INFO:teuthology.orchestra.run.trial035.stdout:Complete! 2026-02-24T07:30:53.048 DEBUG:teuthology.orchestra.run.trial035:> sudo yum -y install python3-rgw 2026-02-24T07:30:53.371 INFO:teuthology.orchestra.run.trial035.stdout:Last metadata expiration check: 0:01:32 ago on Tue 24 Feb 2026 07:29:21 AM UTC. 2026-02-24T07:30:53.437 INFO:teuthology.orchestra.run.trial035.stdout:Package python3-rgw-2:20.3.0-5440.gff63f3d9.el9.x86_64 is already installed. 2026-02-24T07:30:53.454 INFO:teuthology.orchestra.run.trial035.stdout:Dependencies resolved. 2026-02-24T07:30:53.455 INFO:teuthology.orchestra.run.trial035.stdout:Nothing to do. 2026-02-24T07:30:53.455 INFO:teuthology.orchestra.run.trial035.stdout:Complete! 2026-02-24T07:30:53.477 DEBUG:teuthology.orchestra.run.trial035:> sudo yum -y install python3-cephfs 2026-02-24T07:30:53.797 INFO:teuthology.orchestra.run.trial035.stdout:Last metadata expiration check: 0:01:32 ago on Tue 24 Feb 2026 07:29:21 AM UTC. 2026-02-24T07:30:53.864 INFO:teuthology.orchestra.run.trial035.stdout:Package python3-cephfs-2:20.3.0-5440.gff63f3d9.el9.x86_64 is already installed. 2026-02-24T07:30:53.881 INFO:teuthology.orchestra.run.trial035.stdout:Dependencies resolved. 2026-02-24T07:30:53.882 INFO:teuthology.orchestra.run.trial035.stdout:Nothing to do. 2026-02-24T07:30:53.882 INFO:teuthology.orchestra.run.trial035.stdout:Complete! 2026-02-24T07:30:53.903 DEBUG:teuthology.orchestra.run.trial035:> sudo yum -y install python3-rbd 2026-02-24T07:30:54.218 INFO:teuthology.orchestra.run.trial035.stdout:Last metadata expiration check: 0:01:33 ago on Tue 24 Feb 2026 07:29:21 AM UTC. 2026-02-24T07:30:54.283 INFO:teuthology.orchestra.run.trial035.stdout:Package python3-rbd-2:20.3.0-5440.gff63f3d9.el9.x86_64 is already installed. 2026-02-24T07:30:54.301 INFO:teuthology.orchestra.run.trial035.stdout:Dependencies resolved. 2026-02-24T07:30:54.302 INFO:teuthology.orchestra.run.trial035.stdout:Nothing to do. 2026-02-24T07:30:54.302 INFO:teuthology.orchestra.run.trial035.stdout:Complete! 2026-02-24T07:30:54.324 DEBUG:teuthology.orchestra.run.trial035:> sudo yum -y install rbd-fuse 2026-02-24T07:30:54.642 INFO:teuthology.orchestra.run.trial035.stdout:Last metadata expiration check: 0:01:33 ago on Tue 24 Feb 2026 07:29:21 AM UTC. 2026-02-24T07:30:54.726 INFO:teuthology.orchestra.run.trial035.stdout:Dependencies resolved. 2026-02-24T07:30:54.726 INFO:teuthology.orchestra.run.trial035.stdout:================================================================================ 2026-02-24T07:30:54.726 INFO:teuthology.orchestra.run.trial035.stdout: Package Architecture Version Repository Size 2026-02-24T07:30:54.726 INFO:teuthology.orchestra.run.trial035.stdout:================================================================================ 2026-02-24T07:30:54.726 INFO:teuthology.orchestra.run.trial035.stdout:Installing: 2026-02-24T07:30:54.726 INFO:teuthology.orchestra.run.trial035.stdout: rbd-fuse x86_64 2:20.3.0-5440.gff63f3d9.el9 ceph 86 k 2026-02-24T07:30:54.726 INFO:teuthology.orchestra.run.trial035.stdout: 2026-02-24T07:30:54.726 INFO:teuthology.orchestra.run.trial035.stdout:Transaction Summary 2026-02-24T07:30:54.726 INFO:teuthology.orchestra.run.trial035.stdout:================================================================================ 2026-02-24T07:30:54.726 INFO:teuthology.orchestra.run.trial035.stdout:Install 1 Package 2026-02-24T07:30:54.726 INFO:teuthology.orchestra.run.trial035.stdout: 2026-02-24T07:30:54.726 INFO:teuthology.orchestra.run.trial035.stdout:Total download size: 86 k 2026-02-24T07:30:54.726 INFO:teuthology.orchestra.run.trial035.stdout:Installed size: 236 k 2026-02-24T07:30:54.726 INFO:teuthology.orchestra.run.trial035.stdout:Downloading Packages: 2026-02-24T07:30:54.811 INFO:teuthology.orchestra.run.trial035.stdout:rbd-fuse-20.3.0-5440.gff63f3d9.el9.x86_64.rpm 1.0 MB/s | 86 kB 00:00 2026-02-24T07:30:54.811 INFO:teuthology.orchestra.run.trial035.stdout:-------------------------------------------------------------------------------- 2026-02-24T07:30:54.811 INFO:teuthology.orchestra.run.trial035.stdout:Total 1.0 MB/s | 86 kB 00:00 2026-02-24T07:30:54.811 INFO:teuthology.orchestra.run.trial035.stdout:Running transaction check 2026-02-24T07:30:54.815 INFO:teuthology.orchestra.run.trial035.stdout:Transaction check succeeded. 2026-02-24T07:30:54.815 INFO:teuthology.orchestra.run.trial035.stdout:Running transaction test 2026-02-24T07:30:54.846 INFO:teuthology.orchestra.run.trial035.stdout:Transaction test succeeded. 2026-02-24T07:30:54.846 INFO:teuthology.orchestra.run.trial035.stdout:Running transaction 2026-02-24T07:30:54.898 INFO:teuthology.orchestra.run.trial035.stdout: Preparing : 1/1 2026-02-24T07:30:54.915 INFO:teuthology.orchestra.run.trial035.stdout: Installing : rbd-fuse-2:20.3.0-5440.gff63f3d9.el9.x86_64 1/1 2026-02-24T07:30:55.003 INFO:teuthology.orchestra.run.trial035.stdout: Running scriptlet: rbd-fuse-2:20.3.0-5440.gff63f3d9.el9.x86_64 1/1 2026-02-24T07:30:55.066 INFO:teuthology.orchestra.run.trial035.stdout: Verifying : rbd-fuse-2:20.3.0-5440.gff63f3d9.el9.x86_64 1/1 2026-02-24T07:30:55.067 INFO:teuthology.orchestra.run.trial035.stdout: 2026-02-24T07:30:55.067 INFO:teuthology.orchestra.run.trial035.stdout:Installed: 2026-02-24T07:30:55.067 INFO:teuthology.orchestra.run.trial035.stdout: rbd-fuse-2:20.3.0-5440.gff63f3d9.el9.x86_64 2026-02-24T07:30:55.067 INFO:teuthology.orchestra.run.trial035.stdout: 2026-02-24T07:30:55.067 INFO:teuthology.orchestra.run.trial035.stdout:Complete! 2026-02-24T07:30:55.096 DEBUG:teuthology.orchestra.run.trial035:> sudo yum -y install rbd-mirror 2026-02-24T07:30:55.421 INFO:teuthology.orchestra.run.trial035.stdout:Last metadata expiration check: 0:01:34 ago on Tue 24 Feb 2026 07:29:21 AM UTC. 2026-02-24T07:30:55.505 INFO:teuthology.orchestra.run.trial035.stdout:Dependencies resolved. 2026-02-24T07:30:55.506 INFO:teuthology.orchestra.run.trial035.stdout:================================================================================ 2026-02-24T07:30:55.506 INFO:teuthology.orchestra.run.trial035.stdout: Package Arch Version Repo Size 2026-02-24T07:30:55.506 INFO:teuthology.orchestra.run.trial035.stdout:================================================================================ 2026-02-24T07:30:55.506 INFO:teuthology.orchestra.run.trial035.stdout:Installing: 2026-02-24T07:30:55.506 INFO:teuthology.orchestra.run.trial035.stdout: rbd-mirror x86_64 2:20.3.0-5440.gff63f3d9.el9 ceph 2.6 M 2026-02-24T07:30:55.506 INFO:teuthology.orchestra.run.trial035.stdout: 2026-02-24T07:30:55.506 INFO:teuthology.orchestra.run.trial035.stdout:Transaction Summary 2026-02-24T07:30:55.506 INFO:teuthology.orchestra.run.trial035.stdout:================================================================================ 2026-02-24T07:30:55.507 INFO:teuthology.orchestra.run.trial035.stdout:Install 1 Package 2026-02-24T07:30:55.507 INFO:teuthology.orchestra.run.trial035.stdout: 2026-02-24T07:30:55.507 INFO:teuthology.orchestra.run.trial035.stdout:Total download size: 2.6 M 2026-02-24T07:30:55.507 INFO:teuthology.orchestra.run.trial035.stdout:Installed size: 11 M 2026-02-24T07:30:55.507 INFO:teuthology.orchestra.run.trial035.stdout:Downloading Packages: 2026-02-24T07:30:55.647 INFO:teuthology.orchestra.run.trial035.stdout:rbd-mirror-20.3.0-5440.gff63f3d9.el9.x86_64.rpm 19 MB/s | 2.6 MB 00:00 2026-02-24T07:30:55.647 INFO:teuthology.orchestra.run.trial035.stdout:-------------------------------------------------------------------------------- 2026-02-24T07:30:55.647 INFO:teuthology.orchestra.run.trial035.stdout:Total 18 MB/s | 2.6 MB 00:00 2026-02-24T07:30:55.647 INFO:teuthology.orchestra.run.trial035.stdout:Running transaction check 2026-02-24T07:30:55.651 INFO:teuthology.orchestra.run.trial035.stdout:Transaction check succeeded. 2026-02-24T07:30:55.652 INFO:teuthology.orchestra.run.trial035.stdout:Running transaction test 2026-02-24T07:30:55.691 INFO:teuthology.orchestra.run.trial035.stdout:Transaction test succeeded. 2026-02-24T07:30:55.691 INFO:teuthology.orchestra.run.trial035.stdout:Running transaction 2026-02-24T07:30:55.855 INFO:teuthology.orchestra.run.trial035.stdout: Preparing : 1/1 2026-02-24T07:30:55.856 INFO:teuthology.orchestra.run.trial035.stdout: Installing : rbd-mirror-2:20.3.0-5440.gff63f3d9.el9.x86_64 1/1 2026-02-24T07:30:55.880 INFO:teuthology.orchestra.run.trial035.stdout: Running scriptlet: rbd-mirror-2:20.3.0-5440.gff63f3d9.el9.x86_64 1/1 2026-02-24T07:30:55.880 INFO:teuthology.orchestra.run.trial035.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-24T07:30:55.880 INFO:teuthology.orchestra.run.trial035.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2026-02-24T07:30:55.880 INFO:teuthology.orchestra.run.trial035.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2026-02-24T07:30:55.881 INFO:teuthology.orchestra.run.trial035.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2026-02-24T07:30:55.881 INFO:teuthology.orchestra.run.trial035.stdout: 2026-02-24T07:30:56.220 INFO:teuthology.orchestra.run.trial035.stdout: Verifying : rbd-mirror-2:20.3.0-5440.gff63f3d9.el9.x86_64 1/1 2026-02-24T07:30:56.220 INFO:teuthology.orchestra.run.trial035.stdout: 2026-02-24T07:30:56.220 INFO:teuthology.orchestra.run.trial035.stdout:Installed: 2026-02-24T07:30:56.220 INFO:teuthology.orchestra.run.trial035.stdout: rbd-mirror-2:20.3.0-5440.gff63f3d9.el9.x86_64 2026-02-24T07:30:56.220 INFO:teuthology.orchestra.run.trial035.stdout: 2026-02-24T07:30:56.220 INFO:teuthology.orchestra.run.trial035.stdout:Complete! 2026-02-24T07:30:56.249 DEBUG:teuthology.orchestra.run.trial035:> sudo yum -y install rbd-nbd 2026-02-24T07:30:56.568 INFO:teuthology.orchestra.run.trial035.stdout:Last metadata expiration check: 0:01:35 ago on Tue 24 Feb 2026 07:29:21 AM UTC. 2026-02-24T07:30:56.653 INFO:teuthology.orchestra.run.trial035.stdout:Dependencies resolved. 2026-02-24T07:30:56.653 INFO:teuthology.orchestra.run.trial035.stdout:================================================================================ 2026-02-24T07:30:56.653 INFO:teuthology.orchestra.run.trial035.stdout: Package Architecture Version Repository Size 2026-02-24T07:30:56.654 INFO:teuthology.orchestra.run.trial035.stdout:================================================================================ 2026-02-24T07:30:56.654 INFO:teuthology.orchestra.run.trial035.stdout:Installing: 2026-02-24T07:30:56.654 INFO:teuthology.orchestra.run.trial035.stdout: rbd-nbd x86_64 2:20.3.0-5440.gff63f3d9.el9 ceph 166 k 2026-02-24T07:30:56.654 INFO:teuthology.orchestra.run.trial035.stdout: 2026-02-24T07:30:56.654 INFO:teuthology.orchestra.run.trial035.stdout:Transaction Summary 2026-02-24T07:30:56.654 INFO:teuthology.orchestra.run.trial035.stdout:================================================================================ 2026-02-24T07:30:56.654 INFO:teuthology.orchestra.run.trial035.stdout:Install 1 Package 2026-02-24T07:30:56.654 INFO:teuthology.orchestra.run.trial035.stdout: 2026-02-24T07:30:56.654 INFO:teuthology.orchestra.run.trial035.stdout:Total download size: 166 k 2026-02-24T07:30:56.654 INFO:teuthology.orchestra.run.trial035.stdout:Installed size: 496 k 2026-02-24T07:30:56.655 INFO:teuthology.orchestra.run.trial035.stdout:Downloading Packages: 2026-02-24T07:30:56.740 INFO:teuthology.orchestra.run.trial035.stdout:rbd-nbd-20.3.0-5440.gff63f3d9.el9.x86_64.rpm 1.9 MB/s | 166 kB 00:00 2026-02-24T07:30:56.741 INFO:teuthology.orchestra.run.trial035.stdout:-------------------------------------------------------------------------------- 2026-02-24T07:30:56.741 INFO:teuthology.orchestra.run.trial035.stdout:Total 1.9 MB/s | 166 kB 00:00 2026-02-24T07:30:56.742 INFO:teuthology.orchestra.run.trial035.stdout:Running transaction check 2026-02-24T07:30:56.745 INFO:teuthology.orchestra.run.trial035.stdout:Transaction check succeeded. 2026-02-24T07:30:56.745 INFO:teuthology.orchestra.run.trial035.stdout:Running transaction test 2026-02-24T07:30:56.777 INFO:teuthology.orchestra.run.trial035.stdout:Transaction test succeeded. 2026-02-24T07:30:56.777 INFO:teuthology.orchestra.run.trial035.stdout:Running transaction 2026-02-24T07:30:56.833 INFO:teuthology.orchestra.run.trial035.stdout: Preparing : 1/1 2026-02-24T07:30:56.850 INFO:teuthology.orchestra.run.trial035.stdout: Installing : rbd-nbd-2:20.3.0-5440.gff63f3d9.el9.x86_64 1/1 2026-02-24T07:30:56.937 INFO:teuthology.orchestra.run.trial035.stdout: Running scriptlet: rbd-nbd-2:20.3.0-5440.gff63f3d9.el9.x86_64 1/1 2026-02-24T07:30:57.000 INFO:teuthology.orchestra.run.trial035.stdout: Verifying : rbd-nbd-2:20.3.0-5440.gff63f3d9.el9.x86_64 1/1 2026-02-24T07:30:57.000 INFO:teuthology.orchestra.run.trial035.stdout: 2026-02-24T07:30:57.000 INFO:teuthology.orchestra.run.trial035.stdout:Installed: 2026-02-24T07:30:57.001 INFO:teuthology.orchestra.run.trial035.stdout: rbd-nbd-2:20.3.0-5440.gff63f3d9.el9.x86_64 2026-02-24T07:30:57.001 INFO:teuthology.orchestra.run.trial035.stdout: 2026-02-24T07:30:57.001 INFO:teuthology.orchestra.run.trial035.stdout:Complete! 2026-02-24T07:30:57.028 DEBUG:teuthology.orchestra.run.trial035:> sudo yum -y install ceph-osd-crimson 2026-02-24T07:30:57.348 INFO:teuthology.orchestra.run.trial035.stdout:Last metadata expiration check: 0:01:36 ago on Tue 24 Feb 2026 07:29:21 AM UTC. 2026-02-24T07:30:57.417 INFO:teuthology.orchestra.run.trial035.stdout:Package ceph-osd-crimson-2:20.3.0-5440.gff63f3d9.el9.x86_64 is already installed. 2026-02-24T07:30:57.435 INFO:teuthology.orchestra.run.trial035.stdout:Dependencies resolved. 2026-02-24T07:30:57.436 INFO:teuthology.orchestra.run.trial035.stdout:Nothing to do. 2026-02-24T07:30:57.436 INFO:teuthology.orchestra.run.trial035.stdout:Complete! 2026-02-24T07:30:57.457 DEBUG:teuthology.orchestra.run.trial035:> sudo yum -y install ceph-osd-crimson-debuginfo 2026-02-24T07:30:57.776 INFO:teuthology.orchestra.run.trial035.stdout:Last metadata expiration check: 0:01:36 ago on Tue 24 Feb 2026 07:29:21 AM UTC. 2026-02-24T07:30:57.859 INFO:teuthology.orchestra.run.trial035.stdout:Dependencies resolved. 2026-02-24T07:30:57.860 INFO:teuthology.orchestra.run.trial035.stdout:================================================================================ 2026-02-24T07:30:57.860 INFO:teuthology.orchestra.run.trial035.stdout: Package Arch Version Repo Size 2026-02-24T07:30:57.860 INFO:teuthology.orchestra.run.trial035.stdout:================================================================================ 2026-02-24T07:30:57.860 INFO:teuthology.orchestra.run.trial035.stdout:Installing: 2026-02-24T07:30:57.860 INFO:teuthology.orchestra.run.trial035.stdout: ceph-osd-crimson-debuginfo x86_64 2:20.3.0-5440.gff63f3d9.el9 ceph 429 M 2026-02-24T07:30:57.860 INFO:teuthology.orchestra.run.trial035.stdout:Installing weak dependencies: 2026-02-24T07:30:57.860 INFO:teuthology.orchestra.run.trial035.stdout: ceph-debugsource x86_64 2:20.3.0-5440.gff63f3d9.el9 ceph 12 M 2026-02-24T07:30:57.860 INFO:teuthology.orchestra.run.trial035.stdout: 2026-02-24T07:30:57.860 INFO:teuthology.orchestra.run.trial035.stdout:Transaction Summary 2026-02-24T07:30:57.860 INFO:teuthology.orchestra.run.trial035.stdout:================================================================================ 2026-02-24T07:30:57.860 INFO:teuthology.orchestra.run.trial035.stdout:Install 2 Packages 2026-02-24T07:30:57.860 INFO:teuthology.orchestra.run.trial035.stdout: 2026-02-24T07:30:57.860 INFO:teuthology.orchestra.run.trial035.stdout:Total download size: 441 M 2026-02-24T07:30:57.860 INFO:teuthology.orchestra.run.trial035.stdout:Installed size: 2.7 G 2026-02-24T07:30:57.860 INFO:teuthology.orchestra.run.trial035.stdout:Downloading Packages: 2026-02-24T07:30:58.536 INFO:teuthology.orchestra.run.trial035.stdout:(1/2): ceph-debugsource-20.3.0-5440.gff63f3d9.e 18 MB/s | 12 MB 00:00 2026-02-24T07:31:08.371 INFO:teuthology.orchestra.run.trial035.stdout:(2/2): ceph-osd-crimson-debuginfo-20.3.0-5440.g 41 MB/s | 429 MB 00:10 2026-02-24T07:31:08.372 INFO:teuthology.orchestra.run.trial035.stdout:-------------------------------------------------------------------------------- 2026-02-24T07:31:08.373 INFO:teuthology.orchestra.run.trial035.stdout:Total 42 MB/s | 441 MB 00:10 2026-02-24T07:31:08.373 INFO:teuthology.orchestra.run.trial035.stdout:Running transaction check 2026-02-24T07:31:08.378 INFO:teuthology.orchestra.run.trial035.stdout:Transaction check succeeded. 2026-02-24T07:31:08.378 INFO:teuthology.orchestra.run.trial035.stdout:Running transaction test 2026-02-24T07:31:09.736 INFO:teuthology.orchestra.run.trial035.stdout:Transaction test succeeded. 2026-02-24T07:31:09.736 INFO:teuthology.orchestra.run.trial035.stdout:Running transaction 2026-02-24T07:31:13.292 INFO:teuthology.orchestra.run.trial035.stdout: Preparing : 1/1 2026-02-24T07:31:35.315 INFO:teuthology.orchestra.run.trial035.stdout: Installing : ceph-debugsource-2:20.3.0-5440.gff63f3d9.el9.x86_64 1/2 2026-02-24T07:31:35.334 INFO:teuthology.orchestra.run.trial035.stdout: Installing : ceph-osd-crimson-debuginfo-2:20.3.0-5440.gff63f3d9.e 2/2 2026-02-24T07:31:37.425 INFO:teuthology.orchestra.run.trial035.stdout: Running scriptlet: ceph-osd-crimson-debuginfo-2:20.3.0-5440.gff63f3d9.e 2/2 2026-02-24T07:31:37.425 INFO:teuthology.orchestra.run.trial035.stdout: Verifying : ceph-debugsource-2:20.3.0-5440.gff63f3d9.el9.x86_64 1/2 2026-02-24T07:31:37.488 INFO:teuthology.orchestra.run.trial035.stdout: Verifying : ceph-osd-crimson-debuginfo-2:20.3.0-5440.gff63f3d9.e 2/2 2026-02-24T07:31:37.488 INFO:teuthology.orchestra.run.trial035.stdout: 2026-02-24T07:31:37.489 INFO:teuthology.orchestra.run.trial035.stdout:Installed: 2026-02-24T07:31:37.489 INFO:teuthology.orchestra.run.trial035.stdout: ceph-debugsource-2:20.3.0-5440.gff63f3d9.el9.x86_64 2026-02-24T07:31:37.489 INFO:teuthology.orchestra.run.trial035.stdout: ceph-osd-crimson-debuginfo-2:20.3.0-5440.gff63f3d9.el9.x86_64 2026-02-24T07:31:37.489 INFO:teuthology.orchestra.run.trial035.stdout: 2026-02-24T07:31:37.489 INFO:teuthology.orchestra.run.trial035.stdout:Complete! 2026-02-24T07:31:37.567 DEBUG:teuthology.parallel:result is None 2026-02-24T07:38:58.937 INFO:teuthology.orchestra.run.trial091.stdout:Extra Packages for Enterprise Linux 36 kB/s | 20 MB 09:44 2026-02-24T07:39:02.100 INFO:teuthology.orchestra.run.trial091.stdout:lab-extras 1.7 MB/s | 50 kB 00:00 2026-02-24T07:39:03.318 INFO:teuthology.orchestra.run.trial091.stdout:Dependencies resolved. 2026-02-24T07:39:03.319 INFO:teuthology.orchestra.run.trial091.stdout:================================================================================ 2026-02-24T07:39:03.319 INFO:teuthology.orchestra.run.trial091.stdout: Package Arch Version Repository Size 2026-02-24T07:39:03.319 INFO:teuthology.orchestra.run.trial091.stdout:================================================================================ 2026-02-24T07:39:03.320 INFO:teuthology.orchestra.run.trial091.stdout:Installing: 2026-02-24T07:39:03.320 INFO:teuthology.orchestra.run.trial091.stdout: ceph-radosgw x86_64 2:20.3.0-5440.gff63f3d9.el9 ceph 11 M 2026-02-24T07:39:03.320 INFO:teuthology.orchestra.run.trial091.stdout:Upgrading: 2026-02-24T07:39:03.320 INFO:teuthology.orchestra.run.trial091.stdout: librados2 x86_64 2:20.3.0-5440.gff63f3d9.el9 ceph 3.3 M 2026-02-24T07:39:03.320 INFO:teuthology.orchestra.run.trial091.stdout: librbd1 x86_64 2:20.3.0-5440.gff63f3d9.el9 ceph 2.7 M 2026-02-24T07:39:03.320 INFO:teuthology.orchestra.run.trial091.stdout:Installing dependencies: 2026-02-24T07:39:03.320 INFO:teuthology.orchestra.run.trial091.stdout: boost-program-options x86_64 1.75.0-13.el9 appstream 104 k 2026-02-24T07:39:03.320 INFO:teuthology.orchestra.run.trial091.stdout: ceph-base x86_64 2:20.3.0-5440.gff63f3d9.el9 ceph 5.1 M 2026-02-24T07:39:03.320 INFO:teuthology.orchestra.run.trial091.stdout: ceph-common x86_64 2:20.3.0-5440.gff63f3d9.el9 ceph 21 M 2026-02-24T07:39:03.320 INFO:teuthology.orchestra.run.trial091.stdout: ceph-selinux x86_64 2:20.3.0-5440.gff63f3d9.el9 ceph 25 k 2026-02-24T07:39:03.320 INFO:teuthology.orchestra.run.trial091.stdout: gperftools-libs x86_64 2.9.1-3.el9 epel 308 k 2026-02-24T07:39:03.320 INFO:teuthology.orchestra.run.trial091.stdout: libarrow x86_64 9.0.0-15.el9 epel 4.4 M 2026-02-24T07:39:03.320 INFO:teuthology.orchestra.run.trial091.stdout: libarrow-doc noarch 9.0.0-15.el9 epel 25 k 2026-02-24T07:39:03.320 INFO:teuthology.orchestra.run.trial091.stdout: libcephfs2 x86_64 2:20.3.0-5440.gff63f3d9.el9 ceph 859 k 2026-02-24T07:39:03.320 INFO:teuthology.orchestra.run.trial091.stdout: libnbd x86_64 1.20.3-4.el9 appstream 164 k 2026-02-24T07:39:03.320 INFO:teuthology.orchestra.run.trial091.stdout: liboath x86_64 2.6.12-1.el9 epel 49 k 2026-02-24T07:39:03.320 INFO:teuthology.orchestra.run.trial091.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2026-02-24T07:39:03.320 INFO:teuthology.orchestra.run.trial091.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2026-02-24T07:39:03.320 INFO:teuthology.orchestra.run.trial091.stdout: libradosstriper1 x86_64 2:20.3.0-5440.gff63f3d9.el9 ceph 263 k 2026-02-24T07:39:03.320 INFO:teuthology.orchestra.run.trial091.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2026-02-24T07:39:03.320 INFO:teuthology.orchestra.run.trial091.stdout: librgw2 x86_64 2:20.3.0-5440.gff63f3d9.el9 ceph 5.6 M 2026-02-24T07:39:03.320 INFO:teuthology.orchestra.run.trial091.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2026-02-24T07:39:03.320 INFO:teuthology.orchestra.run.trial091.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2026-02-24T07:39:03.320 INFO:teuthology.orchestra.run.trial091.stdout: mailcap noarch 2.1.49-5.el9 baseos 33 k 2026-02-24T07:39:03.320 INFO:teuthology.orchestra.run.trial091.stdout: parquet-libs x86_64 9.0.0-15.el9 epel 838 k 2026-02-24T07:39:03.320 INFO:teuthology.orchestra.run.trial091.stdout: pciutils x86_64 3.7.0-7.el9 baseos 93 k 2026-02-24T07:39:03.320 INFO:teuthology.orchestra.run.trial091.stdout: python3-ceph-argparse x86_64 2:20.3.0-5440.gff63f3d9.el9 ceph 45 k 2026-02-24T07:39:03.320 INFO:teuthology.orchestra.run.trial091.stdout: python3-ceph-common x86_64 2:20.3.0-5440.gff63f3d9.el9 ceph 186 k 2026-02-24T07:39:03.320 INFO:teuthology.orchestra.run.trial091.stdout: python3-cephfs x86_64 2:20.3.0-5440.gff63f3d9.el9 ceph 204 k 2026-02-24T07:39:03.320 INFO:teuthology.orchestra.run.trial091.stdout: python3-rados x86_64 2:20.3.0-5440.gff63f3d9.el9 ceph 324 k 2026-02-24T07:39:03.320 INFO:teuthology.orchestra.run.trial091.stdout: python3-rbd x86_64 2:20.3.0-5440.gff63f3d9.el9 ceph 306 k 2026-02-24T07:39:03.320 INFO:teuthology.orchestra.run.trial091.stdout: python3-rgw x86_64 2:20.3.0-5440.gff63f3d9.el9 ceph 100 k 2026-02-24T07:39:03.320 INFO:teuthology.orchestra.run.trial091.stdout: qatlib x86_64 25.08.0-2.el9 appstream 240 k 2026-02-24T07:39:03.320 INFO:teuthology.orchestra.run.trial091.stdout: qatzip-libs x86_64 1.3.1-1.el9 appstream 66 k 2026-02-24T07:39:03.320 INFO:teuthology.orchestra.run.trial091.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2026-02-24T07:39:03.320 INFO:teuthology.orchestra.run.trial091.stdout: thrift x86_64 0.15.0-4.el9 epel 1.6 M 2026-02-24T07:39:03.320 INFO:teuthology.orchestra.run.trial091.stdout:Installing weak dependencies: 2026-02-24T07:39:03.320 INFO:teuthology.orchestra.run.trial091.stdout: libcephfs-daemon x86_64 2:20.3.0-5440.gff63f3d9.el9 ceph 39 k 2026-02-24T07:39:03.321 INFO:teuthology.orchestra.run.trial091.stdout: libcephfs-proxy2 x86_64 2:20.3.0-5440.gff63f3d9.el9 ceph 26 k 2026-02-24T07:39:03.321 INFO:teuthology.orchestra.run.trial091.stdout: nvme-cli x86_64 2.16-1.el9 baseos 1.2 M 2026-02-24T07:39:03.321 INFO:teuthology.orchestra.run.trial091.stdout: qatlib-service x86_64 25.08.0-2.el9 appstream 37 k 2026-02-24T07:39:03.321 INFO:teuthology.orchestra.run.trial091.stdout: 2026-02-24T07:39:03.321 INFO:teuthology.orchestra.run.trial091.stdout:Transaction Summary 2026-02-24T07:39:03.321 INFO:teuthology.orchestra.run.trial091.stdout:================================================================================ 2026-02-24T07:39:03.321 INFO:teuthology.orchestra.run.trial091.stdout:Install 35 Packages 2026-02-24T07:39:03.321 INFO:teuthology.orchestra.run.trial091.stdout:Upgrade 2 Packages 2026-02-24T07:39:03.321 INFO:teuthology.orchestra.run.trial091.stdout: 2026-02-24T07:39:03.321 INFO:teuthology.orchestra.run.trial091.stdout:Total download size: 61 M 2026-02-24T07:39:03.321 INFO:teuthology.orchestra.run.trial091.stdout:Downloading Packages: 2026-02-24T07:39:05.069 INFO:teuthology.orchestra.run.trial091.stdout:(1/37): ceph-base-20.3.0-5440.gff63f3d9.el9.x86 14 MB/s | 5.1 MB 00:00 2026-02-24T07:39:05.102 INFO:teuthology.orchestra.run.trial091.stdout:(2/37): ceph-selinux-20.3.0-5440.gff63f3d9.el9. 766 kB/s | 25 kB 00:00 2026-02-24T07:39:05.137 INFO:teuthology.orchestra.run.trial091.stdout:(3/37): libcephfs-daemon-20.3.0-5440.gff63f3d9. 1.1 MB/s | 39 kB 00:00 2026-02-24T07:39:05.173 INFO:teuthology.orchestra.run.trial091.stdout:(4/37): libcephfs-proxy2-20.3.0-5440.gff63f3d9. 731 kB/s | 26 kB 00:00 2026-02-24T07:39:05.226 INFO:teuthology.orchestra.run.trial091.stdout:(5/37): libcephfs2-20.3.0-5440.gff63f3d9.el9.x8 16 MB/s | 859 kB 00:00 2026-02-24T07:39:05.270 INFO:teuthology.orchestra.run.trial091.stdout:(6/37): libradosstriper1-20.3.0-5440.gff63f3d9. 5.8 MB/s | 263 kB 00:00 2026-02-24T07:39:05.428 INFO:teuthology.orchestra.run.trial091.stdout:(7/37): ceph-radosgw-20.3.0-5440.gff63f3d9.el9. 15 MB/s | 11 MB 00:00 2026-02-24T07:39:05.470 INFO:teuthology.orchestra.run.trial091.stdout:(8/37): python3-ceph-argparse-20.3.0-5440.gff63 1.0 MB/s | 45 kB 00:00 2026-02-24T07:39:05.507 INFO:teuthology.orchestra.run.trial091.stdout:(9/37): librgw2-20.3.0-5440.gff63f3d9.el9.x86_6 24 MB/s | 5.6 MB 00:00 2026-02-24T07:39:05.519 INFO:teuthology.orchestra.run.trial091.stdout:(10/37): python3-ceph-common-20.3.0-5440.gff63f 3.8 MB/s | 186 kB 00:00 2026-02-24T07:39:05.539 INFO:teuthology.orchestra.run.trial091.stdout:(11/37): python3-cephfs-20.3.0-5440.gff63f3d9.e 6.3 MB/s | 204 kB 00:00 2026-02-24T07:39:05.554 INFO:teuthology.orchestra.run.trial091.stdout:(12/37): python3-rados-20.3.0-5440.gff63f3d9.el 8.9 MB/s | 324 kB 00:00 2026-02-24T07:39:05.576 INFO:teuthology.orchestra.run.trial091.stdout:(13/37): python3-rbd-20.3.0-5440.gff63f3d9.el9. 8.0 MB/s | 306 kB 00:00 2026-02-24T07:39:05.589 INFO:teuthology.orchestra.run.trial091.stdout:(14/37): python3-rgw-20.3.0-5440.gff63f3d9.el9. 2.9 MB/s | 100 kB 00:00 2026-02-24T07:39:05.665 INFO:teuthology.orchestra.run.trial091.stdout:(15/37): ceph-common-20.3.0-5440.gff63f3d9.el9. 22 MB/s | 21 MB 00:00 2026-02-24T07:39:06.145 INFO:teuthology.orchestra.run.trial091.stdout:(16/37): mailcap-2.1.49-5.el9.noarch.rpm 58 kB/s | 33 kB 00:00 2026-02-24T07:39:06.196 INFO:teuthology.orchestra.run.trial091.stdout:(17/37): boost-program-options-1.75.0-13.el9.x8 2.0 MB/s | 104 kB 00:00 2026-02-24T07:39:06.227 INFO:teuthology.orchestra.run.trial091.stdout:(18/37): libnbd-1.20.3-4.el9.x86_64.rpm 5.1 MB/s | 164 kB 00:00 2026-02-24T07:39:06.249 INFO:teuthology.orchestra.run.trial091.stdout:(19/37): libpmemobj-1.12.1-1.el9.x86_64.rpm 7.0 MB/s | 160 kB 00:00 2026-02-24T07:39:06.273 INFO:teuthology.orchestra.run.trial091.stdout:(20/37): librabbitmq-0.11.0-7.el9.x86_64.rpm 1.9 MB/s | 45 kB 00:00 2026-02-24T07:39:06.303 INFO:teuthology.orchestra.run.trial091.stdout:(21/37): librdkafka-1.6.1-102.el9.x86_64.rpm 22 MB/s | 662 kB 00:00 2026-02-24T07:39:06.326 INFO:teuthology.orchestra.run.trial091.stdout:(22/37): lttng-ust-2.12.0-6.el9.x86_64.rpm 12 MB/s | 292 kB 00:00 2026-02-24T07:39:06.351 INFO:teuthology.orchestra.run.trial091.stdout:(23/37): qatlib-25.08.0-2.el9.x86_64.rpm 9.3 MB/s | 240 kB 00:00 2026-02-24T07:39:06.373 INFO:teuthology.orchestra.run.trial091.stdout:(24/37): qatlib-service-25.08.0-2.el9.x86_64.rp 1.6 MB/s | 37 kB 00:00 2026-02-24T07:39:06.378 INFO:teuthology.orchestra.run.trial091.stdout:(25/37): pciutils-3.7.0-7.el9.x86_64.rpm 131 kB/s | 93 kB 00:00 2026-02-24T07:39:06.396 INFO:teuthology.orchestra.run.trial091.stdout:(26/37): qatzip-libs-1.3.1-1.el9.x86_64.rpm 2.8 MB/s | 66 kB 00:00 2026-02-24T07:39:06.529 INFO:teuthology.orchestra.run.trial091.stdout:(27/37): gperftools-libs-2.9.1-3.el9.x86_64.rpm 2.0 MB/s | 308 kB 00:00 2026-02-24T07:39:06.592 INFO:teuthology.orchestra.run.trial091.stdout:(28/37): libarrow-doc-9.0.0-15.el9.noarch.rpm 389 kB/s | 25 kB 00:00 2026-02-24T07:39:06.675 INFO:teuthology.orchestra.run.trial091.stdout:(29/37): libarrow-9.0.0-15.el9.x86_64.rpm 16 MB/s | 4.4 MB 00:00 2026-02-24T07:39:06.676 INFO:teuthology.orchestra.run.trial091.stdout:(30/37): liboath-2.6.12-1.el9.x86_64.rpm 584 kB/s | 49 kB 00:00 2026-02-24T07:39:06.760 INFO:teuthology.orchestra.run.trial091.stdout:(31/37): libunwind-1.6.2-1.el9.x86_64.rpm 796 kB/s | 67 kB 00:00 2026-02-24T07:39:06.844 INFO:teuthology.orchestra.run.trial091.stdout:(32/37): parquet-libs-9.0.0-15.el9.x86_64.rpm 4.9 MB/s | 838 kB 00:00 2026-02-24T07:39:06.866 INFO:teuthology.orchestra.run.trial091.stdout:(33/37): re2-20211101-20.el9.x86_64.rpm 1.8 MB/s | 191 kB 00:00 2026-02-24T07:39:06.958 INFO:teuthology.orchestra.run.trial091.stdout:(34/37): librados2-20.3.0-5440.gff63f3d9.el9.x8 36 MB/s | 3.3 MB 00:00 2026-02-24T07:39:07.009 INFO:teuthology.orchestra.run.trial091.stdout:(35/37): librbd1-20.3.0-5440.gff63f3d9.el9.x86_ 53 MB/s | 2.7 MB 00:00 2026-02-24T07:39:07.035 INFO:teuthology.orchestra.run.trial091.stdout:(36/37): thrift-0.15.0-4.el9.x86_64.rpm 8.3 MB/s | 1.6 MB 00:00 2026-02-24T07:39:07.583 INFO:teuthology.orchestra.run.trial091.stdout:(37/37): nvme-cli-2.16-1.el9.x86_64.rpm 591 kB/s | 1.2 MB 00:01 2026-02-24T07:39:07.588 INFO:teuthology.orchestra.run.trial091.stdout:-------------------------------------------------------------------------------- 2026-02-24T07:39:07.589 INFO:teuthology.orchestra.run.trial091.stdout:Total 14 MB/s | 61 MB 00:04 2026-02-24T07:39:07.685 INFO:teuthology.orchestra.run.trial091.stdout:Running transaction check 2026-02-24T07:39:07.702 INFO:teuthology.orchestra.run.trial091.stdout:Transaction check succeeded. 2026-02-24T07:39:07.702 INFO:teuthology.orchestra.run.trial091.stdout:Running transaction test 2026-02-24T07:39:07.948 INFO:teuthology.orchestra.run.trial091.stdout:Transaction test succeeded. 2026-02-24T07:39:07.948 INFO:teuthology.orchestra.run.trial091.stdout:Running transaction 2026-02-24T07:39:08.237 INFO:teuthology.orchestra.run.trial091.stdout: Preparing : 1/1 2026-02-24T07:39:08.375 INFO:teuthology.orchestra.run.trial091.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 1/39 2026-02-24T07:39:08.380 INFO:teuthology.orchestra.run.trial091.stdout: Upgrading : librados2-2:20.3.0-5440.gff63f3d9.el9.x86_64 2/39 2026-02-24T07:39:08.429 INFO:teuthology.orchestra.run.trial091.stdout: Running scriptlet: librados2-2:20.3.0-5440.gff63f3d9.el9.x86_64 2/39 2026-02-24T07:39:08.430 INFO:teuthology.orchestra.run.trial091.stdout: Installing : libcephfs2-2:20.3.0-5440.gff63f3d9.el9.x86_64 3/39 2026-02-24T07:39:08.442 INFO:teuthology.orchestra.run.trial091.stdout: Running scriptlet: libcephfs2-2:20.3.0-5440.gff63f3d9.el9.x86_64 3/39 2026-02-24T07:39:08.445 INFO:teuthology.orchestra.run.trial091.stdout: Installing : libcephfs-daemon-2:20.3.0-5440.gff63f3d9.el9.x86_6 4/39 2026-02-24T07:39:08.445 INFO:teuthology.orchestra.run.trial091.stdout: Installing : libcephfs-proxy2-2:20.3.0-5440.gff63f3d9.el9.x86_6 5/39 2026-02-24T07:39:08.472 INFO:teuthology.orchestra.run.trial091.stdout: Running scriptlet: libcephfs-proxy2-2:20.3.0-5440.gff63f3d9.el9.x86_6 5/39 2026-02-24T07:39:08.475 INFO:teuthology.orchestra.run.trial091.stdout: Installing : python3-rados-2:20.3.0-5440.gff63f3d9.el9.x86_64 6/39 2026-02-24T07:39:08.484 INFO:teuthology.orchestra.run.trial091.stdout: Installing : liboath-2.6.12-1.el9.x86_64 7/39 2026-02-24T07:39:08.486 INFO:teuthology.orchestra.run.trial091.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 8/39 2026-02-24T07:39:08.489 INFO:teuthology.orchestra.run.trial091.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 9/39 2026-02-24T07:39:08.493 INFO:teuthology.orchestra.run.trial091.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 10/39 2026-02-24T07:39:08.504 INFO:teuthology.orchestra.run.trial091.stdout: Installing : python3-ceph-argparse-2:20.3.0-5440.gff63f3d9.el9. 11/39 2026-02-24T07:39:08.517 INFO:teuthology.orchestra.run.trial091.stdout: Installing : python3-cephfs-2:20.3.0-5440.gff63f3d9.el9.x86_64 12/39 2026-02-24T07:39:08.517 INFO:teuthology.orchestra.run.trial091.stdout: Installing : libradosstriper1-2:20.3.0-5440.gff63f3d9.el9.x86_6 13/39 2026-02-24T07:39:08.538 INFO:teuthology.orchestra.run.trial091.stdout: Running scriptlet: libradosstriper1-2:20.3.0-5440.gff63f3d9.el9.x86_6 13/39 2026-02-24T07:39:08.542 INFO:teuthology.orchestra.run.trial091.stdout: Installing : thrift-0.15.0-4.el9.x86_64 14/39 2026-02-24T07:39:08.574 INFO:teuthology.orchestra.run.trial091.stdout: Installing : re2-1:20211101-20.el9.x86_64 15/39 2026-02-24T07:39:08.578 INFO:teuthology.orchestra.run.trial091.stdout: Installing : libarrow-9.0.0-15.el9.x86_64 16/39 2026-02-24T07:39:08.584 INFO:teuthology.orchestra.run.trial091.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 17/39 2026-02-24T07:39:08.586 INFO:teuthology.orchestra.run.trial091.stdout: Installing : gperftools-libs-2.9.1-3.el9.x86_64 18/39 2026-02-24T07:39:08.591 INFO:teuthology.orchestra.run.trial091.stdout: Installing : libarrow-doc-9.0.0-15.el9.noarch 19/39 2026-02-24T07:39:08.705 INFO:teuthology.orchestra.run.trial091.stdout: Installing : libnbd-1.20.3-4.el9.x86_64 20/39 2026-02-24T07:39:08.706 INFO:teuthology.orchestra.run.trial091.stdout: Upgrading : librbd1-2:20.3.0-5440.gff63f3d9.el9.x86_64 21/39 2026-02-24T07:39:08.732 INFO:teuthology.orchestra.run.trial091.stdout: Running scriptlet: librbd1-2:20.3.0-5440.gff63f3d9.el9.x86_64 21/39 2026-02-24T07:39:08.734 INFO:teuthology.orchestra.run.trial091.stdout: Installing : python3-rbd-2:20.3.0-5440.gff63f3d9.el9.x86_64 22/39 2026-02-24T07:39:08.741 INFO:teuthology.orchestra.run.trial091.stdout: Installing : boost-program-options-1.75.0-13.el9.x86_64 23/39 2026-02-24T07:39:08.988 INFO:teuthology.orchestra.run.trial091.stdout: Installing : parquet-libs-9.0.0-15.el9.x86_64 24/39 2026-02-24T07:39:08.989 INFO:teuthology.orchestra.run.trial091.stdout: Installing : librgw2-2:20.3.0-5440.gff63f3d9.el9.x86_64 25/39 2026-02-24T07:39:09.005 INFO:teuthology.orchestra.run.trial091.stdout: Running scriptlet: librgw2-2:20.3.0-5440.gff63f3d9.el9.x86_64 25/39 2026-02-24T07:39:09.010 INFO:teuthology.orchestra.run.trial091.stdout: Installing : python3-rgw-2:20.3.0-5440.gff63f3d9.el9.x86_64 26/39 2026-02-24T07:39:09.014 INFO:teuthology.orchestra.run.trial091.stdout: Installing : pciutils-3.7.0-7.el9.x86_64 27/39 2026-02-24T07:39:09.017 INFO:teuthology.orchestra.run.trial091.stdout: Installing : qatlib-25.08.0-2.el9.x86_64 28/39 2026-02-24T07:39:09.017 INFO:teuthology.orchestra.run.trial091.stdout: Installing : qatlib-service-25.08.0-2.el9.x86_64 29/39 2026-02-24T07:39:09.034 INFO:teuthology.orchestra.run.trial091.stdout: Running scriptlet: qatlib-service-25.08.0-2.el9.x86_64 29/39 2026-02-24T07:39:09.146 INFO:teuthology.orchestra.run.trial091.stdout: Installing : qatzip-libs-1.3.1-1.el9.x86_64 30/39 2026-02-24T07:39:09.148 INFO:teuthology.orchestra.run.trial091.stdout: Installing : nvme-cli-2.16-1.el9.x86_64 31/39 2026-02-24T07:39:09.443 INFO:teuthology.orchestra.run.trial091.stdout: Running scriptlet: nvme-cli-2.16-1.el9.x86_64 31/39 2026-02-24T07:39:09.443 INFO:teuthology.orchestra.run.trial091.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmefc-boot-connections.service → /usr/lib/systemd/system/nvmefc-boot-connections.service. 2026-02-24T07:39:09.444 INFO:teuthology.orchestra.run.trial091.stdout: 2026-02-24T07:39:09.488 INFO:teuthology.orchestra.run.trial091.stdout: Installing : mailcap-2.1.49-5.el9.noarch 32/39 2026-02-24T07:39:09.490 INFO:teuthology.orchestra.run.trial091.stdout: Installing : python3-ceph-common-2:20.3.0-5440.gff63f3d9.el9.x8 33/39 2026-02-24T07:39:10.431 INFO:teuthology.orchestra.run.trial091.stdout: Running scriptlet: ceph-common-2:20.3.0-5440.gff63f3d9.el9.x86_64 34/39 2026-02-24T07:39:10.434 INFO:teuthology.orchestra.run.trial091.stdout: Installing : ceph-common-2:20.3.0-5440.gff63f3d9.el9.x86_64 34/39 2026-02-24T07:39:10.694 INFO:teuthology.orchestra.run.trial091.stdout: Running scriptlet: ceph-common-2:20.3.0-5440.gff63f3d9.el9.x86_64 34/39 2026-02-24T07:39:10.697 INFO:teuthology.orchestra.run.trial091.stdout: Installing : ceph-base-2:20.3.0-5440.gff63f3d9.el9.x86_64 35/39 2026-02-24T07:39:10.738 INFO:teuthology.orchestra.run.trial091.stdout: Running scriptlet: ceph-base-2:20.3.0-5440.gff63f3d9.el9.x86_64 35/39 2026-02-24T07:39:10.738 INFO:teuthology.orchestra.run.trial091.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2026-02-24T07:39:10.738 INFO:teuthology.orchestra.run.trial091.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2026-02-24T07:39:10.738 INFO:teuthology.orchestra.run.trial091.stdout: 2026-02-24T07:39:10.741 INFO:teuthology.orchestra.run.trial091.stdout: Installing : ceph-selinux-2:20.3.0-5440.gff63f3d9.el9.x86_64 36/39 2026-02-24T07:39:15.209 INFO:teuthology.orchestra.run.trial091.stdout: Running scriptlet: ceph-selinux-2:20.3.0-5440.gff63f3d9.el9.x86_64 36/39 2026-02-24T07:39:15.209 INFO:teuthology.orchestra.run.trial091.stdout:skipping the directory /sys 2026-02-24T07:39:15.209 INFO:teuthology.orchestra.run.trial091.stdout:skipping the directory /proc 2026-02-24T07:39:15.209 INFO:teuthology.orchestra.run.trial091.stdout:skipping the directory /mnt 2026-02-24T07:39:15.209 INFO:teuthology.orchestra.run.trial091.stdout:skipping the directory /var/tmp 2026-02-24T07:39:15.209 INFO:teuthology.orchestra.run.trial091.stdout:skipping the directory /home 2026-02-24T07:39:15.209 INFO:teuthology.orchestra.run.trial091.stdout:skipping the directory /root 2026-02-24T07:39:15.209 INFO:teuthology.orchestra.run.trial091.stdout:skipping the directory /tmp 2026-02-24T07:39:15.209 INFO:teuthology.orchestra.run.trial091.stdout: 2026-02-24T07:39:15.701 INFO:teuthology.orchestra.run.trial091.stdout: Installing : ceph-radosgw-2:20.3.0-5440.gff63f3d9.el9.x86_64 37/39 2026-02-24T07:39:15.725 INFO:teuthology.orchestra.run.trial091.stdout: Running scriptlet: ceph-radosgw-2:20.3.0-5440.gff63f3d9.el9.x86_64 37/39 2026-02-24T07:39:15.725 INFO:teuthology.orchestra.run.trial091.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-24T07:39:15.725 INFO:teuthology.orchestra.run.trial091.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2026-02-24T07:39:15.725 INFO:teuthology.orchestra.run.trial091.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2026-02-24T07:39:15.725 INFO:teuthology.orchestra.run.trial091.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2026-02-24T07:39:15.726 INFO:teuthology.orchestra.run.trial091.stdout: 2026-02-24T07:39:15.727 INFO:teuthology.orchestra.run.trial091.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 38/39 2026-02-24T07:39:15.739 INFO:teuthology.orchestra.run.trial091.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 38/39 2026-02-24T07:39:15.739 INFO:teuthology.orchestra.run.trial091.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 39/39 2026-02-24T07:39:15.792 INFO:teuthology.orchestra.run.trial091.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 39/39 2026-02-24T07:39:15.792 INFO:teuthology.orchestra.run.trial091.stdout:Creating group 'qat' with GID 992. 2026-02-24T07:39:15.792 INFO:teuthology.orchestra.run.trial091.stdout: 2026-02-24T07:39:16.087 INFO:teuthology.orchestra.run.trial091.stdout: Verifying : ceph-base-2:20.3.0-5440.gff63f3d9.el9.x86_64 1/39 2026-02-24T07:39:16.087 INFO:teuthology.orchestra.run.trial091.stdout: Verifying : ceph-common-2:20.3.0-5440.gff63f3d9.el9.x86_64 2/39 2026-02-24T07:39:16.088 INFO:teuthology.orchestra.run.trial091.stdout: Verifying : ceph-radosgw-2:20.3.0-5440.gff63f3d9.el9.x86_64 3/39 2026-02-24T07:39:16.088 INFO:teuthology.orchestra.run.trial091.stdout: Verifying : ceph-selinux-2:20.3.0-5440.gff63f3d9.el9.x86_64 4/39 2026-02-24T07:39:16.088 INFO:teuthology.orchestra.run.trial091.stdout: Verifying : libcephfs-daemon-2:20.3.0-5440.gff63f3d9.el9.x86_6 5/39 2026-02-24T07:39:16.088 INFO:teuthology.orchestra.run.trial091.stdout: Verifying : libcephfs-proxy2-2:20.3.0-5440.gff63f3d9.el9.x86_6 6/39 2026-02-24T07:39:16.088 INFO:teuthology.orchestra.run.trial091.stdout: Verifying : libcephfs2-2:20.3.0-5440.gff63f3d9.el9.x86_64 7/39 2026-02-24T07:39:16.088 INFO:teuthology.orchestra.run.trial091.stdout: Verifying : libradosstriper1-2:20.3.0-5440.gff63f3d9.el9.x86_6 8/39 2026-02-24T07:39:16.088 INFO:teuthology.orchestra.run.trial091.stdout: Verifying : librgw2-2:20.3.0-5440.gff63f3d9.el9.x86_64 9/39 2026-02-24T07:39:16.088 INFO:teuthology.orchestra.run.trial091.stdout: Verifying : python3-ceph-argparse-2:20.3.0-5440.gff63f3d9.el9. 10/39 2026-02-24T07:39:16.088 INFO:teuthology.orchestra.run.trial091.stdout: Verifying : python3-ceph-common-2:20.3.0-5440.gff63f3d9.el9.x8 11/39 2026-02-24T07:39:16.088 INFO:teuthology.orchestra.run.trial091.stdout: Verifying : python3-cephfs-2:20.3.0-5440.gff63f3d9.el9.x86_64 12/39 2026-02-24T07:39:16.088 INFO:teuthology.orchestra.run.trial091.stdout: Verifying : python3-rados-2:20.3.0-5440.gff63f3d9.el9.x86_64 13/39 2026-02-24T07:39:16.089 INFO:teuthology.orchestra.run.trial091.stdout: Verifying : python3-rbd-2:20.3.0-5440.gff63f3d9.el9.x86_64 14/39 2026-02-24T07:39:16.089 INFO:teuthology.orchestra.run.trial091.stdout: Verifying : python3-rgw-2:20.3.0-5440.gff63f3d9.el9.x86_64 15/39 2026-02-24T07:39:16.089 INFO:teuthology.orchestra.run.trial091.stdout: Verifying : mailcap-2.1.49-5.el9.noarch 16/39 2026-02-24T07:39:16.089 INFO:teuthology.orchestra.run.trial091.stdout: Verifying : nvme-cli-2.16-1.el9.x86_64 17/39 2026-02-24T07:39:16.089 INFO:teuthology.orchestra.run.trial091.stdout: Verifying : pciutils-3.7.0-7.el9.x86_64 18/39 2026-02-24T07:39:16.089 INFO:teuthology.orchestra.run.trial091.stdout: Verifying : boost-program-options-1.75.0-13.el9.x86_64 19/39 2026-02-24T07:39:16.089 INFO:teuthology.orchestra.run.trial091.stdout: Verifying : libnbd-1.20.3-4.el9.x86_64 20/39 2026-02-24T07:39:16.089 INFO:teuthology.orchestra.run.trial091.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 21/39 2026-02-24T07:39:16.089 INFO:teuthology.orchestra.run.trial091.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 22/39 2026-02-24T07:39:16.089 INFO:teuthology.orchestra.run.trial091.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 23/39 2026-02-24T07:39:16.089 INFO:teuthology.orchestra.run.trial091.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 24/39 2026-02-24T07:39:16.089 INFO:teuthology.orchestra.run.trial091.stdout: Verifying : qatlib-25.08.0-2.el9.x86_64 25/39 2026-02-24T07:39:16.090 INFO:teuthology.orchestra.run.trial091.stdout: Verifying : qatlib-service-25.08.0-2.el9.x86_64 26/39 2026-02-24T07:39:16.090 INFO:teuthology.orchestra.run.trial091.stdout: Verifying : qatzip-libs-1.3.1-1.el9.x86_64 27/39 2026-02-24T07:39:16.090 INFO:teuthology.orchestra.run.trial091.stdout: Verifying : gperftools-libs-2.9.1-3.el9.x86_64 28/39 2026-02-24T07:39:16.090 INFO:teuthology.orchestra.run.trial091.stdout: Verifying : libarrow-9.0.0-15.el9.x86_64 29/39 2026-02-24T07:39:16.090 INFO:teuthology.orchestra.run.trial091.stdout: Verifying : libarrow-doc-9.0.0-15.el9.noarch 30/39 2026-02-24T07:39:16.090 INFO:teuthology.orchestra.run.trial091.stdout: Verifying : liboath-2.6.12-1.el9.x86_64 31/39 2026-02-24T07:39:16.090 INFO:teuthology.orchestra.run.trial091.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 32/39 2026-02-24T07:39:16.090 INFO:teuthology.orchestra.run.trial091.stdout: Verifying : parquet-libs-9.0.0-15.el9.x86_64 33/39 2026-02-24T07:39:16.090 INFO:teuthology.orchestra.run.trial091.stdout: Verifying : re2-1:20211101-20.el9.x86_64 34/39 2026-02-24T07:39:16.090 INFO:teuthology.orchestra.run.trial091.stdout: Verifying : thrift-0.15.0-4.el9.x86_64 35/39 2026-02-24T07:39:16.090 INFO:teuthology.orchestra.run.trial091.stdout: Verifying : librados2-2:20.3.0-5440.gff63f3d9.el9.x86_64 36/39 2026-02-24T07:39:16.090 INFO:teuthology.orchestra.run.trial091.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 37/39 2026-02-24T07:39:16.090 INFO:teuthology.orchestra.run.trial091.stdout: Verifying : librbd1-2:20.3.0-5440.gff63f3d9.el9.x86_64 38/39 2026-02-24T07:39:16.155 INFO:teuthology.orchestra.run.trial091.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 39/39 2026-02-24T07:39:16.155 INFO:teuthology.orchestra.run.trial091.stdout: 2026-02-24T07:39:16.155 INFO:teuthology.orchestra.run.trial091.stdout:Upgraded: 2026-02-24T07:39:16.155 INFO:teuthology.orchestra.run.trial091.stdout: librados2-2:20.3.0-5440.gff63f3d9.el9.x86_64 2026-02-24T07:39:16.155 INFO:teuthology.orchestra.run.trial091.stdout: librbd1-2:20.3.0-5440.gff63f3d9.el9.x86_64 2026-02-24T07:39:16.156 INFO:teuthology.orchestra.run.trial091.stdout:Installed: 2026-02-24T07:39:16.156 INFO:teuthology.orchestra.run.trial091.stdout: boost-program-options-1.75.0-13.el9.x86_64 2026-02-24T07:39:16.156 INFO:teuthology.orchestra.run.trial091.stdout: ceph-base-2:20.3.0-5440.gff63f3d9.el9.x86_64 2026-02-24T07:39:16.156 INFO:teuthology.orchestra.run.trial091.stdout: ceph-common-2:20.3.0-5440.gff63f3d9.el9.x86_64 2026-02-24T07:39:16.156 INFO:teuthology.orchestra.run.trial091.stdout: ceph-radosgw-2:20.3.0-5440.gff63f3d9.el9.x86_64 2026-02-24T07:39:16.156 INFO:teuthology.orchestra.run.trial091.stdout: ceph-selinux-2:20.3.0-5440.gff63f3d9.el9.x86_64 2026-02-24T07:39:16.156 INFO:teuthology.orchestra.run.trial091.stdout: gperftools-libs-2.9.1-3.el9.x86_64 2026-02-24T07:39:16.156 INFO:teuthology.orchestra.run.trial091.stdout: libarrow-9.0.0-15.el9.x86_64 2026-02-24T07:39:16.156 INFO:teuthology.orchestra.run.trial091.stdout: libarrow-doc-9.0.0-15.el9.noarch 2026-02-24T07:39:16.156 INFO:teuthology.orchestra.run.trial091.stdout: libcephfs-daemon-2:20.3.0-5440.gff63f3d9.el9.x86_64 2026-02-24T07:39:16.156 INFO:teuthology.orchestra.run.trial091.stdout: libcephfs-proxy2-2:20.3.0-5440.gff63f3d9.el9.x86_64 2026-02-24T07:39:16.157 INFO:teuthology.orchestra.run.trial091.stdout: libcephfs2-2:20.3.0-5440.gff63f3d9.el9.x86_64 2026-02-24T07:39:16.157 INFO:teuthology.orchestra.run.trial091.stdout: libnbd-1.20.3-4.el9.x86_64 2026-02-24T07:39:16.157 INFO:teuthology.orchestra.run.trial091.stdout: liboath-2.6.12-1.el9.x86_64 2026-02-24T07:39:16.157 INFO:teuthology.orchestra.run.trial091.stdout: libpmemobj-1.12.1-1.el9.x86_64 2026-02-24T07:39:16.157 INFO:teuthology.orchestra.run.trial091.stdout: librabbitmq-0.11.0-7.el9.x86_64 2026-02-24T07:39:16.157 INFO:teuthology.orchestra.run.trial091.stdout: libradosstriper1-2:20.3.0-5440.gff63f3d9.el9.x86_64 2026-02-24T07:39:16.157 INFO:teuthology.orchestra.run.trial091.stdout: librdkafka-1.6.1-102.el9.x86_64 2026-02-24T07:39:16.157 INFO:teuthology.orchestra.run.trial091.stdout: librgw2-2:20.3.0-5440.gff63f3d9.el9.x86_64 2026-02-24T07:39:16.157 INFO:teuthology.orchestra.run.trial091.stdout: libunwind-1.6.2-1.el9.x86_64 2026-02-24T07:39:16.157 INFO:teuthology.orchestra.run.trial091.stdout: lttng-ust-2.12.0-6.el9.x86_64 2026-02-24T07:39:16.157 INFO:teuthology.orchestra.run.trial091.stdout: mailcap-2.1.49-5.el9.noarch 2026-02-24T07:39:16.157 INFO:teuthology.orchestra.run.trial091.stdout: nvme-cli-2.16-1.el9.x86_64 2026-02-24T07:39:16.158 INFO:teuthology.orchestra.run.trial091.stdout: parquet-libs-9.0.0-15.el9.x86_64 2026-02-24T07:39:16.158 INFO:teuthology.orchestra.run.trial091.stdout: pciutils-3.7.0-7.el9.x86_64 2026-02-24T07:39:16.158 INFO:teuthology.orchestra.run.trial091.stdout: python3-ceph-argparse-2:20.3.0-5440.gff63f3d9.el9.x86_64 2026-02-24T07:39:16.158 INFO:teuthology.orchestra.run.trial091.stdout: python3-ceph-common-2:20.3.0-5440.gff63f3d9.el9.x86_64 2026-02-24T07:39:16.158 INFO:teuthology.orchestra.run.trial091.stdout: python3-cephfs-2:20.3.0-5440.gff63f3d9.el9.x86_64 2026-02-24T07:39:16.158 INFO:teuthology.orchestra.run.trial091.stdout: python3-rados-2:20.3.0-5440.gff63f3d9.el9.x86_64 2026-02-24T07:39:16.158 INFO:teuthology.orchestra.run.trial091.stdout: python3-rbd-2:20.3.0-5440.gff63f3d9.el9.x86_64 2026-02-24T07:39:16.158 INFO:teuthology.orchestra.run.trial091.stdout: python3-rgw-2:20.3.0-5440.gff63f3d9.el9.x86_64 2026-02-24T07:39:16.158 INFO:teuthology.orchestra.run.trial091.stdout: qatlib-25.08.0-2.el9.x86_64 2026-02-24T07:39:16.158 INFO:teuthology.orchestra.run.trial091.stdout: qatlib-service-25.08.0-2.el9.x86_64 2026-02-24T07:39:16.158 INFO:teuthology.orchestra.run.trial091.stdout: qatzip-libs-1.3.1-1.el9.x86_64 2026-02-24T07:39:16.158 INFO:teuthology.orchestra.run.trial091.stdout: re2-1:20211101-20.el9.x86_64 2026-02-24T07:39:16.159 INFO:teuthology.orchestra.run.trial091.stdout: thrift-0.15.0-4.el9.x86_64 2026-02-24T07:39:16.159 INFO:teuthology.orchestra.run.trial091.stdout: 2026-02-24T07:39:16.159 INFO:teuthology.orchestra.run.trial091.stdout:Complete! 2026-02-24T07:39:16.198 DEBUG:teuthology.orchestra.run.trial091:> sudo yum -y install ceph-test 2026-02-24T07:39:16.520 INFO:teuthology.orchestra.run.trial091.stdout:Last metadata expiration check: 0:00:14 ago on Tue 24 Feb 2026 07:39:02 AM UTC. 2026-02-24T07:39:16.604 INFO:teuthology.orchestra.run.trial091.stdout:Dependencies resolved. 2026-02-24T07:39:16.604 INFO:teuthology.orchestra.run.trial091.stdout:================================================================================ 2026-02-24T07:39:16.604 INFO:teuthology.orchestra.run.trial091.stdout: Package Arch Version Repository Size 2026-02-24T07:39:16.604 INFO:teuthology.orchestra.run.trial091.stdout:================================================================================ 2026-02-24T07:39:16.604 INFO:teuthology.orchestra.run.trial091.stdout:Installing: 2026-02-24T07:39:16.605 INFO:teuthology.orchestra.run.trial091.stdout: ceph-test x86_64 2:20.3.0-5440.gff63f3d9.el9 ceph 126 M 2026-02-24T07:39:16.605 INFO:teuthology.orchestra.run.trial091.stdout:Installing dependencies: 2026-02-24T07:39:16.605 INFO:teuthology.orchestra.run.trial091.stdout: hwloc-libs x86_64 2.4.1-6.el9 baseos 2.1 M 2026-02-24T07:39:16.605 INFO:teuthology.orchestra.run.trial091.stdout: libasan8 x86_64 15.2.1-7.el9 appstream 525 k 2026-02-24T07:39:16.605 INFO:teuthology.orchestra.run.trial091.stdout: libatomic x86_64 11.5.0-14.el9 baseos 23 k 2026-02-24T07:39:16.605 INFO:teuthology.orchestra.run.trial091.stdout: libcephsqlite x86_64 2:20.3.0-5440.gff63f3d9.el9 ceph 153 k 2026-02-24T07:39:16.605 INFO:teuthology.orchestra.run.trial091.stdout: libubsan x86_64 11.5.0-14.el9 appstream 176 k 2026-02-24T07:39:16.605 INFO:teuthology.orchestra.run.trial091.stdout: libxslt x86_64 1.1.34-12.el9 appstream 233 k 2026-02-24T07:39:16.605 INFO:teuthology.orchestra.run.trial091.stdout: lksctp-tools x86_64 1.0.19-2.el9 baseos 94 k 2026-02-24T07:39:16.605 INFO:teuthology.orchestra.run.trial091.stdout: protobuf x86_64 3.14.0-17.el9 appstream 1.0 M 2026-02-24T07:39:16.605 INFO:teuthology.orchestra.run.trial091.stdout: socat x86_64 1.7.4.1-8.el9 appstream 303 k 2026-02-24T07:39:16.605 INFO:teuthology.orchestra.run.trial091.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2026-02-24T07:39:16.605 INFO:teuthology.orchestra.run.trial091.stdout: yaml-cpp x86_64 0.6.3-5.el9 epel 122 k 2026-02-24T07:39:16.605 INFO:teuthology.orchestra.run.trial091.stdout: 2026-02-24T07:39:16.605 INFO:teuthology.orchestra.run.trial091.stdout:Transaction Summary 2026-02-24T07:39:16.605 INFO:teuthology.orchestra.run.trial091.stdout:================================================================================ 2026-02-24T07:39:16.605 INFO:teuthology.orchestra.run.trial091.stdout:Install 12 Packages 2026-02-24T07:39:16.605 INFO:teuthology.orchestra.run.trial091.stdout: 2026-02-24T07:39:16.606 INFO:teuthology.orchestra.run.trial091.stdout:Total download size: 131 M 2026-02-24T07:39:16.606 INFO:teuthology.orchestra.run.trial091.stdout:Installed size: 977 M 2026-02-24T07:39:16.606 INFO:teuthology.orchestra.run.trial091.stdout:Downloading Packages: 2026-02-24T07:39:17.076 INFO:teuthology.orchestra.run.trial091.stdout:(1/12): libcephsqlite-20.3.0-5440.gff63f3d9.el9 1.7 MB/s | 153 kB 00:00 2026-02-24T07:39:17.153 INFO:teuthology.orchestra.run.trial091.stdout:(2/12): libatomic-11.5.0-14.el9.x86_64.rpm 297 kB/s | 23 kB 00:00 2026-02-24T07:39:17.222 INFO:teuthology.orchestra.run.trial091.stdout:(3/12): hwloc-libs-2.4.1-6.el9.x86_64.rpm 9.0 MB/s | 2.1 MB 00:00 2026-02-24T07:39:17.226 INFO:teuthology.orchestra.run.trial091.stdout:(4/12): lksctp-tools-1.0.19-2.el9.x86_64.rpm 1.3 MB/s | 94 kB 00:00 2026-02-24T07:39:17.311 INFO:teuthology.orchestra.run.trial091.stdout:(5/12): libubsan-11.5.0-14.el9.x86_64.rpm 2.0 MB/s | 176 kB 00:00 2026-02-24T07:39:17.327 INFO:teuthology.orchestra.run.trial091.stdout:(6/12): libasan8-15.2.1-7.el9.x86_64.rpm 4.8 MB/s | 525 kB 00:00 2026-02-24T07:39:17.357 INFO:teuthology.orchestra.run.trial091.stdout:(7/12): libxslt-1.1.34-12.el9.x86_64.rpm 5.0 MB/s | 233 kB 00:00 2026-02-24T07:39:17.373 INFO:teuthology.orchestra.run.trial091.stdout:(8/12): protobuf-3.14.0-17.el9.x86_64.rpm 22 MB/s | 1.0 MB 00:00 2026-02-24T07:39:17.381 INFO:teuthology.orchestra.run.trial091.stdout:(9/12): socat-1.7.4.1-8.el9.x86_64.rpm 12 MB/s | 303 kB 00:00 2026-02-24T07:39:17.395 INFO:teuthology.orchestra.run.trial091.stdout:(10/12): xmlstarlet-1.6.1-20.el9.x86_64.rpm 2.8 MB/s | 64 kB 00:00 2026-02-24T07:39:17.436 INFO:teuthology.orchestra.run.trial091.stdout:(11/12): yaml-cpp-0.6.3-5.el9.x86_64.rpm 2.2 MB/s | 122 kB 00:00 2026-02-24T07:39:19.431 INFO:teuthology.orchestra.run.trial091.stdout:(12/12): ceph-test-20.3.0-5440.gff63f3d9.el9.x8 52 MB/s | 126 MB 00:02 2026-02-24T07:39:19.433 INFO:teuthology.orchestra.run.trial091.stdout:-------------------------------------------------------------------------------- 2026-02-24T07:39:19.433 INFO:teuthology.orchestra.run.trial091.stdout:Total 46 MB/s | 131 MB 00:02 2026-02-24T07:39:19.518 INFO:teuthology.orchestra.run.trial091.stdout:Running transaction check 2026-02-24T07:39:19.530 INFO:teuthology.orchestra.run.trial091.stdout:Transaction check succeeded. 2026-02-24T07:39:19.530 INFO:teuthology.orchestra.run.trial091.stdout:Running transaction test 2026-02-24T07:39:19.953 INFO:teuthology.orchestra.run.trial091.stdout:Transaction test succeeded. 2026-02-24T07:39:19.953 INFO:teuthology.orchestra.run.trial091.stdout:Running transaction 2026-02-24T07:39:20.406 INFO:teuthology.orchestra.run.trial091.stdout: Preparing : 1/1 2026-02-24T07:39:20.414 INFO:teuthology.orchestra.run.trial091.stdout: Installing : yaml-cpp-0.6.3-5.el9.x86_64 1/12 2026-02-24T07:39:20.423 INFO:teuthology.orchestra.run.trial091.stdout: Installing : socat-1.7.4.1-8.el9.x86_64 2/12 2026-02-24T07:39:20.428 INFO:teuthology.orchestra.run.trial091.stdout: Installing : protobuf-3.14.0-17.el9.x86_64 3/12 2026-02-24T07:39:20.432 INFO:teuthology.orchestra.run.trial091.stdout: Installing : libxslt-1.1.34-12.el9.x86_64 4/12 2026-02-24T07:39:20.434 INFO:teuthology.orchestra.run.trial091.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 5/12 2026-02-24T07:39:20.439 INFO:teuthology.orchestra.run.trial091.stdout: Installing : libubsan-11.5.0-14.el9.x86_64 6/12 2026-02-24T07:39:20.440 INFO:teuthology.orchestra.run.trial091.stdout: Installing : libasan8-15.2.1-7.el9.x86_64 7/12 2026-02-24T07:39:20.458 INFO:teuthology.orchestra.run.trial091.stdout: Running scriptlet: libasan8-15.2.1-7.el9.x86_64 7/12 2026-02-24T07:39:20.460 INFO:teuthology.orchestra.run.trial091.stdout: Installing : lksctp-tools-1.0.19-2.el9.x86_64 8/12 2026-02-24T07:39:20.471 INFO:teuthology.orchestra.run.trial091.stdout: Installing : libatomic-11.5.0-14.el9.x86_64 9/12 2026-02-24T07:39:20.481 INFO:teuthology.orchestra.run.trial091.stdout: Installing : hwloc-libs-2.4.1-6.el9.x86_64 10/12 2026-02-24T07:39:20.482 INFO:teuthology.orchestra.run.trial091.stdout: Installing : libcephsqlite-2:20.3.0-5440.gff63f3d9.el9.x86_64 11/12 2026-02-24T07:39:27.488 INFO:teuthology.orchestra.run.trial091.stdout: Running scriptlet: libcephsqlite-2:20.3.0-5440.gff63f3d9.el9.x86_64 11/12 2026-02-24T07:39:27.507 INFO:teuthology.orchestra.run.trial091.stdout: Installing : ceph-test-2:20.3.0-5440.gff63f3d9.el9.x86_64 12/12 2026-02-24T07:39:28.297 INFO:teuthology.orchestra.run.trial091.stdout: Running scriptlet: ceph-test-2:20.3.0-5440.gff63f3d9.el9.x86_64 12/12 2026-02-24T07:39:28.297 INFO:teuthology.orchestra.run.trial091.stdout: Verifying : ceph-test-2:20.3.0-5440.gff63f3d9.el9.x86_64 1/12 2026-02-24T07:39:28.297 INFO:teuthology.orchestra.run.trial091.stdout: Verifying : libcephsqlite-2:20.3.0-5440.gff63f3d9.el9.x86_64 2/12 2026-02-24T07:39:28.298 INFO:teuthology.orchestra.run.trial091.stdout: Verifying : hwloc-libs-2.4.1-6.el9.x86_64 3/12 2026-02-24T07:39:28.298 INFO:teuthology.orchestra.run.trial091.stdout: Verifying : libatomic-11.5.0-14.el9.x86_64 4/12 2026-02-24T07:39:28.298 INFO:teuthology.orchestra.run.trial091.stdout: Verifying : lksctp-tools-1.0.19-2.el9.x86_64 5/12 2026-02-24T07:39:28.298 INFO:teuthology.orchestra.run.trial091.stdout: Verifying : libasan8-15.2.1-7.el9.x86_64 6/12 2026-02-24T07:39:28.298 INFO:teuthology.orchestra.run.trial091.stdout: Verifying : libubsan-11.5.0-14.el9.x86_64 7/12 2026-02-24T07:39:28.298 INFO:teuthology.orchestra.run.trial091.stdout: Verifying : libxslt-1.1.34-12.el9.x86_64 8/12 2026-02-24T07:39:28.298 INFO:teuthology.orchestra.run.trial091.stdout: Verifying : protobuf-3.14.0-17.el9.x86_64 9/12 2026-02-24T07:39:28.298 INFO:teuthology.orchestra.run.trial091.stdout: Verifying : socat-1.7.4.1-8.el9.x86_64 10/12 2026-02-24T07:39:28.298 INFO:teuthology.orchestra.run.trial091.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 11/12 2026-02-24T07:39:28.356 INFO:teuthology.orchestra.run.trial091.stdout: Verifying : yaml-cpp-0.6.3-5.el9.x86_64 12/12 2026-02-24T07:39:28.357 INFO:teuthology.orchestra.run.trial091.stdout: 2026-02-24T07:39:28.357 INFO:teuthology.orchestra.run.trial091.stdout:Installed: 2026-02-24T07:39:28.357 INFO:teuthology.orchestra.run.trial091.stdout: ceph-test-2:20.3.0-5440.gff63f3d9.el9.x86_64 2026-02-24T07:39:28.357 INFO:teuthology.orchestra.run.trial091.stdout: hwloc-libs-2.4.1-6.el9.x86_64 2026-02-24T07:39:28.357 INFO:teuthology.orchestra.run.trial091.stdout: libasan8-15.2.1-7.el9.x86_64 2026-02-24T07:39:28.357 INFO:teuthology.orchestra.run.trial091.stdout: libatomic-11.5.0-14.el9.x86_64 2026-02-24T07:39:28.357 INFO:teuthology.orchestra.run.trial091.stdout: libcephsqlite-2:20.3.0-5440.gff63f3d9.el9.x86_64 2026-02-24T07:39:28.357 INFO:teuthology.orchestra.run.trial091.stdout: libubsan-11.5.0-14.el9.x86_64 2026-02-24T07:39:28.357 INFO:teuthology.orchestra.run.trial091.stdout: libxslt-1.1.34-12.el9.x86_64 2026-02-24T07:39:28.357 INFO:teuthology.orchestra.run.trial091.stdout: lksctp-tools-1.0.19-2.el9.x86_64 2026-02-24T07:39:28.358 INFO:teuthology.orchestra.run.trial091.stdout: protobuf-3.14.0-17.el9.x86_64 2026-02-24T07:39:28.358 INFO:teuthology.orchestra.run.trial091.stdout: socat-1.7.4.1-8.el9.x86_64 2026-02-24T07:39:28.358 INFO:teuthology.orchestra.run.trial091.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2026-02-24T07:39:28.358 INFO:teuthology.orchestra.run.trial091.stdout: yaml-cpp-0.6.3-5.el9.x86_64 2026-02-24T07:39:28.358 INFO:teuthology.orchestra.run.trial091.stdout: 2026-02-24T07:39:28.358 INFO:teuthology.orchestra.run.trial091.stdout:Complete! 2026-02-24T07:39:28.401 DEBUG:teuthology.orchestra.run.trial091:> sudo yum -y install ceph 2026-02-24T07:39:28.713 INFO:teuthology.orchestra.run.trial091.stdout:Last metadata expiration check: 0:00:26 ago on Tue 24 Feb 2026 07:39:02 AM UTC. 2026-02-24T07:39:28.802 INFO:teuthology.orchestra.run.trial091.stdout:Dependencies resolved. 2026-02-24T07:39:28.804 INFO:teuthology.orchestra.run.trial091.stdout:======================================================================================= 2026-02-24T07:39:28.804 INFO:teuthology.orchestra.run.trial091.stdout: Package Arch Version Repository Size 2026-02-24T07:39:28.804 INFO:teuthology.orchestra.run.trial091.stdout:======================================================================================= 2026-02-24T07:39:28.805 INFO:teuthology.orchestra.run.trial091.stdout:Installing: 2026-02-24T07:39:28.805 INFO:teuthology.orchestra.run.trial091.stdout: ceph x86_64 2:20.3.0-5440.gff63f3d9.el9 ceph 6.5 k 2026-02-24T07:39:28.805 INFO:teuthology.orchestra.run.trial091.stdout:Installing dependencies: 2026-02-24T07:39:28.805 INFO:teuthology.orchestra.run.trial091.stdout: abseil-cpp x86_64 20211102.0-4.el9 epel 551 k 2026-02-24T07:39:28.805 INFO:teuthology.orchestra.run.trial091.stdout: ceph-grafana-dashboards noarch 2:20.3.0-5440.gff63f3d9.el9 ceph-noarch 57 k 2026-02-24T07:39:28.805 INFO:teuthology.orchestra.run.trial091.stdout: ceph-mds x86_64 2:20.3.0-5440.gff63f3d9.el9 ceph 2.0 M 2026-02-24T07:39:28.805 INFO:teuthology.orchestra.run.trial091.stdout: ceph-mgr x86_64 2:20.3.0-5440.gff63f3d9.el9 ceph 937 k 2026-02-24T07:39:28.805 INFO:teuthology.orchestra.run.trial091.stdout: ceph-mgr-modules-core noarch 2:20.3.0-5440.gff63f3d9.el9 ceph-noarch 298 k 2026-02-24T07:39:28.806 INFO:teuthology.orchestra.run.trial091.stdout: ceph-mon x86_64 2:20.3.0-5440.gff63f3d9.el9 ceph 4.9 M 2026-02-24T07:39:28.806 INFO:teuthology.orchestra.run.trial091.stdout: ceph-osd x86_64 2:20.3.0-5440.gff63f3d9.el9 ceph 163 k 2026-02-24T07:39:28.806 INFO:teuthology.orchestra.run.trial091.stdout: ceph-osd-classic x86_64 2:20.3.0-5440.gff63f3d9.el9 ceph 15 M 2026-02-24T07:39:28.806 INFO:teuthology.orchestra.run.trial091.stdout: ceph-osd-crimson x86_64 2:20.3.0-5440.gff63f3d9.el9 ceph 87 M 2026-02-24T07:39:28.806 INFO:teuthology.orchestra.run.trial091.stdout: ceph-prometheus-alerts noarch 2:20.3.0-5440.gff63f3d9.el9 ceph-noarch 17 k 2026-02-24T07:39:28.806 INFO:teuthology.orchestra.run.trial091.stdout: cephadm noarch 2:20.3.0-5440.gff63f3d9.el9 ceph-noarch 1.0 M 2026-02-24T07:39:28.806 INFO:teuthology.orchestra.run.trial091.stdout: cryptsetup x86_64 2.8.1-3.el9 baseos 351 k 2026-02-24T07:39:28.806 INFO:teuthology.orchestra.run.trial091.stdout: flexiblas x86_64 3.0.4-9.el9 appstream 30 k 2026-02-24T07:39:28.806 INFO:teuthology.orchestra.run.trial091.stdout: flexiblas-netlib x86_64 3.0.4-9.el9 appstream 3.0 M 2026-02-24T07:39:28.806 INFO:teuthology.orchestra.run.trial091.stdout: flexiblas-openblas-openmp x86_64 3.0.4-9.el9 appstream 15 k 2026-02-24T07:39:28.807 INFO:teuthology.orchestra.run.trial091.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2026-02-24T07:39:28.807 INFO:teuthology.orchestra.run.trial091.stdout: ledmon-libs x86_64 1.1.0-3.el9 baseos 40 k 2026-02-24T07:39:28.807 INFO:teuthology.orchestra.run.trial091.stdout: libconfig x86_64 1.7.2-9.el9 baseos 72 k 2026-02-24T07:39:28.807 INFO:teuthology.orchestra.run.trial091.stdout: libgfortran x86_64 11.5.0-14.el9 baseos 794 k 2026-02-24T07:39:28.807 INFO:teuthology.orchestra.run.trial091.stdout: libquadmath x86_64 11.5.0-14.el9 baseos 184 k 2026-02-24T07:39:28.807 INFO:teuthology.orchestra.run.trial091.stdout: libstoragemgmt x86_64 1.10.1-1.el9 appstream 246 k 2026-02-24T07:39:28.807 INFO:teuthology.orchestra.run.trial091.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2026-02-24T07:39:28.807 INFO:teuthology.orchestra.run.trial091.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2026-02-24T07:39:28.807 INFO:teuthology.orchestra.run.trial091.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2026-02-24T07:39:28.807 INFO:teuthology.orchestra.run.trial091.stdout: openblas x86_64 0.3.29-1.el9 appstream 42 k 2026-02-24T07:39:28.808 INFO:teuthology.orchestra.run.trial091.stdout: openblas-openmp x86_64 0.3.29-1.el9 appstream 5.3 M 2026-02-24T07:39:28.808 INFO:teuthology.orchestra.run.trial091.stdout: protobuf-compiler x86_64 3.14.0-17.el9 crb 862 k 2026-02-24T07:39:28.808 INFO:teuthology.orchestra.run.trial091.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2026-02-24T07:39:28.808 INFO:teuthology.orchestra.run.trial091.stdout: python3-autocommand noarch 2.2.2-8.el9 epel 29 k 2026-02-24T07:39:28.808 INFO:teuthology.orchestra.run.trial091.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2026-02-24T07:39:28.808 INFO:teuthology.orchestra.run.trial091.stdout: python3-backports-tarfile noarch 1.2.0-1.el9 epel 60 k 2026-02-24T07:39:28.808 INFO:teuthology.orchestra.run.trial091.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2026-02-24T07:39:28.808 INFO:teuthology.orchestra.run.trial091.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2026-02-24T07:39:28.808 INFO:teuthology.orchestra.run.trial091.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2026-02-24T07:39:28.808 INFO:teuthology.orchestra.run.trial091.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2026-02-24T07:39:28.808 INFO:teuthology.orchestra.run.trial091.stdout: python3-cheroot noarch 10.0.1-4.el9 epel 173 k 2026-02-24T07:39:28.809 INFO:teuthology.orchestra.run.trial091.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2026-02-24T07:39:28.809 INFO:teuthology.orchestra.run.trial091.stdout: python3-cryptography x86_64 36.0.1-5.el9 baseos 1.2 M 2026-02-24T07:39:28.809 INFO:teuthology.orchestra.run.trial091.stdout: python3-devel x86_64 3.9.25-3.el9 appstream 244 k 2026-02-24T07:39:28.809 INFO:teuthology.orchestra.run.trial091.stdout: python3-google-auth noarch 1:2.45.0-1.el9 epel 254 k 2026-02-24T07:39:28.809 INFO:teuthology.orchestra.run.trial091.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2026-02-24T07:39:28.809 INFO:teuthology.orchestra.run.trial091.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2026-02-24T07:39:28.809 INFO:teuthology.orchestra.run.trial091.stdout: python3-isodate noarch 0.6.1-3.el9 epel 56 k 2026-02-24T07:39:28.809 INFO:teuthology.orchestra.run.trial091.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2026-02-24T07:39:28.809 INFO:teuthology.orchestra.run.trial091.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2026-02-24T07:39:28.809 INFO:teuthology.orchestra.run.trial091.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2026-02-24T07:39:28.809 INFO:teuthology.orchestra.run.trial091.stdout: python3-jaraco-context noarch 6.0.1-3.el9 epel 20 k 2026-02-24T07:39:28.810 INFO:teuthology.orchestra.run.trial091.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2026-02-24T07:39:28.810 INFO:teuthology.orchestra.run.trial091.stdout: python3-jaraco-text noarch 4.0.0-2.el9 epel 26 k 2026-02-24T07:39:28.810 INFO:teuthology.orchestra.run.trial091.stdout: python3-jinja2 noarch 2.11.3-8.el9 appstream 249 k 2026-02-24T07:39:28.810 INFO:teuthology.orchestra.run.trial091.stdout: python3-jmespath noarch 1.0.1-1.el9 appstream 48 k 2026-02-24T07:39:28.810 INFO:teuthology.orchestra.run.trial091.stdout: python3-kubernetes noarch 1:26.1.0-3.el9 epel 1.0 M 2026-02-24T07:39:28.810 INFO:teuthology.orchestra.run.trial091.stdout: python3-libstoragemgmt x86_64 1.10.1-1.el9 appstream 177 k 2026-02-24T07:39:28.810 INFO:teuthology.orchestra.run.trial091.stdout: python3-lxml x86_64 4.6.5-3.el9 appstream 1.2 M 2026-02-24T07:39:28.810 INFO:teuthology.orchestra.run.trial091.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2026-02-24T07:39:28.810 INFO:teuthology.orchestra.run.trial091.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2026-02-24T07:39:28.810 INFO:teuthology.orchestra.run.trial091.stdout: python3-msgpack x86_64 1.0.3-2.el9 epel 86 k 2026-02-24T07:39:28.811 INFO:teuthology.orchestra.run.trial091.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2026-02-24T07:39:28.811 INFO:teuthology.orchestra.run.trial091.stdout: python3-numpy x86_64 1:1.23.5-2.el9 appstream 6.1 M 2026-02-24T07:39:28.811 INFO:teuthology.orchestra.run.trial091.stdout: python3-numpy-f2py x86_64 1:1.23.5-2.el9 appstream 442 k 2026-02-24T07:39:28.811 INFO:teuthology.orchestra.run.trial091.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2026-02-24T07:39:28.811 INFO:teuthology.orchestra.run.trial091.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2026-02-24T07:39:28.811 INFO:teuthology.orchestra.run.trial091.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2026-02-24T07:39:28.811 INFO:teuthology.orchestra.run.trial091.stdout: python3-protobuf noarch 3.14.0-17.el9 appstream 267 k 2026-02-24T07:39:28.811 INFO:teuthology.orchestra.run.trial091.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2026-02-24T07:39:28.811 INFO:teuthology.orchestra.run.trial091.stdout: python3-pyasn1 noarch 0.4.8-7.el9 appstream 157 k 2026-02-24T07:39:28.811 INFO:teuthology.orchestra.run.trial091.stdout: python3-pyasn1-modules noarch 0.4.8-7.el9 appstream 277 k 2026-02-24T07:39:28.811 INFO:teuthology.orchestra.run.trial091.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2026-02-24T07:39:28.812 INFO:teuthology.orchestra.run.trial091.stdout: python3-pyparsing noarch 2.4.7-9.el9 baseos 150 k 2026-02-24T07:39:28.812 INFO:teuthology.orchestra.run.trial091.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2026-02-24T07:39:28.812 INFO:teuthology.orchestra.run.trial091.stdout: python3-requests noarch 2.25.1-10.el9 baseos 126 k 2026-02-24T07:39:28.812 INFO:teuthology.orchestra.run.trial091.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2026-02-24T07:39:28.812 INFO:teuthology.orchestra.run.trial091.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2026-02-24T07:39:28.812 INFO:teuthology.orchestra.run.trial091.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2026-02-24T07:39:28.812 INFO:teuthology.orchestra.run.trial091.stdout: python3-scipy x86_64 1.9.3-2.el9 appstream 19 M 2026-02-24T07:39:28.812 INFO:teuthology.orchestra.run.trial091.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2026-02-24T07:39:28.812 INFO:teuthology.orchestra.run.trial091.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2026-02-24T07:39:28.812 INFO:teuthology.orchestra.run.trial091.stdout: python3-typing-extensions noarch 4.15.0-1.el9 epel 86 k 2026-02-24T07:39:28.812 INFO:teuthology.orchestra.run.trial091.stdout: python3-urllib3 noarch 1.26.5-7.el9 baseos 218 k 2026-02-24T07:39:28.813 INFO:teuthology.orchestra.run.trial091.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2026-02-24T07:39:28.813 INFO:teuthology.orchestra.run.trial091.stdout: python3-xmlsec x86_64 1.3.13-1.el9 epel 48 k 2026-02-24T07:39:28.813 INFO:teuthology.orchestra.run.trial091.stdout: python3-xmltodict noarch 0.12.0-15.el9 epel 22 k 2026-02-24T07:39:28.813 INFO:teuthology.orchestra.run.trial091.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2026-02-24T07:39:28.813 INFO:teuthology.orchestra.run.trial091.stdout: unzip x86_64 6.0-59.el9 baseos 182 k 2026-02-24T07:39:28.813 INFO:teuthology.orchestra.run.trial091.stdout: xmlsec1 x86_64 1.2.29-13.el9 appstream 189 k 2026-02-24T07:39:28.813 INFO:teuthology.orchestra.run.trial091.stdout: xmlsec1-openssl x86_64 1.2.29-13.el9 appstream 90 k 2026-02-24T07:39:28.813 INFO:teuthology.orchestra.run.trial091.stdout: zip x86_64 3.0-35.el9 baseos 266 k 2026-02-24T07:39:28.813 INFO:teuthology.orchestra.run.trial091.stdout:Installing weak dependencies: 2026-02-24T07:39:28.813 INFO:teuthology.orchestra.run.trial091.stdout: ceph-mgr-cephadm noarch 2:20.3.0-5440.gff63f3d9.el9 ceph-noarch 184 k 2026-02-24T07:39:28.814 INFO:teuthology.orchestra.run.trial091.stdout: ceph-mgr-dashboard noarch 2:20.3.0-5440.gff63f3d9.el9 ceph-noarch 51 M 2026-02-24T07:39:28.814 INFO:teuthology.orchestra.run.trial091.stdout: ceph-mgr-diskprediction-local noarch 2:20.3.0-5440.gff63f3d9.el9 ceph-noarch 7.4 M 2026-02-24T07:39:28.814 INFO:teuthology.orchestra.run.trial091.stdout: ceph-mgr-k8sevents noarch 2:20.3.0-5440.gff63f3d9.el9 ceph-noarch 22 k 2026-02-24T07:39:28.814 INFO:teuthology.orchestra.run.trial091.stdout: ceph-mgr-rook noarch 2:20.3.0-5440.gff63f3d9.el9 ceph-noarch 50 k 2026-02-24T07:39:28.814 INFO:teuthology.orchestra.run.trial091.stdout: ceph-volume noarch 2:20.3.0-5440.gff63f3d9.el9 ceph-noarch 301 k 2026-02-24T07:39:28.814 INFO:teuthology.orchestra.run.trial091.stdout: python3-influxdb noarch 5.3.1-1.el9 epel 139 k 2026-02-24T07:39:28.814 INFO:teuthology.orchestra.run.trial091.stdout: python3-saml noarch 1.16.0-1.el9 epel 125 k 2026-02-24T07:39:28.814 INFO:teuthology.orchestra.run.trial091.stdout: 2026-02-24T07:39:28.814 INFO:teuthology.orchestra.run.trial091.stdout:Transaction Summary 2026-02-24T07:39:28.815 INFO:teuthology.orchestra.run.trial091.stdout:======================================================================================= 2026-02-24T07:39:28.815 INFO:teuthology.orchestra.run.trial091.stdout:Install 96 Packages 2026-02-24T07:39:28.815 INFO:teuthology.orchestra.run.trial091.stdout: 2026-02-24T07:39:28.816 INFO:teuthology.orchestra.run.trial091.stdout:Total download size: 225 M 2026-02-24T07:39:28.816 INFO:teuthology.orchestra.run.trial091.stdout:Installed size: 1.5 G 2026-02-24T07:39:28.816 INFO:teuthology.orchestra.run.trial091.stdout:Downloading Packages: 2026-02-24T07:39:29.359 INFO:teuthology.orchestra.run.trial091.stdout:(1/96): ceph-20.3.0-5440.gff63f3d9.el9.x86_64.r 113 kB/s | 6.5 kB 00:00 2026-02-24T07:39:29.446 INFO:teuthology.orchestra.run.trial091.stdout:(2/96): ceph-mgr-20.3.0-5440.gff63f3d9.el9.x86_ 6.3 MB/s | 937 kB 00:00 2026-02-24T07:39:29.469 INFO:teuthology.orchestra.run.trial091.stdout:(3/96): ceph-mds-20.3.0-5440.gff63f3d9.el9.x86_ 12 MB/s | 2.0 MB 00:00 2026-02-24T07:39:29.472 INFO:teuthology.orchestra.run.trial091.stdout:(4/96): ceph-osd-20.3.0-5440.gff63f3d9.el9.x86_ 6.2 MB/s | 163 kB 00:00 2026-02-24T07:39:29.626 INFO:teuthology.orchestra.run.trial091.stdout:(5/96): ceph-mon-20.3.0-5440.gff63f3d9.el9.x86_ 18 MB/s | 4.9 MB 00:00 2026-02-24T07:39:29.670 INFO:teuthology.orchestra.run.trial091.stdout:(6/96): ceph-grafana-dashboards-20.3.0-5440.gff 1.3 MB/s | 57 kB 00:00 2026-02-24T07:39:29.718 INFO:teuthology.orchestra.run.trial091.stdout:(7/96): ceph-mgr-cephadm-20.3.0-5440.gff63f3d9. 3.8 MB/s | 184 kB 00:00 2026-02-24T07:39:30.085 INFO:teuthology.orchestra.run.trial091.stdout:(8/96): ceph-osd-classic-20.3.0-5440.gff63f3d9. 24 MB/s | 15 MB 00:00 2026-02-24T07:39:30.454 INFO:teuthology.orchestra.run.trial091.stdout:(9/96): ceph-mgr-diskprediction-local-20.3.0-54 20 MB/s | 7.4 MB 00:00 2026-02-24T07:39:30.490 INFO:teuthology.orchestra.run.trial091.stdout:(10/96): ceph-mgr-k8sevents-20.3.0-5440.gff63f3 617 kB/s | 22 kB 00:00 2026-02-24T07:39:30.532 INFO:teuthology.orchestra.run.trial091.stdout:(11/96): ceph-mgr-modules-core-20.3.0-5440.gff6 7.1 MB/s | 298 kB 00:00 2026-02-24T07:39:30.568 INFO:teuthology.orchestra.run.trial091.stdout:(12/96): ceph-mgr-rook-20.3.0-5440.gff63f3d9.el 1.4 MB/s | 50 kB 00:00 2026-02-24T07:39:30.604 INFO:teuthology.orchestra.run.trial091.stdout:(13/96): ceph-prometheus-alerts-20.3.0-5440.gff 480 kB/s | 17 kB 00:00 2026-02-24T07:39:30.646 INFO:teuthology.orchestra.run.trial091.stdout:(14/96): ceph-volume-20.3.0-5440.gff63f3d9.el9. 7.1 MB/s | 301 kB 00:00 2026-02-24T07:39:30.702 INFO:teuthology.orchestra.run.trial091.stdout:(15/96): cephadm-20.3.0-5440.gff63f3d9.el9.noar 18 MB/s | 1.0 MB 00:00 2026-02-24T07:39:30.870 INFO:teuthology.orchestra.run.trial091.stdout:(16/96): cryptsetup-2.8.1-3.el9.x86_64.rpm 2.0 MB/s | 351 kB 00:00 2026-02-24T07:39:30.896 INFO:teuthology.orchestra.run.trial091.stdout:(17/96): ledmon-libs-1.1.0-3.el9.x86_64.rpm 1.5 MB/s | 40 kB 00:00 2026-02-24T07:39:30.923 INFO:teuthology.orchestra.run.trial091.stdout:(18/96): libconfig-1.7.2-9.el9.x86_64.rpm 2.6 MB/s | 72 kB 00:00 2026-02-24T07:39:30.988 INFO:teuthology.orchestra.run.trial091.stdout:(19/96): libgfortran-11.5.0-14.el9.x86_64.rpm 12 MB/s | 794 kB 00:00 2026-02-24T07:39:31.016 INFO:teuthology.orchestra.run.trial091.stdout:(20/96): libquadmath-11.5.0-14.el9.x86_64.rpm 6.3 MB/s | 184 kB 00:00 2026-02-24T07:39:31.045 INFO:teuthology.orchestra.run.trial091.stdout:(21/96): python3-cffi-1.14.5-5.el9.x86_64.rpm 8.5 MB/s | 253 kB 00:00 2026-02-24T07:39:31.098 INFO:teuthology.orchestra.run.trial091.stdout:(22/96): python3-cryptography-36.0.1-5.el9.x86_ 24 MB/s | 1.2 MB 00:00 2026-02-24T07:39:31.125 INFO:teuthology.orchestra.run.trial091.stdout:(23/96): python3-ply-3.11-14.el9.noarch.rpm 3.8 MB/s | 106 kB 00:00 2026-02-24T07:39:31.153 INFO:teuthology.orchestra.run.trial091.stdout:(24/96): python3-pycparser-2.20-6.el9.noarch.rp 4.8 MB/s | 135 kB 00:00 2026-02-24T07:39:31.181 INFO:teuthology.orchestra.run.trial091.stdout:(25/96): python3-pyparsing-2.4.7-9.el9.noarch.r 5.3 MB/s | 150 kB 00:00 2026-02-24T07:39:31.209 INFO:teuthology.orchestra.run.trial091.stdout:(26/96): python3-requests-2.25.1-10.el9.noarch. 4.4 MB/s | 126 kB 00:00 2026-02-24T07:39:31.237 INFO:teuthology.orchestra.run.trial091.stdout:(27/96): python3-urllib3-1.26.5-7.el9.noarch.rp 7.5 MB/s | 218 kB 00:00 2026-02-24T07:39:31.266 INFO:teuthology.orchestra.run.trial091.stdout:(28/96): unzip-6.0-59.el9.x86_64.rpm 6.2 MB/s | 182 kB 00:00 2026-02-24T07:39:31.294 INFO:teuthology.orchestra.run.trial091.stdout:(29/96): zip-3.0-35.el9.x86_64.rpm 9.3 MB/s | 266 kB 00:00 2026-02-24T07:39:31.698 INFO:teuthology.orchestra.run.trial091.stdout:(30/96): flexiblas-3.0.4-9.el9.x86_64.rpm 73 kB/s | 30 kB 00:00 2026-02-24T07:39:32.109 INFO:teuthology.orchestra.run.trial091.stdout:(31/96): ceph-mgr-dashboard-20.3.0-5440.gff63f3 21 MB/s | 51 MB 00:02 2026-02-24T07:39:32.513 INFO:teuthology.orchestra.run.trial091.stdout:(32/96): flexiblas-openblas-openmp-3.0.4-9.el9. 37 kB/s | 15 kB 00:00 2026-02-24T07:39:32.912 INFO:teuthology.orchestra.run.trial091.stdout:(33/96): libstoragemgmt-1.10.1-1.el9.x86_64.rpm 618 kB/s | 246 kB 00:00 2026-02-24T07:39:33.153 INFO:teuthology.orchestra.run.trial091.stdout:(34/96): lua-5.4.4-4.el9.x86_64.rpm 782 kB/s | 188 kB 00:00 2026-02-24T07:39:33.313 INFO:teuthology.orchestra.run.trial091.stdout:(35/96): ceph-osd-crimson-20.3.0-5440.gff63f3d9 23 MB/s | 87 MB 00:03 2026-02-24T07:39:33.314 INFO:teuthology.orchestra.run.trial091.stdout:(36/96): openblas-0.3.29-1.el9.x86_64.rpm 261 kB/s | 42 kB 00:00 2026-02-24T07:39:35.166 INFO:teuthology.orchestra.run.trial091.stdout:(37/96): flexiblas-netlib-3.0.4-9.el9.x86_64.rp 883 kB/s | 3.0 MB 00:03 2026-02-24T07:39:35.482 INFO:teuthology.orchestra.run.trial091.stdout:(38/96): python3-devel-3.9.25-3.el9.x86_64.rpm 771 kB/s | 244 kB 00:00 2026-02-24T07:39:35.800 INFO:teuthology.orchestra.run.trial091.stdout:(39/96): python3-jinja2-2.11.3-8.el9.noarch.rpm 784 kB/s | 249 kB 00:00 2026-02-24T07:39:35.883 INFO:teuthology.orchestra.run.trial091.stdout:(40/96): python3-jmespath-1.0.1-1.el9.noarch.rp 571 kB/s | 48 kB 00:00 2026-02-24T07:39:36.122 INFO:teuthology.orchestra.run.trial091.stdout:(41/96): python3-libstoragemgmt-1.10.1-1.el9.x8 741 kB/s | 177 kB 00:00 2026-02-24T07:39:37.529 INFO:teuthology.orchestra.run.trial091.stdout:(42/96): python3-lxml-4.6.5-3.el9.x86_64.rpm 890 kB/s | 1.2 MB 00:01 2026-02-24T07:39:37.613 INFO:teuthology.orchestra.run.trial091.stdout:(43/96): python3-markupsafe-1.1.1-12.el9.x86_64 415 kB/s | 35 kB 00:00 2026-02-24T07:39:39.372 INFO:teuthology.orchestra.run.trial091.stdout:(44/96): openblas-openmp-0.3.29-1.el9.x86_64.rp 893 kB/s | 5.3 MB 00:06 2026-02-24T07:39:39.928 INFO:teuthology.orchestra.run.trial091.stdout:(45/96): python3-numpy-f2py-1.23.5-2.el9.x86_64 795 kB/s | 442 kB 00:00 2026-02-24T07:39:40.013 INFO:teuthology.orchestra.run.trial091.stdout:[MIRROR] python3-packaging-20.9-5.el9.noarch.rpm: Status code: 503 for https://ftp2.osuosl.org/pub/centos-stream/9-stream/AppStream/x86_64/os/Packages/python3-packaging-20.9-5.el9.noarch.rpm (IP: 140.211.166.134) 2026-02-24T07:39:40.355 INFO:teuthology.orchestra.run.trial091.stdout:[MIRROR] python3-packaging-20.9-5.el9.noarch.rpm: Status code: 503 for https://ftp2.osuosl.org/pub/centos-stream/9-stream/AppStream/x86_64/os/Packages/python3-packaging-20.9-5.el9.noarch.rpm (IP: 140.211.166.134) 2026-02-24T07:39:40.642 INFO:teuthology.orchestra.run.trial091.stdout:(46/96): python3-babel-2.9.1-2.el9.noarch.rpm 833 kB/s | 6.0 MB 00:07 2026-02-24T07:39:40.722 INFO:teuthology.orchestra.run.trial091.stdout:[MIRROR] python3-packaging-20.9-5.el9.noarch.rpm: Status code: 503 for https://ftp2.osuosl.org/pub/centos-stream/9-stream/AppStream/x86_64/os/Packages/python3-packaging-20.9-5.el9.noarch.rpm (IP: 140.211.166.134) 2026-02-24T07:39:40.728 INFO:teuthology.orchestra.run.trial091.stdout:[MIRROR] python3-protobuf-3.14.0-17.el9.noarch.rpm: Status code: 503 for https://ftp2.osuosl.org/pub/centos-stream/9-stream/AppStream/x86_64/os/Packages/python3-protobuf-3.14.0-17.el9.noarch.rpm (IP: 140.211.166.134) 2026-02-24T07:39:41.055 INFO:teuthology.orchestra.run.trial091.stdout:[MIRROR] python3-protobuf-3.14.0-17.el9.noarch.rpm: Status code: 503 for https://ftp2.osuosl.org/pub/centos-stream/9-stream/AppStream/x86_64/os/Packages/python3-protobuf-3.14.0-17.el9.noarch.rpm (IP: 140.211.166.134) 2026-02-24T07:39:41.126 INFO:teuthology.orchestra.run.trial091.stdout:[MIRROR] python3-packaging-20.9-5.el9.noarch.rpm: Status code: 503 for https://ftp2.osuosl.org/pub/centos-stream/9-stream/AppStream/x86_64/os/Packages/python3-packaging-20.9-5.el9.noarch.rpm (IP: 140.211.166.134) 2026-02-24T07:39:41.163 INFO:teuthology.orchestra.run.trial091.stdout:(47/96): python3-packaging-20.9-5.el9.noarch.rp 62 kB/s | 77 kB 00:01 2026-02-24T07:39:41.783 INFO:teuthology.orchestra.run.trial091.stdout:(48/96): python3-protobuf-3.14.0-17.el9.noarch. 234 kB/s | 267 kB 00:01 2026-02-24T07:39:41.793 INFO:teuthology.orchestra.run.trial091.stdout:(49/96): python3-pyasn1-0.4.8-7.el9.noarch.rpm 250 kB/s | 157 kB 00:00 2026-02-24T07:39:41.876 INFO:teuthology.orchestra.run.trial091.stdout:(50/96): python3-requests-oauthlib-1.3.0-12.el9 654 kB/s | 54 kB 00:00 2026-02-24T07:39:42.177 INFO:teuthology.orchestra.run.trial091.stdout:(51/96): python3-pyasn1-modules-0.4.8-7.el9.noa 704 kB/s | 277 kB 00:00 2026-02-24T07:39:42.261 INFO:teuthology.orchestra.run.trial091.stdout:(52/96): python3-toml-0.10.2-6.el9.noarch.rpm 499 kB/s | 42 kB 00:00 2026-02-24T07:39:42.501 INFO:teuthology.orchestra.run.trial091.stdout:(53/96): xmlsec1-1.2.29-13.el9.x86_64.rpm 785 kB/s | 189 kB 00:00 2026-02-24T07:39:42.663 INFO:teuthology.orchestra.run.trial091.stdout:(54/96): xmlsec1-openssl-1.2.29-13.el9.x86_64.r 557 kB/s | 90 kB 00:00 2026-02-24T07:39:42.762 INFO:teuthology.orchestra.run.trial091.stdout:(55/96): lua-devel-5.4.4-4.el9.x86_64.rpm 227 kB/s | 22 kB 00:00 2026-02-24T07:39:43.794 INFO:teuthology.orchestra.run.trial091.stdout:(56/96): protobuf-compiler-3.14.0-17.el9.x86_64 835 kB/s | 862 kB 00:01 2026-02-24T07:39:44.129 INFO:teuthology.orchestra.run.trial091.stdout:(57/96): abseil-cpp-20211102.0-4.el9.x86_64.rpm 1.6 MB/s | 551 kB 00:00 2026-02-24T07:39:44.178 INFO:teuthology.orchestra.run.trial091.stdout:(58/96): grpc-data-1.46.7-10.el9.noarch.rpm 401 kB/s | 19 kB 00:00 2026-02-24T07:39:44.227 INFO:teuthology.orchestra.run.trial091.stdout:(59/96): luarocks-3.9.2-5.el9.noarch.rpm 3.0 MB/s | 151 kB 00:00 2026-02-24T07:39:44.286 INFO:teuthology.orchestra.run.trial091.stdout:(60/96): python3-asyncssh-2.13.2-5.el9.noarch.r 9.1 MB/s | 548 kB 00:00 2026-02-24T07:39:44.335 INFO:teuthology.orchestra.run.trial091.stdout:(61/96): python3-autocommand-2.2.2-8.el9.noarch 604 kB/s | 29 kB 00:00 2026-02-24T07:39:44.384 INFO:teuthology.orchestra.run.trial091.stdout:(62/96): python3-backports-tarfile-1.2.0-1.el9. 1.2 MB/s | 60 kB 00:00 2026-02-24T07:39:44.435 INFO:teuthology.orchestra.run.trial091.stdout:(63/96): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 859 kB/s | 43 kB 00:00 2026-02-24T07:39:44.484 INFO:teuthology.orchestra.run.trial091.stdout:(64/96): python3-cachetools-4.2.4-1.el9.noarch. 658 kB/s | 32 kB 00:00 2026-02-24T07:39:44.532 INFO:teuthology.orchestra.run.trial091.stdout:(65/96): python3-certifi-2023.05.07-4.el9.noarc 294 kB/s | 14 kB 00:00 2026-02-24T07:39:44.582 INFO:teuthology.orchestra.run.trial091.stdout:(66/96): python3-cheroot-10.0.1-4.el9.noarch.rp 3.4 MB/s | 173 kB 00:00 2026-02-24T07:39:44.638 INFO:teuthology.orchestra.run.trial091.stdout:(67/96): python3-cherrypy-18.6.1-2.el9.noarch.r 6.3 MB/s | 358 kB 00:00 2026-02-24T07:39:44.690 INFO:teuthology.orchestra.run.trial091.stdout:(68/96): python3-numpy-1.23.5-2.el9.x86_64.rpm 887 kB/s | 6.1 MB 00:07 2026-02-24T07:39:44.691 INFO:teuthology.orchestra.run.trial091.stdout:(69/96): python3-google-auth-2.45.0-1.el9.noarc 4.6 MB/s | 254 kB 00:00 2026-02-24T07:39:44.742 INFO:teuthology.orchestra.run.trial091.stdout:(70/96): python3-grpcio-tools-1.46.7-10.el9.x86 2.8 MB/s | 144 kB 00:00 2026-02-24T07:39:44.792 INFO:teuthology.orchestra.run.trial091.stdout:(71/96): python3-influxdb-5.3.1-1.el9.noarch.rp 2.7 MB/s | 139 kB 00:00 2026-02-24T07:39:44.846 INFO:teuthology.orchestra.run.trial091.stdout:(72/96): python3-isodate-0.6.1-3.el9.noarch.rpm 1.0 MB/s | 56 kB 00:00 2026-02-24T07:39:44.895 INFO:teuthology.orchestra.run.trial091.stdout:(73/96): python3-jaraco-8.2.1-3.el9.noarch.rpm 220 kB/s | 11 kB 00:00 2026-02-24T07:39:44.945 INFO:teuthology.orchestra.run.trial091.stdout:(74/96): python3-jaraco-classes-3.2.1-5.el9.noa 356 kB/s | 18 kB 00:00 2026-02-24T07:39:44.992 INFO:teuthology.orchestra.run.trial091.stdout:(75/96): python3-jaraco-collections-3.0.0-8.el9 486 kB/s | 23 kB 00:00 2026-02-24T07:39:45.040 INFO:teuthology.orchestra.run.trial091.stdout:(76/96): python3-jaraco-context-6.0.1-3.el9.noa 411 kB/s | 20 kB 00:00 2026-02-24T07:39:45.088 INFO:teuthology.orchestra.run.trial091.stdout:(77/96): python3-jaraco-functools-3.5.0-2.el9.n 404 kB/s | 19 kB 00:00 2026-02-24T07:39:45.120 INFO:teuthology.orchestra.run.trial091.stdout:(78/96): python3-grpcio-1.46.7-10.el9.x86_64.rp 4.7 MB/s | 2.0 MB 00:00 2026-02-24T07:39:45.136 INFO:teuthology.orchestra.run.trial091.stdout:(79/96): python3-jaraco-text-4.0.0-2.el9.noarch 555 kB/s | 26 kB 00:00 2026-02-24T07:39:45.180 INFO:teuthology.orchestra.run.trial091.stdout:(80/96): python3-kubernetes-26.1.0-3.el9.noarch 17 MB/s | 1.0 MB 00:00 2026-02-24T07:39:45.185 INFO:teuthology.orchestra.run.trial091.stdout:(81/96): python3-more-itertools-8.12.0-2.el9.no 1.6 MB/s | 79 kB 00:00 2026-02-24T07:39:45.228 INFO:teuthology.orchestra.run.trial091.stdout:(82/96): python3-msgpack-1.0.3-2.el9.x86_64.rpm 1.7 MB/s | 86 kB 00:00 2026-02-24T07:39:45.233 INFO:teuthology.orchestra.run.trial091.stdout:(83/96): python3-natsort-7.1.1-5.el9.noarch.rpm 1.2 MB/s | 58 kB 00:00 2026-02-24T07:39:45.275 INFO:teuthology.orchestra.run.trial091.stdout:(84/96): python3-portend-3.1.0-2.el9.noarch.rpm 349 kB/s | 16 kB 00:00 2026-02-24T07:39:45.283 INFO:teuthology.orchestra.run.trial091.stdout:(85/96): python3-pyOpenSSL-21.0.0-1.el9.noarch. 1.8 MB/s | 90 kB 00:00 2026-02-24T07:39:45.323 INFO:teuthology.orchestra.run.trial091.stdout:(86/96): python3-repoze-lru-0.7-16.el9.noarch.r 643 kB/s | 31 kB 00:00 2026-02-24T07:39:45.333 INFO:teuthology.orchestra.run.trial091.stdout:(87/96): python3-routes-2.5.1-5.el9.noarch.rpm 3.6 MB/s | 188 kB 00:00 2026-02-24T07:39:45.374 INFO:teuthology.orchestra.run.trial091.stdout:(88/96): python3-rsa-4.9-2.el9.noarch.rpm 1.1 MB/s | 59 kB 00:00 2026-02-24T07:39:45.387 INFO:teuthology.orchestra.run.trial091.stdout:(89/96): python3-saml-1.16.0-1.el9.noarch.rpm 2.3 MB/s | 125 kB 00:00 2026-02-24T07:39:45.427 INFO:teuthology.orchestra.run.trial091.stdout:(90/96): python3-tempora-5.0.0-2.el9.noarch.rpm 668 kB/s | 36 kB 00:00 2026-02-24T07:39:45.443 INFO:teuthology.orchestra.run.trial091.stdout:(91/96): python3-typing-extensions-4.15.0-1.el9 1.5 MB/s | 86 kB 00:00 2026-02-24T07:39:45.483 INFO:teuthology.orchestra.run.trial091.stdout:(92/96): python3-websocket-client-1.2.3-2.el9.n 1.6 MB/s | 90 kB 00:00 2026-02-24T07:39:45.491 INFO:teuthology.orchestra.run.trial091.stdout:(93/96): python3-xmlsec-1.3.13-1.el9.x86_64.rpm 1.0 MB/s | 48 kB 00:00 2026-02-24T07:39:45.531 INFO:teuthology.orchestra.run.trial091.stdout:(94/96): python3-xmltodict-0.12.0-15.el9.noarch 462 kB/s | 22 kB 00:00 2026-02-24T07:39:45.540 INFO:teuthology.orchestra.run.trial091.stdout:(95/96): python3-zc-lockfile-2.0-10.el9.noarch. 408 kB/s | 20 kB 00:00 2026-02-24T07:40:03.685 INFO:teuthology.orchestra.run.trial091.stdout:(96/96): python3-scipy-1.9.3-2.el9.x86_64.rpm 904 kB/s | 19 MB 00:21 2026-02-24T07:40:03.688 INFO:teuthology.orchestra.run.trial091.stdout:-------------------------------------------------------------------------------- 2026-02-24T07:40:03.688 INFO:teuthology.orchestra.run.trial091.stdout:Total 6.4 MB/s | 225 MB 00:34 2026-02-24T07:40:04.094 INFO:teuthology.orchestra.run.trial091.stdout:Running transaction check 2026-02-24T07:40:04.134 INFO:teuthology.orchestra.run.trial091.stdout:Transaction check succeeded. 2026-02-24T07:40:04.134 INFO:teuthology.orchestra.run.trial091.stdout:Running transaction test 2026-02-24T07:40:04.930 INFO:teuthology.orchestra.run.trial091.stdout:Transaction test succeeded. 2026-02-24T07:40:04.931 INFO:teuthology.orchestra.run.trial091.stdout:Running transaction 2026-02-24T07:40:05.759 INFO:teuthology.orchestra.run.trial091.stdout: Preparing : 1/1 2026-02-24T07:40:05.760 INFO:teuthology.orchestra.run.trial091.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 1/96 2026-02-24T07:40:05.764 INFO:teuthology.orchestra.run.trial091.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 2/96 2026-02-24T07:40:05.786 INFO:teuthology.orchestra.run.trial091.stdout: Installing : xmlsec1-1.2.29-13.el9.x86_64 3/96 2026-02-24T07:40:05.819 INFO:teuthology.orchestra.run.trial091.stdout: Installing : python3-pyasn1-0.4.8-7.el9.noarch 4/96 2026-02-24T07:40:05.826 INFO:teuthology.orchestra.run.trial091.stdout: Installing : python3-lxml-4.6.5-3.el9.x86_64 5/96 2026-02-24T07:40:05.828 INFO:teuthology.orchestra.run.trial091.stdout: Installing : lua-5.4.4-4.el9.x86_64 6/96 2026-02-24T07:40:05.832 INFO:teuthology.orchestra.run.trial091.stdout: Installing : flexiblas-3.0.4-9.el9.x86_64 7/96 2026-02-24T07:40:05.855 INFO:teuthology.orchestra.run.trial091.stdout: Installing : unzip-6.0-59.el9.x86_64 8/96 2026-02-24T07:40:05.868 INFO:teuthology.orchestra.run.trial091.stdout: Installing : python3-urllib3-1.26.5-7.el9.noarch 9/96 2026-02-24T07:40:05.872 INFO:teuthology.orchestra.run.trial091.stdout: Installing : python3-requests-2.25.1-10.el9.noarch 10/96 2026-02-24T07:40:05.878 INFO:teuthology.orchestra.run.trial091.stdout: Installing : libquadmath-11.5.0-14.el9.x86_64 11/96 2026-02-24T07:40:05.879 INFO:teuthology.orchestra.run.trial091.stdout: Installing : libgfortran-11.5.0-14.el9.x86_64 12/96 2026-02-24T07:40:05.891 INFO:teuthology.orchestra.run.trial091.stdout: Installing : ledmon-libs-1.1.0-3.el9.x86_64 13/96 2026-02-24T07:40:05.896 INFO:teuthology.orchestra.run.trial091.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 14/96 2026-02-24T07:40:05.921 INFO:teuthology.orchestra.run.trial091.stdout: Installing : zip-3.0-35.el9.x86_64 15/96 2026-02-24T07:40:05.924 INFO:teuthology.orchestra.run.trial091.stdout: Installing : luarocks-3.9.2-5.el9.noarch 16/96 2026-02-24T07:40:05.974 INFO:teuthology.orchestra.run.trial091.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 17/96 2026-02-24T07:40:05.987 INFO:teuthology.orchestra.run.trial091.stdout: Installing : python3-pyasn1-modules-0.4.8-7.el9.noarch 18/96 2026-02-24T07:40:05.989 INFO:teuthology.orchestra.run.trial091.stdout: Installing : python3-rsa-4.9-2.el9.noarch 19/96 2026-02-24T07:40:05.994 INFO:teuthology.orchestra.run.trial091.stdout: Installing : xmlsec1-openssl-1.2.29-13.el9.x86_64 20/96 2026-02-24T07:40:05.999 INFO:teuthology.orchestra.run.trial091.stdout: Installing : python3-xmlsec-1.3.13-1.el9.x86_64 21/96 2026-02-24T07:40:06.004 INFO:teuthology.orchestra.run.trial091.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 22/96 2026-02-24T07:40:06.007 INFO:teuthology.orchestra.run.trial091.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 23/96 2026-02-24T07:40:06.022 INFO:teuthology.orchestra.run.trial091.stdout: Installing : python3-xmltodict-0.12.0-15.el9.noarch 24/96 2026-02-24T07:40:06.026 INFO:teuthology.orchestra.run.trial091.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 25/96 2026-02-24T07:40:06.031 INFO:teuthology.orchestra.run.trial091.stdout: Installing : python3-typing-extensions-4.15.0-1.el9.noarch 26/96 2026-02-24T07:40:06.042 INFO:teuthology.orchestra.run.trial091.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 27/96 2026-02-24T07:40:06.051 INFO:teuthology.orchestra.run.trial091.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 28/96 2026-02-24T07:40:06.058 INFO:teuthology.orchestra.run.trial091.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 29/96 2026-02-24T07:40:06.077 INFO:teuthology.orchestra.run.trial091.stdout: Installing : python3-msgpack-1.0.3-2.el9.x86_64 30/96 2026-02-24T07:40:06.088 INFO:teuthology.orchestra.run.trial091.stdout: Installing : python3-influxdb-5.3.1-1.el9.noarch 31/96 2026-02-24T07:40:06.104 INFO:teuthology.orchestra.run.trial091.stdout: Installing : python3-isodate-0.6.1-3.el9.noarch 32/96 2026-02-24T07:40:06.108 INFO:teuthology.orchestra.run.trial091.stdout: Installing : python3-saml-1.16.0-1.el9.noarch 33/96 2026-02-24T07:40:06.116 INFO:teuthology.orchestra.run.trial091.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 34/96 2026-02-24T07:40:06.156 INFO:teuthology.orchestra.run.trial091.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 35/96 2026-02-24T07:40:06.480 INFO:teuthology.orchestra.run.trial091.stdout: Installing : python3-google-auth-1:2.45.0-1.el9.noarch 36/96 2026-02-24T07:40:06.490 INFO:teuthology.orchestra.run.trial091.stdout: Installing : python3-kubernetes-1:26.1.0-3.el9.noarch 37/96 2026-02-24T07:40:06.494 INFO:teuthology.orchestra.run.trial091.stdout: Installing : python3-backports-tarfile-1.2.0-1.el9.noarch 38/96 2026-02-24T07:40:06.499 INFO:teuthology.orchestra.run.trial091.stdout: Installing : python3-jaraco-context-6.0.1-3.el9.noarch 39/96 2026-02-24T07:40:06.501 INFO:teuthology.orchestra.run.trial091.stdout: Installing : python3-autocommand-2.2.2-8.el9.noarch 40/96 2026-02-24T07:40:06.529 INFO:teuthology.orchestra.run.trial091.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 41/96 2026-02-24T07:40:06.573 INFO:teuthology.orchestra.run.trial091.stdout: Installing : abseil-cpp-20211102.0-4.el9.x86_64 42/96 2026-02-24T07:40:06.582 INFO:teuthology.orchestra.run.trial091.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 43/96 2026-02-24T07:40:06.587 INFO:teuthology.orchestra.run.trial091.stdout: Installing : protobuf-compiler-3.14.0-17.el9.x86_64 44/96 2026-02-24T07:40:06.591 INFO:teuthology.orchestra.run.trial091.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 45/96 2026-02-24T07:40:06.596 INFO:teuthology.orchestra.run.trial091.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 46/96 2026-02-24T07:40:06.600 INFO:teuthology.orchestra.run.trial091.stdout: Installing : python3-jaraco-text-4.0.0-2.el9.noarch 47/96 2026-02-24T07:40:06.607 INFO:teuthology.orchestra.run.trial091.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 48/96 2026-02-24T07:40:06.610 INFO:teuthology.orchestra.run.trial091.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 49/96 2026-02-24T07:40:06.638 INFO:teuthology.orchestra.run.trial091.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 50/96 2026-02-24T07:40:06.648 INFO:teuthology.orchestra.run.trial091.stdout: Installing : python3-protobuf-3.14.0-17.el9.noarch 51/96 2026-02-24T07:40:06.654 INFO:teuthology.orchestra.run.trial091.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 52/96 2026-02-24T07:40:06.660 INFO:teuthology.orchestra.run.trial091.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 53/96 2026-02-24T07:40:06.695 INFO:teuthology.orchestra.run.trial091.stdout: Installing : python3-jmespath-1.0.1-1.el9.noarch 54/96 2026-02-24T07:40:06.919 INFO:teuthology.orchestra.run.trial091.stdout: Installing : python3-devel-3.9.25-3.el9.x86_64 55/96 2026-02-24T07:40:06.943 INFO:teuthology.orchestra.run.trial091.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 56/96 2026-02-24T07:40:06.946 INFO:teuthology.orchestra.run.trial091.stdout: Installing : python3-jinja2-2.11.3-8.el9.noarch 57/96 2026-02-24T07:40:07.005 INFO:teuthology.orchestra.run.trial091.stdout: Installing : openblas-0.3.29-1.el9.x86_64 58/96 2026-02-24T07:40:07.007 INFO:teuthology.orchestra.run.trial091.stdout: Installing : openblas-openmp-0.3.29-1.el9.x86_64 59/96 2026-02-24T07:40:07.029 INFO:teuthology.orchestra.run.trial091.stdout: Installing : flexiblas-openblas-openmp-3.0.4-9.el9.x86_64 60/96 2026-02-24T07:40:07.355 INFO:teuthology.orchestra.run.trial091.stdout: Installing : flexiblas-netlib-3.0.4-9.el9.x86_64 61/96 2026-02-24T07:40:07.428 INFO:teuthology.orchestra.run.trial091.stdout: Installing : python3-numpy-1:1.23.5-2.el9.x86_64 62/96 2026-02-24T07:40:08.099 INFO:teuthology.orchestra.run.trial091.stdout: Installing : python3-numpy-f2py-1:1.23.5-2.el9.x86_64 63/96 2026-02-24T07:40:08.117 INFO:teuthology.orchestra.run.trial091.stdout: Installing : python3-scipy-1.9.3-2.el9.x86_64 64/96 2026-02-24T07:40:08.126 INFO:teuthology.orchestra.run.trial091.stdout: Installing : python3-pyparsing-2.4.7-9.el9.noarch 65/96 2026-02-24T07:40:08.133 INFO:teuthology.orchestra.run.trial091.stdout: Installing : python3-packaging-20.9-5.el9.noarch 66/96 2026-02-24T07:40:08.146 INFO:teuthology.orchestra.run.trial091.stdout: Installing : python3-ply-3.11-14.el9.noarch 67/96 2026-02-24T07:40:08.162 INFO:teuthology.orchestra.run.trial091.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 68/96 2026-02-24T07:40:08.238 INFO:teuthology.orchestra.run.trial091.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 69/96 2026-02-24T07:40:08.246 INFO:teuthology.orchestra.run.trial091.stdout: Installing : python3-cryptography-36.0.1-5.el9.x86_64 70/96 2026-02-24T07:40:08.269 INFO:teuthology.orchestra.run.trial091.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 71/96 2026-02-24T07:40:08.299 INFO:teuthology.orchestra.run.trial091.stdout: Installing : python3-cheroot-10.0.1-4.el9.noarch 72/96 2026-02-24T07:40:08.350 INFO:teuthology.orchestra.run.trial091.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 73/96 2026-02-24T07:40:08.356 INFO:teuthology.orchestra.run.trial091.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 74/96 2026-02-24T07:40:08.359 INFO:teuthology.orchestra.run.trial091.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 75/96 2026-02-24T07:40:08.360 INFO:teuthology.orchestra.run.trial091.stdout: Installing : libconfig-1.7.2-9.el9.x86_64 76/96 2026-02-24T07:40:08.393 INFO:teuthology.orchestra.run.trial091.stdout: Running scriptlet: libstoragemgmt-1.10.1-1.el9.x86_64 77/96 2026-02-24T07:40:08.394 INFO:teuthology.orchestra.run.trial091.stdout:Creating group 'libstoragemgmt' with GID 991. 2026-02-24T07:40:08.394 INFO:teuthology.orchestra.run.trial091.stdout:Creating user 'libstoragemgmt' (daemon account for libstoragemgmt) with UID 991 and GID 991. 2026-02-24T07:40:08.394 INFO:teuthology.orchestra.run.trial091.stdout: 2026-02-24T07:40:08.403 INFO:teuthology.orchestra.run.trial091.stdout: Installing : libstoragemgmt-1.10.1-1.el9.x86_64 77/96 2026-02-24T07:40:08.431 INFO:teuthology.orchestra.run.trial091.stdout: Running scriptlet: libstoragemgmt-1.10.1-1.el9.x86_64 77/96 2026-02-24T07:40:08.431 INFO:teuthology.orchestra.run.trial091.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/libstoragemgmt.service → /usr/lib/systemd/system/libstoragemgmt.service. 2026-02-24T07:40:08.431 INFO:teuthology.orchestra.run.trial091.stdout: 2026-02-24T07:40:08.468 INFO:teuthology.orchestra.run.trial091.stdout: Installing : python3-libstoragemgmt-1.10.1-1.el9.x86_64 78/96 2026-02-24T07:40:09.057 INFO:teuthology.orchestra.run.trial091.stdout: Installing : cryptsetup-2.8.1-3.el9.x86_64 79/96 2026-02-24T07:40:14.401 INFO:teuthology.orchestra.run.trial091.stdout: Installing : ceph-osd-classic-2:20.3.0-5440.gff63f3d9.el9.x86_6 80/96 2026-02-24T07:40:14.465 INFO:teuthology.orchestra.run.trial091.stdout: Installing : ceph-osd-crimson-2:20.3.0-5440.gff63f3d9.el9.x86_6 81/96 2026-02-24T07:40:14.467 INFO:teuthology.orchestra.run.trial091.stdout: Installing : ceph-volume-2:20.3.0-5440.gff63f3d9.el9.noarch 82/96 2026-02-24T07:40:14.481 INFO:teuthology.orchestra.run.trial091.stdout: Running scriptlet: ceph-volume-2:20.3.0-5440.gff63f3d9.el9.noarch 82/96 2026-02-24T07:40:14.481 INFO:teuthology.orchestra.run.trial091.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-24T07:40:14.482 INFO:teuthology.orchestra.run.trial091.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2026-02-24T07:40:14.482 INFO:teuthology.orchestra.run.trial091.stdout: 2026-02-24T07:40:14.494 INFO:teuthology.orchestra.run.trial091.stdout: Installing : ceph-osd-2:20.3.0-5440.gff63f3d9.el9.x86_64 83/96 2026-02-24T07:40:14.519 INFO:teuthology.orchestra.run.trial091.stdout: Running scriptlet: ceph-osd-2:20.3.0-5440.gff63f3d9.el9.x86_64 83/96 2026-02-24T07:40:14.519 INFO:teuthology.orchestra.run.trial091.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-24T07:40:14.519 INFO:teuthology.orchestra.run.trial091.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2026-02-24T07:40:14.519 INFO:teuthology.orchestra.run.trial091.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2026-02-24T07:40:14.519 INFO:teuthology.orchestra.run.trial091.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2026-02-24T07:40:14.520 INFO:teuthology.orchestra.run.trial091.stdout: 2026-02-24T07:40:14.575 INFO:teuthology.orchestra.run.trial091.stdout: Running scriptlet: cephadm-2:20.3.0-5440.gff63f3d9.el9.noarch 84/96 2026-02-24T07:40:14.577 INFO:teuthology.orchestra.run.trial091.stdout: Installing : cephadm-2:20.3.0-5440.gff63f3d9.el9.noarch 84/96 2026-02-24T07:40:14.586 INFO:teuthology.orchestra.run.trial091.stdout: Installing : ceph-prometheus-alerts-2:20.3.0-5440.gff63f3d9.el9 85/96 2026-02-24T07:40:14.610 INFO:teuthology.orchestra.run.trial091.stdout: Installing : ceph-grafana-dashboards-2:20.3.0-5440.gff63f3d9.el 86/96 2026-02-24T07:40:14.612 INFO:teuthology.orchestra.run.trial091.stdout: Installing : ceph-mgr-cephadm-2:20.3.0-5440.gff63f3d9.el9.noarc 87/96 2026-02-24T07:40:17.509 INFO:teuthology.orchestra.run.trial091.stdout: Running scriptlet: ceph-mgr-cephadm-2:20.3.0-5440.gff63f3d9.el9.noarc 87/96 2026-02-24T07:40:17.516 INFO:teuthology.orchestra.run.trial091.stdout: Installing : ceph-mgr-dashboard-2:20.3.0-5440.gff63f3d9.el9.noa 88/96 2026-02-24T07:40:17.978 INFO:teuthology.orchestra.run.trial091.stdout: Running scriptlet: ceph-mgr-dashboard-2:20.3.0-5440.gff63f3d9.el9.noa 88/96 2026-02-24T07:40:17.979 INFO:teuthology.orchestra.run.trial091.stdout: Installing : ceph-mgr-diskprediction-local-2:20.3.0-5440.gff63f 89/96 2026-02-24T07:40:17.992 INFO:teuthology.orchestra.run.trial091.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:20.3.0-5440.gff63f 89/96 2026-02-24T07:40:17.993 INFO:teuthology.orchestra.run.trial091.stdout: Installing : ceph-mgr-k8sevents-2:20.3.0-5440.gff63f3d9.el9.noa 90/96 2026-02-24T07:40:18.055 INFO:teuthology.orchestra.run.trial091.stdout: Running scriptlet: ceph-mgr-k8sevents-2:20.3.0-5440.gff63f3d9.el9.noa 90/96 2026-02-24T07:40:18.095 INFO:teuthology.orchestra.run.trial091.stdout: Installing : ceph-mgr-modules-core-2:20.3.0-5440.gff63f3d9.el9. 91/96 2026-02-24T07:40:18.097 INFO:teuthology.orchestra.run.trial091.stdout: Installing : ceph-mgr-2:20.3.0-5440.gff63f3d9.el9.x86_64 92/96 2026-02-24T07:40:18.118 INFO:teuthology.orchestra.run.trial091.stdout: Running scriptlet: ceph-mgr-2:20.3.0-5440.gff63f3d9.el9.x86_64 92/96 2026-02-24T07:40:18.119 INFO:teuthology.orchestra.run.trial091.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-24T07:40:18.119 INFO:teuthology.orchestra.run.trial091.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2026-02-24T07:40:18.119 INFO:teuthology.orchestra.run.trial091.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2026-02-24T07:40:18.119 INFO:teuthology.orchestra.run.trial091.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2026-02-24T07:40:18.119 INFO:teuthology.orchestra.run.trial091.stdout: 2026-02-24T07:40:18.131 INFO:teuthology.orchestra.run.trial091.stdout: Installing : ceph-mgr-rook-2:20.3.0-5440.gff63f3d9.el9.noarch 93/96 2026-02-24T07:40:18.348 INFO:teuthology.orchestra.run.trial091.stdout: Running scriptlet: ceph-mgr-rook-2:20.3.0-5440.gff63f3d9.el9.noarch 93/96 2026-02-24T07:40:18.349 INFO:teuthology.orchestra.run.trial091.stdout: Installing : ceph-mon-2:20.3.0-5440.gff63f3d9.el9.x86_64 94/96 2026-02-24T07:40:18.371 INFO:teuthology.orchestra.run.trial091.stdout: Running scriptlet: ceph-mon-2:20.3.0-5440.gff63f3d9.el9.x86_64 94/96 2026-02-24T07:40:18.371 INFO:teuthology.orchestra.run.trial091.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-24T07:40:18.371 INFO:teuthology.orchestra.run.trial091.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2026-02-24T07:40:18.371 INFO:teuthology.orchestra.run.trial091.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2026-02-24T07:40:18.371 INFO:teuthology.orchestra.run.trial091.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2026-02-24T07:40:18.371 INFO:teuthology.orchestra.run.trial091.stdout: 2026-02-24T07:40:18.454 INFO:teuthology.orchestra.run.trial091.stdout: Installing : ceph-mds-2:20.3.0-5440.gff63f3d9.el9.x86_64 95/96 2026-02-24T07:40:18.475 INFO:teuthology.orchestra.run.trial091.stdout: Running scriptlet: ceph-mds-2:20.3.0-5440.gff63f3d9.el9.x86_64 95/96 2026-02-24T07:40:18.475 INFO:teuthology.orchestra.run.trial091.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-24T07:40:18.476 INFO:teuthology.orchestra.run.trial091.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2026-02-24T07:40:18.476 INFO:teuthology.orchestra.run.trial091.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2026-02-24T07:40:18.476 INFO:teuthology.orchestra.run.trial091.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2026-02-24T07:40:18.476 INFO:teuthology.orchestra.run.trial091.stdout: 2026-02-24T07:40:18.477 INFO:teuthology.orchestra.run.trial091.stdout: Installing : ceph-2:20.3.0-5440.gff63f3d9.el9.x86_64 96/96 2026-02-24T07:40:18.483 INFO:teuthology.orchestra.run.trial091.stdout: Running scriptlet: ceph-osd-classic-2:20.3.0-5440.gff63f3d9.el9.x86_6 96/96 2026-02-24T07:40:18.507 INFO:teuthology.orchestra.run.trial091.stdout: Running scriptlet: ceph-osd-crimson-2:20.3.0-5440.gff63f3d9.el9.x86_6 96/96 2026-02-24T07:40:19.872 INFO:teuthology.orchestra.run.trial091.stdout: Running scriptlet: ceph-2:20.3.0-5440.gff63f3d9.el9.x86_64 96/96 2026-02-24T07:40:19.872 INFO:teuthology.orchestra.run.trial091.stdout: Verifying : ceph-2:20.3.0-5440.gff63f3d9.el9.x86_64 1/96 2026-02-24T07:40:19.872 INFO:teuthology.orchestra.run.trial091.stdout: Verifying : ceph-mds-2:20.3.0-5440.gff63f3d9.el9.x86_64 2/96 2026-02-24T07:40:19.872 INFO:teuthology.orchestra.run.trial091.stdout: Verifying : ceph-mgr-2:20.3.0-5440.gff63f3d9.el9.x86_64 3/96 2026-02-24T07:40:19.872 INFO:teuthology.orchestra.run.trial091.stdout: Verifying : ceph-mon-2:20.3.0-5440.gff63f3d9.el9.x86_64 4/96 2026-02-24T07:40:19.872 INFO:teuthology.orchestra.run.trial091.stdout: Verifying : ceph-osd-2:20.3.0-5440.gff63f3d9.el9.x86_64 5/96 2026-02-24T07:40:19.872 INFO:teuthology.orchestra.run.trial091.stdout: Verifying : ceph-osd-classic-2:20.3.0-5440.gff63f3d9.el9.x86_6 6/96 2026-02-24T07:40:19.873 INFO:teuthology.orchestra.run.trial091.stdout: Verifying : ceph-osd-crimson-2:20.3.0-5440.gff63f3d9.el9.x86_6 7/96 2026-02-24T07:40:19.873 INFO:teuthology.orchestra.run.trial091.stdout: Verifying : ceph-grafana-dashboards-2:20.3.0-5440.gff63f3d9.el 8/96 2026-02-24T07:40:19.873 INFO:teuthology.orchestra.run.trial091.stdout: Verifying : ceph-mgr-cephadm-2:20.3.0-5440.gff63f3d9.el9.noarc 9/96 2026-02-24T07:40:19.873 INFO:teuthology.orchestra.run.trial091.stdout: Verifying : ceph-mgr-dashboard-2:20.3.0-5440.gff63f3d9.el9.noa 10/96 2026-02-24T07:40:19.873 INFO:teuthology.orchestra.run.trial091.stdout: Verifying : ceph-mgr-diskprediction-local-2:20.3.0-5440.gff63f 11/96 2026-02-24T07:40:19.873 INFO:teuthology.orchestra.run.trial091.stdout: Verifying : ceph-mgr-k8sevents-2:20.3.0-5440.gff63f3d9.el9.noa 12/96 2026-02-24T07:40:19.873 INFO:teuthology.orchestra.run.trial091.stdout: Verifying : ceph-mgr-modules-core-2:20.3.0-5440.gff63f3d9.el9. 13/96 2026-02-24T07:40:19.873 INFO:teuthology.orchestra.run.trial091.stdout: Verifying : ceph-mgr-rook-2:20.3.0-5440.gff63f3d9.el9.noarch 14/96 2026-02-24T07:40:19.873 INFO:teuthology.orchestra.run.trial091.stdout: Verifying : ceph-prometheus-alerts-2:20.3.0-5440.gff63f3d9.el9 15/96 2026-02-24T07:40:19.873 INFO:teuthology.orchestra.run.trial091.stdout: Verifying : ceph-volume-2:20.3.0-5440.gff63f3d9.el9.noarch 16/96 2026-02-24T07:40:19.873 INFO:teuthology.orchestra.run.trial091.stdout: Verifying : cephadm-2:20.3.0-5440.gff63f3d9.el9.noarch 17/96 2026-02-24T07:40:19.873 INFO:teuthology.orchestra.run.trial091.stdout: Verifying : cryptsetup-2.8.1-3.el9.x86_64 18/96 2026-02-24T07:40:19.874 INFO:teuthology.orchestra.run.trial091.stdout: Verifying : ledmon-libs-1.1.0-3.el9.x86_64 19/96 2026-02-24T07:40:19.874 INFO:teuthology.orchestra.run.trial091.stdout: Verifying : libconfig-1.7.2-9.el9.x86_64 20/96 2026-02-24T07:40:19.874 INFO:teuthology.orchestra.run.trial091.stdout: Verifying : libgfortran-11.5.0-14.el9.x86_64 21/96 2026-02-24T07:40:19.874 INFO:teuthology.orchestra.run.trial091.stdout: Verifying : libquadmath-11.5.0-14.el9.x86_64 22/96 2026-02-24T07:40:19.874 INFO:teuthology.orchestra.run.trial091.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 23/96 2026-02-24T07:40:19.874 INFO:teuthology.orchestra.run.trial091.stdout: Verifying : python3-cryptography-36.0.1-5.el9.x86_64 24/96 2026-02-24T07:40:19.874 INFO:teuthology.orchestra.run.trial091.stdout: Verifying : python3-ply-3.11-14.el9.noarch 25/96 2026-02-24T07:40:19.874 INFO:teuthology.orchestra.run.trial091.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 26/96 2026-02-24T07:40:19.874 INFO:teuthology.orchestra.run.trial091.stdout: Verifying : python3-pyparsing-2.4.7-9.el9.noarch 27/96 2026-02-24T07:40:19.874 INFO:teuthology.orchestra.run.trial091.stdout: Verifying : python3-requests-2.25.1-10.el9.noarch 28/96 2026-02-24T07:40:19.874 INFO:teuthology.orchestra.run.trial091.stdout: Verifying : python3-urllib3-1.26.5-7.el9.noarch 29/96 2026-02-24T07:40:19.874 INFO:teuthology.orchestra.run.trial091.stdout: Verifying : unzip-6.0-59.el9.x86_64 30/96 2026-02-24T07:40:19.875 INFO:teuthology.orchestra.run.trial091.stdout: Verifying : zip-3.0-35.el9.x86_64 31/96 2026-02-24T07:40:19.875 INFO:teuthology.orchestra.run.trial091.stdout: Verifying : flexiblas-3.0.4-9.el9.x86_64 32/96 2026-02-24T07:40:19.875 INFO:teuthology.orchestra.run.trial091.stdout: Verifying : flexiblas-netlib-3.0.4-9.el9.x86_64 33/96 2026-02-24T07:40:19.875 INFO:teuthology.orchestra.run.trial091.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-9.el9.x86_64 34/96 2026-02-24T07:40:19.875 INFO:teuthology.orchestra.run.trial091.stdout: Verifying : libstoragemgmt-1.10.1-1.el9.x86_64 35/96 2026-02-24T07:40:19.875 INFO:teuthology.orchestra.run.trial091.stdout: Verifying : lua-5.4.4-4.el9.x86_64 36/96 2026-02-24T07:40:19.875 INFO:teuthology.orchestra.run.trial091.stdout: Verifying : openblas-0.3.29-1.el9.x86_64 37/96 2026-02-24T07:40:19.875 INFO:teuthology.orchestra.run.trial091.stdout: Verifying : openblas-openmp-0.3.29-1.el9.x86_64 38/96 2026-02-24T07:40:19.875 INFO:teuthology.orchestra.run.trial091.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 39/96 2026-02-24T07:40:19.875 INFO:teuthology.orchestra.run.trial091.stdout: Verifying : python3-devel-3.9.25-3.el9.x86_64 40/96 2026-02-24T07:40:19.875 INFO:teuthology.orchestra.run.trial091.stdout: Verifying : python3-jinja2-2.11.3-8.el9.noarch 41/96 2026-02-24T07:40:19.875 INFO:teuthology.orchestra.run.trial091.stdout: Verifying : python3-jmespath-1.0.1-1.el9.noarch 42/96 2026-02-24T07:40:19.876 INFO:teuthology.orchestra.run.trial091.stdout: Verifying : python3-libstoragemgmt-1.10.1-1.el9.x86_64 43/96 2026-02-24T07:40:19.876 INFO:teuthology.orchestra.run.trial091.stdout: Verifying : python3-lxml-4.6.5-3.el9.x86_64 44/96 2026-02-24T07:40:19.876 INFO:teuthology.orchestra.run.trial091.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 45/96 2026-02-24T07:40:19.876 INFO:teuthology.orchestra.run.trial091.stdout: Verifying : python3-numpy-1:1.23.5-2.el9.x86_64 46/96 2026-02-24T07:40:19.876 INFO:teuthology.orchestra.run.trial091.stdout: Verifying : python3-numpy-f2py-1:1.23.5-2.el9.x86_64 47/96 2026-02-24T07:40:19.876 INFO:teuthology.orchestra.run.trial091.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 48/96 2026-02-24T07:40:19.876 INFO:teuthology.orchestra.run.trial091.stdout: Verifying : python3-protobuf-3.14.0-17.el9.noarch 49/96 2026-02-24T07:40:19.876 INFO:teuthology.orchestra.run.trial091.stdout: Verifying : python3-pyasn1-0.4.8-7.el9.noarch 50/96 2026-02-24T07:40:19.876 INFO:teuthology.orchestra.run.trial091.stdout: Verifying : python3-pyasn1-modules-0.4.8-7.el9.noarch 51/96 2026-02-24T07:40:19.876 INFO:teuthology.orchestra.run.trial091.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 52/96 2026-02-24T07:40:19.876 INFO:teuthology.orchestra.run.trial091.stdout: Verifying : python3-scipy-1.9.3-2.el9.x86_64 53/96 2026-02-24T07:40:19.876 INFO:teuthology.orchestra.run.trial091.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 54/96 2026-02-24T07:40:19.876 INFO:teuthology.orchestra.run.trial091.stdout: Verifying : xmlsec1-1.2.29-13.el9.x86_64 55/96 2026-02-24T07:40:19.877 INFO:teuthology.orchestra.run.trial091.stdout: Verifying : xmlsec1-openssl-1.2.29-13.el9.x86_64 56/96 2026-02-24T07:40:19.877 INFO:teuthology.orchestra.run.trial091.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 57/96 2026-02-24T07:40:19.877 INFO:teuthology.orchestra.run.trial091.stdout: Verifying : protobuf-compiler-3.14.0-17.el9.x86_64 58/96 2026-02-24T07:40:19.877 INFO:teuthology.orchestra.run.trial091.stdout: Verifying : abseil-cpp-20211102.0-4.el9.x86_64 59/96 2026-02-24T07:40:19.877 INFO:teuthology.orchestra.run.trial091.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 60/96 2026-02-24T07:40:19.877 INFO:teuthology.orchestra.run.trial091.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 61/96 2026-02-24T07:40:19.877 INFO:teuthology.orchestra.run.trial091.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 62/96 2026-02-24T07:40:19.877 INFO:teuthology.orchestra.run.trial091.stdout: Verifying : python3-autocommand-2.2.2-8.el9.noarch 63/96 2026-02-24T07:40:19.877 INFO:teuthology.orchestra.run.trial091.stdout: Verifying : python3-backports-tarfile-1.2.0-1.el9.noarch 64/96 2026-02-24T07:40:19.877 INFO:teuthology.orchestra.run.trial091.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 65/96 2026-02-24T07:40:19.877 INFO:teuthology.orchestra.run.trial091.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 66/96 2026-02-24T07:40:19.877 INFO:teuthology.orchestra.run.trial091.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 67/96 2026-02-24T07:40:19.878 INFO:teuthology.orchestra.run.trial091.stdout: Verifying : python3-cheroot-10.0.1-4.el9.noarch 68/96 2026-02-24T07:40:19.878 INFO:teuthology.orchestra.run.trial091.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 69/96 2026-02-24T07:40:19.878 INFO:teuthology.orchestra.run.trial091.stdout: Verifying : python3-google-auth-1:2.45.0-1.el9.noarch 70/96 2026-02-24T07:40:19.878 INFO:teuthology.orchestra.run.trial091.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 71/96 2026-02-24T07:40:19.878 INFO:teuthology.orchestra.run.trial091.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 72/96 2026-02-24T07:40:19.878 INFO:teuthology.orchestra.run.trial091.stdout: Verifying : python3-influxdb-5.3.1-1.el9.noarch 73/96 2026-02-24T07:40:19.878 INFO:teuthology.orchestra.run.trial091.stdout: Verifying : python3-isodate-0.6.1-3.el9.noarch 74/96 2026-02-24T07:40:19.878 INFO:teuthology.orchestra.run.trial091.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 75/96 2026-02-24T07:40:19.878 INFO:teuthology.orchestra.run.trial091.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 76/96 2026-02-24T07:40:19.878 INFO:teuthology.orchestra.run.trial091.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 77/96 2026-02-24T07:40:19.878 INFO:teuthology.orchestra.run.trial091.stdout: Verifying : python3-jaraco-context-6.0.1-3.el9.noarch 78/96 2026-02-24T07:40:19.878 INFO:teuthology.orchestra.run.trial091.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 79/96 2026-02-24T07:40:19.878 INFO:teuthology.orchestra.run.trial091.stdout: Verifying : python3-jaraco-text-4.0.0-2.el9.noarch 80/96 2026-02-24T07:40:19.879 INFO:teuthology.orchestra.run.trial091.stdout: Verifying : python3-kubernetes-1:26.1.0-3.el9.noarch 81/96 2026-02-24T07:40:19.879 INFO:teuthology.orchestra.run.trial091.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 82/96 2026-02-24T07:40:19.879 INFO:teuthology.orchestra.run.trial091.stdout: Verifying : python3-msgpack-1.0.3-2.el9.x86_64 83/96 2026-02-24T07:40:19.879 INFO:teuthology.orchestra.run.trial091.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 84/96 2026-02-24T07:40:19.879 INFO:teuthology.orchestra.run.trial091.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 85/96 2026-02-24T07:40:19.879 INFO:teuthology.orchestra.run.trial091.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 86/96 2026-02-24T07:40:19.879 INFO:teuthology.orchestra.run.trial091.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 87/96 2026-02-24T07:40:19.879 INFO:teuthology.orchestra.run.trial091.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 88/96 2026-02-24T07:40:19.879 INFO:teuthology.orchestra.run.trial091.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 89/96 2026-02-24T07:40:19.879 INFO:teuthology.orchestra.run.trial091.stdout: Verifying : python3-saml-1.16.0-1.el9.noarch 90/96 2026-02-24T07:40:19.879 INFO:teuthology.orchestra.run.trial091.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 91/96 2026-02-24T07:40:19.879 INFO:teuthology.orchestra.run.trial091.stdout: Verifying : python3-typing-extensions-4.15.0-1.el9.noarch 92/96 2026-02-24T07:40:19.880 INFO:teuthology.orchestra.run.trial091.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 93/96 2026-02-24T07:40:19.880 INFO:teuthology.orchestra.run.trial091.stdout: Verifying : python3-xmlsec-1.3.13-1.el9.x86_64 94/96 2026-02-24T07:40:19.880 INFO:teuthology.orchestra.run.trial091.stdout: Verifying : python3-xmltodict-0.12.0-15.el9.noarch 95/96 2026-02-24T07:40:19.942 INFO:teuthology.orchestra.run.trial091.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 96/96 2026-02-24T07:40:19.942 INFO:teuthology.orchestra.run.trial091.stdout: 2026-02-24T07:40:19.942 INFO:teuthology.orchestra.run.trial091.stdout:Installed: 2026-02-24T07:40:19.942 INFO:teuthology.orchestra.run.trial091.stdout: abseil-cpp-20211102.0-4.el9.x86_64 2026-02-24T07:40:19.942 INFO:teuthology.orchestra.run.trial091.stdout: ceph-2:20.3.0-5440.gff63f3d9.el9.x86_64 2026-02-24T07:40:19.942 INFO:teuthology.orchestra.run.trial091.stdout: ceph-grafana-dashboards-2:20.3.0-5440.gff63f3d9.el9.noarch 2026-02-24T07:40:19.943 INFO:teuthology.orchestra.run.trial091.stdout: ceph-mds-2:20.3.0-5440.gff63f3d9.el9.x86_64 2026-02-24T07:40:19.943 INFO:teuthology.orchestra.run.trial091.stdout: ceph-mgr-2:20.3.0-5440.gff63f3d9.el9.x86_64 2026-02-24T07:40:19.943 INFO:teuthology.orchestra.run.trial091.stdout: ceph-mgr-cephadm-2:20.3.0-5440.gff63f3d9.el9.noarch 2026-02-24T07:40:19.943 INFO:teuthology.orchestra.run.trial091.stdout: ceph-mgr-dashboard-2:20.3.0-5440.gff63f3d9.el9.noarch 2026-02-24T07:40:19.943 INFO:teuthology.orchestra.run.trial091.stdout: ceph-mgr-diskprediction-local-2:20.3.0-5440.gff63f3d9.el9.noarch 2026-02-24T07:40:19.943 INFO:teuthology.orchestra.run.trial091.stdout: ceph-mgr-k8sevents-2:20.3.0-5440.gff63f3d9.el9.noarch 2026-02-24T07:40:19.943 INFO:teuthology.orchestra.run.trial091.stdout: ceph-mgr-modules-core-2:20.3.0-5440.gff63f3d9.el9.noarch 2026-02-24T07:40:19.943 INFO:teuthology.orchestra.run.trial091.stdout: ceph-mgr-rook-2:20.3.0-5440.gff63f3d9.el9.noarch 2026-02-24T07:40:19.943 INFO:teuthology.orchestra.run.trial091.stdout: ceph-mon-2:20.3.0-5440.gff63f3d9.el9.x86_64 2026-02-24T07:40:19.943 INFO:teuthology.orchestra.run.trial091.stdout: ceph-osd-2:20.3.0-5440.gff63f3d9.el9.x86_64 2026-02-24T07:40:19.943 INFO:teuthology.orchestra.run.trial091.stdout: ceph-osd-classic-2:20.3.0-5440.gff63f3d9.el9.x86_64 2026-02-24T07:40:19.944 INFO:teuthology.orchestra.run.trial091.stdout: ceph-osd-crimson-2:20.3.0-5440.gff63f3d9.el9.x86_64 2026-02-24T07:40:19.944 INFO:teuthology.orchestra.run.trial091.stdout: ceph-prometheus-alerts-2:20.3.0-5440.gff63f3d9.el9.noarch 2026-02-24T07:40:19.944 INFO:teuthology.orchestra.run.trial091.stdout: ceph-volume-2:20.3.0-5440.gff63f3d9.el9.noarch 2026-02-24T07:40:19.944 INFO:teuthology.orchestra.run.trial091.stdout: cephadm-2:20.3.0-5440.gff63f3d9.el9.noarch 2026-02-24T07:40:19.944 INFO:teuthology.orchestra.run.trial091.stdout: cryptsetup-2.8.1-3.el9.x86_64 2026-02-24T07:40:19.944 INFO:teuthology.orchestra.run.trial091.stdout: flexiblas-3.0.4-9.el9.x86_64 2026-02-24T07:40:19.944 INFO:teuthology.orchestra.run.trial091.stdout: flexiblas-netlib-3.0.4-9.el9.x86_64 2026-02-24T07:40:19.944 INFO:teuthology.orchestra.run.trial091.stdout: flexiblas-openblas-openmp-3.0.4-9.el9.x86_64 2026-02-24T07:40:19.944 INFO:teuthology.orchestra.run.trial091.stdout: grpc-data-1.46.7-10.el9.noarch 2026-02-24T07:40:19.944 INFO:teuthology.orchestra.run.trial091.stdout: ledmon-libs-1.1.0-3.el9.x86_64 2026-02-24T07:40:19.945 INFO:teuthology.orchestra.run.trial091.stdout: libconfig-1.7.2-9.el9.x86_64 2026-02-24T07:40:19.945 INFO:teuthology.orchestra.run.trial091.stdout: libgfortran-11.5.0-14.el9.x86_64 2026-02-24T07:40:19.945 INFO:teuthology.orchestra.run.trial091.stdout: libquadmath-11.5.0-14.el9.x86_64 2026-02-24T07:40:19.945 INFO:teuthology.orchestra.run.trial091.stdout: libstoragemgmt-1.10.1-1.el9.x86_64 2026-02-24T07:40:19.945 INFO:teuthology.orchestra.run.trial091.stdout: lua-5.4.4-4.el9.x86_64 2026-02-24T07:40:19.945 INFO:teuthology.orchestra.run.trial091.stdout: lua-devel-5.4.4-4.el9.x86_64 2026-02-24T07:40:19.945 INFO:teuthology.orchestra.run.trial091.stdout: luarocks-3.9.2-5.el9.noarch 2026-02-24T07:40:19.945 INFO:teuthology.orchestra.run.trial091.stdout: openblas-0.3.29-1.el9.x86_64 2026-02-24T07:40:19.945 INFO:teuthology.orchestra.run.trial091.stdout: openblas-openmp-0.3.29-1.el9.x86_64 2026-02-24T07:40:19.945 INFO:teuthology.orchestra.run.trial091.stdout: protobuf-compiler-3.14.0-17.el9.x86_64 2026-02-24T07:40:19.945 INFO:teuthology.orchestra.run.trial091.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2026-02-24T07:40:19.946 INFO:teuthology.orchestra.run.trial091.stdout: python3-autocommand-2.2.2-8.el9.noarch 2026-02-24T07:40:19.946 INFO:teuthology.orchestra.run.trial091.stdout: python3-babel-2.9.1-2.el9.noarch 2026-02-24T07:40:19.946 INFO:teuthology.orchestra.run.trial091.stdout: python3-backports-tarfile-1.2.0-1.el9.noarch 2026-02-24T07:40:19.946 INFO:teuthology.orchestra.run.trial091.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2026-02-24T07:40:19.946 INFO:teuthology.orchestra.run.trial091.stdout: python3-cachetools-4.2.4-1.el9.noarch 2026-02-24T07:40:19.946 INFO:teuthology.orchestra.run.trial091.stdout: python3-certifi-2023.05.07-4.el9.noarch 2026-02-24T07:40:19.946 INFO:teuthology.orchestra.run.trial091.stdout: python3-cffi-1.14.5-5.el9.x86_64 2026-02-24T07:40:19.946 INFO:teuthology.orchestra.run.trial091.stdout: python3-cheroot-10.0.1-4.el9.noarch 2026-02-24T07:40:19.946 INFO:teuthology.orchestra.run.trial091.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2026-02-24T07:40:19.946 INFO:teuthology.orchestra.run.trial091.stdout: python3-cryptography-36.0.1-5.el9.x86_64 2026-02-24T07:40:19.947 INFO:teuthology.orchestra.run.trial091.stdout: python3-devel-3.9.25-3.el9.x86_64 2026-02-24T07:40:19.947 INFO:teuthology.orchestra.run.trial091.stdout: python3-google-auth-1:2.45.0-1.el9.noarch 2026-02-24T07:40:19.947 INFO:teuthology.orchestra.run.trial091.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2026-02-24T07:40:19.947 INFO:teuthology.orchestra.run.trial091.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2026-02-24T07:40:19.947 INFO:teuthology.orchestra.run.trial091.stdout: python3-influxdb-5.3.1-1.el9.noarch 2026-02-24T07:40:19.947 INFO:teuthology.orchestra.run.trial091.stdout: python3-isodate-0.6.1-3.el9.noarch 2026-02-24T07:40:19.947 INFO:teuthology.orchestra.run.trial091.stdout: python3-jaraco-8.2.1-3.el9.noarch 2026-02-24T07:40:19.947 INFO:teuthology.orchestra.run.trial091.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2026-02-24T07:40:19.947 INFO:teuthology.orchestra.run.trial091.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2026-02-24T07:40:19.948 INFO:teuthology.orchestra.run.trial091.stdout: python3-jaraco-context-6.0.1-3.el9.noarch 2026-02-24T07:40:19.948 INFO:teuthology.orchestra.run.trial091.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2026-02-24T07:40:19.948 INFO:teuthology.orchestra.run.trial091.stdout: python3-jaraco-text-4.0.0-2.el9.noarch 2026-02-24T07:40:19.948 INFO:teuthology.orchestra.run.trial091.stdout: python3-jinja2-2.11.3-8.el9.noarch 2026-02-24T07:40:19.948 INFO:teuthology.orchestra.run.trial091.stdout: python3-jmespath-1.0.1-1.el9.noarch 2026-02-24T07:40:19.948 INFO:teuthology.orchestra.run.trial091.stdout: python3-kubernetes-1:26.1.0-3.el9.noarch 2026-02-24T07:40:19.948 INFO:teuthology.orchestra.run.trial091.stdout: python3-libstoragemgmt-1.10.1-1.el9.x86_64 2026-02-24T07:40:19.948 INFO:teuthology.orchestra.run.trial091.stdout: python3-lxml-4.6.5-3.el9.x86_64 2026-02-24T07:40:19.948 INFO:teuthology.orchestra.run.trial091.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2026-02-24T07:40:19.948 INFO:teuthology.orchestra.run.trial091.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2026-02-24T07:40:19.948 INFO:teuthology.orchestra.run.trial091.stdout: python3-msgpack-1.0.3-2.el9.x86_64 2026-02-24T07:40:19.948 INFO:teuthology.orchestra.run.trial091.stdout: python3-natsort-7.1.1-5.el9.noarch 2026-02-24T07:40:19.948 INFO:teuthology.orchestra.run.trial091.stdout: python3-numpy-1:1.23.5-2.el9.x86_64 2026-02-24T07:40:19.949 INFO:teuthology.orchestra.run.trial091.stdout: python3-numpy-f2py-1:1.23.5-2.el9.x86_64 2026-02-24T07:40:19.949 INFO:teuthology.orchestra.run.trial091.stdout: python3-packaging-20.9-5.el9.noarch 2026-02-24T07:40:19.949 INFO:teuthology.orchestra.run.trial091.stdout: python3-ply-3.11-14.el9.noarch 2026-02-24T07:40:19.949 INFO:teuthology.orchestra.run.trial091.stdout: python3-portend-3.1.0-2.el9.noarch 2026-02-24T07:40:19.949 INFO:teuthology.orchestra.run.trial091.stdout: python3-protobuf-3.14.0-17.el9.noarch 2026-02-24T07:40:19.949 INFO:teuthology.orchestra.run.trial091.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2026-02-24T07:40:19.949 INFO:teuthology.orchestra.run.trial091.stdout: python3-pyasn1-0.4.8-7.el9.noarch 2026-02-24T07:40:19.949 INFO:teuthology.orchestra.run.trial091.stdout: python3-pyasn1-modules-0.4.8-7.el9.noarch 2026-02-24T07:40:19.949 INFO:teuthology.orchestra.run.trial091.stdout: python3-pycparser-2.20-6.el9.noarch 2026-02-24T07:40:19.949 INFO:teuthology.orchestra.run.trial091.stdout: python3-pyparsing-2.4.7-9.el9.noarch 2026-02-24T07:40:19.950 INFO:teuthology.orchestra.run.trial091.stdout: python3-repoze-lru-0.7-16.el9.noarch 2026-02-24T07:40:19.950 INFO:teuthology.orchestra.run.trial091.stdout: python3-requests-2.25.1-10.el9.noarch 2026-02-24T07:40:19.950 INFO:teuthology.orchestra.run.trial091.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2026-02-24T07:40:19.950 INFO:teuthology.orchestra.run.trial091.stdout: python3-routes-2.5.1-5.el9.noarch 2026-02-24T07:40:19.950 INFO:teuthology.orchestra.run.trial091.stdout: python3-rsa-4.9-2.el9.noarch 2026-02-24T07:40:19.950 INFO:teuthology.orchestra.run.trial091.stdout: python3-saml-1.16.0-1.el9.noarch 2026-02-24T07:40:19.950 INFO:teuthology.orchestra.run.trial091.stdout: python3-scipy-1.9.3-2.el9.x86_64 2026-02-24T07:40:19.950 INFO:teuthology.orchestra.run.trial091.stdout: python3-tempora-5.0.0-2.el9.noarch 2026-02-24T07:40:19.950 INFO:teuthology.orchestra.run.trial091.stdout: python3-toml-0.10.2-6.el9.noarch 2026-02-24T07:40:19.950 INFO:teuthology.orchestra.run.trial091.stdout: python3-typing-extensions-4.15.0-1.el9.noarch 2026-02-24T07:40:19.950 INFO:teuthology.orchestra.run.trial091.stdout: python3-urllib3-1.26.5-7.el9.noarch 2026-02-24T07:40:19.951 INFO:teuthology.orchestra.run.trial091.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2026-02-24T07:40:19.951 INFO:teuthology.orchestra.run.trial091.stdout: python3-xmlsec-1.3.13-1.el9.x86_64 2026-02-24T07:40:19.951 INFO:teuthology.orchestra.run.trial091.stdout: python3-xmltodict-0.12.0-15.el9.noarch 2026-02-24T07:40:19.951 INFO:teuthology.orchestra.run.trial091.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2026-02-24T07:40:19.951 INFO:teuthology.orchestra.run.trial091.stdout: unzip-6.0-59.el9.x86_64 2026-02-24T07:40:19.951 INFO:teuthology.orchestra.run.trial091.stdout: xmlsec1-1.2.29-13.el9.x86_64 2026-02-24T07:40:19.951 INFO:teuthology.orchestra.run.trial091.stdout: xmlsec1-openssl-1.2.29-13.el9.x86_64 2026-02-24T07:40:19.951 INFO:teuthology.orchestra.run.trial091.stdout: zip-3.0-35.el9.x86_64 2026-02-24T07:40:19.951 INFO:teuthology.orchestra.run.trial091.stdout: 2026-02-24T07:40:19.951 INFO:teuthology.orchestra.run.trial091.stdout:Complete! 2026-02-24T07:40:20.008 DEBUG:teuthology.orchestra.run.trial091:> sudo yum -y install ceph-base 2026-02-24T07:40:20.330 INFO:teuthology.orchestra.run.trial091.stdout:Last metadata expiration check: 0:01:18 ago on Tue 24 Feb 2026 07:39:02 AM UTC. 2026-02-24T07:40:20.394 INFO:teuthology.orchestra.run.trial091.stdout:Package ceph-base-2:20.3.0-5440.gff63f3d9.el9.x86_64 is already installed. 2026-02-24T07:40:20.413 INFO:teuthology.orchestra.run.trial091.stdout:Dependencies resolved. 2026-02-24T07:40:20.413 INFO:teuthology.orchestra.run.trial091.stdout:Nothing to do. 2026-02-24T07:40:20.414 INFO:teuthology.orchestra.run.trial091.stdout:Complete! 2026-02-24T07:40:20.435 DEBUG:teuthology.orchestra.run.trial091:> sudo yum -y install cephadm 2026-02-24T07:40:20.746 INFO:teuthology.orchestra.run.trial091.stdout:Last metadata expiration check: 0:01:18 ago on Tue 24 Feb 2026 07:39:02 AM UTC. 2026-02-24T07:40:20.811 INFO:teuthology.orchestra.run.trial091.stdout:Package cephadm-2:20.3.0-5440.gff63f3d9.el9.noarch is already installed. 2026-02-24T07:40:20.830 INFO:teuthology.orchestra.run.trial091.stdout:Dependencies resolved. 2026-02-24T07:40:20.830 INFO:teuthology.orchestra.run.trial091.stdout:Nothing to do. 2026-02-24T07:40:20.830 INFO:teuthology.orchestra.run.trial091.stdout:Complete! 2026-02-24T07:40:20.851 DEBUG:teuthology.orchestra.run.trial091:> sudo yum -y install ceph-immutable-object-cache 2026-02-24T07:40:21.165 INFO:teuthology.orchestra.run.trial091.stdout:Last metadata expiration check: 0:01:19 ago on Tue 24 Feb 2026 07:39:02 AM UTC. 2026-02-24T07:40:21.249 INFO:teuthology.orchestra.run.trial091.stdout:Dependencies resolved. 2026-02-24T07:40:21.249 INFO:teuthology.orchestra.run.trial091.stdout:================================================================================ 2026-02-24T07:40:21.249 INFO:teuthology.orchestra.run.trial091.stdout: Package Arch Version Repo Size 2026-02-24T07:40:21.249 INFO:teuthology.orchestra.run.trial091.stdout:================================================================================ 2026-02-24T07:40:21.249 INFO:teuthology.orchestra.run.trial091.stdout:Installing: 2026-02-24T07:40:21.249 INFO:teuthology.orchestra.run.trial091.stdout: ceph-immutable-object-cache x86_64 2:20.3.0-5440.gff63f3d9.el9 ceph 149 k 2026-02-24T07:40:21.249 INFO:teuthology.orchestra.run.trial091.stdout: 2026-02-24T07:40:21.249 INFO:teuthology.orchestra.run.trial091.stdout:Transaction Summary 2026-02-24T07:40:21.249 INFO:teuthology.orchestra.run.trial091.stdout:================================================================================ 2026-02-24T07:40:21.249 INFO:teuthology.orchestra.run.trial091.stdout:Install 1 Package 2026-02-24T07:40:21.249 INFO:teuthology.orchestra.run.trial091.stdout: 2026-02-24T07:40:21.249 INFO:teuthology.orchestra.run.trial091.stdout:Total download size: 149 k 2026-02-24T07:40:21.249 INFO:teuthology.orchestra.run.trial091.stdout:Installed size: 455 k 2026-02-24T07:40:21.249 INFO:teuthology.orchestra.run.trial091.stdout:Downloading Packages: 2026-02-24T07:40:21.444 INFO:teuthology.orchestra.run.trial091.stdout:ceph-immutable-object-cache-20.3.0-5440.gff63f3 772 kB/s | 149 kB 00:00 2026-02-24T07:40:21.444 INFO:teuthology.orchestra.run.trial091.stdout:-------------------------------------------------------------------------------- 2026-02-24T07:40:21.444 INFO:teuthology.orchestra.run.trial091.stdout:Total 768 kB/s | 149 kB 00:00 2026-02-24T07:40:21.444 INFO:teuthology.orchestra.run.trial091.stdout:Running transaction check 2026-02-24T07:40:21.449 INFO:teuthology.orchestra.run.trial091.stdout:Transaction check succeeded. 2026-02-24T07:40:21.449 INFO:teuthology.orchestra.run.trial091.stdout:Running transaction test 2026-02-24T07:40:21.480 INFO:teuthology.orchestra.run.trial091.stdout:Transaction test succeeded. 2026-02-24T07:40:21.480 INFO:teuthology.orchestra.run.trial091.stdout:Running transaction 2026-02-24T07:40:21.535 INFO:teuthology.orchestra.run.trial091.stdout: Preparing : 1/1 2026-02-24T07:40:21.536 INFO:teuthology.orchestra.run.trial091.stdout: Installing : ceph-immutable-object-cache-2:20.3.0-5440.gff63f3d9. 1/1 2026-02-24T07:40:21.559 INFO:teuthology.orchestra.run.trial091.stdout: Running scriptlet: ceph-immutable-object-cache-2:20.3.0-5440.gff63f3d9. 1/1 2026-02-24T07:40:21.560 INFO:teuthology.orchestra.run.trial091.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-24T07:40:21.560 INFO:teuthology.orchestra.run.trial091.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2026-02-24T07:40:21.560 INFO:teuthology.orchestra.run.trial091.stdout: 2026-02-24T07:40:21.867 INFO:teuthology.orchestra.run.trial091.stdout: Verifying : ceph-immutable-object-cache-2:20.3.0-5440.gff63f3d9. 1/1 2026-02-24T07:40:21.868 INFO:teuthology.orchestra.run.trial091.stdout: 2026-02-24T07:40:21.868 INFO:teuthology.orchestra.run.trial091.stdout:Installed: 2026-02-24T07:40:21.868 INFO:teuthology.orchestra.run.trial091.stdout: ceph-immutable-object-cache-2:20.3.0-5440.gff63f3d9.el9.x86_64 2026-02-24T07:40:21.868 INFO:teuthology.orchestra.run.trial091.stdout: 2026-02-24T07:40:21.868 INFO:teuthology.orchestra.run.trial091.stdout:Complete! 2026-02-24T07:40:21.896 DEBUG:teuthology.orchestra.run.trial091:> sudo yum -y install ceph-mgr 2026-02-24T07:40:22.210 INFO:teuthology.orchestra.run.trial091.stdout:Last metadata expiration check: 0:01:20 ago on Tue 24 Feb 2026 07:39:02 AM UTC. 2026-02-24T07:40:22.276 INFO:teuthology.orchestra.run.trial091.stdout:Package ceph-mgr-2:20.3.0-5440.gff63f3d9.el9.x86_64 is already installed. 2026-02-24T07:40:22.294 INFO:teuthology.orchestra.run.trial091.stdout:Dependencies resolved. 2026-02-24T07:40:22.294 INFO:teuthology.orchestra.run.trial091.stdout:Nothing to do. 2026-02-24T07:40:22.295 INFO:teuthology.orchestra.run.trial091.stdout:Complete! 2026-02-24T07:40:22.316 DEBUG:teuthology.orchestra.run.trial091:> sudo yum -y install ceph-mgr-dashboard 2026-02-24T07:40:22.625 INFO:teuthology.orchestra.run.trial091.stdout:Last metadata expiration check: 0:01:20 ago on Tue 24 Feb 2026 07:39:02 AM UTC. 2026-02-24T07:40:22.691 INFO:teuthology.orchestra.run.trial091.stdout:Package ceph-mgr-dashboard-2:20.3.0-5440.gff63f3d9.el9.noarch is already installed. 2026-02-24T07:40:22.709 INFO:teuthology.orchestra.run.trial091.stdout:Dependencies resolved. 2026-02-24T07:40:22.710 INFO:teuthology.orchestra.run.trial091.stdout:Nothing to do. 2026-02-24T07:40:22.710 INFO:teuthology.orchestra.run.trial091.stdout:Complete! 2026-02-24T07:40:22.732 DEBUG:teuthology.orchestra.run.trial091:> sudo yum -y install ceph-mgr-diskprediction-local 2026-02-24T07:40:23.049 INFO:teuthology.orchestra.run.trial091.stdout:Last metadata expiration check: 0:01:21 ago on Tue 24 Feb 2026 07:39:02 AM UTC. 2026-02-24T07:40:23.114 INFO:teuthology.orchestra.run.trial091.stdout:Package ceph-mgr-diskprediction-local-2:20.3.0-5440.gff63f3d9.el9.noarch is already installed. 2026-02-24T07:40:23.132 INFO:teuthology.orchestra.run.trial091.stdout:Dependencies resolved. 2026-02-24T07:40:23.133 INFO:teuthology.orchestra.run.trial091.stdout:Nothing to do. 2026-02-24T07:40:23.133 INFO:teuthology.orchestra.run.trial091.stdout:Complete! 2026-02-24T07:40:23.153 DEBUG:teuthology.orchestra.run.trial091:> sudo yum -y install ceph-mgr-rook 2026-02-24T07:40:23.466 INFO:teuthology.orchestra.run.trial091.stdout:Last metadata expiration check: 0:01:21 ago on Tue 24 Feb 2026 07:39:02 AM UTC. 2026-02-24T07:40:23.531 INFO:teuthology.orchestra.run.trial091.stdout:Package ceph-mgr-rook-2:20.3.0-5440.gff63f3d9.el9.noarch is already installed. 2026-02-24T07:40:23.550 INFO:teuthology.orchestra.run.trial091.stdout:Dependencies resolved. 2026-02-24T07:40:23.550 INFO:teuthology.orchestra.run.trial091.stdout:Nothing to do. 2026-02-24T07:40:23.550 INFO:teuthology.orchestra.run.trial091.stdout:Complete! 2026-02-24T07:40:23.571 DEBUG:teuthology.orchestra.run.trial091:> sudo yum -y install ceph-mgr-cephadm 2026-02-24T07:40:23.889 INFO:teuthology.orchestra.run.trial091.stdout:Last metadata expiration check: 0:01:21 ago on Tue 24 Feb 2026 07:39:02 AM UTC. 2026-02-24T07:40:23.954 INFO:teuthology.orchestra.run.trial091.stdout:Package ceph-mgr-cephadm-2:20.3.0-5440.gff63f3d9.el9.noarch is already installed. 2026-02-24T07:40:23.973 INFO:teuthology.orchestra.run.trial091.stdout:Dependencies resolved. 2026-02-24T07:40:23.974 INFO:teuthology.orchestra.run.trial091.stdout:Nothing to do. 2026-02-24T07:40:23.974 INFO:teuthology.orchestra.run.trial091.stdout:Complete! 2026-02-24T07:40:23.995 DEBUG:teuthology.orchestra.run.trial091:> sudo yum -y install ceph-osd 2026-02-24T07:40:24.308 INFO:teuthology.orchestra.run.trial091.stdout:Last metadata expiration check: 0:01:22 ago on Tue 24 Feb 2026 07:39:02 AM UTC. 2026-02-24T07:40:24.373 INFO:teuthology.orchestra.run.trial091.stdout:Package ceph-osd-2:20.3.0-5440.gff63f3d9.el9.x86_64 is already installed. 2026-02-24T07:40:24.392 INFO:teuthology.orchestra.run.trial091.stdout:Dependencies resolved. 2026-02-24T07:40:24.392 INFO:teuthology.orchestra.run.trial091.stdout:Nothing to do. 2026-02-24T07:40:24.392 INFO:teuthology.orchestra.run.trial091.stdout:Complete! 2026-02-24T07:40:24.414 DEBUG:teuthology.orchestra.run.trial091:> sudo yum -y install ceph-osd-classic 2026-02-24T07:40:24.732 INFO:teuthology.orchestra.run.trial091.stdout:Last metadata expiration check: 0:01:22 ago on Tue 24 Feb 2026 07:39:02 AM UTC. 2026-02-24T07:40:24.801 INFO:teuthology.orchestra.run.trial091.stdout:Package ceph-osd-classic-2:20.3.0-5440.gff63f3d9.el9.x86_64 is already installed. 2026-02-24T07:40:24.819 INFO:teuthology.orchestra.run.trial091.stdout:Dependencies resolved. 2026-02-24T07:40:24.820 INFO:teuthology.orchestra.run.trial091.stdout:Nothing to do. 2026-02-24T07:40:24.820 INFO:teuthology.orchestra.run.trial091.stdout:Complete! 2026-02-24T07:40:24.842 DEBUG:teuthology.orchestra.run.trial091:> sudo yum -y install ceph-fuse 2026-02-24T07:40:25.157 INFO:teuthology.orchestra.run.trial091.stdout:Last metadata expiration check: 0:01:23 ago on Tue 24 Feb 2026 07:39:02 AM UTC. 2026-02-24T07:40:25.240 INFO:teuthology.orchestra.run.trial091.stdout:Dependencies resolved. 2026-02-24T07:40:25.240 INFO:teuthology.orchestra.run.trial091.stdout:================================================================================ 2026-02-24T07:40:25.240 INFO:teuthology.orchestra.run.trial091.stdout: Package Arch Version Repo Size 2026-02-24T07:40:25.241 INFO:teuthology.orchestra.run.trial091.stdout:================================================================================ 2026-02-24T07:40:25.241 INFO:teuthology.orchestra.run.trial091.stdout:Installing: 2026-02-24T07:40:25.241 INFO:teuthology.orchestra.run.trial091.stdout: ceph-fuse x86_64 2:20.3.0-5440.gff63f3d9.el9 ceph 914 k 2026-02-24T07:40:25.241 INFO:teuthology.orchestra.run.trial091.stdout: 2026-02-24T07:40:25.241 INFO:teuthology.orchestra.run.trial091.stdout:Transaction Summary 2026-02-24T07:40:25.241 INFO:teuthology.orchestra.run.trial091.stdout:================================================================================ 2026-02-24T07:40:25.241 INFO:teuthology.orchestra.run.trial091.stdout:Install 1 Package 2026-02-24T07:40:25.241 INFO:teuthology.orchestra.run.trial091.stdout: 2026-02-24T07:40:25.241 INFO:teuthology.orchestra.run.trial091.stdout:Total download size: 914 k 2026-02-24T07:40:25.241 INFO:teuthology.orchestra.run.trial091.stdout:Installed size: 2.9 M 2026-02-24T07:40:25.241 INFO:teuthology.orchestra.run.trial091.stdout:Downloading Packages: 2026-02-24T07:40:25.578 INFO:teuthology.orchestra.run.trial091.stdout:ceph-fuse-20.3.0-5440.gff63f3d9.el9.x86_64.rpm 2.6 MB/s | 914 kB 00:00 2026-02-24T07:40:25.579 INFO:teuthology.orchestra.run.trial091.stdout:-------------------------------------------------------------------------------- 2026-02-24T07:40:25.579 INFO:teuthology.orchestra.run.trial091.stdout:Total 2.6 MB/s | 914 kB 00:00 2026-02-24T07:40:25.579 INFO:teuthology.orchestra.run.trial091.stdout:Running transaction check 2026-02-24T07:40:25.583 INFO:teuthology.orchestra.run.trial091.stdout:Transaction check succeeded. 2026-02-24T07:40:25.583 INFO:teuthology.orchestra.run.trial091.stdout:Running transaction test 2026-02-24T07:40:25.617 INFO:teuthology.orchestra.run.trial091.stdout:Transaction test succeeded. 2026-02-24T07:40:25.617 INFO:teuthology.orchestra.run.trial091.stdout:Running transaction 2026-02-24T07:40:25.708 INFO:teuthology.orchestra.run.trial091.stdout: Preparing : 1/1 2026-02-24T07:40:25.726 INFO:teuthology.orchestra.run.trial091.stdout: Installing : ceph-fuse-2:20.3.0-5440.gff63f3d9.el9.x86_64 1/1 2026-02-24T07:40:25.961 INFO:teuthology.orchestra.run.trial091.stdout: Running scriptlet: ceph-fuse-2:20.3.0-5440.gff63f3d9.el9.x86_64 1/1 2026-02-24T07:40:26.023 INFO:teuthology.orchestra.run.trial091.stdout: Verifying : ceph-fuse-2:20.3.0-5440.gff63f3d9.el9.x86_64 1/1 2026-02-24T07:40:26.024 INFO:teuthology.orchestra.run.trial091.stdout: 2026-02-24T07:40:26.024 INFO:teuthology.orchestra.run.trial091.stdout:Installed: 2026-02-24T07:40:26.024 INFO:teuthology.orchestra.run.trial091.stdout: ceph-fuse-2:20.3.0-5440.gff63f3d9.el9.x86_64 2026-02-24T07:40:26.024 INFO:teuthology.orchestra.run.trial091.stdout: 2026-02-24T07:40:26.024 INFO:teuthology.orchestra.run.trial091.stdout:Complete! 2026-02-24T07:40:26.052 DEBUG:teuthology.orchestra.run.trial091:> sudo yum -y install ceph-volume 2026-02-24T07:40:26.368 INFO:teuthology.orchestra.run.trial091.stdout:Last metadata expiration check: 0:01:24 ago on Tue 24 Feb 2026 07:39:02 AM UTC. 2026-02-24T07:40:26.432 INFO:teuthology.orchestra.run.trial091.stdout:Package ceph-volume-2:20.3.0-5440.gff63f3d9.el9.noarch is already installed. 2026-02-24T07:40:26.451 INFO:teuthology.orchestra.run.trial091.stdout:Dependencies resolved. 2026-02-24T07:40:26.452 INFO:teuthology.orchestra.run.trial091.stdout:Nothing to do. 2026-02-24T07:40:26.452 INFO:teuthology.orchestra.run.trial091.stdout:Complete! 2026-02-24T07:40:26.473 DEBUG:teuthology.orchestra.run.trial091:> sudo yum -y install librados-devel 2026-02-24T07:40:26.787 INFO:teuthology.orchestra.run.trial091.stdout:Last metadata expiration check: 0:01:24 ago on Tue 24 Feb 2026 07:39:02 AM UTC. 2026-02-24T07:40:26.871 INFO:teuthology.orchestra.run.trial091.stdout:Dependencies resolved. 2026-02-24T07:40:26.872 INFO:teuthology.orchestra.run.trial091.stdout:================================================================================ 2026-02-24T07:40:26.872 INFO:teuthology.orchestra.run.trial091.stdout: Package Arch Version Repo Size 2026-02-24T07:40:26.872 INFO:teuthology.orchestra.run.trial091.stdout:================================================================================ 2026-02-24T07:40:26.872 INFO:teuthology.orchestra.run.trial091.stdout:Installing: 2026-02-24T07:40:26.872 INFO:teuthology.orchestra.run.trial091.stdout: librados-devel x86_64 2:20.3.0-5440.gff63f3d9.el9 ceph 124 k 2026-02-24T07:40:26.872 INFO:teuthology.orchestra.run.trial091.stdout: 2026-02-24T07:40:26.872 INFO:teuthology.orchestra.run.trial091.stdout:Transaction Summary 2026-02-24T07:40:26.872 INFO:teuthology.orchestra.run.trial091.stdout:================================================================================ 2026-02-24T07:40:26.872 INFO:teuthology.orchestra.run.trial091.stdout:Install 1 Package 2026-02-24T07:40:26.872 INFO:teuthology.orchestra.run.trial091.stdout: 2026-02-24T07:40:26.872 INFO:teuthology.orchestra.run.trial091.stdout:Total download size: 124 k 2026-02-24T07:40:26.872 INFO:teuthology.orchestra.run.trial091.stdout:Installed size: 434 k 2026-02-24T07:40:26.872 INFO:teuthology.orchestra.run.trial091.stdout:Downloading Packages: 2026-02-24T07:40:26.958 INFO:teuthology.orchestra.run.trial091.stdout:librados-devel-20.3.0-5440.gff63f3d9.el9.x86_64 1.4 MB/s | 124 kB 00:00 2026-02-24T07:40:26.959 INFO:teuthology.orchestra.run.trial091.stdout:-------------------------------------------------------------------------------- 2026-02-24T07:40:26.959 INFO:teuthology.orchestra.run.trial091.stdout:Total 1.4 MB/s | 124 kB 00:00 2026-02-24T07:40:26.959 INFO:teuthology.orchestra.run.trial091.stdout:Running transaction check 2026-02-24T07:40:26.962 INFO:teuthology.orchestra.run.trial091.stdout:Transaction check succeeded. 2026-02-24T07:40:26.962 INFO:teuthology.orchestra.run.trial091.stdout:Running transaction test 2026-02-24T07:40:26.996 INFO:teuthology.orchestra.run.trial091.stdout:Transaction test succeeded. 2026-02-24T07:40:26.996 INFO:teuthology.orchestra.run.trial091.stdout:Running transaction 2026-02-24T07:40:27.050 INFO:teuthology.orchestra.run.trial091.stdout: Preparing : 1/1 2026-02-24T07:40:27.068 INFO:teuthology.orchestra.run.trial091.stdout: Installing : librados-devel-2:20.3.0-5440.gff63f3d9.el9.x86_64 1/1 2026-02-24T07:40:27.157 INFO:teuthology.orchestra.run.trial091.stdout: Running scriptlet: librados-devel-2:20.3.0-5440.gff63f3d9.el9.x86_64 1/1 2026-02-24T07:40:27.223 INFO:teuthology.orchestra.run.trial091.stdout: Verifying : librados-devel-2:20.3.0-5440.gff63f3d9.el9.x86_64 1/1 2026-02-24T07:40:27.223 INFO:teuthology.orchestra.run.trial091.stdout: 2026-02-24T07:40:27.223 INFO:teuthology.orchestra.run.trial091.stdout:Installed: 2026-02-24T07:40:27.223 INFO:teuthology.orchestra.run.trial091.stdout: librados-devel-2:20.3.0-5440.gff63f3d9.el9.x86_64 2026-02-24T07:40:27.223 INFO:teuthology.orchestra.run.trial091.stdout: 2026-02-24T07:40:27.223 INFO:teuthology.orchestra.run.trial091.stdout:Complete! 2026-02-24T07:40:27.251 DEBUG:teuthology.orchestra.run.trial091:> sudo yum -y install libcephfs2 2026-02-24T07:40:27.569 INFO:teuthology.orchestra.run.trial091.stdout:Last metadata expiration check: 0:01:25 ago on Tue 24 Feb 2026 07:39:02 AM UTC. 2026-02-24T07:40:27.635 INFO:teuthology.orchestra.run.trial091.stdout:Package libcephfs2-2:20.3.0-5440.gff63f3d9.el9.x86_64 is already installed. 2026-02-24T07:40:27.653 INFO:teuthology.orchestra.run.trial091.stdout:Dependencies resolved. 2026-02-24T07:40:27.654 INFO:teuthology.orchestra.run.trial091.stdout:Nothing to do. 2026-02-24T07:40:27.654 INFO:teuthology.orchestra.run.trial091.stdout:Complete! 2026-02-24T07:40:27.676 DEBUG:teuthology.orchestra.run.trial091:> sudo yum -y install libcephfs-devel 2026-02-24T07:40:27.994 INFO:teuthology.orchestra.run.trial091.stdout:Last metadata expiration check: 0:01:25 ago on Tue 24 Feb 2026 07:39:02 AM UTC. 2026-02-24T07:40:28.080 INFO:teuthology.orchestra.run.trial091.stdout:Dependencies resolved. 2026-02-24T07:40:28.080 INFO:teuthology.orchestra.run.trial091.stdout:================================================================================ 2026-02-24T07:40:28.080 INFO:teuthology.orchestra.run.trial091.stdout: Package Arch Version Repo Size 2026-02-24T07:40:28.080 INFO:teuthology.orchestra.run.trial091.stdout:================================================================================ 2026-02-24T07:40:28.080 INFO:teuthology.orchestra.run.trial091.stdout:Installing: 2026-02-24T07:40:28.080 INFO:teuthology.orchestra.run.trial091.stdout: libcephfs-devel x86_64 2:20.3.0-5440.gff63f3d9.el9 ceph 37 k 2026-02-24T07:40:28.081 INFO:teuthology.orchestra.run.trial091.stdout: 2026-02-24T07:40:28.081 INFO:teuthology.orchestra.run.trial091.stdout:Transaction Summary 2026-02-24T07:40:28.081 INFO:teuthology.orchestra.run.trial091.stdout:================================================================================ 2026-02-24T07:40:28.081 INFO:teuthology.orchestra.run.trial091.stdout:Install 1 Package 2026-02-24T07:40:28.081 INFO:teuthology.orchestra.run.trial091.stdout: 2026-02-24T07:40:28.081 INFO:teuthology.orchestra.run.trial091.stdout:Total download size: 37 k 2026-02-24T07:40:28.081 INFO:teuthology.orchestra.run.trial091.stdout:Installed size: 171 k 2026-02-24T07:40:28.081 INFO:teuthology.orchestra.run.trial091.stdout:Downloading Packages: 2026-02-24T07:40:28.143 INFO:teuthology.orchestra.run.trial091.stdout:libcephfs-devel-20.3.0-5440.gff63f3d9.el9.x86_6 594 kB/s | 37 kB 00:00 2026-02-24T07:40:28.144 INFO:teuthology.orchestra.run.trial091.stdout:-------------------------------------------------------------------------------- 2026-02-24T07:40:28.144 INFO:teuthology.orchestra.run.trial091.stdout:Total 587 kB/s | 37 kB 00:00 2026-02-24T07:40:28.144 INFO:teuthology.orchestra.run.trial091.stdout:Running transaction check 2026-02-24T07:40:28.146 INFO:teuthology.orchestra.run.trial091.stdout:Transaction check succeeded. 2026-02-24T07:40:28.146 INFO:teuthology.orchestra.run.trial091.stdout:Running transaction test 2026-02-24T07:40:28.148 INFO:teuthology.orchestra.run.trial091.stdout:Transaction test succeeded. 2026-02-24T07:40:28.148 INFO:teuthology.orchestra.run.trial091.stdout:Running transaction 2026-02-24T07:40:28.171 INFO:teuthology.orchestra.run.trial091.stdout: Preparing : 1/1 2026-02-24T07:40:28.188 INFO:teuthology.orchestra.run.trial091.stdout: Installing : libcephfs-devel-2:20.3.0-5440.gff63f3d9.el9.x86_64 1/1 2026-02-24T07:40:28.249 INFO:teuthology.orchestra.run.trial091.stdout: Running scriptlet: libcephfs-devel-2:20.3.0-5440.gff63f3d9.el9.x86_64 1/1 2026-02-24T07:40:28.311 INFO:teuthology.orchestra.run.trial091.stdout: Verifying : libcephfs-devel-2:20.3.0-5440.gff63f3d9.el9.x86_64 1/1 2026-02-24T07:40:28.311 INFO:teuthology.orchestra.run.trial091.stdout: 2026-02-24T07:40:28.311 INFO:teuthology.orchestra.run.trial091.stdout:Installed: 2026-02-24T07:40:28.311 INFO:teuthology.orchestra.run.trial091.stdout: libcephfs-devel-2:20.3.0-5440.gff63f3d9.el9.x86_64 2026-02-24T07:40:28.311 INFO:teuthology.orchestra.run.trial091.stdout: 2026-02-24T07:40:28.311 INFO:teuthology.orchestra.run.trial091.stdout:Complete! 2026-02-24T07:40:28.338 DEBUG:teuthology.orchestra.run.trial091:> sudo yum -y install librados2 2026-02-24T07:40:28.655 INFO:teuthology.orchestra.run.trial091.stdout:Last metadata expiration check: 0:01:26 ago on Tue 24 Feb 2026 07:39:02 AM UTC. 2026-02-24T07:40:28.720 INFO:teuthology.orchestra.run.trial091.stdout:Package librados2-2:20.3.0-5440.gff63f3d9.el9.x86_64 is already installed. 2026-02-24T07:40:28.738 INFO:teuthology.orchestra.run.trial091.stdout:Dependencies resolved. 2026-02-24T07:40:28.739 INFO:teuthology.orchestra.run.trial091.stdout:Nothing to do. 2026-02-24T07:40:28.739 INFO:teuthology.orchestra.run.trial091.stdout:Complete! 2026-02-24T07:40:28.761 DEBUG:teuthology.orchestra.run.trial091:> sudo yum -y install librbd1 2026-02-24T07:40:29.076 INFO:teuthology.orchestra.run.trial091.stdout:Last metadata expiration check: 0:01:27 ago on Tue 24 Feb 2026 07:39:02 AM UTC. 2026-02-24T07:40:29.141 INFO:teuthology.orchestra.run.trial091.stdout:Package librbd1-2:20.3.0-5440.gff63f3d9.el9.x86_64 is already installed. 2026-02-24T07:40:29.158 INFO:teuthology.orchestra.run.trial091.stdout:Dependencies resolved. 2026-02-24T07:40:29.159 INFO:teuthology.orchestra.run.trial091.stdout:Nothing to do. 2026-02-24T07:40:29.159 INFO:teuthology.orchestra.run.trial091.stdout:Complete! 2026-02-24T07:40:29.180 DEBUG:teuthology.orchestra.run.trial091:> sudo yum -y install python3-rados 2026-02-24T07:40:29.495 INFO:teuthology.orchestra.run.trial091.stdout:Last metadata expiration check: 0:01:27 ago on Tue 24 Feb 2026 07:39:02 AM UTC. 2026-02-24T07:40:29.560 INFO:teuthology.orchestra.run.trial091.stdout:Package python3-rados-2:20.3.0-5440.gff63f3d9.el9.x86_64 is already installed. 2026-02-24T07:40:29.577 INFO:teuthology.orchestra.run.trial091.stdout:Dependencies resolved. 2026-02-24T07:40:29.578 INFO:teuthology.orchestra.run.trial091.stdout:Nothing to do. 2026-02-24T07:40:29.578 INFO:teuthology.orchestra.run.trial091.stdout:Complete! 2026-02-24T07:40:29.599 DEBUG:teuthology.orchestra.run.trial091:> sudo yum -y install python3-rgw 2026-02-24T07:40:29.915 INFO:teuthology.orchestra.run.trial091.stdout:Last metadata expiration check: 0:01:27 ago on Tue 24 Feb 2026 07:39:02 AM UTC. 2026-02-24T07:40:29.982 INFO:teuthology.orchestra.run.trial091.stdout:Package python3-rgw-2:20.3.0-5440.gff63f3d9.el9.x86_64 is already installed. 2026-02-24T07:40:30.000 INFO:teuthology.orchestra.run.trial091.stdout:Dependencies resolved. 2026-02-24T07:40:30.000 INFO:teuthology.orchestra.run.trial091.stdout:Nothing to do. 2026-02-24T07:40:30.000 INFO:teuthology.orchestra.run.trial091.stdout:Complete! 2026-02-24T07:40:30.021 DEBUG:teuthology.orchestra.run.trial091:> sudo yum -y install python3-cephfs 2026-02-24T07:40:30.335 INFO:teuthology.orchestra.run.trial091.stdout:Last metadata expiration check: 0:01:28 ago on Tue 24 Feb 2026 07:39:02 AM UTC. 2026-02-24T07:40:30.401 INFO:teuthology.orchestra.run.trial091.stdout:Package python3-cephfs-2:20.3.0-5440.gff63f3d9.el9.x86_64 is already installed. 2026-02-24T07:40:30.418 INFO:teuthology.orchestra.run.trial091.stdout:Dependencies resolved. 2026-02-24T07:40:30.418 INFO:teuthology.orchestra.run.trial091.stdout:Nothing to do. 2026-02-24T07:40:30.418 INFO:teuthology.orchestra.run.trial091.stdout:Complete! 2026-02-24T07:40:30.440 DEBUG:teuthology.orchestra.run.trial091:> sudo yum -y install python3-rbd 2026-02-24T07:40:30.756 INFO:teuthology.orchestra.run.trial091.stdout:Last metadata expiration check: 0:01:28 ago on Tue 24 Feb 2026 07:39:02 AM UTC. 2026-02-24T07:40:30.822 INFO:teuthology.orchestra.run.trial091.stdout:Package python3-rbd-2:20.3.0-5440.gff63f3d9.el9.x86_64 is already installed. 2026-02-24T07:40:30.839 INFO:teuthology.orchestra.run.trial091.stdout:Dependencies resolved. 2026-02-24T07:40:30.840 INFO:teuthology.orchestra.run.trial091.stdout:Nothing to do. 2026-02-24T07:40:30.840 INFO:teuthology.orchestra.run.trial091.stdout:Complete! 2026-02-24T07:40:30.862 DEBUG:teuthology.orchestra.run.trial091:> sudo yum -y install rbd-fuse 2026-02-24T07:40:31.181 INFO:teuthology.orchestra.run.trial091.stdout:Last metadata expiration check: 0:01:29 ago on Tue 24 Feb 2026 07:39:02 AM UTC. 2026-02-24T07:40:31.264 INFO:teuthology.orchestra.run.trial091.stdout:Dependencies resolved. 2026-02-24T07:40:31.265 INFO:teuthology.orchestra.run.trial091.stdout:================================================================================ 2026-02-24T07:40:31.265 INFO:teuthology.orchestra.run.trial091.stdout: Package Architecture Version Repository Size 2026-02-24T07:40:31.265 INFO:teuthology.orchestra.run.trial091.stdout:================================================================================ 2026-02-24T07:40:31.265 INFO:teuthology.orchestra.run.trial091.stdout:Installing: 2026-02-24T07:40:31.265 INFO:teuthology.orchestra.run.trial091.stdout: rbd-fuse x86_64 2:20.3.0-5440.gff63f3d9.el9 ceph 86 k 2026-02-24T07:40:31.265 INFO:teuthology.orchestra.run.trial091.stdout: 2026-02-24T07:40:31.265 INFO:teuthology.orchestra.run.trial091.stdout:Transaction Summary 2026-02-24T07:40:31.265 INFO:teuthology.orchestra.run.trial091.stdout:================================================================================ 2026-02-24T07:40:31.266 INFO:teuthology.orchestra.run.trial091.stdout:Install 1 Package 2026-02-24T07:40:31.266 INFO:teuthology.orchestra.run.trial091.stdout: 2026-02-24T07:40:31.266 INFO:teuthology.orchestra.run.trial091.stdout:Total download size: 86 k 2026-02-24T07:40:31.266 INFO:teuthology.orchestra.run.trial091.stdout:Installed size: 236 k 2026-02-24T07:40:31.266 INFO:teuthology.orchestra.run.trial091.stdout:Downloading Packages: 2026-02-24T07:40:31.341 INFO:teuthology.orchestra.run.trial091.stdout:rbd-fuse-20.3.0-5440.gff63f3d9.el9.x86_64.rpm 1.1 MB/s | 86 kB 00:00 2026-02-24T07:40:31.342 INFO:teuthology.orchestra.run.trial091.stdout:-------------------------------------------------------------------------------- 2026-02-24T07:40:31.342 INFO:teuthology.orchestra.run.trial091.stdout:Total 1.1 MB/s | 86 kB 00:00 2026-02-24T07:40:31.342 INFO:teuthology.orchestra.run.trial091.stdout:Running transaction check 2026-02-24T07:40:31.346 INFO:teuthology.orchestra.run.trial091.stdout:Transaction check succeeded. 2026-02-24T07:40:31.346 INFO:teuthology.orchestra.run.trial091.stdout:Running transaction test 2026-02-24T07:40:31.377 INFO:teuthology.orchestra.run.trial091.stdout:Transaction test succeeded. 2026-02-24T07:40:31.377 INFO:teuthology.orchestra.run.trial091.stdout:Running transaction 2026-02-24T07:40:31.429 INFO:teuthology.orchestra.run.trial091.stdout: Preparing : 1/1 2026-02-24T07:40:31.446 INFO:teuthology.orchestra.run.trial091.stdout: Installing : rbd-fuse-2:20.3.0-5440.gff63f3d9.el9.x86_64 1/1 2026-02-24T07:40:31.534 INFO:teuthology.orchestra.run.trial091.stdout: Running scriptlet: rbd-fuse-2:20.3.0-5440.gff63f3d9.el9.x86_64 1/1 2026-02-24T07:40:31.596 INFO:teuthology.orchestra.run.trial091.stdout: Verifying : rbd-fuse-2:20.3.0-5440.gff63f3d9.el9.x86_64 1/1 2026-02-24T07:40:31.596 INFO:teuthology.orchestra.run.trial091.stdout: 2026-02-24T07:40:31.596 INFO:teuthology.orchestra.run.trial091.stdout:Installed: 2026-02-24T07:40:31.596 INFO:teuthology.orchestra.run.trial091.stdout: rbd-fuse-2:20.3.0-5440.gff63f3d9.el9.x86_64 2026-02-24T07:40:31.596 INFO:teuthology.orchestra.run.trial091.stdout: 2026-02-24T07:40:31.596 INFO:teuthology.orchestra.run.trial091.stdout:Complete! 2026-02-24T07:40:31.624 DEBUG:teuthology.orchestra.run.trial091:> sudo yum -y install rbd-mirror 2026-02-24T07:40:31.941 INFO:teuthology.orchestra.run.trial091.stdout:Last metadata expiration check: 0:01:29 ago on Tue 24 Feb 2026 07:39:02 AM UTC. 2026-02-24T07:40:32.025 INFO:teuthology.orchestra.run.trial091.stdout:Dependencies resolved. 2026-02-24T07:40:32.025 INFO:teuthology.orchestra.run.trial091.stdout:================================================================================ 2026-02-24T07:40:32.025 INFO:teuthology.orchestra.run.trial091.stdout: Package Arch Version Repo Size 2026-02-24T07:40:32.025 INFO:teuthology.orchestra.run.trial091.stdout:================================================================================ 2026-02-24T07:40:32.025 INFO:teuthology.orchestra.run.trial091.stdout:Installing: 2026-02-24T07:40:32.025 INFO:teuthology.orchestra.run.trial091.stdout: rbd-mirror x86_64 2:20.3.0-5440.gff63f3d9.el9 ceph 2.6 M 2026-02-24T07:40:32.026 INFO:teuthology.orchestra.run.trial091.stdout: 2026-02-24T07:40:32.026 INFO:teuthology.orchestra.run.trial091.stdout:Transaction Summary 2026-02-24T07:40:32.026 INFO:teuthology.orchestra.run.trial091.stdout:================================================================================ 2026-02-24T07:40:32.026 INFO:teuthology.orchestra.run.trial091.stdout:Install 1 Package 2026-02-24T07:40:32.026 INFO:teuthology.orchestra.run.trial091.stdout: 2026-02-24T07:40:32.026 INFO:teuthology.orchestra.run.trial091.stdout:Total download size: 2.6 M 2026-02-24T07:40:32.026 INFO:teuthology.orchestra.run.trial091.stdout:Installed size: 11 M 2026-02-24T07:40:32.026 INFO:teuthology.orchestra.run.trial091.stdout:Downloading Packages: 2026-02-24T07:40:33.444 INFO:teuthology.orchestra.run.trial091.stdout:rbd-mirror-20.3.0-5440.gff63f3d9.el9.x86_64.rpm 1.8 MB/s | 2.6 MB 00:01 2026-02-24T07:40:33.444 INFO:teuthology.orchestra.run.trial091.stdout:-------------------------------------------------------------------------------- 2026-02-24T07:40:33.445 INFO:teuthology.orchestra.run.trial091.stdout:Total 1.8 MB/s | 2.6 MB 00:01 2026-02-24T07:40:33.445 INFO:teuthology.orchestra.run.trial091.stdout:Running transaction check 2026-02-24T07:40:33.450 INFO:teuthology.orchestra.run.trial091.stdout:Transaction check succeeded. 2026-02-24T07:40:33.450 INFO:teuthology.orchestra.run.trial091.stdout:Running transaction test 2026-02-24T07:40:33.489 INFO:teuthology.orchestra.run.trial091.stdout:Transaction test succeeded. 2026-02-24T07:40:33.490 INFO:teuthology.orchestra.run.trial091.stdout:Running transaction 2026-02-24T07:40:33.656 INFO:teuthology.orchestra.run.trial091.stdout: Preparing : 1/1 2026-02-24T07:40:33.657 INFO:teuthology.orchestra.run.trial091.stdout: Installing : rbd-mirror-2:20.3.0-5440.gff63f3d9.el9.x86_64 1/1 2026-02-24T07:40:33.680 INFO:teuthology.orchestra.run.trial091.stdout: Running scriptlet: rbd-mirror-2:20.3.0-5440.gff63f3d9.el9.x86_64 1/1 2026-02-24T07:40:33.680 INFO:teuthology.orchestra.run.trial091.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-24T07:40:33.680 INFO:teuthology.orchestra.run.trial091.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2026-02-24T07:40:33.680 INFO:teuthology.orchestra.run.trial091.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2026-02-24T07:40:33.681 INFO:teuthology.orchestra.run.trial091.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2026-02-24T07:40:33.681 INFO:teuthology.orchestra.run.trial091.stdout: 2026-02-24T07:40:33.982 INFO:teuthology.orchestra.run.trial091.stdout: Verifying : rbd-mirror-2:20.3.0-5440.gff63f3d9.el9.x86_64 1/1 2026-02-24T07:40:33.982 INFO:teuthology.orchestra.run.trial091.stdout: 2026-02-24T07:40:33.982 INFO:teuthology.orchestra.run.trial091.stdout:Installed: 2026-02-24T07:40:33.982 INFO:teuthology.orchestra.run.trial091.stdout: rbd-mirror-2:20.3.0-5440.gff63f3d9.el9.x86_64 2026-02-24T07:40:33.982 INFO:teuthology.orchestra.run.trial091.stdout: 2026-02-24T07:40:33.983 INFO:teuthology.orchestra.run.trial091.stdout:Complete! 2026-02-24T07:40:34.009 DEBUG:teuthology.orchestra.run.trial091:> sudo yum -y install rbd-nbd 2026-02-24T07:40:34.327 INFO:teuthology.orchestra.run.trial091.stdout:Last metadata expiration check: 0:01:32 ago on Tue 24 Feb 2026 07:39:02 AM UTC. 2026-02-24T07:40:34.411 INFO:teuthology.orchestra.run.trial091.stdout:Dependencies resolved. 2026-02-24T07:40:34.411 INFO:teuthology.orchestra.run.trial091.stdout:================================================================================ 2026-02-24T07:40:34.411 INFO:teuthology.orchestra.run.trial091.stdout: Package Architecture Version Repository Size 2026-02-24T07:40:34.411 INFO:teuthology.orchestra.run.trial091.stdout:================================================================================ 2026-02-24T07:40:34.411 INFO:teuthology.orchestra.run.trial091.stdout:Installing: 2026-02-24T07:40:34.411 INFO:teuthology.orchestra.run.trial091.stdout: rbd-nbd x86_64 2:20.3.0-5440.gff63f3d9.el9 ceph 166 k 2026-02-24T07:40:34.411 INFO:teuthology.orchestra.run.trial091.stdout: 2026-02-24T07:40:34.412 INFO:teuthology.orchestra.run.trial091.stdout:Transaction Summary 2026-02-24T07:40:34.412 INFO:teuthology.orchestra.run.trial091.stdout:================================================================================ 2026-02-24T07:40:34.412 INFO:teuthology.orchestra.run.trial091.stdout:Install 1 Package 2026-02-24T07:40:34.412 INFO:teuthology.orchestra.run.trial091.stdout: 2026-02-24T07:40:34.412 INFO:teuthology.orchestra.run.trial091.stdout:Total download size: 166 k 2026-02-24T07:40:34.412 INFO:teuthology.orchestra.run.trial091.stdout:Installed size: 496 k 2026-02-24T07:40:34.412 INFO:teuthology.orchestra.run.trial091.stdout:Downloading Packages: 2026-02-24T07:40:34.785 INFO:teuthology.orchestra.run.trial091.stdout:rbd-nbd-20.3.0-5440.gff63f3d9.el9.x86_64.rpm 445 kB/s | 166 kB 00:00 2026-02-24T07:40:34.786 INFO:teuthology.orchestra.run.trial091.stdout:-------------------------------------------------------------------------------- 2026-02-24T07:40:34.786 INFO:teuthology.orchestra.run.trial091.stdout:Total 443 kB/s | 166 kB 00:00 2026-02-24T07:40:34.787 INFO:teuthology.orchestra.run.trial091.stdout:Running transaction check 2026-02-24T07:40:34.791 INFO:teuthology.orchestra.run.trial091.stdout:Transaction check succeeded. 2026-02-24T07:40:34.791 INFO:teuthology.orchestra.run.trial091.stdout:Running transaction test 2026-02-24T07:40:34.823 INFO:teuthology.orchestra.run.trial091.stdout:Transaction test succeeded. 2026-02-24T07:40:34.823 INFO:teuthology.orchestra.run.trial091.stdout:Running transaction 2026-02-24T07:40:34.879 INFO:teuthology.orchestra.run.trial091.stdout: Preparing : 1/1 2026-02-24T07:40:34.896 INFO:teuthology.orchestra.run.trial091.stdout: Installing : rbd-nbd-2:20.3.0-5440.gff63f3d9.el9.x86_64 1/1 2026-02-24T07:40:34.985 INFO:teuthology.orchestra.run.trial091.stdout: Running scriptlet: rbd-nbd-2:20.3.0-5440.gff63f3d9.el9.x86_64 1/1 2026-02-24T07:40:35.047 INFO:teuthology.orchestra.run.trial091.stdout: Verifying : rbd-nbd-2:20.3.0-5440.gff63f3d9.el9.x86_64 1/1 2026-02-24T07:40:35.047 INFO:teuthology.orchestra.run.trial091.stdout: 2026-02-24T07:40:35.047 INFO:teuthology.orchestra.run.trial091.stdout:Installed: 2026-02-24T07:40:35.047 INFO:teuthology.orchestra.run.trial091.stdout: rbd-nbd-2:20.3.0-5440.gff63f3d9.el9.x86_64 2026-02-24T07:40:35.047 INFO:teuthology.orchestra.run.trial091.stdout: 2026-02-24T07:40:35.047 INFO:teuthology.orchestra.run.trial091.stdout:Complete! 2026-02-24T07:40:35.076 DEBUG:teuthology.orchestra.run.trial091:> sudo yum -y install ceph-osd-crimson 2026-02-24T07:40:35.395 INFO:teuthology.orchestra.run.trial091.stdout:Last metadata expiration check: 0:01:33 ago on Tue 24 Feb 2026 07:39:02 AM UTC. 2026-02-24T07:40:35.460 INFO:teuthology.orchestra.run.trial091.stdout:Package ceph-osd-crimson-2:20.3.0-5440.gff63f3d9.el9.x86_64 is already installed. 2026-02-24T07:40:35.477 INFO:teuthology.orchestra.run.trial091.stdout:Dependencies resolved. 2026-02-24T07:40:35.478 INFO:teuthology.orchestra.run.trial091.stdout:Nothing to do. 2026-02-24T07:40:35.478 INFO:teuthology.orchestra.run.trial091.stdout:Complete! 2026-02-24T07:40:35.501 DEBUG:teuthology.orchestra.run.trial091:> sudo yum -y install ceph-osd-crimson-debuginfo 2026-02-24T07:40:35.819 INFO:teuthology.orchestra.run.trial091.stdout:Last metadata expiration check: 0:01:33 ago on Tue 24 Feb 2026 07:39:02 AM UTC. 2026-02-24T07:40:35.901 INFO:teuthology.orchestra.run.trial091.stdout:Dependencies resolved. 2026-02-24T07:40:35.902 INFO:teuthology.orchestra.run.trial091.stdout:================================================================================ 2026-02-24T07:40:35.902 INFO:teuthology.orchestra.run.trial091.stdout: Package Arch Version Repo Size 2026-02-24T07:40:35.902 INFO:teuthology.orchestra.run.trial091.stdout:================================================================================ 2026-02-24T07:40:35.902 INFO:teuthology.orchestra.run.trial091.stdout:Installing: 2026-02-24T07:40:35.902 INFO:teuthology.orchestra.run.trial091.stdout: ceph-osd-crimson-debuginfo x86_64 2:20.3.0-5440.gff63f3d9.el9 ceph 429 M 2026-02-24T07:40:35.902 INFO:teuthology.orchestra.run.trial091.stdout:Installing weak dependencies: 2026-02-24T07:40:35.902 INFO:teuthology.orchestra.run.trial091.stdout: ceph-debugsource x86_64 2:20.3.0-5440.gff63f3d9.el9 ceph 12 M 2026-02-24T07:40:35.902 INFO:teuthology.orchestra.run.trial091.stdout: 2026-02-24T07:40:35.902 INFO:teuthology.orchestra.run.trial091.stdout:Transaction Summary 2026-02-24T07:40:35.902 INFO:teuthology.orchestra.run.trial091.stdout:================================================================================ 2026-02-24T07:40:35.902 INFO:teuthology.orchestra.run.trial091.stdout:Install 2 Packages 2026-02-24T07:40:35.902 INFO:teuthology.orchestra.run.trial091.stdout: 2026-02-24T07:40:35.902 INFO:teuthology.orchestra.run.trial091.stdout:Total download size: 441 M 2026-02-24T07:40:35.902 INFO:teuthology.orchestra.run.trial091.stdout:Installed size: 2.7 G 2026-02-24T07:40:35.903 INFO:teuthology.orchestra.run.trial091.stdout:Downloading Packages: 2026-02-24T07:40:39.278 INFO:teuthology.orchestra.run.trial091.stdout:(1/2): ceph-debugsource-20.3.0-5440.gff63f3d9.e 3.6 MB/s | 12 MB 00:03 2026-02-24T07:40:57.520 INFO:teuthology.orchestra.run.trial091.stdout:(2/2): ceph-osd-crimson-debuginfo-20.3.0-5440.g 20 MB/s | 429 MB 00:21 2026-02-24T07:40:57.521 INFO:teuthology.orchestra.run.trial091.stdout:-------------------------------------------------------------------------------- 2026-02-24T07:40:57.522 INFO:teuthology.orchestra.run.trial091.stdout:Total 20 MB/s | 441 MB 00:21 2026-02-24T07:40:57.522 INFO:teuthology.orchestra.run.trial091.stdout:Running transaction check 2026-02-24T07:40:57.526 INFO:teuthology.orchestra.run.trial091.stdout:Transaction check succeeded. 2026-02-24T07:40:57.527 INFO:teuthology.orchestra.run.trial091.stdout:Running transaction test 2026-02-24T07:40:58.888 INFO:teuthology.orchestra.run.trial091.stdout:Transaction test succeeded. 2026-02-24T07:40:58.889 INFO:teuthology.orchestra.run.trial091.stdout:Running transaction 2026-02-24T07:41:02.465 INFO:teuthology.orchestra.run.trial091.stdout: Preparing : 1/1 2026-02-24T07:41:24.630 INFO:teuthology.orchestra.run.trial091.stdout: Installing : ceph-debugsource-2:20.3.0-5440.gff63f3d9.el9.x86_64 1/2 2026-02-24T07:41:24.649 INFO:teuthology.orchestra.run.trial091.stdout: Installing : ceph-osd-crimson-debuginfo-2:20.3.0-5440.gff63f3d9.e 2/2 2026-02-24T07:41:26.620 INFO:teuthology.orchestra.run.trial091.stdout: Running scriptlet: ceph-osd-crimson-debuginfo-2:20.3.0-5440.gff63f3d9.e 2/2 2026-02-24T07:41:26.620 INFO:teuthology.orchestra.run.trial091.stdout: Verifying : ceph-debugsource-2:20.3.0-5440.gff63f3d9.el9.x86_64 1/2 2026-02-24T07:41:26.684 INFO:teuthology.orchestra.run.trial091.stdout: Verifying : ceph-osd-crimson-debuginfo-2:20.3.0-5440.gff63f3d9.e 2/2 2026-02-24T07:41:26.684 INFO:teuthology.orchestra.run.trial091.stdout: 2026-02-24T07:41:26.684 INFO:teuthology.orchestra.run.trial091.stdout:Installed: 2026-02-24T07:41:26.684 INFO:teuthology.orchestra.run.trial091.stdout: ceph-debugsource-2:20.3.0-5440.gff63f3d9.el9.x86_64 2026-02-24T07:41:26.684 INFO:teuthology.orchestra.run.trial091.stdout: ceph-osd-crimson-debuginfo-2:20.3.0-5440.gff63f3d9.el9.x86_64 2026-02-24T07:41:26.684 INFO:teuthology.orchestra.run.trial091.stdout: 2026-02-24T07:41:26.685 INFO:teuthology.orchestra.run.trial091.stdout:Complete! 2026-02-24T07:41:26.757 DEBUG:teuthology.parallel:result is None 2026-02-24T07:41:26.758 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=debug&distros=centos%2F9%2Fx86_64&sha1=ff63f3d997e38f69382fd23ac0206f07f7b05017 2026-02-24T07:41:26.908 DEBUG:teuthology.orchestra.run.trial035:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2026-02-24T07:41:26.935 INFO:teuthology.orchestra.run.trial035.stdout:20.3.0-5440.gff63f3d9.el9 2026-02-24T07:41:26.935 INFO:teuthology.packaging:The installed version of ceph is 20.3.0-5440.gff63f3d9.el9 2026-02-24T07:41:26.935 INFO:teuthology.task.install:The correct ceph version 20.3.0-5440.gff63f3d9 is installed. 2026-02-24T07:41:26.938 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=debug&distros=centos%2F9%2Fx86_64&sha1=ff63f3d997e38f69382fd23ac0206f07f7b05017 2026-02-24T07:41:27.099 DEBUG:teuthology.orchestra.run.trial091:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2026-02-24T07:41:27.120 INFO:teuthology.orchestra.run.trial091.stdout:20.3.0-5440.gff63f3d9.el9 2026-02-24T07:41:27.121 INFO:teuthology.packaging:The installed version of ceph is 20.3.0-5440.gff63f3d9.el9 2026-02-24T07:41:27.121 INFO:teuthology.task.install:The correct ceph version 20.3.0-5440.gff63f3d9 is installed. 2026-02-24T07:41:27.124 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2026-02-24T07:41:27.124 DEBUG:teuthology.orchestra.run.trial035:> set -ex 2026-02-24T07:41:27.124 DEBUG:teuthology.orchestra.run.trial035:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2026-02-24T07:41:27.153 DEBUG:teuthology.orchestra.run.trial091:> set -ex 2026-02-24T07:41:27.153 DEBUG:teuthology.orchestra.run.trial091:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2026-02-24T07:41:27.185 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2026-02-24T07:41:27.186 DEBUG:teuthology.orchestra.run.trial035:> set -ex 2026-02-24T07:41:27.186 DEBUG:teuthology.orchestra.run.trial035:> sudo dd of=/usr/bin/daemon-helper 2026-02-24T07:41:27.217 DEBUG:teuthology.orchestra.run.trial035:> sudo chmod a=rx -- /usr/bin/daemon-helper 2026-02-24T07:41:27.282 DEBUG:teuthology.orchestra.run.trial091:> set -ex 2026-02-24T07:41:27.282 DEBUG:teuthology.orchestra.run.trial091:> sudo dd of=/usr/bin/daemon-helper 2026-02-24T07:41:27.305 DEBUG:teuthology.orchestra.run.trial091:> sudo chmod a=rx -- /usr/bin/daemon-helper 2026-02-24T07:41:27.368 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2026-02-24T07:41:27.369 DEBUG:teuthology.orchestra.run.trial035:> set -ex 2026-02-24T07:41:27.369 DEBUG:teuthology.orchestra.run.trial035:> sudo dd of=/usr/bin/adjust-ulimits 2026-02-24T07:41:27.395 DEBUG:teuthology.orchestra.run.trial035:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2026-02-24T07:41:27.457 DEBUG:teuthology.orchestra.run.trial091:> set -ex 2026-02-24T07:41:27.457 DEBUG:teuthology.orchestra.run.trial091:> sudo dd of=/usr/bin/adjust-ulimits 2026-02-24T07:41:27.482 DEBUG:teuthology.orchestra.run.trial091:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2026-02-24T07:41:27.547 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2026-02-24T07:41:27.547 DEBUG:teuthology.orchestra.run.trial035:> set -ex 2026-02-24T07:41:27.547 DEBUG:teuthology.orchestra.run.trial035:> sudo dd of=/usr/bin/stdin-killer 2026-02-24T07:41:27.573 DEBUG:teuthology.orchestra.run.trial035:> sudo chmod a=rx -- /usr/bin/stdin-killer 2026-02-24T07:41:27.637 DEBUG:teuthology.orchestra.run.trial091:> set -ex 2026-02-24T07:41:27.637 DEBUG:teuthology.orchestra.run.trial091:> sudo dd of=/usr/bin/stdin-killer 2026-02-24T07:41:27.662 DEBUG:teuthology.orchestra.run.trial091:> sudo chmod a=rx -- /usr/bin/stdin-killer 2026-02-24T07:41:27.725 INFO:teuthology.run_tasks:Running task pexec... 2026-02-24T07:41:27.741 INFO:teuthology.task.pexec:Executing custom commands... 2026-02-24T07:41:27.742 DEBUG:teuthology.orchestra.run.trial035:> TESTDIR=/home/ubuntu/cephtest bash -s 2026-02-24T07:41:27.742 DEBUG:teuthology.orchestra.run.trial091:> TESTDIR=/home/ubuntu/cephtest bash -s 2026-02-24T07:41:27.745 INFO:teuthology.task.pexec:Running commands on host ubuntu@trial035.front.sepia.ceph.com 2026-02-24T07:41:27.746 INFO:teuthology.task.pexec:sudo update-alternatives --set ceph-osd /usr/bin/ceph-osd-crimson 2026-02-24T07:41:27.767 INFO:teuthology.task.pexec:Running commands on host ubuntu@trial091.front.sepia.ceph.com 2026-02-24T07:41:27.767 INFO:teuthology.task.pexec:sudo update-alternatives --set ceph-osd /usr/bin/ceph-osd-crimson 2026-02-24T07:41:27.795 DEBUG:teuthology.parallel:result is None 2026-02-24T07:41:27.817 DEBUG:teuthology.parallel:result is None 2026-02-24T07:41:27.817 INFO:teuthology.run_tasks:Running task cephadm... 2026-02-24T07:41:27.906 INFO:tasks.cephadm:Config: {'conf': {'mgr': {'debug mgr': 20, 'debug ms': 1, 'debug osd': 10}, 'client.0': {'debug ms': 1, 'debug objecter': 20, 'debug rados': 20}, 'global': {'crimson_allow_pg_split': False, 'enable experimental unrecoverable data corrupting features': 'crimson', 'ms client mode': 'crc', 'ms cluster mode': 'crc', 'ms mon client mode': 'crc', 'ms mon cluster mode': 'crc', 'ms mon service mode': 'crc', 'ms service mode': 'crc', 'osd_max_pg_log_entries': 2, 'osd_min_pg_log_entries': 1, 'osd_pg_log_trim_min': 0, 'osd_pool_default_min_size': 2, 'osd_pool_default_size': 3}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon min osdmap epochs': 50, 'mon osdmap full prune interval': 2, 'mon osdmap full prune min': 15, 'mon osdmap full prune txsize': 2, 'osd pool default crimson': True, 'paxos service trim min': 10}, 'osd': {'crimson cpu num': 4, 'crimson osd obc lru size': 10, 'debug ms': 20, 'debug osd': 20, 'debug seastore': 20, 'debug seastore backref': 20, 'debug seastore cache': 20, 'debug seastore cleaner': 20, 'debug seastore device': 20, 'debug seastore epm': 20, 'debug seastore fixedkv tree': 20, 'debug seastore journal': 20, 'debug seastore lba': 20, 'debug seastore odata': 20, 'debug seastore omap': 20, 'debug seastore onode': 20, 'debug seastore t': 20, 'debug seastore tm': 20, 'osd blocked scrub grace period': 3600, 'osd debug reject backfill probability': 0.3, 'osd delete sleep': 1, 'osd max backfills': 3, 'osd max markdown count': 1000, 'osd objectstore': 'seastore', 'osd scrub max interval': 120, 'osd scrub min interval': 60, 'osd shutdown pgref assert': True, 'osd snap trim sleep': 2, 'seastore cachepin size pershard': '64M', 'seastore main device type': 'RANDOM_BLOCK_SSD', 'seastore max concurrent transactions': 8}}, 'osd_type': 'crimson', 'crimson_compat': True, 'flavor': 'debug', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'but it is still running', 'objects unfound and apparently lost', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', 'SLOW_OPS', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'timeout on replica', 'late reservation from', 'MON_DOWN', 'OSDMAP_FLAGS', 'OSD_DOWN', 'PG_DEGRADED', 'PG_AVAILABILITY', 'POOL_APP_NOT_ENABLED', 'mons down', 'mon down', 'out of quorum', 'noscrub', 'nodeep-scrub', 'Degraded data redundancy', 'is down', 'osds down', 'pg .*? is .*?degraded.*?, acting', 'pg .*? is stuck', 'pg degraded', 'PG_BACKFILL_FULL', 'Low space hindering backfill .*? backfill_toofull', 'OSD_ROOT_DOWN', 'pgs degraded', 'pgs undersized', 'is active.*backfill_toofull.*'], 'sha1': 'ff63f3d997e38f69382fd23ac0206f07f7b05017'} 2026-02-24T07:41:27.906 INFO:tasks.cephadm:Cluster image is quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 2026-02-24T07:41:27.908 INFO:tasks.cephadm:Cluster fsid is 3a5983d2-1154-11f1-9c05-d404e6e7d460 2026-02-24T07:41:27.908 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2026-02-24T07:41:27.908 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '10.20.193.35'} 2026-02-24T07:41:27.908 INFO:tasks.cephadm:First mon is mon.a on trial035 2026-02-24T07:41:27.908 INFO:tasks.cephadm:First mgr is x 2026-02-24T07:41:27.908 INFO:tasks.cephadm:Normalizing hostnames... 2026-02-24T07:41:27.908 DEBUG:teuthology.orchestra.run.trial035:> sudo hostname $(hostname -s) 2026-02-24T07:41:27.934 DEBUG:teuthology.orchestra.run.trial091:> sudo hostname $(hostname -s) 2026-02-24T07:41:27.959 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra 2026-02-24T07:41:27.960 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=debug&distros=centos%2F9%2Fx86_64&sha1=ff63f3d997e38f69382fd23ac0206f07f7b05017 2026-02-24T07:41:28.088 INFO:tasks.cephadm:builder_project result: [{'url': 'https://1.chacra.ceph.com/r/ceph/wip-shraddhaag-enable-cephadm-tests/ff63f3d997e38f69382fd23ac0206f07f7b05017/centos/9/flavors/debug/', 'chacra_url': 'https://1.chacra.ceph.com/repos/ceph/wip-shraddhaag-enable-cephadm-tests/ff63f3d997e38f69382fd23ac0206f07f7b05017/centos/9/flavors/debug/', 'ref': 'wip-shraddhaag-enable-cephadm-tests', 'sha1': 'ff63f3d997e38f69382fd23ac0206f07f7b05017', 'distro': 'centos', 'distro_version': '9', 'distro_codename': None, 'modified': '2026-02-24 05:52:36.050311', 'status': 'ready', 'flavor': 'debug', 'project': 'ceph', 'archs': ['x86_64', 'source'], 'extra': {'version': '20.3.0-5440-gff63f3d9', 'package_manager_version': '20.3.0-5440.gff63f3d9', 'build_url': 'https://jenkins.ceph.com/job/ceph-dev-pipeline/3219/', 'root_build_cause': '', 'node_name': '10.20.192.97+toko07', 'job_name': 'ceph-dev-pipeline'}}] 2026-02-24T07:41:28.206 INFO:tasks.util.chacra:got chacra host 1.chacra.ceph.com, ref wip-shraddhaag-enable-cephadm-tests, sha1 ff63f3d997e38f69382fd23ac0206f07f7b05017 from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F9%2Fx86_64&flavor=debug&sha1=ff63f3d997e38f69382fd23ac0206f07f7b05017 2026-02-24T07:41:28.208 INFO:tasks.cephadm:Discovered cachra url: https://1.chacra.ceph.com/binaries/ceph/wip-shraddhaag-enable-cephadm-tests/ff63f3d997e38f69382fd23ac0206f07f7b05017/centos/9/x86_64/flavors/debug/cephadm 2026-02-24T07:41:28.208 INFO:tasks.cephadm:Downloading cephadm from url: https://1.chacra.ceph.com/binaries/ceph/wip-shraddhaag-enable-cephadm-tests/ff63f3d997e38f69382fd23ac0206f07f7b05017/centos/9/x86_64/flavors/debug/cephadm 2026-02-24T07:41:28.208 DEBUG:teuthology.orchestra.run.trial035:> curl --silent -L https://1.chacra.ceph.com/binaries/ceph/wip-shraddhaag-enable-cephadm-tests/ff63f3d997e38f69382fd23ac0206f07f7b05017/centos/9/x86_64/flavors/debug/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2026-02-24T07:41:28.964 INFO:teuthology.orchestra.run.trial035.stdout:-rw-r--r--. 1 ubuntu ubuntu 1076667 Feb 24 07:41 /home/ubuntu/cephtest/cephadm 2026-02-24T07:41:28.964 DEBUG:teuthology.orchestra.run.trial091:> curl --silent -L https://1.chacra.ceph.com/binaries/ceph/wip-shraddhaag-enable-cephadm-tests/ff63f3d997e38f69382fd23ac0206f07f7b05017/centos/9/x86_64/flavors/debug/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2026-02-24T07:41:29.734 INFO:teuthology.orchestra.run.trial091.stdout:-rw-r--r--. 1 ubuntu ubuntu 1076667 Feb 24 07:41 /home/ubuntu/cephtest/cephadm 2026-02-24T07:41:29.734 DEBUG:teuthology.orchestra.run.trial035:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2026-02-24T07:41:29.752 DEBUG:teuthology.orchestra.run.trial091:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2026-02-24T07:41:29.789 INFO:tasks.cephadm:Pulling image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 on all hosts... 2026-02-24T07:41:29.789 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 pull 2026-02-24T07:41:29.794 DEBUG:teuthology.orchestra.run.trial091:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 pull 2026-02-24T07:41:29.933 INFO:teuthology.orchestra.run.trial035.stderr:Pulling container image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017... 2026-02-24T07:41:29.960 INFO:teuthology.orchestra.run.trial091.stderr:Pulling container image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017... 2026-02-24T07:41:40.358 INFO:teuthology.orchestra.run.trial091.stdout:{ 2026-02-24T07:41:40.358 INFO:teuthology.orchestra.run.trial091.stdout: "ceph_version": "ceph version 20.3.0-5440-gff63f3d9 (ff63f3d997e38f69382fd23ac0206f07f7b05017) tentacle (dev)", 2026-02-24T07:41:40.359 INFO:teuthology.orchestra.run.trial091.stdout: "image_id": "3992f70354a3fdc3bbd8e07cbd698e424b71669298ffe35e8f0565a46973aa85", 2026-02-24T07:41:40.359 INFO:teuthology.orchestra.run.trial091.stdout: "repo_digests": [ 2026-02-24T07:41:40.359 INFO:teuthology.orchestra.run.trial091.stdout: "quay.ceph.io/ceph-ci/ceph@sha256:9a5db51bca6d139691bb8c7054be8834f3ba065d1c0aaaecbafb190e389fb148" 2026-02-24T07:41:40.359 INFO:teuthology.orchestra.run.trial091.stdout: ] 2026-02-24T07:41:40.359 INFO:teuthology.orchestra.run.trial091.stdout:} 2026-02-24T07:41:40.370 INFO:teuthology.orchestra.run.trial035.stdout:{ 2026-02-24T07:41:40.370 INFO:teuthology.orchestra.run.trial035.stdout: "ceph_version": "ceph version 20.3.0-5440-gff63f3d9 (ff63f3d997e38f69382fd23ac0206f07f7b05017) tentacle (dev)", 2026-02-24T07:41:40.370 INFO:teuthology.orchestra.run.trial035.stdout: "image_id": "3992f70354a3fdc3bbd8e07cbd698e424b71669298ffe35e8f0565a46973aa85", 2026-02-24T07:41:40.371 INFO:teuthology.orchestra.run.trial035.stdout: "repo_digests": [ 2026-02-24T07:41:40.371 INFO:teuthology.orchestra.run.trial035.stdout: "quay.ceph.io/ceph-ci/ceph@sha256:9a5db51bca6d139691bb8c7054be8834f3ba065d1c0aaaecbafb190e389fb148" 2026-02-24T07:41:40.371 INFO:teuthology.orchestra.run.trial035.stdout: ] 2026-02-24T07:41:40.371 INFO:teuthology.orchestra.run.trial035.stdout:} 2026-02-24T07:41:40.388 DEBUG:teuthology.orchestra.run.trial035:> sudo mkdir -p /etc/ceph 2026-02-24T07:41:40.453 DEBUG:teuthology.orchestra.run.trial091:> sudo mkdir -p /etc/ceph 2026-02-24T07:41:40.480 DEBUG:teuthology.orchestra.run.trial035:> sudo chmod 777 /etc/ceph 2026-02-24T07:41:40.506 DEBUG:teuthology.orchestra.run.trial091:> sudo chmod 777 /etc/ceph 2026-02-24T07:41:40.544 INFO:tasks.cephadm:Writing seed config... 2026-02-24T07:41:40.545 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2026-02-24T07:41:40.545 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2026-02-24T07:41:40.546 INFO:tasks.cephadm: override: [mgr] debug osd = 10 2026-02-24T07:41:40.546 INFO:tasks.cephadm: override: [client.0] debug ms = 1 2026-02-24T07:41:40.546 INFO:tasks.cephadm: override: [client.0] debug objecter = 20 2026-02-24T07:41:40.546 INFO:tasks.cephadm: override: [client.0] debug rados = 20 2026-02-24T07:41:40.546 INFO:tasks.cephadm: override: [global] crimson_allow_pg_split = False 2026-02-24T07:41:40.546 INFO:tasks.cephadm: override: [global] enable experimental unrecoverable data corrupting features = crimson 2026-02-24T07:41:40.546 INFO:tasks.cephadm: override: [global] ms client mode = crc 2026-02-24T07:41:40.546 INFO:tasks.cephadm: override: [global] ms cluster mode = crc 2026-02-24T07:41:40.546 INFO:tasks.cephadm: override: [global] ms mon client mode = crc 2026-02-24T07:41:40.546 INFO:tasks.cephadm: override: [global] ms mon cluster mode = crc 2026-02-24T07:41:40.546 INFO:tasks.cephadm: override: [global] ms mon service mode = crc 2026-02-24T07:41:40.546 INFO:tasks.cephadm: override: [global] ms service mode = crc 2026-02-24T07:41:40.546 INFO:tasks.cephadm: override: [global] osd_max_pg_log_entries = 2 2026-02-24T07:41:40.546 INFO:tasks.cephadm: override: [global] osd_min_pg_log_entries = 1 2026-02-24T07:41:40.546 INFO:tasks.cephadm: override: [global] osd_pg_log_trim_min = 0 2026-02-24T07:41:40.546 INFO:tasks.cephadm: override: [global] osd_pool_default_min_size = 2 2026-02-24T07:41:40.546 INFO:tasks.cephadm: override: [global] osd_pool_default_size = 3 2026-02-24T07:41:40.546 INFO:tasks.cephadm: override: [mon] debug mon = 20 2026-02-24T07:41:40.546 INFO:tasks.cephadm: override: [mon] debug ms = 1 2026-02-24T07:41:40.546 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2026-02-24T07:41:40.546 INFO:tasks.cephadm: override: [mon] mon min osdmap epochs = 50 2026-02-24T07:41:40.546 INFO:tasks.cephadm: override: [mon] mon osdmap full prune interval = 2 2026-02-24T07:41:40.546 INFO:tasks.cephadm: override: [mon] mon osdmap full prune min = 15 2026-02-24T07:41:40.546 INFO:tasks.cephadm: override: [mon] mon osdmap full prune txsize = 2 2026-02-24T07:41:40.546 INFO:tasks.cephadm: override: [mon] osd pool default crimson = True 2026-02-24T07:41:40.546 INFO:tasks.cephadm: override: [mon] paxos service trim min = 10 2026-02-24T07:41:40.546 INFO:tasks.cephadm: override: [osd] crimson cpu num = 4 2026-02-24T07:41:40.546 INFO:tasks.cephadm: override: [osd] crimson osd obc lru size = 10 2026-02-24T07:41:40.546 INFO:tasks.cephadm: override: [osd] debug ms = 20 2026-02-24T07:41:40.546 INFO:tasks.cephadm: override: [osd] debug osd = 20 2026-02-24T07:41:40.546 INFO:tasks.cephadm: override: [osd] debug seastore = 20 2026-02-24T07:41:40.546 INFO:tasks.cephadm: override: [osd] debug seastore backref = 20 2026-02-24T07:41:40.546 INFO:tasks.cephadm: override: [osd] debug seastore cache = 20 2026-02-24T07:41:40.546 INFO:tasks.cephadm: override: [osd] debug seastore cleaner = 20 2026-02-24T07:41:40.546 INFO:tasks.cephadm: override: [osd] debug seastore device = 20 2026-02-24T07:41:40.546 INFO:tasks.cephadm: override: [osd] debug seastore epm = 20 2026-02-24T07:41:40.546 INFO:tasks.cephadm: override: [osd] debug seastore fixedkv tree = 20 2026-02-24T07:41:40.546 INFO:tasks.cephadm: override: [osd] debug seastore journal = 20 2026-02-24T07:41:40.546 INFO:tasks.cephadm: override: [osd] debug seastore lba = 20 2026-02-24T07:41:40.546 INFO:tasks.cephadm: override: [osd] debug seastore odata = 20 2026-02-24T07:41:40.546 INFO:tasks.cephadm: override: [osd] debug seastore omap = 20 2026-02-24T07:41:40.547 INFO:tasks.cephadm: override: [osd] debug seastore onode = 20 2026-02-24T07:41:40.547 INFO:tasks.cephadm: override: [osd] debug seastore t = 20 2026-02-24T07:41:40.547 INFO:tasks.cephadm: override: [osd] debug seastore tm = 20 2026-02-24T07:41:40.547 INFO:tasks.cephadm: override: [osd] osd blocked scrub grace period = 3600 2026-02-24T07:41:40.547 INFO:tasks.cephadm: override: [osd] osd debug reject backfill probability = 0.3 2026-02-24T07:41:40.547 INFO:tasks.cephadm: override: [osd] osd delete sleep = 1 2026-02-24T07:41:40.547 INFO:tasks.cephadm: override: [osd] osd max backfills = 3 2026-02-24T07:41:40.547 INFO:tasks.cephadm: override: [osd] osd max markdown count = 1000 2026-02-24T07:41:40.547 INFO:tasks.cephadm: override: [osd] osd objectstore = seastore 2026-02-24T07:41:40.547 INFO:tasks.cephadm: override: [osd] osd scrub max interval = 120 2026-02-24T07:41:40.547 INFO:tasks.cephadm: override: [osd] osd scrub min interval = 60 2026-02-24T07:41:40.547 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2026-02-24T07:41:40.547 INFO:tasks.cephadm: override: [osd] osd snap trim sleep = 2 2026-02-24T07:41:40.547 INFO:tasks.cephadm: override: [osd] seastore cachepin size pershard = 64M 2026-02-24T07:41:40.547 INFO:tasks.cephadm: override: [osd] seastore main device type = RANDOM_BLOCK_SSD 2026-02-24T07:41:40.547 INFO:tasks.cephadm: override: [osd] seastore max concurrent transactions = 8 2026-02-24T07:41:40.547 DEBUG:teuthology.orchestra.run.trial035:> set -ex 2026-02-24T07:41:40.547 DEBUG:teuthology.orchestra.run.trial035:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2026-02-24T07:41:40.563 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log to file = true mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=isa technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = 3a5983d2-1154-11f1-9c05-d404e6e7d460 crimson_allow_pg_split = False enable experimental unrecoverable data corrupting features = crimson ms client mode = crc ms cluster mode = crc ms mon client mode = crc ms mon cluster mode = crc ms mon service mode = crc ms service mode = crc osd_max_pg_log_entries = 2 osd_min_pg_log_entries = 1 osd_pg_log_trim_min = 0 osd_pool_default_min_size = 2 osd_pool_default_size = 3 [osd] osd scrub load threshold = 5.0 osd scrub max interval = 120 osd mclock profile = high_recovery_ops osd mclock skip benchmark = true osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true crimson cpu num = 4 crimson osd obc lru size = 10 debug ms = 20 debug osd = 20 debug seastore = 20 debug seastore backref = 20 debug seastore cache = 20 debug seastore cleaner = 20 debug seastore device = 20 debug seastore epm = 20 debug seastore fixedkv tree = 20 debug seastore journal = 20 debug seastore lba = 20 debug seastore odata = 20 debug seastore omap = 20 debug seastore onode = 20 debug seastore t = 20 debug seastore tm = 20 osd blocked scrub grace period = 3600 osd debug reject backfill probability = 0.3 osd delete sleep = 1 osd max backfills = 3 osd max markdown count = 1000 osd objectstore = seastore osd scrub min interval = 60 osd snap trim sleep = 2 seastore cachepin size pershard = 64M seastore main device type = RANDOM_BLOCK_SSD seastore max concurrent transactions = 8 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false debug mgr = 20 debug ms = 1 debug osd = 10 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon min osdmap epochs = 50 mon osdmap full prune interval = 2 mon osdmap full prune min = 15 mon osdmap full prune txsize = 2 osd pool default crimson = True paxos service trim min = 10 [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client.0] debug ms = 1 debug objecter = 20 debug rados = 20 2026-02-24T07:41:40.563 DEBUG:teuthology.orchestra.run.trial035:mon.a> sudo journalctl -f -n 0 -u ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460@mon.a.service 2026-02-24T07:41:40.605 DEBUG:teuthology.orchestra.run.trial035:mgr.x> sudo journalctl -f -n 0 -u ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460@mgr.x.service 2026-02-24T07:41:40.649 INFO:tasks.cephadm:Bootstrapping... 2026-02-24T07:41:40.650 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 -v bootstrap --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id x --orphan-initial-daemons --skip-monitoring-stack --mon-ip 10.20.193.35 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2026-02-24T07:41:40.788 INFO:teuthology.orchestra.run.trial035.stdout:-------------------------------------------------------------------------------- 2026-02-24T07:41:40.789 INFO:teuthology.orchestra.run.trial035.stdout:cephadm ['--image', 'quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017', '-v', 'bootstrap', '--fsid', '3a5983d2-1154-11f1-9c05-d404e6e7d460', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'x', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '10.20.193.35', '--skip-admin-label'] 2026-02-24T07:41:40.789 INFO:teuthology.orchestra.run.trial035.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2026-02-24T07:41:40.789 INFO:teuthology.orchestra.run.trial035.stdout:Verifying podman|docker is present... 2026-02-24T07:41:40.803 INFO:teuthology.orchestra.run.trial035.stdout:/bin/podman: stdout 5.8.0 2026-02-24T07:41:40.803 INFO:teuthology.orchestra.run.trial035.stdout:Verifying lvm2 is present... 2026-02-24T07:41:40.804 INFO:teuthology.orchestra.run.trial035.stdout:Verifying time synchronization is in place... 2026-02-24T07:41:40.810 INFO:teuthology.orchestra.run.trial035.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2026-02-24T07:41:40.810 INFO:teuthology.orchestra.run.trial035.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2026-02-24T07:41:40.816 INFO:teuthology.orchestra.run.trial035.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2026-02-24T07:41:40.816 INFO:teuthology.orchestra.run.trial035.stdout:systemctl: stdout inactive 2026-02-24T07:41:40.822 INFO:teuthology.orchestra.run.trial035.stdout:systemctl: stdout enabled 2026-02-24T07:41:40.827 INFO:teuthology.orchestra.run.trial035.stdout:systemctl: stdout active 2026-02-24T07:41:40.827 INFO:teuthology.orchestra.run.trial035.stdout:Unit chronyd.service is enabled and running 2026-02-24T07:41:40.827 INFO:teuthology.orchestra.run.trial035.stdout:Repeating the final host check... 2026-02-24T07:41:40.841 INFO:teuthology.orchestra.run.trial035.stdout:/bin/podman: stdout 5.8.0 2026-02-24T07:41:40.841 INFO:teuthology.orchestra.run.trial035.stdout:podman (/bin/podman) version 5.8.0 is present 2026-02-24T07:41:40.841 INFO:teuthology.orchestra.run.trial035.stdout:systemctl is present 2026-02-24T07:41:40.841 INFO:teuthology.orchestra.run.trial035.stdout:lvcreate is present 2026-02-24T07:41:40.847 INFO:teuthology.orchestra.run.trial035.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2026-02-24T07:41:40.847 INFO:teuthology.orchestra.run.trial035.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2026-02-24T07:41:40.852 INFO:teuthology.orchestra.run.trial035.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2026-02-24T07:41:40.853 INFO:teuthology.orchestra.run.trial035.stdout:systemctl: stdout inactive 2026-02-24T07:41:40.858 INFO:teuthology.orchestra.run.trial035.stdout:systemctl: stdout enabled 2026-02-24T07:41:40.863 INFO:teuthology.orchestra.run.trial035.stdout:systemctl: stdout active 2026-02-24T07:41:40.863 INFO:teuthology.orchestra.run.trial035.stdout:Unit chronyd.service is enabled and running 2026-02-24T07:41:40.863 INFO:teuthology.orchestra.run.trial035.stdout:Host looks OK 2026-02-24T07:41:40.863 INFO:teuthology.orchestra.run.trial035.stdout:Cluster fsid: 3a5983d2-1154-11f1-9c05-d404e6e7d460 2026-02-24T07:41:40.863 INFO:teuthology.orchestra.run.trial035.stdout:Acquiring lock 140482526961040 on /run/cephadm/3a5983d2-1154-11f1-9c05-d404e6e7d460.lock 2026-02-24T07:41:40.863 INFO:teuthology.orchestra.run.trial035.stdout:Lock 140482526961040 acquired on /run/cephadm/3a5983d2-1154-11f1-9c05-d404e6e7d460.lock 2026-02-24T07:41:40.863 INFO:teuthology.orchestra.run.trial035.stdout:Verifying IP 10.20.193.35 port 3300 ... 2026-02-24T07:41:40.864 INFO:teuthology.orchestra.run.trial035.stdout:Verifying IP 10.20.193.35 port 6789 ... 2026-02-24T07:41:40.864 INFO:teuthology.orchestra.run.trial035.stdout:Base mon IP(s) is [10.20.193.35:3300, 10.20.193.35:6789], mon addrv is [v2:10.20.193.35:3300,v1:10.20.193.35:6789] 2026-02-24T07:41:40.867 INFO:teuthology.orchestra.run.trial035.stdout:/sbin/ip: stdout default via 10.20.192.1 dev enp1s0f0 proto dhcp src 10.20.193.35 metric 102 2026-02-24T07:41:40.867 INFO:teuthology.orchestra.run.trial035.stdout:/sbin/ip: stdout 10.20.192.0/20 dev enp1s0f0 proto kernel scope link src 10.20.193.35 metric 102 2026-02-24T07:41:40.867 INFO:teuthology.orchestra.run.trial035.stdout:/sbin/ip: stdout 169.254.3.0/24 dev enp7s0f4u2u2c2 proto kernel scope link src 169.254.3.1 metric 101 2026-02-24T07:41:40.870 INFO:teuthology.orchestra.run.trial035.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2026-02-24T07:41:40.870 INFO:teuthology.orchestra.run.trial035.stdout:/sbin/ip: stdout fe80::/64 dev enp1s0f0 proto kernel metric 256 pref medium 2026-02-24T07:41:40.870 INFO:teuthology.orchestra.run.trial035.stdout:/sbin/ip: stdout fe80::/64 dev enp7s0f4u2u2c2 proto kernel metric 1024 pref medium 2026-02-24T07:41:40.872 INFO:teuthology.orchestra.run.trial035.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2026-02-24T07:41:40.872 INFO:teuthology.orchestra.run.trial035.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2026-02-24T07:41:40.872 INFO:teuthology.orchestra.run.trial035.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2026-02-24T07:41:40.872 INFO:teuthology.orchestra.run.trial035.stdout:/sbin/ip: stdout 2: enp7s0f4u2u2c2: mtu 1500 state UNKNOWN qlen 1000 2026-02-24T07:41:40.873 INFO:teuthology.orchestra.run.trial035.stdout:/sbin/ip: stdout inet6 fe80::1e62:6be1:2613:cbd3/64 scope link noprefixroute 2026-02-24T07:41:40.873 INFO:teuthology.orchestra.run.trial035.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2026-02-24T07:41:40.873 INFO:teuthology.orchestra.run.trial035.stdout:/sbin/ip: stdout 3: enp1s0f0: mtu 1500 state UP qlen 1000 2026-02-24T07:41:40.873 INFO:teuthology.orchestra.run.trial035.stdout:/sbin/ip: stdout inet6 fe80::925a:8ff:fe77:5e6a/64 scope link 2026-02-24T07:41:40.873 INFO:teuthology.orchestra.run.trial035.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2026-02-24T07:41:40.873 INFO:teuthology.orchestra.run.trial035.stdout:Mon IP `10.20.193.35` is in CIDR network `10.20.192.0/20` 2026-02-24T07:41:40.873 INFO:teuthology.orchestra.run.trial035.stdout:Mon IP `10.20.193.35` is in CIDR network `10.20.192.0/20` 2026-02-24T07:41:40.873 INFO:teuthology.orchestra.run.trial035.stdout:Inferred mon public CIDR from local network configuration ['10.20.192.0/20', '10.20.192.0/20'] 2026-02-24T07:41:40.873 INFO:teuthology.orchestra.run.trial035.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2026-02-24T07:41:40.873 INFO:teuthology.orchestra.run.trial035.stdout:Pulling container image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017... 2026-02-24T07:41:41.131 INFO:teuthology.orchestra.run.trial035.stdout:/bin/podman: stdout 3992f70354a3fdc3bbd8e07cbd698e424b71669298ffe35e8f0565a46973aa85 2026-02-24T07:41:41.131 INFO:teuthology.orchestra.run.trial035.stdout:/bin/podman: stderr Trying to pull quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017... 2026-02-24T07:41:41.131 INFO:teuthology.orchestra.run.trial035.stdout:/bin/podman: stderr Getting image source signatures 2026-02-24T07:41:41.131 INFO:teuthology.orchestra.run.trial035.stdout:/bin/podman: stderr Copying blob sha256:34ff0ae581cc4b523b1cc31578790f86bdcdd88215c5534f4c04451410e82346 2026-02-24T07:41:41.132 INFO:teuthology.orchestra.run.trial035.stdout:/bin/podman: stderr Copying blob sha256:8e380faede39ebd4286247457b408d979ab568aafd8389c42ec304b8cfba4e92 2026-02-24T07:41:41.132 INFO:teuthology.orchestra.run.trial035.stdout:/bin/podman: stderr Copying config sha256:3992f70354a3fdc3bbd8e07cbd698e424b71669298ffe35e8f0565a46973aa85 2026-02-24T07:41:41.132 INFO:teuthology.orchestra.run.trial035.stdout:/bin/podman: stderr Writing manifest to image destination 2026-02-24T07:41:41.265 INFO:teuthology.orchestra.run.trial035.stdout:ceph: stdout ceph version 20.3.0-5440-gff63f3d9 (ff63f3d997e38f69382fd23ac0206f07f7b05017) tentacle (dev) 2026-02-24T07:41:41.265 INFO:teuthology.orchestra.run.trial035.stdout:Ceph version: ceph version 20.3.0-5440-gff63f3d9 (ff63f3d997e38f69382fd23ac0206f07f7b05017) tentacle (dev) 2026-02-24T07:41:41.265 INFO:teuthology.orchestra.run.trial035.stdout:Extracting ceph user uid/gid from container image... 2026-02-24T07:41:41.386 INFO:teuthology.orchestra.run.trial035.stdout:stat: stdout 167 167 2026-02-24T07:41:41.386 INFO:teuthology.orchestra.run.trial035.stdout:Creating initial keys... 2026-02-24T07:41:41.534 INFO:teuthology.orchestra.run.trial035.stdout:/usr/bin/ceph-authtool: stdout AQA1Vp1pwZm8HRAAJASS0Wr3vNOJOav36GOxRQ== 2026-02-24T07:41:41.674 INFO:teuthology.orchestra.run.trial035.stdout:/usr/bin/ceph-authtool: stdout AQA1Vp1pNI2GJxAAyO+nD+VuhzolGeXgrt75/A== 2026-02-24T07:41:41.774 INFO:teuthology.orchestra.run.trial035.stdout:/usr/bin/ceph-authtool: stdout AQA1Vp1pPWB5LRAAOYFvS9wBhD2CR/r6oV90xA== 2026-02-24T07:41:41.775 INFO:teuthology.orchestra.run.trial035.stdout:Creating initial monmap... 2026-02-24T07:41:41.918 INFO:teuthology.orchestra.run.trial035.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2026-02-24T07:41:41.918 INFO:teuthology.orchestra.run.trial035.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = tentacle 2026-02-24T07:41:41.918 INFO:teuthology.orchestra.run.trial035.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to 3a5983d2-1154-11f1-9c05-d404e6e7d460 2026-02-24T07:41:41.918 INFO:teuthology.orchestra.run.trial035.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2026-02-24T07:41:41.918 INFO:teuthology.orchestra.run.trial035.stdout:monmaptool for a [v2:10.20.193.35:3300,v1:10.20.193.35:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2026-02-24T07:41:41.918 INFO:teuthology.orchestra.run.trial035.stdout:setting min_mon_release = tentacle 2026-02-24T07:41:41.918 INFO:teuthology.orchestra.run.trial035.stdout:/usr/bin/monmaptool: set fsid to 3a5983d2-1154-11f1-9c05-d404e6e7d460 2026-02-24T07:41:41.918 INFO:teuthology.orchestra.run.trial035.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2026-02-24T07:41:41.918 INFO:teuthology.orchestra.run.trial035.stdout: 2026-02-24T07:41:41.918 INFO:teuthology.orchestra.run.trial035.stdout:Creating mon... 2026-02-24T07:41:42.054 INFO:teuthology.orchestra.run.trial035.stdout:create mon.a on 2026-02-24T07:41:42.225 INFO:teuthology.orchestra.run.trial035.stdout:systemctl: stderr Removed "/etc/systemd/system/multi-user.target.wants/ceph.target". 2026-02-24T07:41:42.321 INFO:teuthology.orchestra.run.trial035.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2026-02-24T07:41:42.426 INFO:teuthology.orchestra.run.trial035.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460.target → /etc/systemd/system/ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460.target. 2026-02-24T07:41:42.427 INFO:teuthology.orchestra.run.trial035.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460.target → /etc/systemd/system/ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460.target. 2026-02-24T07:41:42.547 INFO:teuthology.orchestra.run.trial035.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460@mon.a 2026-02-24T07:41:42.548 INFO:teuthology.orchestra.run.trial035.stdout:systemctl: stderr Failed to reset failed state of unit ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460@mon.a.service: Unit ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460@mon.a.service not loaded. 2026-02-24T07:41:42.663 INFO:teuthology.orchestra.run.trial035.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460.target.wants/ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460@mon.a.service → /etc/systemd/system/ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460@.service. 2026-02-24T07:41:42.796 INFO:teuthology.orchestra.run.trial035.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2026-02-24T07:41:42.796 INFO:teuthology.orchestra.run.trial035.stdout:systemctl: stdout disabled 2026-02-24T07:41:42.803 INFO:teuthology.orchestra.run.trial035.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2026-02-24T07:41:42.803 INFO:teuthology.orchestra.run.trial035.stdout:systemctl: stdout inactive 2026-02-24T07:41:42.803 INFO:teuthology.orchestra.run.trial035.stdout:firewalld.service is not enabled 2026-02-24T07:41:42.803 INFO:teuthology.orchestra.run.trial035.stdout:Not possible to enable service . firewalld.service is not available 2026-02-24T07:41:42.803 INFO:teuthology.orchestra.run.trial035.stdout:Waiting for mon to start... 2026-02-24T07:41:42.804 INFO:teuthology.orchestra.run.trial035.stdout:Waiting for mon... 2026-02-24T07:41:42.807 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:41:42 trial035 ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-mon-a[38849]: 2026-02-24T07:41:42.794+0000 7f49c072cc00 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:41:42.962 INFO:teuthology.orchestra.run.trial035.stdout:/usr/bin/ceph: stdout cluster: 2026-02-24T07:41:42.962 INFO:teuthology.orchestra.run.trial035.stdout:/usr/bin/ceph: stdout id: 3a5983d2-1154-11f1-9c05-d404e6e7d460 2026-02-24T07:41:42.962 INFO:teuthology.orchestra.run.trial035.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2026-02-24T07:41:42.962 INFO:teuthology.orchestra.run.trial035.stdout:/usr/bin/ceph: stdout 2026-02-24T07:41:42.962 INFO:teuthology.orchestra.run.trial035.stdout:/usr/bin/ceph: stdout services: 2026-02-24T07:41:42.963 INFO:teuthology.orchestra.run.trial035.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 0.138091s) [leader: a] 2026-02-24T07:41:42.963 INFO:teuthology.orchestra.run.trial035.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2026-02-24T07:41:42.963 INFO:teuthology.orchestra.run.trial035.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2026-02-24T07:41:42.963 INFO:teuthology.orchestra.run.trial035.stdout:/usr/bin/ceph: stdout 2026-02-24T07:41:42.963 INFO:teuthology.orchestra.run.trial035.stdout:/usr/bin/ceph: stdout data: 2026-02-24T07:41:42.963 INFO:teuthology.orchestra.run.trial035.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2026-02-24T07:41:42.963 INFO:teuthology.orchestra.run.trial035.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2026-02-24T07:41:42.963 INFO:teuthology.orchestra.run.trial035.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2026-02-24T07:41:42.963 INFO:teuthology.orchestra.run.trial035.stdout:/usr/bin/ceph: stdout pgs: 2026-02-24T07:41:42.963 INFO:teuthology.orchestra.run.trial035.stdout:/usr/bin/ceph: stdout 2026-02-24T07:41:42.963 INFO:teuthology.orchestra.run.trial035.stdout:/usr/bin/ceph: stderr 2026-02-24T07:41:42.906+0000 7fe6d2a30640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:41:42.963 INFO:teuthology.orchestra.run.trial035.stdout:/usr/bin/ceph: stderr 2026-02-24T07:41:42.910+0000 7fe6d2a30640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:41:42.964 INFO:teuthology.orchestra.run.trial035.stdout:mon is available 2026-02-24T07:41:42.964 INFO:teuthology.orchestra.run.trial035.stdout:Assimilating anything we can from ceph.conf... 2026-02-24T07:41:43.075 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:41:42 trial035 ceph-mon[38853]: mkfs 3a5983d2-1154-11f1-9c05-d404e6e7d460 2026-02-24T07:41:43.075 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:41:42 trial035 ceph-mon[38853]: mon.a is new leader, mons a in quorum (ranks 0) 2026-02-24T07:41:43.102 INFO:teuthology.orchestra.run.trial035.stdout:/usr/bin/ceph: stdout 2026-02-24T07:41:43.102 INFO:teuthology.orchestra.run.trial035.stdout:/usr/bin/ceph: stdout [global] 2026-02-24T07:41:43.102 INFO:teuthology.orchestra.run.trial035.stdout:/usr/bin/ceph: stdout fsid = 3a5983d2-1154-11f1-9c05-d404e6e7d460 2026-02-24T07:41:43.102 INFO:teuthology.orchestra.run.trial035.stdout:/usr/bin/ceph: stdout mon_cluster_log_file_level = debug 2026-02-24T07:41:43.102 INFO:teuthology.orchestra.run.trial035.stdout:/usr/bin/ceph: stdout mon_host = [v2:10.20.193.35:3300,v1:10.20.193.35:6789] 2026-02-24T07:41:43.102 INFO:teuthology.orchestra.run.trial035.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2026-02-24T07:41:43.102 INFO:teuthology.orchestra.run.trial035.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2026-02-24T07:41:43.103 INFO:teuthology.orchestra.run.trial035.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2026-02-24T07:41:43.103 INFO:teuthology.orchestra.run.trial035.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2026-02-24T07:41:43.103 INFO:teuthology.orchestra.run.trial035.stdout:/usr/bin/ceph: stdout 2026-02-24T07:41:43.103 INFO:teuthology.orchestra.run.trial035.stdout:/usr/bin/ceph: stdout [mgr] 2026-02-24T07:41:43.103 INFO:teuthology.orchestra.run.trial035.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2026-02-24T07:41:43.103 INFO:teuthology.orchestra.run.trial035.stdout:/usr/bin/ceph: stdout 2026-02-24T07:41:43.103 INFO:teuthology.orchestra.run.trial035.stdout:/usr/bin/ceph: stdout [osd] 2026-02-24T07:41:43.103 INFO:teuthology.orchestra.run.trial035.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2026-02-24T07:41:43.103 INFO:teuthology.orchestra.run.trial035.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2026-02-24T07:41:43.103 INFO:teuthology.orchestra.run.trial035.stdout:/usr/bin/ceph: stderr 2026-02-24T07:41:43.047+0000 7f578f9f2640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:41:43.103 INFO:teuthology.orchestra.run.trial035.stdout:/usr/bin/ceph: stderr 2026-02-24T07:41:43.050+0000 7f578f9f2640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:41:43.103 INFO:teuthology.orchestra.run.trial035.stdout:Generating new minimal ceph.conf... 2026-02-24T07:41:43.256 INFO:teuthology.orchestra.run.trial035.stdout:/usr/bin/ceph: stderr 2026-02-24T07:41:43.194+0000 7f30520a1640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:41:43.256 INFO:teuthology.orchestra.run.trial035.stdout:/usr/bin/ceph: stderr 2026-02-24T07:41:43.197+0000 7f30520a1640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:41:43.256 INFO:teuthology.orchestra.run.trial035.stdout:Restarting the monitor... 2026-02-24T07:41:43.338 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:41:43 trial035 systemd[1]: Stopping Ceph mon.a for 3a5983d2-1154-11f1-9c05-d404e6e7d460... 2026-02-24T07:41:43.338 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:41:43 trial035 ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-mon-a[38849]: 2026-02-24T07:41:43.335+0000 7f49beda4640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2026-02-24T07:41:43.338 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:41:43 trial035 ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-mon-a[38849]: 2026-02-24T07:41:43.335+0000 7f49beda4640 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2026-02-24T07:41:43.793 INFO:teuthology.orchestra.run.trial035.stdout:Waiting for mon to start after restart... 2026-02-24T07:41:43.794 INFO:teuthology.orchestra.run.trial035.stdout:Waiting for mon... 2026-02-24T07:41:43.852 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:41:43 trial035 podman[39127]: 2026-02-24 07:41:43.599736878 +0000 UTC m=+0.275798355 container died ddcbd153fab6a88841ad205caada4cb35d38533ef5f0217a0dc44b2986300022 (image=quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017, name=ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-mon-a, ceph=True, io.buildah.version=1.33.7, FROM_IMAGE=quay.io/centos/centos:stream9, OSD_FLAVOR=default, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20260223, org.opencontainers.image.authors=Ceph Release Team , GANESHA_REPO_BASEURL=https://buildlogs.centos.org/centos/$releasever-stream/storage/$basearch/nfsganesha-5/, CEPH_REF=wip-shraddhaag-enable-cephadm-tests, CEPH_GIT_REPO=https://github.com/ceph/ceph-ci.git, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, CEPH_SHA1=ff63f3d997e38f69382fd23ac0206f07f7b05017, org.label-schema.license=GPLv2, org.opencontainers.image.documentation=https://docs.ceph.com/) 2026-02-24T07:41:43.852 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:41:43 trial035 podman[39127]: 2026-02-24 07:41:43.607306248 +0000 UTC m=+0.283367725 container remove ddcbd153fab6a88841ad205caada4cb35d38533ef5f0217a0dc44b2986300022 (image=quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017, name=ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-mon-a, org.opencontainers.image.authors=Ceph Release Team , org.opencontainers.image.documentation=https://docs.ceph.com/, CEPH_GIT_REPO=https://github.com/ceph/ceph-ci.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, io.buildah.version=1.33.7, org.label-schema.license=GPLv2, CEPH_SHA1=ff63f3d997e38f69382fd23ac0206f07f7b05017, FROM_IMAGE=quay.io/centos/centos:stream9, org.label-schema.schema-version=1.0, ceph=True, CEPH_REF=wip-shraddhaag-enable-cephadm-tests, GANESHA_REPO_BASEURL=https://buildlogs.centos.org/centos/$releasever-stream/storage/$basearch/nfsganesha-5/, OSD_FLAVOR=default, org.label-schema.build-date=20260223) 2026-02-24T07:41:43.852 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:41:43 trial035 bash[39127]: ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-mon-a 2026-02-24T07:41:43.852 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:41:43 trial035 systemd[1]: ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460@mon.a.service: Deactivated successfully. 2026-02-24T07:41:43.852 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:41:43 trial035 systemd[1]: Stopped Ceph mon.a for 3a5983d2-1154-11f1-9c05-d404e6e7d460. 2026-02-24T07:41:43.853 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:41:43 trial035 systemd[1]: Starting Ceph mon.a for 3a5983d2-1154-11f1-9c05-d404e6e7d460... 2026-02-24T07:41:43.853 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:41:43 trial035 podman[39256]: 2026-02-24 07:41:43.758489042 +0000 UTC m=+0.014230105 container create ce32af800c6107fb165f396394271002a0b090944a93745241d962596bff7b00 (image=quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017, name=ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-mon-a, org.opencontainers.image.authors=Ceph Release Team , GANESHA_REPO_BASEURL=https://buildlogs.centos.org/centos/$releasever-stream/storage/$basearch/nfsganesha-5/, CEPH_GIT_REPO=https://github.com/ceph/ceph-ci.git, org.label-schema.build-date=20260223, ceph=True, CEPH_REF=wip-shraddhaag-enable-cephadm-tests, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, FROM_IMAGE=quay.io/centos/centos:stream9, io.buildah.version=1.33.7, org.label-schema.vendor=CentOS, CEPH_SHA1=ff63f3d997e38f69382fd23ac0206f07f7b05017, org.opencontainers.image.documentation=https://docs.ceph.com/, OSD_FLAVOR=default, org.label-schema.license=GPLv2) 2026-02-24T07:41:43.853 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:41:43 trial035 podman[39256]: 2026-02-24 07:41:43.786000819 +0000 UTC m=+0.041741872 container init ce32af800c6107fb165f396394271002a0b090944a93745241d962596bff7b00 (image=quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017, name=ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-mon-a, CEPH_SHA1=ff63f3d997e38f69382fd23ac0206f07f7b05017, org.opencontainers.image.authors=Ceph Release Team , OSD_FLAVOR=default, org.opencontainers.image.documentation=https://docs.ceph.com/, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, FROM_IMAGE=quay.io/centos/centos:stream9, io.buildah.version=1.33.7, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, CEPH_GIT_REPO=https://github.com/ceph/ceph-ci.git, org.label-schema.build-date=20260223, CEPH_REF=wip-shraddhaag-enable-cephadm-tests, GANESHA_REPO_BASEURL=https://buildlogs.centos.org/centos/$releasever-stream/storage/$basearch/nfsganesha-5/, org.label-schema.vendor=CentOS) 2026-02-24T07:41:43.853 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:41:43 trial035 podman[39256]: 2026-02-24 07:41:43.788154967 +0000 UTC m=+0.043896030 container start ce32af800c6107fb165f396394271002a0b090944a93745241d962596bff7b00 (image=quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017, name=ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-mon-a, org.label-schema.vendor=CentOS, CEPH_REF=wip-shraddhaag-enable-cephadm-tests, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_SHA1=ff63f3d997e38f69382fd23ac0206f07f7b05017, org.opencontainers.image.authors=Ceph Release Team , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GANESHA_REPO_BASEURL=https://buildlogs.centos.org/centos/$releasever-stream/storage/$basearch/nfsganesha-5/, FROM_IMAGE=quay.io/centos/centos:stream9, CEPH_GIT_REPO=https://github.com/ceph/ceph-ci.git, io.buildah.version=1.33.7, org.label-schema.build-date=20260223, ceph=True, OSD_FLAVOR=default, org.opencontainers.image.documentation=https://docs.ceph.com/) 2026-02-24T07:41:43.853 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:41:43 trial035 bash[39256]: ce32af800c6107fb165f396394271002a0b090944a93745241d962596bff7b00 2026-02-24T07:41:43.853 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:41:43 trial035 podman[39256]: 2026-02-24 07:41:43.75284193 +0000 UTC m=+0.008582983 image pull 3992f70354a3fdc3bbd8e07cbd698e424b71669298ffe35e8f0565a46973aa85 quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 2026-02-24T07:41:43.853 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:41:43 trial035 systemd[1]: Started Ceph mon.a for 3a5983d2-1154-11f1-9c05-d404e6e7d460. 2026-02-24T07:41:43.853 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:41:43 trial035 ceph-mon[39284]: set uid:gid to 167:167 (ceph:ceph) 2026-02-24T07:41:43.853 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:41:43 trial035 ceph-mon[39284]: ceph version 20.3.0-5440-gff63f3d9 (ff63f3d997e38f69382fd23ac0206f07f7b05017) tentacle (dev - RelWithDebInfo), process ceph-mon, pid 2 2026-02-24T07:41:43.853 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:41:43 trial035 ceph-mon[39284]: pidfile_write: ignore empty --pid-file 2026-02-24T07:41:43.853 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:41:43 trial035 ceph-mon[39284]: load: jerasure load: lrc 2026-02-24T07:41:43.853 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:41:43 trial035 ceph-mon[39284]: rocksdb: RocksDB version: 7.9.2 2026-02-24T07:41:43.854 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:41:43 trial035 ceph-mon[39284]: rocksdb: Git sha 0 2026-02-24T07:41:43.854 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:41:43 trial035 ceph-mon[39284]: rocksdb: Compile date 2026-02-24 05:12:44 2026-02-24T07:41:43.854 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:41:43 trial035 ceph-mon[39284]: rocksdb: DB SUMMARY 2026-02-24T07:41:43.854 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:41:43 trial035 ceph-mon[39284]: rocksdb: DB Session ID: 7GRJVY8W4LMJSBR5R02B 2026-02-24T07:41:43.854 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:41:43 trial035 ceph-mon[39284]: rocksdb: CURRENT file: CURRENT 2026-02-24T07:41:43.854 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:41:43 trial035 ceph-mon[39284]: rocksdb: IDENTITY file: IDENTITY 2026-02-24T07:41:43.854 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:41:43 trial035 ceph-mon[39284]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2026-02-24T07:41:43.854 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:41:43 trial035 ceph-mon[39284]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2026-02-24T07:41:43.854 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:41:43 trial035 ceph-mon[39284]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000009.log size: 100590 ; 2026-02-24T07:41:43.854 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:41:43 trial035 ceph-mon[39284]: rocksdb: Options.error_if_exists: 0 2026-02-24T07:41:43.854 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:41:43 trial035 ceph-mon[39284]: rocksdb: Options.create_if_missing: 0 2026-02-24T07:41:43.854 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:41:43 trial035 ceph-mon[39284]: rocksdb: Options.paranoid_checks: 1 2026-02-24T07:41:43.855 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:41:43 trial035 ceph-mon[39284]: rocksdb: Options.flush_verify_memtable_count: 1 2026-02-24T07:41:43.855 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:41:43 trial035 ceph-mon[39284]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2026-02-24T07:41:43.855 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:41:43 trial035 ceph-mon[39284]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2026-02-24T07:41:43.855 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:41:43 trial035 ceph-mon[39284]: rocksdb: Options.env: 0xed5d60 2026-02-24T07:41:43.855 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:41:43 trial035 ceph-mon[39284]: rocksdb: Options.fs: PosixFileSystem 2026-02-24T07:41:43.855 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:41:43 trial035 ceph-mon[39284]: rocksdb: Options.info_log: 0x1a4eec0 2026-02-24T07:41:43.855 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:41:43 trial035 ceph-mon[39284]: rocksdb: Options.max_file_opening_threads: 16 2026-02-24T07:41:43.855 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:41:43 trial035 ceph-mon[39284]: rocksdb: Options.statistics: (nil) 2026-02-24T07:41:43.855 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:41:43 trial035 ceph-mon[39284]: rocksdb: Options.use_fsync: 0 2026-02-24T07:41:43.855 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:41:43 trial035 ceph-mon[39284]: rocksdb: Options.max_log_file_size: 0 2026-02-24T07:41:43.855 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:41:43 trial035 ceph-mon[39284]: rocksdb: Options.max_manifest_file_size: 1073741824 2026-02-24T07:41:43.855 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:41:43 trial035 ceph-mon[39284]: rocksdb: Options.log_file_time_to_roll: 0 2026-02-24T07:41:43.855 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:41:43 trial035 ceph-mon[39284]: rocksdb: Options.keep_log_file_num: 1000 2026-02-24T07:41:43.856 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:41:43 trial035 ceph-mon[39284]: rocksdb: Options.recycle_log_file_num: 0 2026-02-24T07:41:43.856 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:41:43 trial035 ceph-mon[39284]: rocksdb: Options.allow_fallocate: 1 2026-02-24T07:41:43.856 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:41:43 trial035 ceph-mon[39284]: rocksdb: Options.allow_mmap_reads: 0 2026-02-24T07:41:43.856 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:41:43 trial035 ceph-mon[39284]: rocksdb: Options.allow_mmap_writes: 0 2026-02-24T07:41:43.856 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:41:43 trial035 ceph-mon[39284]: rocksdb: Options.use_direct_reads: 0 2026-02-24T07:41:43.856 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:41:43 trial035 ceph-mon[39284]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2026-02-24T07:41:43.856 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:41:43 trial035 ceph-mon[39284]: rocksdb: Options.create_missing_column_families: 0 2026-02-24T07:41:43.856 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:41:43 trial035 ceph-mon[39284]: rocksdb: Options.db_log_dir: 2026-02-24T07:41:43.856 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:41:43 trial035 ceph-mon[39284]: rocksdb: Options.wal_dir: 2026-02-24T07:41:43.856 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:41:43 trial035 ceph-mon[39284]: rocksdb: Options.table_cache_numshardbits: 6 2026-02-24T07:41:43.856 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:41:43 trial035 ceph-mon[39284]: rocksdb: Options.WAL_ttl_seconds: 0 2026-02-24T07:41:43.856 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:41:43 trial035 ceph-mon[39284]: rocksdb: Options.WAL_size_limit_MB: 0 2026-02-24T07:41:43.857 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:41:43 trial035 ceph-mon[39284]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2026-02-24T07:41:43.857 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:41:43 trial035 ceph-mon[39284]: rocksdb: Options.manifest_preallocation_size: 4194304 2026-02-24T07:41:43.857 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:41:43 trial035 ceph-mon[39284]: rocksdb: Options.is_fd_close_on_exec: 1 2026-02-24T07:41:43.857 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:41:43 trial035 ceph-mon[39284]: rocksdb: Options.advise_random_on_open: 1 2026-02-24T07:41:43.857 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:41:43 trial035 ceph-mon[39284]: rocksdb: Options.db_write_buffer_size: 0 2026-02-24T07:41:43.857 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:41:43 trial035 ceph-mon[39284]: rocksdb: Options.write_buffer_manager: 0x1a4a140 2026-02-24T07:41:43.857 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:41:43 trial035 ceph-mon[39284]: rocksdb: Options.access_hint_on_compaction_start: 1 2026-02-24T07:41:43.857 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:41:43 trial035 ceph-mon[39284]: rocksdb: Options.random_access_max_buffer_size: 1048576 2026-02-24T07:41:43.857 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:41:43 trial035 ceph-mon[39284]: rocksdb: Options.use_adaptive_mutex: 0 2026-02-24T07:41:43.857 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:41:43 trial035 ceph-mon[39284]: rocksdb: Options.rate_limiter: (nil) 2026-02-24T07:41:43.857 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:41:43 trial035 ceph-mon[39284]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2026-02-24T07:41:43.857 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:41:43 trial035 ceph-mon[39284]: rocksdb: Options.wal_recovery_mode: 2 2026-02-24T07:41:43.857 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:41:43 trial035 ceph-mon[39284]: rocksdb: Options.enable_thread_tracking: 0 2026-02-24T07:41:43.857 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:41:43 trial035 ceph-mon[39284]: rocksdb: Options.enable_pipelined_write: 0 2026-02-24T07:41:43.858 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:41:43 trial035 ceph-mon[39284]: rocksdb: Options.unordered_write: 0 2026-02-24T07:41:43.858 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:41:43 trial035 ceph-mon[39284]: rocksdb: Options.allow_concurrent_memtable_write: 1 2026-02-24T07:41:43.858 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:41:43 trial035 ceph-mon[39284]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2026-02-24T07:41:43.858 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:41:43 trial035 ceph-mon[39284]: rocksdb: Options.write_thread_max_yield_usec: 100 2026-02-24T07:41:43.858 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:41:43 trial035 ceph-mon[39284]: rocksdb: Options.write_thread_slow_yield_usec: 3 2026-02-24T07:41:43.858 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:41:43 trial035 ceph-mon[39284]: rocksdb: Options.row_cache: None 2026-02-24T07:41:43.858 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:41:43 trial035 ceph-mon[39284]: rocksdb: Options.wal_filter: None 2026-02-24T07:41:43.858 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:41:43 trial035 ceph-mon[39284]: rocksdb: Options.avoid_flush_during_recovery: 0 2026-02-24T07:41:43.858 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:41:43 trial035 ceph-mon[39284]: rocksdb: Options.allow_ingest_behind: 0 2026-02-24T07:41:43.858 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:41:43 trial035 ceph-mon[39284]: rocksdb: Options.two_write_queues: 0 2026-02-24T07:41:43.858 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:41:43 trial035 ceph-mon[39284]: rocksdb: Options.manual_wal_flush: 0 2026-02-24T07:41:43.858 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:41:43 trial035 ceph-mon[39284]: rocksdb: Options.wal_compression: 0 2026-02-24T07:41:43.858 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:41:43 trial035 ceph-mon[39284]: rocksdb: Options.atomic_flush: 0 2026-02-24T07:41:43.859 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:41:43 trial035 ceph-mon[39284]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2026-02-24T07:41:43.859 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:41:43 trial035 ceph-mon[39284]: rocksdb: Options.persist_stats_to_disk: 0 2026-02-24T07:41:43.859 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:41:43 trial035 ceph-mon[39284]: rocksdb: Options.write_dbid_to_manifest: 0 2026-02-24T07:41:43.859 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:41:43 trial035 ceph-mon[39284]: rocksdb: Options.log_readahead_size: 0 2026-02-24T07:41:43.859 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:41:43 trial035 ceph-mon[39284]: rocksdb: Options.file_checksum_gen_factory: Unknown 2026-02-24T07:41:43.859 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:41:43 trial035 ceph-mon[39284]: rocksdb: Options.best_efforts_recovery: 0 2026-02-24T07:41:43.859 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:41:43 trial035 ceph-mon[39284]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2026-02-24T07:41:43.859 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:41:43 trial035 ceph-mon[39284]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2026-02-24T07:41:43.859 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:41:43 trial035 ceph-mon[39284]: rocksdb: Options.allow_data_in_errors: 0 2026-02-24T07:41:43.859 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:41:43 trial035 ceph-mon[39284]: rocksdb: Options.db_host_id: __hostname__ 2026-02-24T07:41:43.859 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:41:43 trial035 ceph-mon[39284]: rocksdb: Options.enforce_single_del_contracts: true 2026-02-24T07:41:43.859 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:41:43 trial035 ceph-mon[39284]: rocksdb: Options.max_background_jobs: 2 2026-02-24T07:41:43.859 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:41:43 trial035 ceph-mon[39284]: rocksdb: Options.max_background_compactions: -1 2026-02-24T07:41:43.859 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:41:43 trial035 ceph-mon[39284]: rocksdb: Options.max_subcompactions: 1 2026-02-24T07:41:43.860 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:41:43 trial035 ceph-mon[39284]: rocksdb: Options.avoid_flush_during_shutdown: 0 2026-02-24T07:41:43.860 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:41:43 trial035 ceph-mon[39284]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2026-02-24T07:41:43.860 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:41:43 trial035 ceph-mon[39284]: rocksdb: Options.delayed_write_rate : 16777216 2026-02-24T07:41:43.860 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:41:43 trial035 ceph-mon[39284]: rocksdb: Options.max_total_wal_size: 0 2026-02-24T07:41:43.860 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:41:43 trial035 ceph-mon[39284]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2026-02-24T07:41:43.860 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:41:43 trial035 ceph-mon[39284]: rocksdb: Options.stats_dump_period_sec: 600 2026-02-24T07:41:43.860 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:41:43 trial035 ceph-mon[39284]: rocksdb: Options.stats_persist_period_sec: 600 2026-02-24T07:41:43.860 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:41:43 trial035 ceph-mon[39284]: rocksdb: Options.stats_history_buffer_size: 1048576 2026-02-24T07:41:43.860 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:41:43 trial035 ceph-mon[39284]: rocksdb: Options.max_open_files: -1 2026-02-24T07:41:43.860 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:41:43 trial035 ceph-mon[39284]: rocksdb: Options.bytes_per_sync: 0 2026-02-24T07:41:43.860 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:41:43 trial035 ceph-mon[39284]: rocksdb: Options.wal_bytes_per_sync: 0 2026-02-24T07:41:43.861 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:41:43 trial035 ceph-mon[39284]: rocksdb: Options.strict_bytes_per_sync: 0 2026-02-24T07:41:43.861 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:41:43 trial035 ceph-mon[39284]: rocksdb: Options.compaction_readahead_size: 0 2026-02-24T07:41:43.861 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:41:43 trial035 ceph-mon[39284]: rocksdb: Options.max_background_flushes: -1 2026-02-24T07:41:43.861 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:41:43 trial035 ceph-mon[39284]: rocksdb: Compression algorithms supported: 2026-02-24T07:41:43.861 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:41:43 trial035 ceph-mon[39284]: rocksdb: kZSTD supported: 0 2026-02-24T07:41:43.861 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:41:43 trial035 ceph-mon[39284]: rocksdb: kXpressCompression supported: 0 2026-02-24T07:41:43.861 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:41:43 trial035 ceph-mon[39284]: rocksdb: kBZip2Compression supported: 0 2026-02-24T07:41:43.861 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:41:43 trial035 ceph-mon[39284]: rocksdb: kZSTDNotFinalCompression supported: 0 2026-02-24T07:41:43.861 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:41:43 trial035 ceph-mon[39284]: rocksdb: kLZ4Compression supported: 1 2026-02-24T07:41:43.861 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:41:43 trial035 ceph-mon[39284]: rocksdb: kZlibCompression supported: 1 2026-02-24T07:41:43.861 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:41:43 trial035 ceph-mon[39284]: rocksdb: kLZ4HCCompression supported: 1 2026-02-24T07:41:43.861 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:41:43 trial035 ceph-mon[39284]: rocksdb: kSnappyCompression supported: 1 2026-02-24T07:41:43.861 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:41:43 trial035 ceph-mon[39284]: rocksdb: Fast CRC32 supported: Supported on x86 2026-02-24T07:41:43.862 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:41:43 trial035 ceph-mon[39284]: rocksdb: DMutex implementation: pthread_mutex_t 2026-02-24T07:41:43.862 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:41:43 trial035 ceph-mon[39284]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2026-02-24T07:41:43.862 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:41:43 trial035 ceph-mon[39284]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2026-02-24T07:41:43.862 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:41:43 trial035 ceph-mon[39284]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2026-02-24T07:41:43.862 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:41:43 trial035 ceph-mon[39284]: rocksdb: Options.merge_operator: 2026-02-24T07:41:43.862 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:41:43 trial035 ceph-mon[39284]: rocksdb: Options.compaction_filter: None 2026-02-24T07:41:43.862 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:41:43 trial035 ceph-mon[39284]: rocksdb: Options.compaction_filter_factory: None 2026-02-24T07:41:43.862 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:41:43 trial035 ceph-mon[39284]: rocksdb: Options.sst_partitioner_factory: None 2026-02-24T07:41:43.862 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:41:43 trial035 ceph-mon[39284]: rocksdb: Options.memtable_factory: SkipListFactory 2026-02-24T07:41:43.862 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:41:43 trial035 ceph-mon[39284]: rocksdb: Options.table_factory: BlockBasedTable 2026-02-24T07:41:43.862 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:41:43 trial035 ceph-mon[39284]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x1a4f600) 2026-02-24T07:41:43.862 INFO:journalctl@ceph.mon.a.trial035.stdout: cache_index_and_filter_blocks: 1 2026-02-24T07:41:43.862 INFO:journalctl@ceph.mon.a.trial035.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2026-02-24T07:41:43.863 INFO:journalctl@ceph.mon.a.trial035.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2026-02-24T07:41:43.863 INFO:journalctl@ceph.mon.a.trial035.stdout: pin_top_level_index_and_filter: 1 2026-02-24T07:41:43.863 INFO:journalctl@ceph.mon.a.trial035.stdout: index_type: 0 2026-02-24T07:41:43.863 INFO:journalctl@ceph.mon.a.trial035.stdout: data_block_index_type: 0 2026-02-24T07:41:43.863 INFO:journalctl@ceph.mon.a.trial035.stdout: index_shortening: 1 2026-02-24T07:41:43.863 INFO:journalctl@ceph.mon.a.trial035.stdout: data_block_hash_table_util_ratio: 0.750000 2026-02-24T07:41:43.863 INFO:journalctl@ceph.mon.a.trial035.stdout: checksum: 4 2026-02-24T07:41:43.863 INFO:journalctl@ceph.mon.a.trial035.stdout: no_block_cache: 0 2026-02-24T07:41:43.863 INFO:journalctl@ceph.mon.a.trial035.stdout: block_cache: 0x1a1da50 2026-02-24T07:41:43.863 INFO:journalctl@ceph.mon.a.trial035.stdout: block_cache_name: BinnedLRUCache 2026-02-24T07:41:43.863 INFO:journalctl@ceph.mon.a.trial035.stdout: block_cache_options: 2026-02-24T07:41:43.863 INFO:journalctl@ceph.mon.a.trial035.stdout: capacity : 536870912 2026-02-24T07:41:43.863 INFO:journalctl@ceph.mon.a.trial035.stdout: num_shard_bits : 4 2026-02-24T07:41:43.864 INFO:journalctl@ceph.mon.a.trial035.stdout: strict_capacity_limit : 0 2026-02-24T07:41:43.864 INFO:journalctl@ceph.mon.a.trial035.stdout: high_pri_pool_ratio: 0.000 2026-02-24T07:41:43.864 INFO:journalctl@ceph.mon.a.trial035.stdout: block_cache_compressed: (nil) 2026-02-24T07:41:43.864 INFO:journalctl@ceph.mon.a.trial035.stdout: persistent_cache: (nil) 2026-02-24T07:41:43.864 INFO:journalctl@ceph.mon.a.trial035.stdout: block_size: 4096 2026-02-24T07:41:43.864 INFO:journalctl@ceph.mon.a.trial035.stdout: block_size_deviation: 10 2026-02-24T07:41:43.864 INFO:journalctl@ceph.mon.a.trial035.stdout: block_restart_interval: 16 2026-02-24T07:41:43.864 INFO:journalctl@ceph.mon.a.trial035.stdout: index_block_restart_interval: 1 2026-02-24T07:41:43.864 INFO:journalctl@ceph.mon.a.trial035.stdout: metadata_block_size: 4096 2026-02-24T07:41:43.864 INFO:journalctl@ceph.mon.a.trial035.stdout: partition_filters: 0 2026-02-24T07:41:43.864 INFO:journalctl@ceph.mon.a.trial035.stdout: use_delta_encoding: 1 2026-02-24T07:41:43.864 INFO:journalctl@ceph.mon.a.trial035.stdout: filter_policy: bloomfilter 2026-02-24T07:41:43.864 INFO:journalctl@ceph.mon.a.trial035.stdout: whole_key_filtering: 1 2026-02-24T07:41:43.865 INFO:journalctl@ceph.mon.a.trial035.stdout: verify_compression: 0 2026-02-24T07:41:43.865 INFO:journalctl@ceph.mon.a.trial035.stdout: read_amp_bytes_per_bit: 0 2026-02-24T07:41:43.865 INFO:journalctl@ceph.mon.a.trial035.stdout: format_version: 5 2026-02-24T07:41:43.865 INFO:journalctl@ceph.mon.a.trial035.stdout: enable_index_compression: 1 2026-02-24T07:41:43.865 INFO:journalctl@ceph.mon.a.trial035.stdout: block_align: 0 2026-02-24T07:41:43.865 INFO:journalctl@ceph.mon.a.trial035.stdout: max_auto_readahead_size: 262144 2026-02-24T07:41:43.865 INFO:journalctl@ceph.mon.a.trial035.stdout: prepopulate_block_cache: 0 2026-02-24T07:41:43.865 INFO:journalctl@ceph.mon.a.trial035.stdout: initial_auto_readahead_size: 8192 2026-02-24T07:41:43.865 INFO:journalctl@ceph.mon.a.trial035.stdout: num_file_reads_for_auto_readahead: 2 2026-02-24T07:41:43.865 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:41:43 trial035 ceph-mon[39284]: rocksdb: Options.write_buffer_size: 33554432 2026-02-24T07:41:43.865 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:41:43 trial035 ceph-mon[39284]: rocksdb: Options.max_write_buffer_number: 2 2026-02-24T07:41:43.865 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:41:43 trial035 ceph-mon[39284]: rocksdb: Options.compression: NoCompression 2026-02-24T07:41:43.865 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:41:43 trial035 ceph-mon[39284]: rocksdb: Options.bottommost_compression: Disabled 2026-02-24T07:41:43.865 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:41:43 trial035 ceph-mon[39284]: rocksdb: Options.prefix_extractor: nullptr 2026-02-24T07:41:43.866 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:41:43 trial035 ceph-mon[39284]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2026-02-24T07:41:43.866 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:41:43 trial035 ceph-mon[39284]: rocksdb: Options.num_levels: 7 2026-02-24T07:41:43.866 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:41:43 trial035 ceph-mon[39284]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2026-02-24T07:41:43.866 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:41:43 trial035 ceph-mon[39284]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2026-02-24T07:41:43.866 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:41:43 trial035 ceph-mon[39284]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2026-02-24T07:41:43.866 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:41:43 trial035 ceph-mon[39284]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2026-02-24T07:41:43.866 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:41:43 trial035 ceph-mon[39284]: rocksdb: Options.bottommost_compression_opts.level: 32767 2026-02-24T07:41:43.866 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:41:43 trial035 ceph-mon[39284]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2026-02-24T07:41:43.866 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:41:43 trial035 ceph-mon[39284]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2026-02-24T07:41:43.866 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:41:43 trial035 ceph-mon[39284]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2026-02-24T07:41:43.866 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:41:43 trial035 ceph-mon[39284]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2026-02-24T07:41:43.866 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:41:43 trial035 ceph-mon[39284]: rocksdb: Options.bottommost_compression_opts.enabled: false 2026-02-24T07:41:43.867 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:41:43 trial035 ceph-mon[39284]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2026-02-24T07:41:43.867 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:41:43 trial035 ceph-mon[39284]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2026-02-24T07:41:43.867 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:41:43 trial035 ceph-mon[39284]: rocksdb: Options.compression_opts.window_bits: -14 2026-02-24T07:41:43.867 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:41:43 trial035 ceph-mon[39284]: rocksdb: Options.compression_opts.level: 32767 2026-02-24T07:41:43.867 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:41:43 trial035 ceph-mon[39284]: rocksdb: Options.compression_opts.strategy: 0 2026-02-24T07:41:43.867 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:41:43 trial035 ceph-mon[39284]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2026-02-24T07:41:43.867 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:41:43 trial035 ceph-mon[39284]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2026-02-24T07:41:43.867 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:41:43 trial035 ceph-mon[39284]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2026-02-24T07:41:43.867 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:41:43 trial035 ceph-mon[39284]: rocksdb: Options.compression_opts.parallel_threads: 1 2026-02-24T07:41:43.867 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:41:43 trial035 ceph-mon[39284]: rocksdb: Options.compression_opts.enabled: false 2026-02-24T07:41:43.867 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:41:43 trial035 ceph-mon[39284]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2026-02-24T07:41:43.867 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:41:43 trial035 ceph-mon[39284]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2026-02-24T07:41:43.867 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:41:43 trial035 ceph-mon[39284]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2026-02-24T07:41:43.868 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:41:43 trial035 ceph-mon[39284]: rocksdb: Options.level0_stop_writes_trigger: 36 2026-02-24T07:41:43.868 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:41:43 trial035 ceph-mon[39284]: rocksdb: Options.target_file_size_base: 67108864 2026-02-24T07:41:43.868 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:41:43 trial035 ceph-mon[39284]: rocksdb: Options.target_file_size_multiplier: 1 2026-02-24T07:41:43.868 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:41:43 trial035 ceph-mon[39284]: rocksdb: Options.max_bytes_for_level_base: 268435456 2026-02-24T07:41:43.868 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:41:43 trial035 ceph-mon[39284]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2026-02-24T07:41:43.868 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:41:43 trial035 ceph-mon[39284]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2026-02-24T07:41:43.868 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:41:43 trial035 ceph-mon[39284]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2026-02-24T07:41:43.868 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:41:43 trial035 ceph-mon[39284]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2026-02-24T07:41:43.868 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:41:43 trial035 ceph-mon[39284]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2026-02-24T07:41:43.868 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:41:43 trial035 ceph-mon[39284]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2026-02-24T07:41:43.868 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:41:43 trial035 ceph-mon[39284]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2026-02-24T07:41:43.868 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:41:43 trial035 ceph-mon[39284]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2026-02-24T07:41:43.868 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:41:43 trial035 ceph-mon[39284]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2026-02-24T07:41:43.869 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:41:43 trial035 ceph-mon[39284]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2026-02-24T07:41:43.869 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:41:43 trial035 ceph-mon[39284]: rocksdb: Options.max_compaction_bytes: 1677721600 2026-02-24T07:41:43.869 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:41:43 trial035 ceph-mon[39284]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2026-02-24T07:41:43.869 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:41:43 trial035 ceph-mon[39284]: rocksdb: Options.arena_block_size: 1048576 2026-02-24T07:41:43.869 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:41:43 trial035 ceph-mon[39284]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2026-02-24T07:41:43.869 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:41:43 trial035 ceph-mon[39284]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2026-02-24T07:41:43.869 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:41:43 trial035 ceph-mon[39284]: rocksdb: Options.disable_auto_compactions: 0 2026-02-24T07:41:43.869 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:41:43 trial035 ceph-mon[39284]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2026-02-24T07:41:43.869 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:41:43 trial035 ceph-mon[39284]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2026-02-24T07:41:43.869 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:41:43 trial035 ceph-mon[39284]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2026-02-24T07:41:43.869 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:41:43 trial035 ceph-mon[39284]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2026-02-24T07:41:43.869 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:41:43 trial035 ceph-mon[39284]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2026-02-24T07:41:43.869 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:41:43 trial035 ceph-mon[39284]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2026-02-24T07:41:43.870 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:41:43 trial035 ceph-mon[39284]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2026-02-24T07:41:43.870 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:41:43 trial035 ceph-mon[39284]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2026-02-24T07:41:43.870 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:41:43 trial035 ceph-mon[39284]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2026-02-24T07:41:43.870 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:41:43 trial035 ceph-mon[39284]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2026-02-24T07:41:43.870 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:41:43 trial035 ceph-mon[39284]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2026-02-24T07:41:43.870 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:41:43 trial035 ceph-mon[39284]: rocksdb: Options.inplace_update_support: 0 2026-02-24T07:41:43.870 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:41:43 trial035 ceph-mon[39284]: rocksdb: Options.inplace_update_num_locks: 10000 2026-02-24T07:41:43.870 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:41:43 trial035 ceph-mon[39284]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2026-02-24T07:41:43.870 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:41:43 trial035 ceph-mon[39284]: rocksdb: Options.memtable_whole_key_filtering: 0 2026-02-24T07:41:43.870 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:41:43 trial035 ceph-mon[39284]: rocksdb: Options.memtable_huge_page_size: 0 2026-02-24T07:41:43.870 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:41:43 trial035 ceph-mon[39284]: rocksdb: Options.bloom_locality: 0 2026-02-24T07:41:43.870 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:41:43 trial035 ceph-mon[39284]: rocksdb: Options.max_successive_merges: 0 2026-02-24T07:41:43.870 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:41:43 trial035 ceph-mon[39284]: rocksdb: Options.optimize_filters_for_hits: 0 2026-02-24T07:41:43.871 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:41:43 trial035 ceph-mon[39284]: rocksdb: Options.paranoid_file_checks: 0 2026-02-24T07:41:43.871 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:41:43 trial035 ceph-mon[39284]: rocksdb: Options.force_consistency_checks: 1 2026-02-24T07:41:43.871 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:41:43 trial035 ceph-mon[39284]: rocksdb: Options.report_bg_io_stats: 0 2026-02-24T07:41:43.871 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:41:43 trial035 ceph-mon[39284]: rocksdb: Options.ttl: 2592000 2026-02-24T07:41:43.871 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:41:43 trial035 ceph-mon[39284]: rocksdb: Options.periodic_compaction_seconds: 0 2026-02-24T07:41:43.871 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:41:43 trial035 ceph-mon[39284]: rocksdb: Options.preclude_last_level_data_seconds: 0 2026-02-24T07:41:43.871 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:41:43 trial035 ceph-mon[39284]: rocksdb: Options.preserve_internal_time_seconds: 0 2026-02-24T07:41:43.871 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:41:43 trial035 ceph-mon[39284]: rocksdb: Options.enable_blob_files: false 2026-02-24T07:41:43.871 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:41:43 trial035 ceph-mon[39284]: rocksdb: Options.min_blob_size: 0 2026-02-24T07:41:43.871 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:41:43 trial035 ceph-mon[39284]: rocksdb: Options.blob_file_size: 268435456 2026-02-24T07:41:43.871 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:41:43 trial035 ceph-mon[39284]: rocksdb: Options.blob_compression_type: NoCompression 2026-02-24T07:41:43.871 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:41:43 trial035 ceph-mon[39284]: rocksdb: Options.enable_blob_garbage_collection: false 2026-02-24T07:41:43.871 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:41:43 trial035 ceph-mon[39284]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2026-02-24T07:41:43.872 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:41:43 trial035 ceph-mon[39284]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2026-02-24T07:41:43.872 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:41:43 trial035 ceph-mon[39284]: rocksdb: Options.blob_compaction_readahead_size: 0 2026-02-24T07:41:43.872 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:41:43 trial035 ceph-mon[39284]: rocksdb: Options.blob_file_starting_level: 0 2026-02-24T07:41:43.872 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:41:43 trial035 ceph-mon[39284]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2026-02-24T07:41:43.872 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:41:43 trial035 ceph-mon[39284]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2026-02-24T07:41:43.872 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:41:43 trial035 ceph-mon[39284]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2026-02-24T07:41:43.872 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:41:43 trial035 ceph-mon[39284]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 89209cce-1837-4d5f-a1c9-c2d7fe4a4d0e 2026-02-24T07:41:43.872 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:41:43 trial035 ceph-mon[39284]: rocksdb: EVENT_LOG_v1 {"time_micros": 1771918903805487, "job": 1, "event": "recovery_started", "wal_files": [9]} 2026-02-24T07:41:43.872 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:41:43 trial035 ceph-mon[39284]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2026-02-24T07:41:43.872 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:41:43 trial035 ceph-mon[39284]: rocksdb: EVENT_LOG_v1 {"time_micros": 1771918903806475, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 95084, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 327, "table_properties": {"data_size": 93022, "index_size": 255, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 773, "raw_key_size": 15830, "raw_average_key_size": 53, "raw_value_size": 84807, "raw_average_value_size": 284, "num_data_blocks": 9, "num_entries": 298, "num_filter_entries": 298, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1771918903, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "89209cce-1837-4d5f-a1c9-c2d7fe4a4d0e", "db_session_id": "7GRJVY8W4LMJSBR5R02B", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2026-02-24T07:41:43.872 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:41:43 trial035 ceph-mon[39284]: rocksdb: EVENT_LOG_v1 {"time_micros": 1771918903806546, "job": 1, "event": "recovery_finished"} 2026-02-24T07:41:43.872 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:41:43 trial035 ceph-mon[39284]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2026-02-24T07:41:43.873 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:41:43 trial035 ceph-mon[39284]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2026-02-24T07:41:43.873 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:41:43 trial035 ceph-mon[39284]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x1a6ae00 2026-02-24T07:41:43.873 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:41:43 trial035 ceph-mon[39284]: rocksdb: DB pointer 0x285c000 2026-02-24T07:41:43.873 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:41:43 trial035 ceph-mon[39284]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2026-02-24T07:41:43.873 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:41:43 trial035 ceph-mon[39284]: rocksdb: [db/db_impl/db_impl.cc:1111] 2026-02-24T07:41:43.873 INFO:journalctl@ceph.mon.a.trial035.stdout: ** DB Stats ** 2026-02-24T07:41:43.873 INFO:journalctl@ceph.mon.a.trial035.stdout: Uptime(secs): 0.0 total, 0.0 interval 2026-02-24T07:41:43.873 INFO:journalctl@ceph.mon.a.trial035.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2026-02-24T07:41:43.873 INFO:journalctl@ceph.mon.a.trial035.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2026-02-24T07:41:43.874 INFO:journalctl@ceph.mon.a.trial035.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2026-02-24T07:41:43.874 INFO:journalctl@ceph.mon.a.trial035.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2026-02-24T07:41:43.874 INFO:journalctl@ceph.mon.a.trial035.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2026-02-24T07:41:43.874 INFO:journalctl@ceph.mon.a.trial035.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2026-02-24T07:41:43.874 INFO:journalctl@ceph.mon.a.trial035.stdout: 2026-02-24T07:41:43.874 INFO:journalctl@ceph.mon.a.trial035.stdout: ** Compaction Stats [default] ** 2026-02-24T07:41:43.874 INFO:journalctl@ceph.mon.a.trial035.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2026-02-24T07:41:43.874 INFO:journalctl@ceph.mon.a.trial035.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2026-02-24T07:41:43.874 INFO:journalctl@ceph.mon.a.trial035.stdout: L0 2/0 94.72 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 121.2 0.00 0.00 1 0.001 0 0 0.0 0.0 2026-02-24T07:41:43.874 INFO:journalctl@ceph.mon.a.trial035.stdout: Sum 2/0 94.72 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 121.2 0.00 0.00 1 0.001 0 0 0.0 0.0 2026-02-24T07:41:43.874 INFO:journalctl@ceph.mon.a.trial035.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 121.2 0.00 0.00 1 0.001 0 0 0.0 0.0 2026-02-24T07:41:43.875 INFO:journalctl@ceph.mon.a.trial035.stdout: 2026-02-24T07:41:43.875 INFO:journalctl@ceph.mon.a.trial035.stdout: ** Compaction Stats [default] ** 2026-02-24T07:41:43.875 INFO:journalctl@ceph.mon.a.trial035.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2026-02-24T07:41:43.875 INFO:journalctl@ceph.mon.a.trial035.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2026-02-24T07:41:43.875 INFO:journalctl@ceph.mon.a.trial035.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 121.2 0.00 0.00 1 0.001 0 0 0.0 0.0 2026-02-24T07:41:43.875 INFO:journalctl@ceph.mon.a.trial035.stdout: 2026-02-24T07:41:43.875 INFO:journalctl@ceph.mon.a.trial035.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2026-02-24T07:41:43.875 INFO:journalctl@ceph.mon.a.trial035.stdout: 2026-02-24T07:41:43.875 INFO:journalctl@ceph.mon.a.trial035.stdout: Uptime(secs): 0.0 total, 0.0 interval 2026-02-24T07:41:43.875 INFO:journalctl@ceph.mon.a.trial035.stdout: Flush(GB): cumulative 0.000, interval 0.000 2026-02-24T07:41:43.875 INFO:journalctl@ceph.mon.a.trial035.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2026-02-24T07:41:43.876 INFO:journalctl@ceph.mon.a.trial035.stdout: AddFile(Total Files): cumulative 0, interval 0 2026-02-24T07:41:43.876 INFO:journalctl@ceph.mon.a.trial035.stdout: AddFile(L0 Files): cumulative 0, interval 0 2026-02-24T07:41:43.876 INFO:journalctl@ceph.mon.a.trial035.stdout: AddFile(Keys): cumulative 0, interval 0 2026-02-24T07:41:43.876 INFO:journalctl@ceph.mon.a.trial035.stdout: Cumulative compaction: 0.00 GB write, 30.95 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2026-02-24T07:41:43.876 INFO:journalctl@ceph.mon.a.trial035.stdout: Interval compaction: 0.00 GB write, 30.95 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2026-02-24T07:41:43.876 INFO:journalctl@ceph.mon.a.trial035.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2026-02-24T07:41:43.876 INFO:journalctl@ceph.mon.a.trial035.stdout: Block cache BinnedLRUCache@0x1a1da50#2 capacity: 512.00 MB usage: 1.45 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1.4e-05 secs_since: 0 2026-02-24T07:41:43.876 INFO:journalctl@ceph.mon.a.trial035.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,1.02 KB,0.000193715%) IndexBlock(2,0.44 KB,8.34465e-05%) Misc(1,0.00 KB,0%) 2026-02-24T07:41:43.876 INFO:journalctl@ceph.mon.a.trial035.stdout: 2026-02-24T07:41:43.876 INFO:journalctl@ceph.mon.a.trial035.stdout: ** File Read Latency Histogram By Level [default] ** 2026-02-24T07:41:43.877 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:41:43 trial035 ceph-mon[39284]: starting mon.a rank 0 at public addrs [v2:10.20.193.35:3300/0,v1:10.20.193.35:6789/0] at bind addrs [v2:10.20.193.35:3300/0,v1:10.20.193.35:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 2026-02-24T07:41:43.877 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:41:43 trial035 ceph-mon[39284]: mon.a@-1(???) e1 preinit fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 2026-02-24T07:41:43.877 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:41:43 trial035 ceph-mon[39284]: mon.a@-1(???).mds e1 new map 2026-02-24T07:41:43.877 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:41:43 trial035 ceph-mon[39284]: mon.a@-1(???).mds e1 print_map 2026-02-24T07:41:43.877 INFO:journalctl@ceph.mon.a.trial035.stdout: e1 2026-02-24T07:41:43.877 INFO:journalctl@ceph.mon.a.trial035.stdout: btime 2026-02-24T07:41:42:805557+0000 2026-02-24T07:41:43.877 INFO:journalctl@ceph.mon.a.trial035.stdout: enable_multiple, ever_enabled_multiple: 1,1 2026-02-24T07:41:43.877 INFO:journalctl@ceph.mon.a.trial035.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2026-02-24T07:41:43.877 INFO:journalctl@ceph.mon.a.trial035.stdout: legacy client fscid: -1 2026-02-24T07:41:43.877 INFO:journalctl@ceph.mon.a.trial035.stdout: 2026-02-24T07:41:43.877 INFO:journalctl@ceph.mon.a.trial035.stdout: No filesystems configured 2026-02-24T07:41:43.878 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:41:43 trial035 ceph-mon[39284]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2026-02-24T07:41:43.878 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:41:43 trial035 ceph-mon[39284]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2026-02-24T07:41:43.878 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:41:43 trial035 ceph-mon[39284]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2026-02-24T07:41:43.878 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:41:43 trial035 ceph-mon[39284]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2026-02-24T07:41:43.878 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:41:43 trial035 ceph-mon[39284]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2026-02-24T07:41:43.878 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:41:43 trial035 ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-mon-a[39280]: 2026-02-24T07:41:43.808+0000 7f4606814c00 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:41:43.945 INFO:teuthology.orchestra.run.trial035.stdout:/usr/bin/ceph: stdout cluster: 2026-02-24T07:41:43.945 INFO:teuthology.orchestra.run.trial035.stdout:/usr/bin/ceph: stdout id: 3a5983d2-1154-11f1-9c05-d404e6e7d460 2026-02-24T07:41:43.946 INFO:teuthology.orchestra.run.trial035.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2026-02-24T07:41:43.946 INFO:teuthology.orchestra.run.trial035.stdout:/usr/bin/ceph: stdout 2026-02-24T07:41:43.946 INFO:teuthology.orchestra.run.trial035.stdout:/usr/bin/ceph: stdout services: 2026-02-24T07:41:43.946 INFO:teuthology.orchestra.run.trial035.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 0.113599s) [leader: a] 2026-02-24T07:41:43.946 INFO:teuthology.orchestra.run.trial035.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2026-02-24T07:41:43.946 INFO:teuthology.orchestra.run.trial035.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2026-02-24T07:41:43.946 INFO:teuthology.orchestra.run.trial035.stdout:/usr/bin/ceph: stdout 2026-02-24T07:41:43.946 INFO:teuthology.orchestra.run.trial035.stdout:/usr/bin/ceph: stdout data: 2026-02-24T07:41:43.946 INFO:teuthology.orchestra.run.trial035.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2026-02-24T07:41:43.946 INFO:teuthology.orchestra.run.trial035.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2026-02-24T07:41:43.946 INFO:teuthology.orchestra.run.trial035.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2026-02-24T07:41:43.947 INFO:teuthology.orchestra.run.trial035.stdout:/usr/bin/ceph: stdout pgs: 2026-02-24T07:41:43.947 INFO:teuthology.orchestra.run.trial035.stdout:/usr/bin/ceph: stdout 2026-02-24T07:41:43.947 INFO:teuthology.orchestra.run.trial035.stdout:/usr/bin/ceph: stderr 2026-02-24T07:41:43.887+0000 7fef8e425640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:41:43.947 INFO:teuthology.orchestra.run.trial035.stdout:/usr/bin/ceph: stderr 2026-02-24T07:41:43.892+0000 7fef8e425640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:41:43.947 INFO:teuthology.orchestra.run.trial035.stdout:mon is available 2026-02-24T07:41:43.947 INFO:teuthology.orchestra.run.trial035.stdout:Setting public_network to 10.20.192.0/20 in global config section 2026-02-24T07:41:44.088 INFO:teuthology.orchestra.run.trial035.stdout:/usr/bin/ceph: stderr 2026-02-24T07:41:44.033+0000 7fd1019fc640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:41:44.088 INFO:teuthology.orchestra.run.trial035.stdout:/usr/bin/ceph: stderr 2026-02-24T07:41:44.037+0000 7fd1019fc640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:41:44.088 INFO:teuthology.orchestra.run.trial035.stdout:Wrote config to /etc/ceph/ceph.conf 2026-02-24T07:41:44.088 INFO:teuthology.orchestra.run.trial035.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2026-02-24T07:41:44.088 INFO:teuthology.orchestra.run.trial035.stdout:Creating mgr... 2026-02-24T07:41:44.089 INFO:teuthology.orchestra.run.trial035.stdout:Verifying port 0.0.0.0:9283 ... 2026-02-24T07:41:44.089 INFO:teuthology.orchestra.run.trial035.stdout:Verifying port 0.0.0.0:8765 ... 2026-02-24T07:41:44.134 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:41:43 trial035 ceph-mon[39284]: mon.a is new leader, mons a in quorum (ranks 0) 2026-02-24T07:41:44.134 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:41:43 trial035 ceph-mon[39284]: monmap epoch 1 2026-02-24T07:41:44.134 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:41:43 trial035 ceph-mon[39284]: fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 2026-02-24T07:41:44.134 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:41:43 trial035 ceph-mon[39284]: last_changed 2026-02-24T07:41:41.890986+0000 2026-02-24T07:41:44.134 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:41:43 trial035 ceph-mon[39284]: created 2026-02-24T07:41:41.890986+0000 2026-02-24T07:41:44.134 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:41:43 trial035 ceph-mon[39284]: min_mon_release 20 (tentacle) 2026-02-24T07:41:44.134 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:41:43 trial035 ceph-mon[39284]: election_strategy: 1 2026-02-24T07:41:44.135 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:41:43 trial035 ceph-mon[39284]: 0: [v2:10.20.193.35:3300/0,v1:10.20.193.35:6789/0] mon.a 2026-02-24T07:41:44.135 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:41:43 trial035 ceph-mon[39284]: fsmap 2026-02-24T07:41:44.135 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:41:43 trial035 ceph-mon[39284]: osdmap e1: 0 total, 0 up, 0 in 2026-02-24T07:41:44.135 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:41:43 trial035 ceph-mon[39284]: mgrmap e1: no daemons active 2026-02-24T07:41:44.220 INFO:teuthology.orchestra.run.trial035.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460@mgr.x 2026-02-24T07:41:44.220 INFO:teuthology.orchestra.run.trial035.stdout:systemctl: stderr Failed to reset failed state of unit ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460@mgr.x.service: Unit ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460@mgr.x.service not loaded. 2026-02-24T07:41:44.333 INFO:teuthology.orchestra.run.trial035.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460.target.wants/ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460@mgr.x.service → /etc/systemd/system/ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460@.service. 2026-02-24T07:41:44.474 INFO:teuthology.orchestra.run.trial035.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2026-02-24T07:41:44.474 INFO:teuthology.orchestra.run.trial035.stdout:systemctl: stdout disabled 2026-02-24T07:41:44.477 INFO:journalctl@ceph.mgr.x.trial035.stdout:Feb 24 07:41:44 trial035 ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-mgr-x[39621]: 2026-02-24T07:41:44.475+0000 7f94f8866fc0 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:41:44.480 INFO:teuthology.orchestra.run.trial035.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2026-02-24T07:41:44.480 INFO:teuthology.orchestra.run.trial035.stdout:systemctl: stdout inactive 2026-02-24T07:41:44.480 INFO:teuthology.orchestra.run.trial035.stdout:firewalld.service is not enabled 2026-02-24T07:41:44.480 INFO:teuthology.orchestra.run.trial035.stdout:Not possible to enable service . firewalld.service is not available 2026-02-24T07:41:44.486 INFO:teuthology.orchestra.run.trial035.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2026-02-24T07:41:44.487 INFO:teuthology.orchestra.run.trial035.stdout:systemctl: stdout disabled 2026-02-24T07:41:44.493 INFO:teuthology.orchestra.run.trial035.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2026-02-24T07:41:44.493 INFO:teuthology.orchestra.run.trial035.stdout:systemctl: stdout inactive 2026-02-24T07:41:44.493 INFO:teuthology.orchestra.run.trial035.stdout:firewalld.service is not enabled 2026-02-24T07:41:44.493 INFO:teuthology.orchestra.run.trial035.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2026-02-24T07:41:44.493 INFO:teuthology.orchestra.run.trial035.stdout:Waiting for mgr to start... 2026-02-24T07:41:44.493 INFO:teuthology.orchestra.run.trial035.stdout:Waiting for mgr... 2026-02-24T07:41:44.657 INFO:teuthology.orchestra.run.trial035.stdout:/usr/bin/ceph: stdout 2026-02-24T07:41:44.657 INFO:teuthology.orchestra.run.trial035.stdout:/usr/bin/ceph: stdout { 2026-02-24T07:41:44.658 INFO:teuthology.orchestra.run.trial035.stdout:/usr/bin/ceph: stdout "fsid": "3a5983d2-1154-11f1-9c05-d404e6e7d460", 2026-02-24T07:41:44.658 INFO:teuthology.orchestra.run.trial035.stdout:/usr/bin/ceph: stdout "health": { 2026-02-24T07:41:44.658 INFO:teuthology.orchestra.run.trial035.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2026-02-24T07:41:44.658 INFO:teuthology.orchestra.run.trial035.stdout:/usr/bin/ceph: stdout "checks": {}, 2026-02-24T07:41:44.658 INFO:teuthology.orchestra.run.trial035.stdout:/usr/bin/ceph: stdout "mutes": [] 2026-02-24T07:41:44.658 INFO:teuthology.orchestra.run.trial035.stdout:/usr/bin/ceph: stdout }, 2026-02-24T07:41:44.658 INFO:teuthology.orchestra.run.trial035.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2026-02-24T07:41:44.658 INFO:teuthology.orchestra.run.trial035.stdout:/usr/bin/ceph: stdout "quorum": [ 2026-02-24T07:41:44.658 INFO:teuthology.orchestra.run.trial035.stdout:/usr/bin/ceph: stdout 0 2026-02-24T07:41:44.658 INFO:teuthology.orchestra.run.trial035.stdout:/usr/bin/ceph: stdout ], 2026-02-24T07:41:44.658 INFO:teuthology.orchestra.run.trial035.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2026-02-24T07:41:44.659 INFO:teuthology.orchestra.run.trial035.stdout:/usr/bin/ceph: stdout "a" 2026-02-24T07:41:44.659 INFO:teuthology.orchestra.run.trial035.stdout:/usr/bin/ceph: stdout ], 2026-02-24T07:41:44.659 INFO:teuthology.orchestra.run.trial035.stdout:/usr/bin/ceph: stdout "quorum_age": 0, 2026-02-24T07:41:44.659 INFO:teuthology.orchestra.run.trial035.stdout:/usr/bin/ceph: stdout "monmap": { 2026-02-24T07:41:44.659 INFO:teuthology.orchestra.run.trial035.stdout:/usr/bin/ceph: stdout "epoch": 1, 2026-02-24T07:41:44.659 INFO:teuthology.orchestra.run.trial035.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "tentacle", 2026-02-24T07:41:44.659 INFO:teuthology.orchestra.run.trial035.stdout:/usr/bin/ceph: stdout "num_mons": 1 2026-02-24T07:41:44.659 INFO:teuthology.orchestra.run.trial035.stdout:/usr/bin/ceph: stdout }, 2026-02-24T07:41:44.659 INFO:teuthology.orchestra.run.trial035.stdout:/usr/bin/ceph: stdout "osdmap": { 2026-02-24T07:41:44.659 INFO:teuthology.orchestra.run.trial035.stdout:/usr/bin/ceph: stdout "epoch": 1, 2026-02-24T07:41:44.659 INFO:teuthology.orchestra.run.trial035.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2026-02-24T07:41:44.659 INFO:teuthology.orchestra.run.trial035.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2026-02-24T07:41:44.659 INFO:teuthology.orchestra.run.trial035.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2026-02-24T07:41:44.660 INFO:teuthology.orchestra.run.trial035.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2026-02-24T07:41:44.660 INFO:teuthology.orchestra.run.trial035.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2026-02-24T07:41:44.660 INFO:teuthology.orchestra.run.trial035.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2026-02-24T07:41:44.660 INFO:teuthology.orchestra.run.trial035.stdout:/usr/bin/ceph: stdout }, 2026-02-24T07:41:44.660 INFO:teuthology.orchestra.run.trial035.stdout:/usr/bin/ceph: stdout "pgmap": { 2026-02-24T07:41:44.660 INFO:teuthology.orchestra.run.trial035.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2026-02-24T07:41:44.660 INFO:teuthology.orchestra.run.trial035.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2026-02-24T07:41:44.660 INFO:teuthology.orchestra.run.trial035.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2026-02-24T07:41:44.660 INFO:teuthology.orchestra.run.trial035.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2026-02-24T07:41:44.660 INFO:teuthology.orchestra.run.trial035.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2026-02-24T07:41:44.660 INFO:teuthology.orchestra.run.trial035.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2026-02-24T07:41:44.660 INFO:teuthology.orchestra.run.trial035.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2026-02-24T07:41:44.660 INFO:teuthology.orchestra.run.trial035.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2026-02-24T07:41:44.661 INFO:teuthology.orchestra.run.trial035.stdout:/usr/bin/ceph: stdout }, 2026-02-24T07:41:44.661 INFO:teuthology.orchestra.run.trial035.stdout:/usr/bin/ceph: stdout "fsmap": { 2026-02-24T07:41:44.661 INFO:teuthology.orchestra.run.trial035.stdout:/usr/bin/ceph: stdout "epoch": 1, 2026-02-24T07:41:44.661 INFO:teuthology.orchestra.run.trial035.stdout:/usr/bin/ceph: stdout "btime": "2026-02-24T07:41:42:805557+0000", 2026-02-24T07:41:44.661 INFO:teuthology.orchestra.run.trial035.stdout:/usr/bin/ceph: stdout "by_rank": [], 2026-02-24T07:41:44.661 INFO:teuthology.orchestra.run.trial035.stdout:/usr/bin/ceph: stdout "up:standby": 0 2026-02-24T07:41:44.661 INFO:teuthology.orchestra.run.trial035.stdout:/usr/bin/ceph: stdout }, 2026-02-24T07:41:44.661 INFO:teuthology.orchestra.run.trial035.stdout:/usr/bin/ceph: stdout "mgrmap": { 2026-02-24T07:41:44.661 INFO:teuthology.orchestra.run.trial035.stdout:/usr/bin/ceph: stdout "available": false, 2026-02-24T07:41:44.661 INFO:teuthology.orchestra.run.trial035.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2026-02-24T07:41:44.661 INFO:teuthology.orchestra.run.trial035.stdout:/usr/bin/ceph: stdout "modules": [ 2026-02-24T07:41:44.661 INFO:teuthology.orchestra.run.trial035.stdout:/usr/bin/ceph: stdout "iostat", 2026-02-24T07:41:44.662 INFO:teuthology.orchestra.run.trial035.stdout:/usr/bin/ceph: stdout "nfs" 2026-02-24T07:41:44.662 INFO:teuthology.orchestra.run.trial035.stdout:/usr/bin/ceph: stdout ], 2026-02-24T07:41:44.662 INFO:teuthology.orchestra.run.trial035.stdout:/usr/bin/ceph: stdout "services": {} 2026-02-24T07:41:44.662 INFO:teuthology.orchestra.run.trial035.stdout:/usr/bin/ceph: stdout }, 2026-02-24T07:41:44.662 INFO:teuthology.orchestra.run.trial035.stdout:/usr/bin/ceph: stdout "servicemap": { 2026-02-24T07:41:44.662 INFO:teuthology.orchestra.run.trial035.stdout:/usr/bin/ceph: stdout "epoch": 1, 2026-02-24T07:41:44.662 INFO:teuthology.orchestra.run.trial035.stdout:/usr/bin/ceph: stdout "modified": "2026-02-24T07:41:42.805950+0000", 2026-02-24T07:41:44.662 INFO:teuthology.orchestra.run.trial035.stdout:/usr/bin/ceph: stdout "services": {} 2026-02-24T07:41:44.662 INFO:teuthology.orchestra.run.trial035.stdout:/usr/bin/ceph: stdout }, 2026-02-24T07:41:44.662 INFO:teuthology.orchestra.run.trial035.stdout:/usr/bin/ceph: stdout "progress_events": {} 2026-02-24T07:41:44.662 INFO:teuthology.orchestra.run.trial035.stdout:/usr/bin/ceph: stdout } 2026-02-24T07:41:44.662 INFO:teuthology.orchestra.run.trial035.stdout:/usr/bin/ceph: stderr 2026-02-24T07:41:44.598+0000 7f3c7bebc640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:41:44.662 INFO:teuthology.orchestra.run.trial035.stdout:/usr/bin/ceph: stderr 2026-02-24T07:41:44.601+0000 7f3c7bebc640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:41:44.663 INFO:teuthology.orchestra.run.trial035.stdout:mgr not available, waiting (1/15)... 2026-02-24T07:41:44.864 INFO:journalctl@ceph.mgr.x.trial035.stdout:Feb 24 07:41:44 trial035 ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-mgr-x[39621]: 2026-02-24T07:41:44.476+0000 7f94f8866fc0 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:41:45.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:41:44 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/4073253852' entity='client.admin' cmd={"prefix": "status"} : dispatch 2026-02-24T07:41:45.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:41:44 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/2425304684' entity='client.admin' 2026-02-24T07:41:45.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:41:44 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/278895983' entity='client.admin' cmd={"prefix": "status", "format": "json-pretty"} : dispatch 2026-02-24T07:41:45.954 INFO:journalctl@ceph.mgr.x.trial035.stdout:Feb 24 07:41:45 trial035 ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-mgr-x[39621]: /lib64/python3.9/site-packages/scipy/__init__.py:73: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2026-02-24T07:41:45.954 INFO:journalctl@ceph.mgr.x.trial035.stdout:Feb 24 07:41:45 trial035 ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-mgr-x[39621]: Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2026-02-24T07:41:45.954 INFO:journalctl@ceph.mgr.x.trial035.stdout:Feb 24 07:41:45 trial035 ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-mgr-x[39621]: from numpy import show_config as show_numpy_config 2026-02-24T07:41:46.816 INFO:teuthology.orchestra.run.trial035.stdout:/usr/bin/ceph: stdout 2026-02-24T07:41:46.816 INFO:teuthology.orchestra.run.trial035.stdout:/usr/bin/ceph: stdout { 2026-02-24T07:41:46.816 INFO:teuthology.orchestra.run.trial035.stdout:/usr/bin/ceph: stdout "fsid": "3a5983d2-1154-11f1-9c05-d404e6e7d460", 2026-02-24T07:41:46.816 INFO:teuthology.orchestra.run.trial035.stdout:/usr/bin/ceph: stdout "health": { 2026-02-24T07:41:46.816 INFO:teuthology.orchestra.run.trial035.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2026-02-24T07:41:46.816 INFO:teuthology.orchestra.run.trial035.stdout:/usr/bin/ceph: stdout "checks": {}, 2026-02-24T07:41:46.816 INFO:teuthology.orchestra.run.trial035.stdout:/usr/bin/ceph: stdout "mutes": [] 2026-02-24T07:41:46.817 INFO:teuthology.orchestra.run.trial035.stdout:/usr/bin/ceph: stdout }, 2026-02-24T07:41:46.817 INFO:teuthology.orchestra.run.trial035.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2026-02-24T07:41:46.817 INFO:teuthology.orchestra.run.trial035.stdout:/usr/bin/ceph: stdout "quorum": [ 2026-02-24T07:41:46.817 INFO:teuthology.orchestra.run.trial035.stdout:/usr/bin/ceph: stdout 0 2026-02-24T07:41:46.817 INFO:teuthology.orchestra.run.trial035.stdout:/usr/bin/ceph: stdout ], 2026-02-24T07:41:46.817 INFO:teuthology.orchestra.run.trial035.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2026-02-24T07:41:46.817 INFO:teuthology.orchestra.run.trial035.stdout:/usr/bin/ceph: stdout "a" 2026-02-24T07:41:46.817 INFO:teuthology.orchestra.run.trial035.stdout:/usr/bin/ceph: stdout ], 2026-02-24T07:41:46.817 INFO:teuthology.orchestra.run.trial035.stdout:/usr/bin/ceph: stdout "quorum_age": 2, 2026-02-24T07:41:46.817 INFO:teuthology.orchestra.run.trial035.stdout:/usr/bin/ceph: stdout "monmap": { 2026-02-24T07:41:46.817 INFO:teuthology.orchestra.run.trial035.stdout:/usr/bin/ceph: stdout "epoch": 1, 2026-02-24T07:41:46.817 INFO:teuthology.orchestra.run.trial035.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "tentacle", 2026-02-24T07:41:46.818 INFO:teuthology.orchestra.run.trial035.stdout:/usr/bin/ceph: stdout "num_mons": 1 2026-02-24T07:41:46.818 INFO:teuthology.orchestra.run.trial035.stdout:/usr/bin/ceph: stdout }, 2026-02-24T07:41:46.818 INFO:teuthology.orchestra.run.trial035.stdout:/usr/bin/ceph: stdout "osdmap": { 2026-02-24T07:41:46.818 INFO:teuthology.orchestra.run.trial035.stdout:/usr/bin/ceph: stdout "epoch": 1, 2026-02-24T07:41:46.818 INFO:teuthology.orchestra.run.trial035.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2026-02-24T07:41:46.818 INFO:teuthology.orchestra.run.trial035.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2026-02-24T07:41:46.818 INFO:teuthology.orchestra.run.trial035.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2026-02-24T07:41:46.818 INFO:teuthology.orchestra.run.trial035.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2026-02-24T07:41:46.818 INFO:teuthology.orchestra.run.trial035.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2026-02-24T07:41:46.818 INFO:teuthology.orchestra.run.trial035.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2026-02-24T07:41:46.818 INFO:teuthology.orchestra.run.trial035.stdout:/usr/bin/ceph: stdout }, 2026-02-24T07:41:46.818 INFO:teuthology.orchestra.run.trial035.stdout:/usr/bin/ceph: stdout "pgmap": { 2026-02-24T07:41:46.819 INFO:teuthology.orchestra.run.trial035.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2026-02-24T07:41:46.819 INFO:teuthology.orchestra.run.trial035.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2026-02-24T07:41:46.819 INFO:teuthology.orchestra.run.trial035.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2026-02-24T07:41:46.819 INFO:teuthology.orchestra.run.trial035.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2026-02-24T07:41:46.819 INFO:teuthology.orchestra.run.trial035.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2026-02-24T07:41:46.819 INFO:teuthology.orchestra.run.trial035.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2026-02-24T07:41:46.819 INFO:teuthology.orchestra.run.trial035.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2026-02-24T07:41:46.819 INFO:teuthology.orchestra.run.trial035.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2026-02-24T07:41:46.819 INFO:teuthology.orchestra.run.trial035.stdout:/usr/bin/ceph: stdout }, 2026-02-24T07:41:46.819 INFO:teuthology.orchestra.run.trial035.stdout:/usr/bin/ceph: stdout "fsmap": { 2026-02-24T07:41:46.819 INFO:teuthology.orchestra.run.trial035.stdout:/usr/bin/ceph: stdout "epoch": 1, 2026-02-24T07:41:46.819 INFO:teuthology.orchestra.run.trial035.stdout:/usr/bin/ceph: stdout "btime": "2026-02-24T07:41:42:805557+0000", 2026-02-24T07:41:46.819 INFO:teuthology.orchestra.run.trial035.stdout:/usr/bin/ceph: stdout "by_rank": [], 2026-02-24T07:41:46.820 INFO:teuthology.orchestra.run.trial035.stdout:/usr/bin/ceph: stdout "up:standby": 0 2026-02-24T07:41:46.820 INFO:teuthology.orchestra.run.trial035.stdout:/usr/bin/ceph: stdout }, 2026-02-24T07:41:46.820 INFO:teuthology.orchestra.run.trial035.stdout:/usr/bin/ceph: stdout "mgrmap": { 2026-02-24T07:41:46.820 INFO:teuthology.orchestra.run.trial035.stdout:/usr/bin/ceph: stdout "available": false, 2026-02-24T07:41:46.820 INFO:teuthology.orchestra.run.trial035.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2026-02-24T07:41:46.820 INFO:teuthology.orchestra.run.trial035.stdout:/usr/bin/ceph: stdout "modules": [ 2026-02-24T07:41:46.820 INFO:teuthology.orchestra.run.trial035.stdout:/usr/bin/ceph: stdout "iostat", 2026-02-24T07:41:46.820 INFO:teuthology.orchestra.run.trial035.stdout:/usr/bin/ceph: stdout "nfs" 2026-02-24T07:41:46.820 INFO:teuthology.orchestra.run.trial035.stdout:/usr/bin/ceph: stdout ], 2026-02-24T07:41:46.820 INFO:teuthology.orchestra.run.trial035.stdout:/usr/bin/ceph: stdout "services": {} 2026-02-24T07:41:46.820 INFO:teuthology.orchestra.run.trial035.stdout:/usr/bin/ceph: stdout }, 2026-02-24T07:41:46.820 INFO:teuthology.orchestra.run.trial035.stdout:/usr/bin/ceph: stdout "servicemap": { 2026-02-24T07:41:46.820 INFO:teuthology.orchestra.run.trial035.stdout:/usr/bin/ceph: stdout "epoch": 1, 2026-02-24T07:41:46.820 INFO:teuthology.orchestra.run.trial035.stdout:/usr/bin/ceph: stdout "modified": "2026-02-24T07:41:42.805950+0000", 2026-02-24T07:41:46.821 INFO:teuthology.orchestra.run.trial035.stdout:/usr/bin/ceph: stdout "services": {} 2026-02-24T07:41:46.821 INFO:teuthology.orchestra.run.trial035.stdout:/usr/bin/ceph: stdout }, 2026-02-24T07:41:46.821 INFO:teuthology.orchestra.run.trial035.stdout:/usr/bin/ceph: stdout "progress_events": {} 2026-02-24T07:41:46.821 INFO:teuthology.orchestra.run.trial035.stdout:/usr/bin/ceph: stdout } 2026-02-24T07:41:46.821 INFO:teuthology.orchestra.run.trial035.stdout:/usr/bin/ceph: stderr 2026-02-24T07:41:46.754+0000 7f6a4658a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:41:46.821 INFO:teuthology.orchestra.run.trial035.stdout:/usr/bin/ceph: stderr 2026-02-24T07:41:46.757+0000 7f6a4658a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:41:46.821 INFO:teuthology.orchestra.run.trial035.stdout:mgr not available, waiting (2/15)... 2026-02-24T07:41:46.954 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:41:46 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/508678359' entity='client.admin' cmd={"prefix": "status", "format": "json-pretty"} : dispatch 2026-02-24T07:41:48.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:41:47 trial035 ceph-mon[39284]: Activating manager daemon x 2026-02-24T07:41:48.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:41:47 trial035 ceph-mon[39284]: mgrmap e2: x(active, starting, since 0.00276061s) 2026-02-24T07:41:48.205 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:41:47 trial035 ceph-mon[39284]: from='mgr.14102 10.20.193.35:0/1724789305' entity='mgr.x' cmd={"prefix": "mds metadata"} : dispatch 2026-02-24T07:41:48.205 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:41:47 trial035 ceph-mon[39284]: from='mgr.14102 10.20.193.35:0/1724789305' entity='mgr.x' cmd={"prefix": "osd metadata"} : dispatch 2026-02-24T07:41:48.205 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:41:47 trial035 ceph-mon[39284]: from='mgr.14102 10.20.193.35:0/1724789305' entity='mgr.x' cmd={"prefix": "mon metadata"} : dispatch 2026-02-24T07:41:48.205 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:41:47 trial035 ceph-mon[39284]: from='mgr.14102 10.20.193.35:0/1724789305' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2026-02-24T07:41:48.205 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:41:47 trial035 ceph-mon[39284]: from='mgr.14102 10.20.193.35:0/1724789305' entity='mgr.x' cmd={"prefix": "mgr metadata", "who": "x", "id": "x"} : dispatch 2026-02-24T07:41:48.205 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:41:47 trial035 ceph-mon[39284]: Manager daemon x is now available 2026-02-24T07:41:48.205 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:41:47 trial035 ceph-mon[39284]: from='mgr.14102 10.20.193.35:0/1724789305' entity='mgr.x' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"} : dispatch 2026-02-24T07:41:48.205 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:41:47 trial035 ceph-mon[39284]: from='mgr.14102 10.20.193.35:0/1724789305' entity='mgr.x' 2026-02-24T07:41:48.205 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:41:47 trial035 ceph-mon[39284]: from='mgr.14102 10.20.193.35:0/1724789305' entity='mgr.x' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"} : dispatch 2026-02-24T07:41:48.206 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:41:47 trial035 ceph-mon[39284]: from='mgr.14102 10.20.193.35:0/1724789305' entity='mgr.x' 2026-02-24T07:41:48.206 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:41:47 trial035 ceph-mon[39284]: from='mgr.14102 10.20.193.35:0/1724789305' entity='mgr.x' 2026-02-24T07:41:49.101 INFO:teuthology.orchestra.run.trial035.stdout:/usr/bin/ceph: stdout 2026-02-24T07:41:49.102 INFO:teuthology.orchestra.run.trial035.stdout:/usr/bin/ceph: stdout { 2026-02-24T07:41:49.102 INFO:teuthology.orchestra.run.trial035.stdout:/usr/bin/ceph: stdout "fsid": "3a5983d2-1154-11f1-9c05-d404e6e7d460", 2026-02-24T07:41:49.102 INFO:teuthology.orchestra.run.trial035.stdout:/usr/bin/ceph: stdout "health": { 2026-02-24T07:41:49.102 INFO:teuthology.orchestra.run.trial035.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2026-02-24T07:41:49.102 INFO:teuthology.orchestra.run.trial035.stdout:/usr/bin/ceph: stdout "checks": {}, 2026-02-24T07:41:49.102 INFO:teuthology.orchestra.run.trial035.stdout:/usr/bin/ceph: stdout "mutes": [] 2026-02-24T07:41:49.102 INFO:teuthology.orchestra.run.trial035.stdout:/usr/bin/ceph: stdout }, 2026-02-24T07:41:49.102 INFO:teuthology.orchestra.run.trial035.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2026-02-24T07:41:49.102 INFO:teuthology.orchestra.run.trial035.stdout:/usr/bin/ceph: stdout "quorum": [ 2026-02-24T07:41:49.103 INFO:teuthology.orchestra.run.trial035.stdout:/usr/bin/ceph: stdout 0 2026-02-24T07:41:49.103 INFO:teuthology.orchestra.run.trial035.stdout:/usr/bin/ceph: stdout ], 2026-02-24T07:41:49.103 INFO:teuthology.orchestra.run.trial035.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2026-02-24T07:41:49.103 INFO:teuthology.orchestra.run.trial035.stdout:/usr/bin/ceph: stdout "a" 2026-02-24T07:41:49.103 INFO:teuthology.orchestra.run.trial035.stdout:/usr/bin/ceph: stdout ], 2026-02-24T07:41:49.103 INFO:teuthology.orchestra.run.trial035.stdout:/usr/bin/ceph: stdout "quorum_age": 5, 2026-02-24T07:41:49.103 INFO:teuthology.orchestra.run.trial035.stdout:/usr/bin/ceph: stdout "monmap": { 2026-02-24T07:41:49.103 INFO:teuthology.orchestra.run.trial035.stdout:/usr/bin/ceph: stdout "epoch": 1, 2026-02-24T07:41:49.103 INFO:teuthology.orchestra.run.trial035.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "tentacle", 2026-02-24T07:41:49.103 INFO:teuthology.orchestra.run.trial035.stdout:/usr/bin/ceph: stdout "num_mons": 1 2026-02-24T07:41:49.103 INFO:teuthology.orchestra.run.trial035.stdout:/usr/bin/ceph: stdout }, 2026-02-24T07:41:49.104 INFO:teuthology.orchestra.run.trial035.stdout:/usr/bin/ceph: stdout "osdmap": { 2026-02-24T07:41:49.104 INFO:teuthology.orchestra.run.trial035.stdout:/usr/bin/ceph: stdout "epoch": 1, 2026-02-24T07:41:49.104 INFO:teuthology.orchestra.run.trial035.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2026-02-24T07:41:49.104 INFO:teuthology.orchestra.run.trial035.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2026-02-24T07:41:49.104 INFO:teuthology.orchestra.run.trial035.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2026-02-24T07:41:49.104 INFO:teuthology.orchestra.run.trial035.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2026-02-24T07:41:49.104 INFO:teuthology.orchestra.run.trial035.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2026-02-24T07:41:49.104 INFO:teuthology.orchestra.run.trial035.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2026-02-24T07:41:49.104 INFO:teuthology.orchestra.run.trial035.stdout:/usr/bin/ceph: stdout }, 2026-02-24T07:41:49.104 INFO:teuthology.orchestra.run.trial035.stdout:/usr/bin/ceph: stdout "pgmap": { 2026-02-24T07:41:49.104 INFO:teuthology.orchestra.run.trial035.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2026-02-24T07:41:49.105 INFO:teuthology.orchestra.run.trial035.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2026-02-24T07:41:49.105 INFO:teuthology.orchestra.run.trial035.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2026-02-24T07:41:49.105 INFO:teuthology.orchestra.run.trial035.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2026-02-24T07:41:49.105 INFO:teuthology.orchestra.run.trial035.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2026-02-24T07:41:49.105 INFO:teuthology.orchestra.run.trial035.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2026-02-24T07:41:49.105 INFO:teuthology.orchestra.run.trial035.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2026-02-24T07:41:49.105 INFO:teuthology.orchestra.run.trial035.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2026-02-24T07:41:49.105 INFO:teuthology.orchestra.run.trial035.stdout:/usr/bin/ceph: stdout }, 2026-02-24T07:41:49.105 INFO:teuthology.orchestra.run.trial035.stdout:/usr/bin/ceph: stdout "fsmap": { 2026-02-24T07:41:49.105 INFO:teuthology.orchestra.run.trial035.stdout:/usr/bin/ceph: stdout "epoch": 1, 2026-02-24T07:41:49.105 INFO:teuthology.orchestra.run.trial035.stdout:/usr/bin/ceph: stdout "btime": "2026-02-24T07:41:42:805557+0000", 2026-02-24T07:41:49.106 INFO:teuthology.orchestra.run.trial035.stdout:/usr/bin/ceph: stdout "by_rank": [], 2026-02-24T07:41:49.106 INFO:teuthology.orchestra.run.trial035.stdout:/usr/bin/ceph: stdout "up:standby": 0 2026-02-24T07:41:49.106 INFO:teuthology.orchestra.run.trial035.stdout:/usr/bin/ceph: stdout }, 2026-02-24T07:41:49.106 INFO:teuthology.orchestra.run.trial035.stdout:/usr/bin/ceph: stdout "mgrmap": { 2026-02-24T07:41:49.106 INFO:teuthology.orchestra.run.trial035.stdout:/usr/bin/ceph: stdout "available": true, 2026-02-24T07:41:49.106 INFO:teuthology.orchestra.run.trial035.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2026-02-24T07:41:49.106 INFO:teuthology.orchestra.run.trial035.stdout:/usr/bin/ceph: stdout "modules": [ 2026-02-24T07:41:49.106 INFO:teuthology.orchestra.run.trial035.stdout:/usr/bin/ceph: stdout "iostat", 2026-02-24T07:41:49.106 INFO:teuthology.orchestra.run.trial035.stdout:/usr/bin/ceph: stdout "nfs" 2026-02-24T07:41:49.106 INFO:teuthology.orchestra.run.trial035.stdout:/usr/bin/ceph: stdout ], 2026-02-24T07:41:49.107 INFO:teuthology.orchestra.run.trial035.stdout:/usr/bin/ceph: stdout "services": {} 2026-02-24T07:41:49.107 INFO:teuthology.orchestra.run.trial035.stdout:/usr/bin/ceph: stdout }, 2026-02-24T07:41:49.107 INFO:teuthology.orchestra.run.trial035.stdout:/usr/bin/ceph: stdout "servicemap": { 2026-02-24T07:41:49.107 INFO:teuthology.orchestra.run.trial035.stdout:/usr/bin/ceph: stdout "epoch": 1, 2026-02-24T07:41:49.107 INFO:teuthology.orchestra.run.trial035.stdout:/usr/bin/ceph: stdout "modified": "2026-02-24T07:41:42.805950+0000", 2026-02-24T07:41:49.107 INFO:teuthology.orchestra.run.trial035.stdout:/usr/bin/ceph: stdout "services": {} 2026-02-24T07:41:49.107 INFO:teuthology.orchestra.run.trial035.stdout:/usr/bin/ceph: stdout }, 2026-02-24T07:41:49.107 INFO:teuthology.orchestra.run.trial035.stdout:/usr/bin/ceph: stdout "progress_events": {} 2026-02-24T07:41:49.107 INFO:teuthology.orchestra.run.trial035.stdout:/usr/bin/ceph: stdout } 2026-02-24T07:41:49.107 INFO:teuthology.orchestra.run.trial035.stdout:/usr/bin/ceph: stderr 2026-02-24T07:41:48.903+0000 7f77c4482640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:41:49.107 INFO:teuthology.orchestra.run.trial035.stdout:/usr/bin/ceph: stderr 2026-02-24T07:41:48.906+0000 7f77c4482640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:41:49.107 INFO:teuthology.orchestra.run.trial035.stdout:mgr is available 2026-02-24T07:41:49.316 INFO:teuthology.orchestra.run.trial035.stdout:/usr/bin/ceph: stdout 2026-02-24T07:41:49.316 INFO:teuthology.orchestra.run.trial035.stdout:/usr/bin/ceph: stdout [global] 2026-02-24T07:41:49.316 INFO:teuthology.orchestra.run.trial035.stdout:/usr/bin/ceph: stdout fsid = 3a5983d2-1154-11f1-9c05-d404e6e7d460 2026-02-24T07:41:49.316 INFO:teuthology.orchestra.run.trial035.stdout:/usr/bin/ceph: stdout mon_cluster_log_file_level = debug 2026-02-24T07:41:49.316 INFO:teuthology.orchestra.run.trial035.stdout:/usr/bin/ceph: stdout mon_host = [v2:10.20.193.35:3300,v1:10.20.193.35:6789] 2026-02-24T07:41:49.316 INFO:teuthology.orchestra.run.trial035.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2026-02-24T07:41:49.316 INFO:teuthology.orchestra.run.trial035.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2026-02-24T07:41:49.317 INFO:teuthology.orchestra.run.trial035.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2026-02-24T07:41:49.317 INFO:teuthology.orchestra.run.trial035.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2026-02-24T07:41:49.317 INFO:teuthology.orchestra.run.trial035.stdout:/usr/bin/ceph: stdout 2026-02-24T07:41:49.317 INFO:teuthology.orchestra.run.trial035.stdout:/usr/bin/ceph: stdout [mgr] 2026-02-24T07:41:49.317 INFO:teuthology.orchestra.run.trial035.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2026-02-24T07:41:49.317 INFO:teuthology.orchestra.run.trial035.stdout:/usr/bin/ceph: stdout 2026-02-24T07:41:49.317 INFO:teuthology.orchestra.run.trial035.stdout:/usr/bin/ceph: stdout [osd] 2026-02-24T07:41:49.317 INFO:teuthology.orchestra.run.trial035.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2026-02-24T07:41:49.317 INFO:teuthology.orchestra.run.trial035.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2026-02-24T07:41:49.317 INFO:teuthology.orchestra.run.trial035.stdout:/usr/bin/ceph: stderr 2026-02-24T07:41:49.181+0000 7f40a4689640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:41:49.317 INFO:teuthology.orchestra.run.trial035.stdout:/usr/bin/ceph: stderr 2026-02-24T07:41:49.185+0000 7f40a4689640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:41:49.317 INFO:teuthology.orchestra.run.trial035.stdout:Enabling cephadm module... 2026-02-24T07:41:49.365 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:41:49 trial035 ceph-mon[39284]: mgrmap e3: x(active, since 1.00501s) 2026-02-24T07:41:49.365 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:41:49 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/1717510267' entity='client.admin' cmd={"prefix": "status", "format": "json-pretty"} : dispatch 2026-02-24T07:41:50.265 INFO:teuthology.orchestra.run.trial035.stdout:/usr/bin/ceph: stderr 2026-02-24T07:41:49.407+0000 7fe3a27f5640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:41:50.265 INFO:teuthology.orchestra.run.trial035.stdout:/usr/bin/ceph: stderr 2026-02-24T07:41:49.411+0000 7fe3a27f5640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:41:50.506 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:41:50 trial035 ceph-mon[39284]: mgrmap e4: x(active, since 2s) 2026-02-24T07:41:50.506 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:41:50 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/768996287' entity='client.admin' cmd={"prefix": "config assimilate-conf"} : dispatch 2026-02-24T07:41:50.506 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:41:50 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/963107956' entity='client.admin' cmd={"prefix": "mgr module enable", "module": "cephadm"} : dispatch 2026-02-24T07:41:50.507 INFO:journalctl@ceph.mgr.x.trial035.stdout:Feb 24 07:41:50 trial035 ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-mgr-x[39621]: ignoring --setuser ceph since I am not root 2026-02-24T07:41:50.507 INFO:journalctl@ceph.mgr.x.trial035.stdout:Feb 24 07:41:50 trial035 ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-mgr-x[39621]: ignoring --setgroup ceph since I am not root 2026-02-24T07:41:50.507 INFO:journalctl@ceph.mgr.x.trial035.stdout:Feb 24 07:41:50 trial035 ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-mgr-x[39621]: 2026-02-24T07:41:50.278+0000 7f16e02dbfc0 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:41:50.507 INFO:journalctl@ceph.mgr.x.trial035.stdout:Feb 24 07:41:50 trial035 ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-mgr-x[39621]: 2026-02-24T07:41:50.282+0000 7f16e02dbfc0 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:41:50.507 INFO:journalctl@ceph.mgr.x.trial035.stdout:Feb 24 07:41:50 trial035 ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-mgr-x[39621]: 2026-02-24T07:41:50.282+0000 7f16e02dbfc0 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:41:50.527 INFO:teuthology.orchestra.run.trial035.stdout:/usr/bin/ceph: stdout { 2026-02-24T07:41:50.527 INFO:teuthology.orchestra.run.trial035.stdout:/usr/bin/ceph: stdout "epoch": 5, 2026-02-24T07:41:50.527 INFO:teuthology.orchestra.run.trial035.stdout:/usr/bin/ceph: stdout "available": true, 2026-02-24T07:41:50.527 INFO:teuthology.orchestra.run.trial035.stdout:/usr/bin/ceph: stdout "active_name": "x", 2026-02-24T07:41:50.527 INFO:teuthology.orchestra.run.trial035.stdout:/usr/bin/ceph: stdout "num_standby": 0 2026-02-24T07:41:50.528 INFO:teuthology.orchestra.run.trial035.stdout:/usr/bin/ceph: stdout } 2026-02-24T07:41:50.528 INFO:teuthology.orchestra.run.trial035.stdout:/usr/bin/ceph: stderr 2026-02-24T07:41:50.359+0000 7f9f3af79640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:41:50.528 INFO:teuthology.orchestra.run.trial035.stdout:/usr/bin/ceph: stderr 2026-02-24T07:41:50.362+0000 7f9f3af79640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:41:50.528 INFO:teuthology.orchestra.run.trial035.stdout:Waiting for the mgr to restart... 2026-02-24T07:41:50.528 INFO:teuthology.orchestra.run.trial035.stdout:Waiting for mgr epoch 5... 2026-02-24T07:41:51.704 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:41:51 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/963107956' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2026-02-24T07:41:51.704 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:41:51 trial035 ceph-mon[39284]: mgrmap e5: x(active, since 3s) 2026-02-24T07:41:51.704 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:41:51 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/2472150649' entity='client.admin' cmd={"prefix": "mgr stat"} : dispatch 2026-02-24T07:41:51.705 INFO:journalctl@ceph.mgr.x.trial035.stdout:Feb 24 07:41:51 trial035 ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-mgr-x[39621]: /lib64/python3.9/site-packages/scipy/__init__.py:73: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2026-02-24T07:41:51.705 INFO:journalctl@ceph.mgr.x.trial035.stdout:Feb 24 07:41:51 trial035 ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-mgr-x[39621]: Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2026-02-24T07:41:51.705 INFO:journalctl@ceph.mgr.x.trial035.stdout:Feb 24 07:41:51 trial035 ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-mgr-x[39621]: from numpy import show_config as show_numpy_config 2026-02-24T07:41:53.454 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:41:53 trial035 ceph-mon[39284]: Active manager daemon x restarted 2026-02-24T07:41:53.454 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:41:53 trial035 ceph-mon[39284]: Activating manager daemon x 2026-02-24T07:41:53.454 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:41:53 trial035 ceph-mon[39284]: osdmap e2: 0 total, 0 up, 0 in 2026-02-24T07:41:53.454 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:41:53 trial035 ceph-mon[39284]: mgrmap e6: x(active, starting, since 0.00268374s) 2026-02-24T07:41:53.454 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:41:53 trial035 ceph-mon[39284]: from='mgr.14120 10.20.193.35:0/3606876491' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2026-02-24T07:41:53.454 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:41:53 trial035 ceph-mon[39284]: from='mgr.14120 10.20.193.35:0/3606876491' entity='mgr.x' cmd={"prefix": "mgr metadata", "who": "x", "id": "x"} : dispatch 2026-02-24T07:41:53.454 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:41:53 trial035 ceph-mon[39284]: from='mgr.14120 10.20.193.35:0/3606876491' entity='mgr.x' cmd={"prefix": "mds metadata"} : dispatch 2026-02-24T07:41:53.454 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:41:53 trial035 ceph-mon[39284]: from='mgr.14120 10.20.193.35:0/3606876491' entity='mgr.x' cmd={"prefix": "osd metadata"} : dispatch 2026-02-24T07:41:53.454 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:41:53 trial035 ceph-mon[39284]: from='mgr.14120 10.20.193.35:0/3606876491' entity='mgr.x' cmd={"prefix": "mon metadata"} : dispatch 2026-02-24T07:41:53.454 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:41:53 trial035 ceph-mon[39284]: Manager daemon x is now available 2026-02-24T07:41:54.034 INFO:teuthology.orchestra.run.trial035.stdout:/usr/bin/ceph: stdout { 2026-02-24T07:41:54.035 INFO:teuthology.orchestra.run.trial035.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2026-02-24T07:41:54.035 INFO:teuthology.orchestra.run.trial035.stdout:/usr/bin/ceph: stdout "initialized": true 2026-02-24T07:41:54.035 INFO:teuthology.orchestra.run.trial035.stdout:/usr/bin/ceph: stdout } 2026-02-24T07:41:54.035 INFO:teuthology.orchestra.run.trial035.stdout:/usr/bin/ceph: stderr 2026-02-24T07:41:50.628+0000 7ffae0331640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:41:54.035 INFO:teuthology.orchestra.run.trial035.stdout:/usr/bin/ceph: stderr 2026-02-24T07:41:50.631+0000 7ffae0331640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:41:54.035 INFO:teuthology.orchestra.run.trial035.stdout:mgr epoch 5 is available 2026-02-24T07:41:54.035 INFO:teuthology.orchestra.run.trial035.stdout:Verifying orchestrator module is enabled... 2026-02-24T07:41:54.954 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:41:54 trial035 ceph-mon[39284]: from='mgr.14120 10.20.193.35:0/3606876491' entity='mgr.x' 2026-02-24T07:41:54.954 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:41:54 trial035 ceph-mon[39284]: from='mgr.14120 10.20.193.35:0/3606876491' entity='mgr.x' 2026-02-24T07:41:54.954 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:41:54 trial035 ceph-mon[39284]: Found migration_current of "None". Setting to last migration. 2026-02-24T07:41:54.954 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:41:54 trial035 ceph-mon[39284]: from='mgr.14120 10.20.193.35:0/3606876491' entity='mgr.x' 2026-02-24T07:41:54.955 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:41:54 trial035 ceph-mon[39284]: from='mgr.14120 10.20.193.35:0/3606876491' entity='mgr.x' 2026-02-24T07:41:54.955 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:41:54 trial035 ceph-mon[39284]: from='mgr.14120 10.20.193.35:0/3606876491' entity='mgr.x' 2026-02-24T07:41:54.955 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:41:54 trial035 ceph-mon[39284]: from='mgr.14120 10.20.193.35:0/3606876491' entity='mgr.x' 2026-02-24T07:41:54.955 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:41:54 trial035 ceph-mon[39284]: from='mgr.14120 10.20.193.35:0/3606876491' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2026-02-24T07:41:54.955 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:41:54 trial035 ceph-mon[39284]: from='mgr.14120 10.20.193.35:0/3606876491' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2026-02-24T07:41:54.955 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:41:54 trial035 ceph-mon[39284]: from='mgr.14120 10.20.193.35:0/3606876491' entity='mgr.x' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"} : dispatch 2026-02-24T07:41:54.955 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:41:54 trial035 ceph-mon[39284]: from='mgr.14120 10.20.193.35:0/3606876491' entity='mgr.x' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"} : dispatch 2026-02-24T07:41:54.955 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:41:54 trial035 ceph-mon[39284]: mgrmap e7: x(active, since 1.00345s) 2026-02-24T07:41:54.955 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:41:54 trial035 ceph-mon[39284]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2026-02-24T07:41:54.955 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:41:54 trial035 ceph-mon[39284]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2026-02-24T07:41:54.955 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:41:54 trial035 ceph-mon[39284]: from='mgr.14120 10.20.193.35:0/3606876491' entity='mgr.x' 2026-02-24T07:41:54.955 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:41:54 trial035 ceph-mon[39284]: from='mgr.14120 10.20.193.35:0/3606876491' entity='mgr.x' 2026-02-24T07:41:54.955 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:41:54 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/594656051' entity='client.admin' cmd={"prefix": "mgr module enable", "module": "orchestrator"} : dispatch 2026-02-24T07:41:54.955 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:41:54 trial035 ceph-mon[39284]: [24/Feb/2026:07:41:54] ENGINE Bus STARTING 2026-02-24T07:41:54.955 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:41:54 trial035 ceph-mon[39284]: from='mgr.14120 10.20.193.35:0/3606876491' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2026-02-24T07:41:55.128 INFO:teuthology.orchestra.run.trial035.stdout:/usr/bin/ceph: stderr 2026-02-24T07:41:54.117+0000 7f0034ec5640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:41:55.128 INFO:teuthology.orchestra.run.trial035.stdout:/usr/bin/ceph: stderr 2026-02-24T07:41:54.122+0000 7f0034ec5640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:41:55.128 INFO:teuthology.orchestra.run.trial035.stdout:/usr/bin/ceph: stderr module 'orchestrator' is already enabled (always-on) 2026-02-24T07:41:55.128 INFO:teuthology.orchestra.run.trial035.stdout:Setting orchestrator backend to cephadm... 2026-02-24T07:41:55.360 INFO:teuthology.orchestra.run.trial035.stdout:/usr/bin/ceph: stderr 2026-02-24T07:41:55.213+0000 7f8d0f927640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:41:55.360 INFO:teuthology.orchestra.run.trial035.stdout:/usr/bin/ceph: stderr 2026-02-24T07:41:55.219+0000 7f8d0f927640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:41:55.583 INFO:teuthology.orchestra.run.trial035.stdout:/usr/bin/ceph: stdout value unchanged 2026-02-24T07:41:55.583 INFO:teuthology.orchestra.run.trial035.stdout:/usr/bin/ceph: stderr 2026-02-24T07:41:55.449+0000 7fcf56f27640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:41:55.583 INFO:teuthology.orchestra.run.trial035.stdout:/usr/bin/ceph: stderr 2026-02-24T07:41:55.453+0000 7fcf56f27640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:41:55.583 INFO:teuthology.orchestra.run.trial035.stdout:Generating ssh key... 2026-02-24T07:41:55.835 INFO:teuthology.orchestra.run.trial035.stdout:/usr/bin/ceph: stderr 2026-02-24T07:41:55.680+0000 7f4726e53640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:41:55.835 INFO:teuthology.orchestra.run.trial035.stdout:/usr/bin/ceph: stderr 2026-02-24T07:41:55.683+0000 7f4725bca640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:41:55.882 INFO:journalctl@ceph.mgr.x.trial035.stdout:Feb 24 07:41:55 trial035 ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-mgr-x[39621]: Generating public/private ed25519 key pair. 2026-02-24T07:41:55.882 INFO:journalctl@ceph.mgr.x.trial035.stdout:Feb 24 07:41:55 trial035 ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-mgr-x[39621]: Your identification has been saved in /tmp/tmpva3y21w9/key 2026-02-24T07:41:55.882 INFO:journalctl@ceph.mgr.x.trial035.stdout:Feb 24 07:41:55 trial035 ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-mgr-x[39621]: Your public key has been saved in /tmp/tmpva3y21w9/key.pub 2026-02-24T07:41:55.882 INFO:journalctl@ceph.mgr.x.trial035.stdout:Feb 24 07:41:55 trial035 ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-mgr-x[39621]: The key fingerprint is: 2026-02-24T07:41:55.882 INFO:journalctl@ceph.mgr.x.trial035.stdout:Feb 24 07:41:55 trial035 ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-mgr-x[39621]: SHA256:0zv1aBohswG1ONw3FYu6VKbVObrDfNBYhaq1ijOkCU0 ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460 2026-02-24T07:41:55.882 INFO:journalctl@ceph.mgr.x.trial035.stdout:Feb 24 07:41:55 trial035 ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-mgr-x[39621]: The key's randomart image is: 2026-02-24T07:41:55.882 INFO:journalctl@ceph.mgr.x.trial035.stdout:Feb 24 07:41:55 trial035 ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-mgr-x[39621]: +--[ED25519 256]--+ 2026-02-24T07:41:55.882 INFO:journalctl@ceph.mgr.x.trial035.stdout:Feb 24 07:41:55 trial035 ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-mgr-x[39621]: | . oo. | 2026-02-24T07:41:55.882 INFO:journalctl@ceph.mgr.x.trial035.stdout:Feb 24 07:41:55 trial035 ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-mgr-x[39621]: | . + . +.+ | 2026-02-24T07:41:55.883 INFO:journalctl@ceph.mgr.x.trial035.stdout:Feb 24 07:41:55 trial035 ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-mgr-x[39621]: | = o B.* | 2026-02-24T07:41:55.883 INFO:journalctl@ceph.mgr.x.trial035.stdout:Feb 24 07:41:55 trial035 ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-mgr-x[39621]: | E o Oo* . | 2026-02-24T07:41:55.883 INFO:journalctl@ceph.mgr.x.trial035.stdout:Feb 24 07:41:55 trial035 ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-mgr-x[39621]: | o So*.o | 2026-02-24T07:41:55.883 INFO:journalctl@ceph.mgr.x.trial035.stdout:Feb 24 07:41:55 trial035 ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-mgr-x[39621]: | . . ...X.* o | 2026-02-24T07:41:55.883 INFO:journalctl@ceph.mgr.x.trial035.stdout:Feb 24 07:41:55 trial035 ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-mgr-x[39621]: | . + .o.O + . | 2026-02-24T07:41:55.883 INFO:journalctl@ceph.mgr.x.trial035.stdout:Feb 24 07:41:55 trial035 ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-mgr-x[39621]: | o + . B | 2026-02-24T07:41:55.883 INFO:journalctl@ceph.mgr.x.trial035.stdout:Feb 24 07:41:55 trial035 ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-mgr-x[39621]: | o . | 2026-02-24T07:41:55.883 INFO:journalctl@ceph.mgr.x.trial035.stdout:Feb 24 07:41:55 trial035 ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-mgr-x[39621]: +----[SHA256]-----+ 2026-02-24T07:41:56.055 INFO:teuthology.orchestra.run.trial035.stdout:/usr/bin/ceph: stdout ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIMyTwY6dp+NwS9foHhJghlyw1hmbgXD0owS8aw7WYPZY ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460 2026-02-24T07:41:56.055 INFO:teuthology.orchestra.run.trial035.stdout:/usr/bin/ceph: stderr 2026-02-24T07:41:55.922+0000 7f3dd3ca2640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:41:56.055 INFO:teuthology.orchestra.run.trial035.stdout:/usr/bin/ceph: stderr 2026-02-24T07:41:55.925+0000 7f3dd2a19640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:41:56.055 INFO:teuthology.orchestra.run.trial035.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2026-02-24T07:41:56.056 INFO:teuthology.orchestra.run.trial035.stdout:Adding key to root@localhost authorized_keys... 2026-02-24T07:41:56.056 INFO:teuthology.orchestra.run.trial035.stdout:Adding host trial035... 2026-02-24T07:41:56.157 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:41:56 trial035 ceph-mon[39284]: [24/Feb/2026:07:41:54] ENGINE Serving on https://10.20.193.35:7150 2026-02-24T07:41:56.157 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:41:56 trial035 ceph-mon[39284]: [24/Feb/2026:07:41:54] ENGINE Client ('10.20.193.35', 50758) lost — peer dropped the TLS connection suddenly, during handshake: (6, 'TLS/SSL connection has been closed (EOF) (_ssl.c:1147)') 2026-02-24T07:41:56.157 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:41:56 trial035 ceph-mon[39284]: [24/Feb/2026:07:41:54] ENGINE Serving on http://10.20.193.35:8765 2026-02-24T07:41:56.157 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:41:56 trial035 ceph-mon[39284]: [24/Feb/2026:07:41:54] ENGINE Bus STARTED 2026-02-24T07:41:56.157 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:41:56 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/594656051' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "orchestrator"}]': finished 2026-02-24T07:41:56.157 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:41:56 trial035 ceph-mon[39284]: mgrmap e8: x(active, since 2s) 2026-02-24T07:41:56.157 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:41:56 trial035 ceph-mon[39284]: from='mgr.14120 10.20.193.35:0/3606876491' entity='mgr.x' 2026-02-24T07:41:56.157 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:41:56 trial035 ceph-mon[39284]: from='mgr.14120 10.20.193.35:0/3606876491' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2026-02-24T07:41:56.158 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:41:56 trial035 ceph-mon[39284]: from='mgr.14120 10.20.193.35:0/3606876491' entity='mgr.x' 2026-02-24T07:41:56.158 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:41:56 trial035 ceph-mon[39284]: from='mgr.14120 10.20.193.35:0/3606876491' entity='mgr.x' 2026-02-24T07:41:57.454 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:41:57 trial035 ceph-mon[39284]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2026-02-24T07:41:57.454 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:41:57 trial035 ceph-mon[39284]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2026-02-24T07:41:57.454 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:41:57 trial035 ceph-mon[39284]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2026-02-24T07:41:57.454 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:41:57 trial035 ceph-mon[39284]: Generating ssh key... 2026-02-24T07:41:57.454 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:41:57 trial035 ceph-mon[39284]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2026-02-24T07:41:57.454 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:41:57 trial035 ceph-mon[39284]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "trial035", "addr": "10.20.193.35", "target": ["mon-mgr", ""]}]: dispatch 2026-02-24T07:41:57.890 INFO:teuthology.orchestra.run.trial035.stdout:/usr/bin/ceph: stdout Added host 'trial035' with addr '10.20.193.35' 2026-02-24T07:41:57.890 INFO:teuthology.orchestra.run.trial035.stdout:/usr/bin/ceph: stderr 2026-02-24T07:41:56.151+0000 7f6185838640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:41:57.890 INFO:teuthology.orchestra.run.trial035.stdout:/usr/bin/ceph: stderr 2026-02-24T07:41:56.156+0000 7f6185838640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:41:57.890 INFO:teuthology.orchestra.run.trial035.stdout:Deploying unmanaged mon service... 2026-02-24T07:41:58.143 INFO:teuthology.orchestra.run.trial035.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2026-02-24T07:41:58.143 INFO:teuthology.orchestra.run.trial035.stdout:/usr/bin/ceph: stderr 2026-02-24T07:41:57.996+0000 7fbf943d4640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:41:58.143 INFO:teuthology.orchestra.run.trial035.stdout:/usr/bin/ceph: stderr 2026-02-24T07:41:58.001+0000 7fbf943d4640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:41:58.143 INFO:teuthology.orchestra.run.trial035.stdout:Deploying unmanaged mgr service... 2026-02-24T07:41:58.356 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:41:58 trial035 ceph-mon[39284]: Deploying cephadm binary to trial035 2026-02-24T07:41:58.356 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:41:58 trial035 ceph-mon[39284]: from='mgr.14120 10.20.193.35:0/3606876491' entity='mgr.x' 2026-02-24T07:41:58.356 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:41:58 trial035 ceph-mon[39284]: from='mgr.14120 10.20.193.35:0/3606876491' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2026-02-24T07:41:58.386 INFO:teuthology.orchestra.run.trial035.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2026-02-24T07:41:58.387 INFO:teuthology.orchestra.run.trial035.stdout:/usr/bin/ceph: stderr 2026-02-24T07:41:58.236+0000 7f6fd4d8f640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:41:58.387 INFO:teuthology.orchestra.run.trial035.stdout:/usr/bin/ceph: stderr 2026-02-24T07:41:58.240+0000 7f6fd4d8f640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:41:58.621 INFO:teuthology.orchestra.run.trial035.stdout:/usr/bin/ceph: stderr 2026-02-24T07:41:58.483+0000 7f9b4e3d2640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:41:58.621 INFO:teuthology.orchestra.run.trial035.stdout:/usr/bin/ceph: stderr 2026-02-24T07:41:58.487+0000 7f9b4e3d2640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:41:58.865 INFO:teuthology.orchestra.run.trial035.stdout:/usr/bin/ceph: stderr 2026-02-24T07:41:58.721+0000 7fd612a66640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:41:58.866 INFO:teuthology.orchestra.run.trial035.stdout:/usr/bin/ceph: stderr 2026-02-24T07:41:58.727+0000 7fd612a66640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:41:58.866 INFO:teuthology.orchestra.run.trial035.stdout:Enabling the dashboard module... 2026-02-24T07:41:59.446 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:41:59 trial035 ceph-mon[39284]: Added host trial035 2026-02-24T07:41:59.446 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:41:59 trial035 ceph-mon[39284]: from='client.14144 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2026-02-24T07:41:59.446 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:41:59 trial035 ceph-mon[39284]: Saving service mon spec with placement count:5 2026-02-24T07:41:59.446 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:41:59 trial035 ceph-mon[39284]: from='mgr.14120 10.20.193.35:0/3606876491' entity='mgr.x' 2026-02-24T07:41:59.446 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:41:59 trial035 ceph-mon[39284]: from='mgr.14120 10.20.193.35:0/3606876491' entity='mgr.x' 2026-02-24T07:41:59.447 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:41:59 trial035 ceph-mon[39284]: from='mgr.14120 10.20.193.35:0/3606876491' entity='mgr.x' 2026-02-24T07:41:59.447 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:41:59 trial035 ceph-mon[39284]: from='mgr.14120 10.20.193.35:0/3606876491' entity='mgr.x' 2026-02-24T07:41:59.447 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:41:59 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/3043493185' entity='client.admin' 2026-02-24T07:41:59.447 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:41:59 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/3344251478' entity='client.admin' 2026-02-24T07:41:59.447 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:41:59 trial035 ceph-mon[39284]: from='mgr.14120 10.20.193.35:0/3606876491' entity='mgr.x' 2026-02-24T07:41:59.447 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:41:59 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/2182893116' entity='client.admin' cmd={"prefix": "mgr module enable", "module": "dashboard"} : dispatch 2026-02-24T07:41:59.945 INFO:teuthology.orchestra.run.trial035.stdout:/usr/bin/ceph: stderr 2026-02-24T07:41:58.956+0000 7fb59ce21640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:41:59.945 INFO:teuthology.orchestra.run.trial035.stdout:/usr/bin/ceph: stderr 2026-02-24T07:41:58.960+0000 7fb59ce21640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:42:00.195 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:00 trial035 ceph-mon[39284]: from='client.14146 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2026-02-24T07:42:00.195 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:00 trial035 ceph-mon[39284]: Saving service mgr spec with placement count:2 2026-02-24T07:42:00.195 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:00 trial035 ceph-mon[39284]: from='mgr.14120 10.20.193.35:0/3606876491' entity='mgr.x' 2026-02-24T07:42:00.195 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:00 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/2182893116' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2026-02-24T07:42:00.195 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:00 trial035 ceph-mon[39284]: mgrmap e9: x(active, since 6s) 2026-02-24T07:42:00.195 INFO:journalctl@ceph.mgr.x.trial035.stdout:Feb 24 07:41:59 trial035 ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-mgr-x[39621]: ignoring --setuser ceph since I am not root 2026-02-24T07:42:00.195 INFO:journalctl@ceph.mgr.x.trial035.stdout:Feb 24 07:41:59 trial035 ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-mgr-x[39621]: ignoring --setgroup ceph since I am not root 2026-02-24T07:42:00.195 INFO:journalctl@ceph.mgr.x.trial035.stdout:Feb 24 07:41:59 trial035 ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-mgr-x[39621]: 2026-02-24T07:41:59.936+0000 7f3d93a8afc0 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:42:00.195 INFO:journalctl@ceph.mgr.x.trial035.stdout:Feb 24 07:41:59 trial035 ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-mgr-x[39621]: 2026-02-24T07:41:59.938+0000 7f3d93a8afc0 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:42:00.195 INFO:journalctl@ceph.mgr.x.trial035.stdout:Feb 24 07:41:59 trial035 ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-mgr-x[39621]: 2026-02-24T07:41:59.938+0000 7f3d93a8afc0 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:42:00.216 INFO:teuthology.orchestra.run.trial035.stdout:/usr/bin/ceph: stdout { 2026-02-24T07:42:00.216 INFO:teuthology.orchestra.run.trial035.stdout:/usr/bin/ceph: stdout "epoch": 9, 2026-02-24T07:42:00.216 INFO:teuthology.orchestra.run.trial035.stdout:/usr/bin/ceph: stdout "available": true, 2026-02-24T07:42:00.216 INFO:teuthology.orchestra.run.trial035.stdout:/usr/bin/ceph: stdout "active_name": "x", 2026-02-24T07:42:00.216 INFO:teuthology.orchestra.run.trial035.stdout:/usr/bin/ceph: stdout "num_standby": 0 2026-02-24T07:42:00.216 INFO:teuthology.orchestra.run.trial035.stdout:/usr/bin/ceph: stdout } 2026-02-24T07:42:00.216 INFO:teuthology.orchestra.run.trial035.stdout:/usr/bin/ceph: stderr 2026-02-24T07:42:00.046+0000 7fd9704f3640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:42:00.216 INFO:teuthology.orchestra.run.trial035.stdout:/usr/bin/ceph: stderr 2026-02-24T07:42:00.050+0000 7fd9704f3640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:42:00.216 INFO:teuthology.orchestra.run.trial035.stdout:Waiting for the mgr to restart... 2026-02-24T07:42:00.216 INFO:teuthology.orchestra.run.trial035.stdout:Waiting for mgr epoch 9... 2026-02-24T07:42:01.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:01 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/3823139393' entity='client.admin' cmd={"prefix": "mgr stat"} : dispatch 2026-02-24T07:42:01.205 INFO:journalctl@ceph.mgr.x.trial035.stdout:Feb 24 07:42:00 trial035 ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-mgr-x[39621]: /lib64/python3.9/site-packages/scipy/__init__.py:73: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2026-02-24T07:42:01.205 INFO:journalctl@ceph.mgr.x.trial035.stdout:Feb 24 07:42:00 trial035 ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-mgr-x[39621]: Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2026-02-24T07:42:01.205 INFO:journalctl@ceph.mgr.x.trial035.stdout:Feb 24 07:42:00 trial035 ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-mgr-x[39621]: from numpy import show_config as show_numpy_config 2026-02-24T07:42:03.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:02 trial035 ceph-mon[39284]: Active manager daemon x restarted 2026-02-24T07:42:03.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:02 trial035 ceph-mon[39284]: Activating manager daemon x 2026-02-24T07:42:03.205 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:02 trial035 ceph-mon[39284]: osdmap e3: 0 total, 0 up, 0 in 2026-02-24T07:42:03.205 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:02 trial035 ceph-mon[39284]: mgrmap e10: x(active, starting, since 0.0028038s) 2026-02-24T07:42:03.205 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:02 trial035 ceph-mon[39284]: from='mgr.14154 10.20.193.35:0/1087484775' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2026-02-24T07:42:03.205 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:02 trial035 ceph-mon[39284]: from='mgr.14154 10.20.193.35:0/1087484775' entity='mgr.x' cmd={"prefix": "mgr metadata", "who": "x", "id": "x"} : dispatch 2026-02-24T07:42:03.205 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:02 trial035 ceph-mon[39284]: from='mgr.14154 10.20.193.35:0/1087484775' entity='mgr.x' cmd={"prefix": "mds metadata"} : dispatch 2026-02-24T07:42:03.205 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:02 trial035 ceph-mon[39284]: from='mgr.14154 10.20.193.35:0/1087484775' entity='mgr.x' cmd={"prefix": "osd metadata"} : dispatch 2026-02-24T07:42:03.205 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:02 trial035 ceph-mon[39284]: from='mgr.14154 10.20.193.35:0/1087484775' entity='mgr.x' cmd={"prefix": "mon metadata"} : dispatch 2026-02-24T07:42:03.205 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:02 trial035 ceph-mon[39284]: Manager daemon x is now available 2026-02-24T07:42:03.735 INFO:teuthology.orchestra.run.trial035.stdout:/usr/bin/ceph: stdout { 2026-02-24T07:42:03.735 INFO:teuthology.orchestra.run.trial035.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2026-02-24T07:42:03.735 INFO:teuthology.orchestra.run.trial035.stdout:/usr/bin/ceph: stdout "initialized": true 2026-02-24T07:42:03.735 INFO:teuthology.orchestra.run.trial035.stdout:/usr/bin/ceph: stdout } 2026-02-24T07:42:03.735 INFO:teuthology.orchestra.run.trial035.stdout:/usr/bin/ceph: stderr 2026-02-24T07:42:00.304+0000 7fe4c25a7640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:42:03.735 INFO:teuthology.orchestra.run.trial035.stdout:/usr/bin/ceph: stderr 2026-02-24T07:42:00.308+0000 7fe4c25a7640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:42:03.735 INFO:teuthology.orchestra.run.trial035.stdout:mgr epoch 9 is available 2026-02-24T07:42:03.735 INFO:teuthology.orchestra.run.trial035.stdout:Using certmgr to generate dashboard self-signed certificate... 2026-02-24T07:42:03.954 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:03 trial035 ceph-mon[39284]: from='mgr.14154 10.20.193.35:0/1087484775' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2026-02-24T07:42:03.954 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:03 trial035 ceph-mon[39284]: from='mgr.14154 10.20.193.35:0/1087484775' entity='mgr.x' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"} : dispatch 2026-02-24T07:42:03.954 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:03 trial035 ceph-mon[39284]: from='mgr.14154 10.20.193.35:0/1087484775' entity='mgr.x' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"} : dispatch 2026-02-24T07:42:03.954 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:03 trial035 ceph-mon[39284]: from='mgr.14154 10.20.193.35:0/1087484775' entity='mgr.x' 2026-02-24T07:42:03.955 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:03 trial035 ceph-mon[39284]: mgrmap e11: x(active, since 1.00513s) 2026-02-24T07:42:04.467 INFO:teuthology.orchestra.run.trial035.stdout:/usr/bin/ceph: stdout {"cert": "-----BEGIN CERTIFICATE-----\nMIIFHzCCAwegAwIBAgIUJEodgftTKZJARf8ybKKAOLdm0Y8wDQYJKoZIhvcNAQEL\nBQAwPDE6MDgGA1UEAwwxY2VwaGFkbS1yb290LTNhNTk4M2QyLTExNTQtMTFmMS05\nYzA1LWQ0MDRlNmU3ZDQ2MDAeFw0yNjAyMjQwNzQyMDRaFw0yOTAyMjMwNzQyMDRa\nMBcxFTATBgNVBAMMDDEwLjIwLjE5My4zNTCCAiIwDQYJKoZIhvcNAQEBBQADggIP\nADCCAgoCggIBALpaTDeHuhEeFMhVcfP6IqdaTwAc9H7kDSi37ItuUAw4U5S8AjaQ\nhZuWNhaPCn9sR9obKmbPQImdG7Ix3Jp9+cAPhv+UrMOxhWcwXXGMSzMCimUGQjqy\nxV3RB9ov7KKAjBhQ2z0fe9kZB5NEE2muvR9Mqqidebz0vY3lvg8ugmc243dxyO6N\nk/yM+7F/dhZdy/DAvclHmK3zmXefjFVdWFBepRsmknhAHRN0tjheRXyZy314d0Ge\n4n7YKT4kRU+EmmkgGUNRcmibhrFC0GwXts7DazdfWOD8/BxrUALhXjX7ek8gFNvr\nRzHwgUVMhPxfiRnsKuDfPLipC/IF5VnpNSL8CxatE8UNkKsVSn8Z6OwYTQnGOWLk\nYiP+1RT8I5KEDQ5Btl3acSp9HEojwj3aRxFIe6vlnJdnJhSF3ALOVy2hJcCabKp5\n47S7eAM9uIaB8A9bOYsYn6mZJiG1QfN7kIkOAMmnWefoH84E3xWAGVt5bFCyc/Vy\nSIhIArxGbDi5tKuv+KHNWTKIDbKabBaEvUQHrpucApxaQOAZ2I0oRrRZTRwr+2W0\nnxMTecM8EISgkt0tp+eB8FTCdN/x06TIwqJ3PaXH0ZlNcjhyRv08/oMEEVodcMP5\nW2fw5yFh5ZcD/gQJZuvTB1zjTnSY2Gq8/9OJLiVpdfkLsXZxSfraM1/VAgMBAAGj\nPjA8MCwGA1UdEQQlMCOCCHRyaWFsMDM1ghFkYXNoYm9hcmRfc2VydmVyc4cEChTB\nIzAMBgNVHRMBAf8EAjAAMA0GCSqGSIb3DQEBCwUAA4ICAQCkmzoEXppMkqZsbGk7\nCyGelF2KRdMPhRBx7M501w5dxvVEnkRV/Qz/rRseKkcSs5SWsd2dbnIMBuEIWyAN\n5k8TWaTfS05Jt6UxDStKUPLhMagJGGxIuNjRqiEYsmGKYzuxfT/48VfYHXXib2+R\nV3Nsfm6e67FYrqzBs5BYH1o9kVxV+UYL5g6IUUt1Mxvdse+3Vqpmx/SwbpA/sNda\nTslOb5Z6sL34Df64RHWu9gR5mIqqrfFMF/AN28Fa6DsxZo/FnwspRt1HZgL2bFRd\nRLRJP9EQyOd/2jNN65DLHYWksCXqY5Qp9JLbSl0vVX8IlOho2nrZ20J62owNZPYD\ntT+lXYHlk8IeyPa7LE1do+eSpsxwdtfLtraYywzsDxcCV8gAS8ue+Ehyk/s50Fvh\nfm4FLmOJNT2kw0dJ3CtDOVBj59Ld4Z5P+MENAVdDAPwLxz/fzsRCdNTbhSY24upQ\njZm3/FCRkbDBSwAbtyOMtddTTLoy05fu6rSl2pnjoIcCthJ1yclf5vBKCl4/U1Sx\n/KDykFOYL1dHp4Bn3M7kmxB42r2EESYx+KmWrilkDcAOnLfLzgyWP+7PJaMnkgY2\nvOjidmWsrHRC4UjHOpLNrz2D5G4K/ZMs3vNUAcQGBn95WEwbK4Bzxd6R9Bwvq5UC\naiSQICihidkcJ00ORXwbaz4TQQ==\n-----END CERTIFICATE-----\n", "key": "-----BEGIN RSA PRIVATE KEY-----\nMIIJKQIBAAKCAgEAulpMN4e6ER4UyFVx8/oip1pPABz0fuQNKLfsi25QDDhTlLwC\nNpCFm5Y2Fo8Kf2xH2hsqZs9AiZ0bsjHcmn35wA+G/5Ssw7GFZzBdcYxLMwKKZQZC\nOrLFXdEH2i/sooCMGFDbPR972RkHk0QTaa69H0yqqJ15vPS9jeW+Dy6CZzbjd3HI\n7o2T/Iz7sX92Fl3L8MC9yUeYrfOZd5+MVV1YUF6lGyaSeEAdE3S2OF5FfJnLfXh3\nQZ7iftgpPiRFT4SaaSAZQ1FyaJuGsULQbBe2zsNrN19Y4Pz8HGtQAuFeNft6TyAU\n2+tHMfCBRUyE/F+JGewq4N88uKkL8gXlWek1IvwLFq0TxQ2QqxVKfxno7BhNCcY5\nYuRiI/7VFPwjkoQNDkG2XdpxKn0cSiPCPdpHEUh7q+Wcl2cmFIXcAs5XLaElwJps\nqnnjtLt4Az24hoHwD1s5ixifqZkmIbVB83uQiQ4AyadZ5+gfzgTfFYAZW3lsULJz\n9XJIiEgCvEZsOLm0q6/4oc1ZMogNsppsFoS9RAeum5wCnFpA4BnYjShGtFlNHCv7\nZbSfExN5wzwQhKCS3S2n54HwVMJ03/HTpMjConc9pcfRmU1yOHJG/Tz+gwQRWh1w\nw/lbZ/DnIWHllwP+BAlm69MHXONOdJjYarz/04kuJWl1+QuxdnFJ+tozX9UCAwEA\nAQKCAgA1BPfmXLFtqVdUYCsZtTMi1ijf6V0OtSehTCL60JnpwCTXjl461AePar9c\nz7m240HWJiB8O78q6A43NsSDGBB0unY4RT6aNumzoCkV40qJ6tEhSRSSysYomQ5Y\n34W9/kDdsjOOQmTkMaDcH2Q+jz6zpwZvDs7SGffMZW8btqnXX+B73V3DsXhHRH1f\n1NnuJyWdC9lzaPFhnIsvY69zoik8RJFJFkJaADPFk1E83GV8u+0Pd7TGcMQUPsAN\nNBicNSgRfNZzNrsMVCOQT2gpON4EGclpabFJ4fCAzW9oQmKhjTkRRhlpMJmC9eBx\nrQGxJCVHwM+6RwlpqH8DjvXgDN/W5RJgxxrmTsEJ94YBcrD17ugdDp1j+7h6SJSG\n748mcSi87ZtZWjMmEKv4JC+l25Eqprsz8UbHT8uvjviQYu2qjhkBv1ZYEWV4K4UH\njSV0qBrJEfAoufqc4CnMy69hhl0sm18adkpmvXYZMHktZOXpwYznOQZy+jVRtD4A\nJpQ/+Y7PiMArI/dTXNDiXfmlEO5hup+DR9qxwgRQ4JR/uW90z5zy33Tv/iEvbWC5\nvsuFrtE1HhXwFFH+X2t11Wd4TGkmphgSywZdKlG1fFXVlJP3Ar5vnQmCoF5ytu49\n0wyTqZuH+Qh8mNd2+LcY+Nrlx5K7zawLY1m4VR/+QlnItInkyQKCAQEA9uTUrI2u\n+Y/CJnNSKKZ9uiG0YrivPVVASfcc2kWcx2Mmv1dHbBDx/mJQGT1uGLZB0NCbY3v+\nbOKnUffDe8CpTu5ioJt1Ad0vl/+Hde9pd81orP+a0j+fqpRClaOxk82KUd9q6meR\nM6u2Wfk1FdST1/mye6kb+cxXAAk1pf0BHdIQKXPzedBaSbhTf4SGRjm9ApFo9dqr\npgtPoLQ6zfpxewu/wJi+slCC+m1UN4avvjl5ZuUVNzG40LmgyDOgIPCVM7qY2KW1\nbqee1Icyln4SZY5chD7f2sDJcmQfDL9jxuk8TlHrzkOiLDOeJseTIhVUTQMPg5y/\nAhyBZSeu+erZCwKCAQEAwTnWj+tAzx4y30yTqCh5TGAOJm2G9idAAlFUwA6W+Mha\n6iKVFABinpsaB1YhdRQPsgVCCaepAXitqteY+635U7uHfTUHx9rkysL+EiFzn69A\nmwsHjudDM19qkTaqY+TkwE7fMxAzLMRmqMBJf+Ntw8ozl4HkFpb+YNRCN+uLOWEK\nhUlmBmVZdqYAlFr8luJjnI+J0AD7NgG8q3uVBiW7njlrL9RXD7DRMgG1kQBIdE8Q\n6jk45xFHUuIxnQtxF7kv2E1bJlkcyImigyaZYiKXPoXQYXcXol1l7+6AKMwnBE6d\nU44ufQi1d9aiRjAqB8+tm5AXsOR2N0TlMufLVhL2nwKCAQAfcfewXn9Jo8irkwaE\nzOe4fDB4XmNbYW9FvGZ9Wl5kerllp10eY24e4zCmdk9+cEAjs14HJrS584gyBWdt\nbJVjyzLel84oRZ6ZMl9zYuHA83WAizGR066nUCjG3ElytLwIP3djkXtJ5HXeMcJU\nHHYUDOZxn5FDmIYAKCwgDwBEZwvN68m32xSUzRg9Zoifix0KpbCDl61DAa/LnQjc\nUSiA7ncmtY2cMmrsMF6Ij+v0Q2e19jlkhT2vv+S9eu3LJDkslHiBJ2HbTRgKo4mK\n51W6f45C/asPNnyxyBS4T+wKWoxmwUJSTATsskpTUBnqPmM1R82DCsS9JRpJ4usD\nQ6kPAoIBAQC0ZQwlJwxFkbsK8FI2Pzsni9IKHF1Q+EnxH/9Qow5dy6fdSVYG62kn\nDETbrpTj2kvg9rm1i9rkLcdtZYVFCyJoAv82bMkL1BegGTXctXBeF97oIP74cn4x\nuxYh5nhjdLH/vGLzKF+untn9GDBGfozeqqtJ+pF4aZWpzuDXlkdP1YJd962RsRM6\nCzclSmDheRBofTW7hP8DZ9G+A02ZBKdJy+9/Iu35DynvVtstOadrNtni4nN2fpIp\nFlYdeS4naNP7TBUaEz4lFyIq8WFn1kV/sKxgIjBlfG8L9o2eCSMwCTkmZeqO/w90\nh2TOxG6pBkHdp0QG+PRJG0AkK3WRsXifAoIBAQCBGh6pSQ7U1wXReCghxe8S5Piw\nZ48AQQDjwN1zjmhqG5c3w6JzFW9ly/OV7haZ/EfqwAXLCXC1rh6kNvfr4RLR2VFS\nA8oRrGUOqgZKyN/JOBgzYImy5S1T/HxxKox89uP4jc5rRMwlsS+EcY+IMxIYam9+\n4x1cuRGRPAHXjO/gUKzUDFU/Ubnn96IpFv+vjXAsknWviSCzZjeRDErGPFmgU5lb\nbGiNnZLo5Pv61r7WoyBakD9rEJgZMEOCnEF8XqQ+T3zMs6eDzqL5Jp9poyniQjcf\nyt5I1rVeg/7H+DRkFpV/6b38d0eAPkC4g8ZG6I5cQdSYdzrNhY946LLLO0jX\n-----END RSA PRIVATE KEY-----\n"} 2026-02-24T07:42:04.467 INFO:teuthology.orchestra.run.trial035.stdout:/usr/bin/ceph: stderr 2026-02-24T07:42:03.834+0000 7f2362ae9640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:42:04.467 INFO:teuthology.orchestra.run.trial035.stdout:/usr/bin/ceph: stderr 2026-02-24T07:42:03.838+0000 7f2362ae9640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:42:04.707 INFO:teuthology.orchestra.run.trial035.stdout:/usr/bin/ceph: stdout SSL certificate updated 2026-02-24T07:42:04.707 INFO:teuthology.orchestra.run.trial035.stdout:/usr/bin/ceph: stderr 2026-02-24T07:42:04.563+0000 7fc1bdf96640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:42:04.707 INFO:teuthology.orchestra.run.trial035.stdout:/usr/bin/ceph: stderr 2026-02-24T07:42:04.566+0000 7fc1bdf96640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:42:04.935 INFO:teuthology.orchestra.run.trial035.stdout:/usr/bin/ceph: stdout SSL certificate key updated 2026-02-24T07:42:04.936 INFO:teuthology.orchestra.run.trial035.stdout:/usr/bin/ceph: stderr 2026-02-24T07:42:04.794+0000 7f6836017640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:42:04.936 INFO:teuthology.orchestra.run.trial035.stdout:/usr/bin/ceph: stderr 2026-02-24T07:42:04.797+0000 7f6836017640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:42:04.936 INFO:teuthology.orchestra.run.trial035.stdout:Creating initial admin user... 2026-02-24T07:42:04.954 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:04 trial035 ceph-mon[39284]: [24/Feb/2026:07:42:03] ENGINE Bus STARTING 2026-02-24T07:42:04.954 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:04 trial035 ceph-mon[39284]: [24/Feb/2026:07:42:03] ENGINE Serving on http://10.20.193.35:8765 2026-02-24T07:42:04.954 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:04 trial035 ceph-mon[39284]: [24/Feb/2026:07:42:03] ENGINE Serving on https://10.20.193.35:7150 2026-02-24T07:42:04.954 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:04 trial035 ceph-mon[39284]: [24/Feb/2026:07:42:03] ENGINE Bus STARTED 2026-02-24T07:42:04.954 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:04 trial035 ceph-mon[39284]: [24/Feb/2026:07:42:03] ENGINE Client ('10.20.193.35', 44530) lost — peer dropped the TLS connection suddenly, during handshake: (6, 'TLS/SSL connection has been closed (EOF) (_ssl.c:1147)') 2026-02-24T07:42:04.954 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:04 trial035 ceph-mon[39284]: from='client.14158 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2026-02-24T07:42:04.954 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:04 trial035 ceph-mon[39284]: from='client.14158 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2026-02-24T07:42:04.954 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:04 trial035 ceph-mon[39284]: from='mgr.14154 10.20.193.35:0/1087484775' entity='mgr.x' 2026-02-24T07:42:04.955 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:04 trial035 ceph-mon[39284]: from='mgr.14154 10.20.193.35:0/1087484775' entity='mgr.x' 2026-02-24T07:42:05.366 INFO:teuthology.orchestra.run.trial035.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$7xEKqKJ0XkxqbmsImj83.eFZj.3LNBlaKWWt.s22Al6X2qiZRm31O", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1771918925, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2026-02-24T07:42:05.366 INFO:teuthology.orchestra.run.trial035.stdout:/usr/bin/ceph: stderr 2026-02-24T07:42:05.023+0000 7f99ba925640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:42:05.366 INFO:teuthology.orchestra.run.trial035.stdout:/usr/bin/ceph: stderr 2026-02-24T07:42:05.029+0000 7f99ba925640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:42:05.366 INFO:teuthology.orchestra.run.trial035.stdout:Fetching dashboard port number... 2026-02-24T07:42:05.586 INFO:teuthology.orchestra.run.trial035.stdout:/usr/bin/ceph: stdout 8443 2026-02-24T07:42:05.586 INFO:teuthology.orchestra.run.trial035.stdout:/usr/bin/ceph: stderr 2026-02-24T07:42:05.453+0000 7fe983475640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:42:05.587 INFO:teuthology.orchestra.run.trial035.stdout:/usr/bin/ceph: stderr 2026-02-24T07:42:05.457+0000 7fe983475640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:42:05.595 INFO:teuthology.orchestra.run.trial035.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2026-02-24T07:42:05.595 INFO:teuthology.orchestra.run.trial035.stdout:systemctl: stdout disabled 2026-02-24T07:42:05.600 INFO:teuthology.orchestra.run.trial035.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2026-02-24T07:42:05.601 INFO:teuthology.orchestra.run.trial035.stdout:systemctl: stdout inactive 2026-02-24T07:42:05.601 INFO:teuthology.orchestra.run.trial035.stdout:firewalld.service is not enabled 2026-02-24T07:42:05.601 INFO:teuthology.orchestra.run.trial035.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2026-02-24T07:42:05.602 INFO:teuthology.orchestra.run.trial035.stdout:Ceph Dashboard is now available at: 2026-02-24T07:42:05.602 INFO:teuthology.orchestra.run.trial035.stdout: 2026-02-24T07:42:05.602 INFO:teuthology.orchestra.run.trial035.stdout: URL: https://trial035:8443/ 2026-02-24T07:42:05.602 INFO:teuthology.orchestra.run.trial035.stdout: User: admin 2026-02-24T07:42:05.602 INFO:teuthology.orchestra.run.trial035.stdout: Password: cn7s156bc3 2026-02-24T07:42:05.602 INFO:teuthology.orchestra.run.trial035.stdout: 2026-02-24T07:42:05.603 INFO:teuthology.orchestra.run.trial035.stdout:Saving cluster configuration to /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/config directory 2026-02-24T07:42:05.873 INFO:teuthology.orchestra.run.trial035.stdout:/usr/bin/ceph: stderr 2026-02-24T07:42:05.687+0000 7fb14b80a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:42:05.873 INFO:teuthology.orchestra.run.trial035.stdout:/usr/bin/ceph: stderr 2026-02-24T07:42:05.691+0000 7fb14b80a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:42:05.873 INFO:teuthology.orchestra.run.trial035.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2026-02-24T07:42:05.873 INFO:teuthology.orchestra.run.trial035.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2026-02-24T07:42:05.873 INFO:teuthology.orchestra.run.trial035.stdout: 2026-02-24T07:42:05.873 INFO:teuthology.orchestra.run.trial035.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2026-02-24T07:42:05.873 INFO:teuthology.orchestra.run.trial035.stdout: 2026-02-24T07:42:05.873 INFO:teuthology.orchestra.run.trial035.stdout:Or, if you are only running a single cluster on this host: 2026-02-24T07:42:05.873 INFO:teuthology.orchestra.run.trial035.stdout: 2026-02-24T07:42:05.873 INFO:teuthology.orchestra.run.trial035.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2026-02-24T07:42:05.873 INFO:teuthology.orchestra.run.trial035.stdout: 2026-02-24T07:42:05.873 INFO:teuthology.orchestra.run.trial035.stdout:Please consider enabling telemetry to help improve Ceph: 2026-02-24T07:42:05.873 INFO:teuthology.orchestra.run.trial035.stdout: 2026-02-24T07:42:05.873 INFO:teuthology.orchestra.run.trial035.stdout: ceph telemetry on 2026-02-24T07:42:05.873 INFO:teuthology.orchestra.run.trial035.stdout: 2026-02-24T07:42:05.874 INFO:teuthology.orchestra.run.trial035.stdout:For more information see: 2026-02-24T07:42:05.874 INFO:teuthology.orchestra.run.trial035.stdout: 2026-02-24T07:42:05.874 INFO:teuthology.orchestra.run.trial035.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2026-02-24T07:42:05.874 INFO:teuthology.orchestra.run.trial035.stdout: 2026-02-24T07:42:05.874 INFO:teuthology.orchestra.run.trial035.stdout:Bootstrap complete. 2026-02-24T07:42:05.881 INFO:teuthology.orchestra.run.trial035.stdout:systemctl: stdout static 2026-02-24T07:42:05.887 INFO:teuthology.orchestra.run.trial035.stdout:Non-zero exit code 3 from systemctl is-active logrotate 2026-02-24T07:42:05.887 INFO:teuthology.orchestra.run.trial035.stdout:systemctl: stdout inactive 2026-02-24T07:42:05.887 INFO:teuthology.orchestra.run.trial035.stdout:Enabling the logrotate.timer service to perform daily log rotation. 2026-02-24T07:42:06.034 INFO:tasks.cephadm:Fetching config... 2026-02-24T07:42:06.034 DEBUG:teuthology.orchestra.run.trial035:> set -ex 2026-02-24T07:42:06.034 DEBUG:teuthology.orchestra.run.trial035:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2026-02-24T07:42:06.050 INFO:tasks.cephadm:Fetching client.admin keyring... 2026-02-24T07:42:06.050 DEBUG:teuthology.orchestra.run.trial035:> set -ex 2026-02-24T07:42:06.050 DEBUG:teuthology.orchestra.run.trial035:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2026-02-24T07:42:06.106 INFO:tasks.cephadm:Fetching mon keyring... 2026-02-24T07:42:06.106 DEBUG:teuthology.orchestra.run.trial035:> set -ex 2026-02-24T07:42:06.106 DEBUG:teuthology.orchestra.run.trial035:> sudo dd if=/var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/keyring of=/dev/stdout 2026-02-24T07:42:06.154 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:05 trial035 ceph-mon[39284]: from='client.14166 -' entity='client.admin' cmd=[{"prefix": "orch certmgr generate-certificates", "module_name": "dashboard", "target": ["mon-mgr", ""]}]: dispatch 2026-02-24T07:42:06.154 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:05 trial035 ceph-mon[39284]: from='client.14168 -' entity='client.admin' cmd=[{"prefix": "dashboard set-ssl-certificate", "target": ["mon-mgr", ""]}]: dispatch 2026-02-24T07:42:06.154 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:05 trial035 ceph-mon[39284]: from='client.14170 -' entity='client.admin' cmd=[{"prefix": "dashboard set-ssl-certificate-key", "target": ["mon-mgr", ""]}]: dispatch 2026-02-24T07:42:06.155 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:05 trial035 ceph-mon[39284]: from='mgr.14154 10.20.193.35:0/1087484775' entity='mgr.x' 2026-02-24T07:42:06.155 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:05 trial035 ceph-mon[39284]: from='mgr.14154 10.20.193.35:0/1087484775' entity='mgr.x' 2026-02-24T07:42:06.155 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:05 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/3486333233' entity='client.admin' cmd={"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} : dispatch 2026-02-24T07:42:06.155 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:05 trial035 ceph-mon[39284]: mgrmap e12: x(active, since 2s) 2026-02-24T07:42:06.155 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:05 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/83589392' entity='client.admin' 2026-02-24T07:42:06.172 INFO:tasks.cephadm:Fetching pub ssh key... 2026-02-24T07:42:06.172 DEBUG:teuthology.orchestra.run.trial035:> set -ex 2026-02-24T07:42:06.173 DEBUG:teuthology.orchestra.run.trial035:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2026-02-24T07:42:06.230 INFO:tasks.cephadm:Installing pub ssh key for root users... 2026-02-24T07:42:06.231 DEBUG:teuthology.orchestra.run.trial035:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIMyTwY6dp+NwS9foHhJghlyw1hmbgXD0owS8aw7WYPZY ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2026-02-24T07:42:06.304 INFO:teuthology.orchestra.run.trial035.stdout:ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIMyTwY6dp+NwS9foHhJghlyw1hmbgXD0owS8aw7WYPZY ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460 2026-02-24T07:42:06.314 DEBUG:teuthology.orchestra.run.trial091:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIMyTwY6dp+NwS9foHhJghlyw1hmbgXD0owS8aw7WYPZY ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2026-02-24T07:42:06.350 INFO:teuthology.orchestra.run.trial091.stdout:ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIMyTwY6dp+NwS9foHhJghlyw1hmbgXD0owS8aw7WYPZY ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460 2026-02-24T07:42:06.360 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2026-02-24T07:42:06.474 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:42:06.578 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:42:06.577+0000 7fb706452640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:42:06.578 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:42:06.577+0000 7fb706452640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:42:06.726 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2026-02-24T07:42:06.727 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph orch client-keyring set client.admin '*' --mode 0755 2026-02-24T07:42:06.844 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:42:06.950 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:42:06.949+0000 7ff382acb640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:42:06.951 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:42:06.949+0000 7ff382acb640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:42:07.025 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:06 trial035 ceph-mon[39284]: from='client.14172 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2026-02-24T07:42:07.025 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:06 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/76892796' entity='client.admin' 2026-02-24T07:42:07.104 INFO:tasks.cephadm:Writing (initial) conf and keyring to trial091 2026-02-24T07:42:07.104 DEBUG:teuthology.orchestra.run.trial091:> set -ex 2026-02-24T07:42:07.104 DEBUG:teuthology.orchestra.run.trial091:> dd of=/etc/ceph/ceph.conf 2026-02-24T07:42:07.122 DEBUG:teuthology.orchestra.run.trial091:> set -ex 2026-02-24T07:42:07.122 DEBUG:teuthology.orchestra.run.trial091:> dd of=/etc/ceph/ceph.client.admin.keyring 2026-02-24T07:42:07.178 INFO:tasks.cephadm:Adding host trial091 to orchestrator... 2026-02-24T07:42:07.178 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph orch host add trial091 2026-02-24T07:42:07.298 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:42:07.405 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:42:07.403+0000 7f4acb630640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:42:07.405 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:42:07.403+0000 7f4acb630640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:42:08.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:08 trial035 ceph-mon[39284]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2026-02-24T07:42:08.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:08 trial035 ceph-mon[39284]: from='mgr.14154 10.20.193.35:0/1087484775' entity='mgr.x' 2026-02-24T07:42:08.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:08 trial035 ceph-mon[39284]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "trial091", "target": ["mon-mgr", ""]}]: dispatch 2026-02-24T07:42:08.205 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:08 trial035 ceph-mon[39284]: from='mgr.14154 10.20.193.35:0/1087484775' entity='mgr.x' 2026-02-24T07:42:08.205 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:08 trial035 ceph-mon[39284]: from='mgr.14154 10.20.193.35:0/1087484775' entity='mgr.x' 2026-02-24T07:42:08.205 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:08 trial035 ceph-mon[39284]: from='mgr.14154 10.20.193.35:0/1087484775' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd/host:trial035", "name": "osd_memory_target"} : dispatch 2026-02-24T07:42:08.205 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:08 trial035 ceph-mon[39284]: from='mgr.14154 10.20.193.35:0/1087484775' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-24T07:42:08.205 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:08 trial035 ceph-mon[39284]: from='mgr.14154 10.20.193.35:0/1087484775' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2026-02-24T07:42:08.205 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:08 trial035 ceph-mon[39284]: Updating trial035:/etc/ceph/ceph.conf 2026-02-24T07:42:09.188 INFO:teuthology.orchestra.run.trial035.stdout:Added host 'trial091' with addr '10.20.193.91' 2026-02-24T07:42:09.245 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph orch host ls --format=json 2026-02-24T07:42:09.363 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:42:09.468 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:42:09.466+0000 7ff0e0751640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:42:09.469 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:42:09.466+0000 7ff0e0751640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:42:09.579 INFO:teuthology.orchestra.run.trial035.stdout: 2026-02-24T07:42:09.579 INFO:teuthology.orchestra.run.trial035.stdout:[{"addr": "10.20.193.35", "hostname": "trial035", "labels": [], "status": ""}, {"addr": "10.20.193.91", "hostname": "trial091", "labels": [], "status": ""}] 2026-02-24T07:42:09.613 INFO:tasks.cephadm:Setting crush tunables to default 2026-02-24T07:42:09.614 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph osd crush tunables default 2026-02-24T07:42:09.704 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:09 trial035 ceph-mon[39284]: Deploying cephadm binary to trial091 2026-02-24T07:42:09.704 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:09 trial035 ceph-mon[39284]: Updating trial035:/var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/config/ceph.conf 2026-02-24T07:42:09.704 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:09 trial035 ceph-mon[39284]: Updating trial035:/etc/ceph/ceph.client.admin.keyring 2026-02-24T07:42:09.704 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:09 trial035 ceph-mon[39284]: Updating trial035:/var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/config/ceph.client.admin.keyring 2026-02-24T07:42:09.704 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:09 trial035 ceph-mon[39284]: from='mgr.14154 10.20.193.35:0/1087484775' entity='mgr.x' 2026-02-24T07:42:09.704 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:09 trial035 ceph-mon[39284]: from='mgr.14154 10.20.193.35:0/1087484775' entity='mgr.x' 2026-02-24T07:42:09.704 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:09 trial035 ceph-mon[39284]: from='mgr.14154 10.20.193.35:0/1087484775' entity='mgr.x' 2026-02-24T07:42:09.704 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:09 trial035 ceph-mon[39284]: from='mgr.14154 10.20.193.35:0/1087484775' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2026-02-24T07:42:09.704 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:09 trial035 ceph-mon[39284]: from='mgr.14154 10.20.193.35:0/1087484775' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-24T07:42:09.704 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:09 trial035 ceph-mon[39284]: from='mgr.14154 10.20.193.35:0/1087484775' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2026-02-24T07:42:09.704 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:09 trial035 ceph-mon[39284]: from='mgr.14154 10.20.193.35:0/1087484775' entity='mgr.x' 2026-02-24T07:42:09.704 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:09 trial035 ceph-mon[39284]: from='mgr.14154 10.20.193.35:0/1087484775' entity='mgr.x' 2026-02-24T07:42:09.704 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:09 trial035 ceph-mon[39284]: from='mgr.14154 10.20.193.35:0/1087484775' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2026-02-24T07:42:09.705 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:09 trial035 ceph-mon[39284]: from='mgr.14154 10.20.193.35:0/1087484775' entity='mgr.x' 2026-02-24T07:42:09.731 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:42:09.829 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:42:09.827+0000 7f34d1791640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:42:09.829 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:42:09.827+0000 7f34d1791640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:42:10.489 INFO:teuthology.orchestra.run.trial035.stderr:adjusted tunables profile to default 2026-02-24T07:42:10.530 INFO:tasks.cephadm:Adding mon.a on trial035 2026-02-24T07:42:10.530 DEBUG:teuthology.orchestra.run.trial091:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph orch apply mon '1;trial035:10.20.193.35=a' 2026-02-24T07:42:10.662 INFO:teuthology.orchestra.run.trial091.stderr:Inferring config /etc/ceph/ceph.conf 2026-02-24T07:42:10.662 INFO:teuthology.orchestra.run.trial091.stderr:Inferring config /etc/ceph/ceph.conf 2026-02-24T07:42:10.704 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:10 trial035 ceph-mon[39284]: Certificate for "cephadm-signed_agent_cert (trial035)" is still valid for 1824 days. 2026-02-24T07:42:10.704 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:10 trial035 ceph-mon[39284]: Added host trial091 2026-02-24T07:42:10.704 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:10 trial035 ceph-mon[39284]: mgrmap e13: x(active, since 6s) 2026-02-24T07:42:10.704 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:10 trial035 ceph-mon[39284]: from='mgr.14154 10.20.193.35:0/1087484775' entity='mgr.x' 2026-02-24T07:42:10.704 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:10 trial035 ceph-mon[39284]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2026-02-24T07:42:10.704 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:10 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/1528118837' entity='client.admin' cmd={"prefix": "osd crush tunables", "profile": "default"} : dispatch 2026-02-24T07:42:10.704 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:10 trial035 ceph-mon[39284]: from='mgr.14154 10.20.193.35:0/1087484775' entity='mgr.x' 2026-02-24T07:42:10.788 INFO:teuthology.orchestra.run.trial091.stderr:2026-02-24T07:42:10.786+0000 7fae583e9640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:42:10.788 INFO:teuthology.orchestra.run.trial091.stderr:2026-02-24T07:42:10.786+0000 7fae583e9640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:42:10.911 INFO:teuthology.orchestra.run.trial091.stdout:Scheduled mon update... 2026-02-24T07:42:10.947 INFO:tasks.cephadm:Waiting for 1 mons in monmap... 2026-02-24T07:42:10.947 DEBUG:teuthology.orchestra.run.trial091:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph mon dump -f json 2026-02-24T07:42:11.064 INFO:teuthology.orchestra.run.trial091.stderr:Inferring config /etc/ceph/ceph.conf 2026-02-24T07:42:11.064 INFO:teuthology.orchestra.run.trial091.stderr:Inferring config /etc/ceph/ceph.conf 2026-02-24T07:42:11.163 INFO:teuthology.orchestra.run.trial091.stderr:2026-02-24T07:42:11.162+0000 7fe325028640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:42:11.163 INFO:teuthology.orchestra.run.trial091.stderr:2026-02-24T07:42:11.162+0000 7fe325028640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:42:11.316 INFO:teuthology.orchestra.run.trial091.stdout: 2026-02-24T07:42:11.316 INFO:teuthology.orchestra.run.trial091.stdout:{"epoch":1,"fsid":"3a5983d2-1154-11f1-9c05-d404e6e7d460","modified":"2026-02-24T07:41:41.890986Z","created":"2026-02-24T07:41:41.890986Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.35:3300","nonce":0},{"type":"v1","addr":"10.20.193.35:6789","nonce":0}]},"addr":"10.20.193.35:6789/0","public_addr":"10.20.193.35:6789/0","priority":0,"weight":0,"time_added":"2026-02-24T07:41:41:891001+0000","crush_location":[]}],"quorum":[0]} 2026-02-24T07:42:11.316 INFO:teuthology.orchestra.run.trial091.stderr:dumped monmap epoch 1 2026-02-24T07:42:11.351 INFO:tasks.cephadm:Generating final ceph.conf file... 2026-02-24T07:42:11.351 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph config generate-minimal-conf 2026-02-24T07:42:11.469 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:42:11.573 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:42:11.571+0000 7f7907153640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:42:11.573 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:42:11.571+0000 7f7907153640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:42:11.684 INFO:teuthology.orchestra.run.trial035.stdout:# minimal ceph.conf for 3a5983d2-1154-11f1-9c05-d404e6e7d460 2026-02-24T07:42:11.684 INFO:teuthology.orchestra.run.trial035.stdout:[global] 2026-02-24T07:42:11.684 INFO:teuthology.orchestra.run.trial035.stdout: fsid = 3a5983d2-1154-11f1-9c05-d404e6e7d460 2026-02-24T07:42:11.684 INFO:teuthology.orchestra.run.trial035.stdout: mon_host = [v2:10.20.193.35:3300/0,v1:10.20.193.35:6789/0] 2026-02-24T07:42:11.684 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:11 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/1528118837' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2026-02-24T07:42:11.685 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:11 trial035 ceph-mon[39284]: osdmap e4: 0 total, 0 up, 0 in 2026-02-24T07:42:11.685 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:11 trial035 ceph-mon[39284]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "1;trial035:10.20.193.35=a", "target": ["mon-mgr", ""]}]: dispatch 2026-02-24T07:42:11.685 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:11 trial035 ceph-mon[39284]: Saving service mon spec with placement trial035:10.20.193.35=a;count:1 2026-02-24T07:42:11.685 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:11 trial035 ceph-mon[39284]: from='mgr.14154 10.20.193.35:0/1087484775' entity='mgr.x' 2026-02-24T07:42:11.685 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:11 trial035 ceph-mon[39284]: from='client.? 10.20.193.91:0/2651303532' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2026-02-24T07:42:11.721 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2026-02-24T07:42:11.721 DEBUG:teuthology.orchestra.run.trial035:> set -ex 2026-02-24T07:42:11.721 DEBUG:teuthology.orchestra.run.trial035:> sudo dd of=/etc/ceph/ceph.conf 2026-02-24T07:42:11.748 DEBUG:teuthology.orchestra.run.trial035:> set -ex 2026-02-24T07:42:11.748 DEBUG:teuthology.orchestra.run.trial035:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2026-02-24T07:42:11.813 DEBUG:teuthology.orchestra.run.trial091:> set -ex 2026-02-24T07:42:11.813 DEBUG:teuthology.orchestra.run.trial091:> sudo dd of=/etc/ceph/ceph.conf 2026-02-24T07:42:11.841 DEBUG:teuthology.orchestra.run.trial091:> set -ex 2026-02-24T07:42:11.844 DEBUG:teuthology.orchestra.run.trial091:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2026-02-24T07:42:11.907 INFO:tasks.cephadm:Adding mgr.x on trial035 2026-02-24T07:42:11.908 DEBUG:teuthology.orchestra.run.trial091:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph orch apply mgr '1;trial035=x' 2026-02-24T07:42:12.061 INFO:teuthology.orchestra.run.trial091.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/config/ceph.conf 2026-02-24T07:42:12.180 INFO:teuthology.orchestra.run.trial091.stderr:2026-02-24T07:42:12.178+0000 7f857fd1c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:42:12.181 INFO:teuthology.orchestra.run.trial091.stderr:2026-02-24T07:42:12.179+0000 7f857fd1c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:42:12.299 INFO:teuthology.orchestra.run.trial091.stdout:Scheduled mgr update... 2026-02-24T07:42:12.336 DEBUG:tasks.cephadm:set 0 configs 2026-02-24T07:42:12.336 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph config dump 2026-02-24T07:42:12.452 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:42:12.582 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:42:12.580+0000 7fe92d2ac640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:42:12.582 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:42:12.580+0000 7fe92d2ac640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:42:12.691 INFO:teuthology.orchestra.run.trial035.stdout:WHO MASK LEVEL OPTION VALUE RO 2026-02-24T07:42:12.692 INFO:teuthology.orchestra.run.trial035.stdout:global dev auth_debug true 2026-02-24T07:42:12.692 INFO:teuthology.orchestra.run.trial035.stdout:global basic container_image quay.ceph.io/ceph-ci/ceph@sha256:9a5db51bca6d139691bb8c7054be8834f3ba065d1c0aaaecbafb190e389fb148 * 2026-02-24T07:42:12.692 INFO:teuthology.orchestra.run.trial035.stdout:global advanced crimson_allow_pg_split false 2026-02-24T07:42:12.692 INFO:teuthology.orchestra.run.trial035.stdout:global dev debug_asserts_on_shutdown true 2026-02-24T07:42:12.692 INFO:teuthology.orchestra.run.trial035.stdout:global advanced enable_experimental_unrecoverable_data_corrupting_features crimson 2026-02-24T07:42:12.692 INFO:teuthology.orchestra.run.trial035.stdout:global basic log_to_file true 2026-02-24T07:42:12.692 INFO:teuthology.orchestra.run.trial035.stdout:global basic log_to_journald false 2026-02-24T07:42:12.692 INFO:teuthology.orchestra.run.trial035.stdout:global basic log_to_stderr false 2026-02-24T07:42:12.692 INFO:teuthology.orchestra.run.trial035.stdout:global advanced mon_allow_pool_delete true 2026-02-24T07:42:12.693 INFO:teuthology.orchestra.run.trial035.stdout:global advanced mon_clock_drift_allowed 1.000000 2026-02-24T07:42:12.693 INFO:teuthology.orchestra.run.trial035.stdout:global advanced mon_cluster_log_to_file true 2026-02-24T07:42:12.693 INFO:teuthology.orchestra.run.trial035.stdout:global advanced mon_max_pg_per_osd 10000 2026-02-24T07:42:12.693 INFO:teuthology.orchestra.run.trial035.stdout:global advanced mon_pg_warn_max_object_skew 0.000000 2026-02-24T07:42:12.693 INFO:teuthology.orchestra.run.trial035.stdout:global advanced mon_warn_on_crush_straw_calc_version_zero false 2026-02-24T07:42:12.693 INFO:teuthology.orchestra.run.trial035.stdout:global advanced mon_warn_on_legacy_crush_tunables false 2026-02-24T07:42:12.693 INFO:teuthology.orchestra.run.trial035.stdout:global advanced mon_warn_on_osd_down_out_interval_zero false 2026-02-24T07:42:12.693 INFO:teuthology.orchestra.run.trial035.stdout:global dev mon_warn_on_pool_pg_num_not_power_of_two false 2026-02-24T07:42:12.693 INFO:teuthology.orchestra.run.trial035.stdout:global advanced mon_warn_on_too_few_osds false 2026-02-24T07:42:12.693 INFO:teuthology.orchestra.run.trial035.stdout:global basic ms_client_mode crc * 2026-02-24T07:42:12.693 INFO:teuthology.orchestra.run.trial035.stdout:global basic ms_cluster_mode crc * 2026-02-24T07:42:12.693 INFO:teuthology.orchestra.run.trial035.stdout:global dev ms_die_on_bug true 2026-02-24T07:42:12.693 INFO:teuthology.orchestra.run.trial035.stdout:global dev ms_die_on_old_message true 2026-02-24T07:42:12.694 INFO:teuthology.orchestra.run.trial035.stdout:global basic ms_mon_client_mode crc * 2026-02-24T07:42:12.694 INFO:teuthology.orchestra.run.trial035.stdout:global basic ms_mon_cluster_mode crc * 2026-02-24T07:42:12.694 INFO:teuthology.orchestra.run.trial035.stdout:global basic ms_mon_service_mode crc * 2026-02-24T07:42:12.694 INFO:teuthology.orchestra.run.trial035.stdout:global basic ms_service_mode crc * 2026-02-24T07:42:12.694 INFO:teuthology.orchestra.run.trial035.stdout:global dev osd_max_pg_log_entries 2 2026-02-24T07:42:12.694 INFO:teuthology.orchestra.run.trial035.stdout:global dev osd_min_pg_log_entries 1 2026-02-24T07:42:12.694 INFO:teuthology.orchestra.run.trial035.stdout:global dev osd_pg_log_trim_min 0 2026-02-24T07:42:12.694 INFO:teuthology.orchestra.run.trial035.stdout:global advanced osd_pool_default_erasure_code_profile plugin=isa technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2026-02-24T07:42:12.694 INFO:teuthology.orchestra.run.trial035.stdout:global advanced osd_pool_default_min_size 2 2026-02-24T07:42:12.694 INFO:teuthology.orchestra.run.trial035.stdout:global advanced osd_pool_default_pg_autoscale_mode off 2026-02-24T07:42:12.694 INFO:teuthology.orchestra.run.trial035.stdout:global advanced osd_pool_default_size 3 2026-02-24T07:42:12.694 INFO:teuthology.orchestra.run.trial035.stdout:global advanced public_network 10.20.192.0/20 * 2026-02-24T07:42:12.695 INFO:teuthology.orchestra.run.trial035.stdout:mon advanced auth_allow_insecure_global_id_reclaim false 2026-02-24T07:42:12.695 INFO:teuthology.orchestra.run.trial035.stdout:mon advanced auth_mon_ticket_ttl 660.000000 2026-02-24T07:42:12.695 INFO:teuthology.orchestra.run.trial035.stdout:mon advanced auth_service_ticket_ttl 240.000000 2026-02-24T07:42:12.695 INFO:teuthology.orchestra.run.trial035.stdout:mon advanced debug_mon 20/20 2026-02-24T07:42:12.695 INFO:teuthology.orchestra.run.trial035.stdout:mon advanced debug_ms 1/1 2026-02-24T07:42:12.695 INFO:teuthology.orchestra.run.trial035.stdout:mon advanced debug_paxos 20/20 2026-02-24T07:42:12.695 INFO:teuthology.orchestra.run.trial035.stdout:mon advanced mon_data_avail_warn 5 2026-02-24T07:42:12.695 INFO:teuthology.orchestra.run.trial035.stdout:mon advanced mon_mgr_mkfs_grace 240 2026-02-24T07:42:12.695 INFO:teuthology.orchestra.run.trial035.stdout:mon advanced mon_min_osdmap_epochs 50 2026-02-24T07:42:12.695 INFO:teuthology.orchestra.run.trial035.stdout:mon dev mon_osd_prime_pg_temp true 2026-02-24T07:42:12.695 INFO:teuthology.orchestra.run.trial035.stdout:mon advanced mon_osd_reporter_subtree_level osd 2026-02-24T07:42:12.695 INFO:teuthology.orchestra.run.trial035.stdout:mon advanced mon_osdmap_full_prune_interval 2 2026-02-24T07:42:12.696 INFO:teuthology.orchestra.run.trial035.stdout:mon advanced mon_osdmap_full_prune_min 15 2026-02-24T07:42:12.696 INFO:teuthology.orchestra.run.trial035.stdout:mon advanced mon_osdmap_full_prune_txsize 2 2026-02-24T07:42:12.696 INFO:teuthology.orchestra.run.trial035.stdout:mon advanced mon_reweight_min_bytes_per_osd 10 2026-02-24T07:42:12.696 INFO:teuthology.orchestra.run.trial035.stdout:mon advanced mon_reweight_min_pgs_per_osd 4 2026-02-24T07:42:12.696 INFO:teuthology.orchestra.run.trial035.stdout:mon advanced mon_warn_on_insecure_global_id_reclaim false 2026-02-24T07:42:12.696 INFO:teuthology.orchestra.run.trial035.stdout:mon advanced mon_warn_on_insecure_global_id_reclaim_allowed false 2026-02-24T07:42:12.696 INFO:teuthology.orchestra.run.trial035.stdout:mon advanced osd_pool_default_crimson true 2026-02-24T07:42:12.696 INFO:teuthology.orchestra.run.trial035.stdout:mon advanced paxos_service_trim_min 10 2026-02-24T07:42:12.696 INFO:teuthology.orchestra.run.trial035.stdout:mgr advanced debug_mgr 20/20 2026-02-24T07:42:12.696 INFO:teuthology.orchestra.run.trial035.stdout:mgr advanced debug_ms 1/1 2026-02-24T07:42:12.696 INFO:teuthology.orchestra.run.trial035.stdout:mgr advanced debug_osd 10/10 2026-02-24T07:42:12.696 INFO:teuthology.orchestra.run.trial035.stdout:mgr advanced mgr/cephadm/allow_ptrace true * 2026-02-24T07:42:12.697 INFO:teuthology.orchestra.run.trial035.stdout:mgr advanced mgr/cephadm/container_init True * 2026-02-24T07:42:12.697 INFO:teuthology.orchestra.run.trial035.stdout:mgr advanced mgr/cephadm/migration_current 8 * 2026-02-24T07:42:12.697 INFO:teuthology.orchestra.run.trial035.stdout:mgr advanced mgr/dashboard/ssl_server_port 8443 * 2026-02-24T07:42:12.697 INFO:teuthology.orchestra.run.trial035.stdout:mgr advanced mgr/orchestrator/orchestrator cephadm 2026-02-24T07:42:12.697 INFO:teuthology.orchestra.run.trial035.stdout:mgr advanced mon_reweight_min_bytes_per_osd 10 2026-02-24T07:42:12.697 INFO:teuthology.orchestra.run.trial035.stdout:mgr advanced mon_reweight_min_pgs_per_osd 4 2026-02-24T07:42:12.697 INFO:teuthology.orchestra.run.trial035.stdout:osd dev bdev_debug_aio true 2026-02-24T07:42:12.697 INFO:teuthology.orchestra.run.trial035.stdout:osd advanced crimson_cpu_num 4 * 2026-02-24T07:42:12.697 INFO:teuthology.orchestra.run.trial035.stdout:osd advanced crimson_osd_obc_lru_size 10 2026-02-24T07:42:12.697 INFO:teuthology.orchestra.run.trial035.stdout:osd advanced debug_ms 20/20 2026-02-24T07:42:12.697 INFO:teuthology.orchestra.run.trial035.stdout:osd advanced debug_osd 20/20 2026-02-24T07:42:12.697 INFO:teuthology.orchestra.run.trial035.stdout:osd advanced debug_seastore 20/20 2026-02-24T07:42:12.697 INFO:teuthology.orchestra.run.trial035.stdout:osd advanced debug_seastore_backref 20/20 2026-02-24T07:42:12.698 INFO:teuthology.orchestra.run.trial035.stdout:osd advanced debug_seastore_cache 20/20 2026-02-24T07:42:12.698 INFO:teuthology.orchestra.run.trial035.stdout:osd advanced debug_seastore_cleaner 20/20 2026-02-24T07:42:12.698 INFO:teuthology.orchestra.run.trial035.stdout:osd advanced debug_seastore_device 20/20 2026-02-24T07:42:12.698 INFO:teuthology.orchestra.run.trial035.stdout:osd advanced debug_seastore_epm 20/20 2026-02-24T07:42:12.698 INFO:teuthology.orchestra.run.trial035.stdout:osd advanced debug_seastore_fixedkv_tree 20/20 2026-02-24T07:42:12.698 INFO:teuthology.orchestra.run.trial035.stdout:osd advanced debug_seastore_journal 20/20 2026-02-24T07:42:12.698 INFO:teuthology.orchestra.run.trial035.stdout:osd advanced debug_seastore_lba 20/20 2026-02-24T07:42:12.698 INFO:teuthology.orchestra.run.trial035.stdout:osd advanced debug_seastore_odata 20/20 2026-02-24T07:42:12.698 INFO:teuthology.orchestra.run.trial035.stdout:osd advanced debug_seastore_omap 20/20 2026-02-24T07:42:12.698 INFO:teuthology.orchestra.run.trial035.stdout:osd advanced debug_seastore_onode 20/20 2026-02-24T07:42:12.698 INFO:teuthology.orchestra.run.trial035.stdout:osd advanced debug_seastore_t 20/20 2026-02-24T07:42:12.698 INFO:teuthology.orchestra.run.trial035.stdout:osd advanced debug_seastore_tm 20/20 2026-02-24T07:42:12.698 INFO:teuthology.orchestra.run.trial035.stdout:osd advanced osd_blocked_scrub_grace_period 3600 2026-02-24T07:42:12.699 INFO:teuthology.orchestra.run.trial035.stdout:osd dev osd_debug_misdirected_ops true 2026-02-24T07:42:12.699 INFO:teuthology.orchestra.run.trial035.stdout:osd dev osd_debug_op_order true 2026-02-24T07:42:12.699 INFO:teuthology.orchestra.run.trial035.stdout:osd dev osd_debug_pg_log_writeout true 2026-02-24T07:42:12.699 INFO:teuthology.orchestra.run.trial035.stdout:osd dev osd_debug_reject_backfill_probability 0.300000 2026-02-24T07:42:12.699 INFO:teuthology.orchestra.run.trial035.stdout:osd dev osd_debug_shutdown true 2026-02-24T07:42:12.699 INFO:teuthology.orchestra.run.trial035.stdout:osd dev osd_debug_verify_cached_snaps true 2026-02-24T07:42:12.699 INFO:teuthology.orchestra.run.trial035.stdout:osd dev osd_debug_verify_missing_on_start true 2026-02-24T07:42:12.699 INFO:teuthology.orchestra.run.trial035.stdout:osd dev osd_debug_verify_stray_on_activate true 2026-02-24T07:42:12.699 INFO:teuthology.orchestra.run.trial035.stdout:osd advanced osd_deep_scrub_update_digest_min_age 30 2026-02-24T07:42:12.699 INFO:teuthology.orchestra.run.trial035.stdout:osd advanced osd_delete_sleep 1.000000 2026-02-24T07:42:12.700 INFO:teuthology.orchestra.run.trial035.stdout:osd advanced osd_max_backfills 3 2026-02-24T07:42:12.700 INFO:teuthology.orchestra.run.trial035.stdout:osd advanced osd_max_markdown_count 1000 2026-02-24T07:42:12.700 INFO:teuthology.orchestra.run.trial035.stdout:osd advanced osd_mclock_profile high_recovery_ops 2026-02-24T07:42:12.700 INFO:teuthology.orchestra.run.trial035.stdout:osd dev osd_mclock_skip_benchmark true 2026-02-24T07:42:12.700 INFO:teuthology.orchestra.run.trial035.stdout:osd advanced osd_memory_target_autotune true 2026-02-24T07:42:12.700 INFO:teuthology.orchestra.run.trial035.stdout:osd advanced osd_objectstore seastore * 2026-02-24T07:42:12.700 INFO:teuthology.orchestra.run.trial035.stdout:osd advanced osd_op_queue debug_random * 2026-02-24T07:42:12.700 INFO:teuthology.orchestra.run.trial035.stdout:osd advanced osd_op_queue_cut_off debug_random * 2026-02-24T07:42:12.700 INFO:teuthology.orchestra.run.trial035.stdout:osd advanced osd_recover_clone_overlap true 2026-02-24T07:42:12.700 INFO:teuthology.orchestra.run.trial035.stdout:osd advanced osd_recovery_max_chunk 1048576 2026-02-24T07:42:12.700 INFO:teuthology.orchestra.run.trial035.stdout:osd advanced osd_scrub_load_threshold 5.000000 2026-02-24T07:42:12.700 INFO:teuthology.orchestra.run.trial035.stdout:osd advanced osd_scrub_max_interval 120.000000 2026-02-24T07:42:12.700 INFO:teuthology.orchestra.run.trial035.stdout:osd advanced osd_scrub_min_interval 60.000000 2026-02-24T07:42:12.701 INFO:teuthology.orchestra.run.trial035.stdout:osd advanced osd_shutdown_pgref_assert true 2026-02-24T07:42:12.701 INFO:teuthology.orchestra.run.trial035.stdout:osd advanced osd_snap_trim_sleep 2.000000 2026-02-24T07:42:12.701 INFO:teuthology.orchestra.run.trial035.stdout:osd advanced seastore_cachepin_size_pershard 67108864 2026-02-24T07:42:12.701 INFO:teuthology.orchestra.run.trial035.stdout:osd dev seastore_main_device_type RANDOM_BLOCK_SSD * 2026-02-24T07:42:12.701 INFO:teuthology.orchestra.run.trial035.stdout:osd advanced seastore_max_concurrent_transactions 8 2026-02-24T07:42:12.701 INFO:teuthology.orchestra.run.trial035.stdout:client.0 advanced debug_ms 1/1 2026-02-24T07:42:12.701 INFO:teuthology.orchestra.run.trial035.stdout:client.0 advanced debug_objecter 20/20 2026-02-24T07:42:12.701 INFO:teuthology.orchestra.run.trial035.stdout:client.0 advanced debug_rados 20/20 2026-02-24T07:42:12.701 INFO:teuthology.orchestra.run.trial035.stdout:client.rgw advanced rgw_cache_enabled true 2026-02-24T07:42:12.701 INFO:teuthology.orchestra.run.trial035.stdout:client.rgw advanced rgw_enable_ops_log true 2026-02-24T07:42:12.701 INFO:teuthology.orchestra.run.trial035.stdout:client.rgw advanced rgw_enable_usage_log true 2026-02-24T07:42:12.729 INFO:tasks.cephadm:Enabling crimson flags... 2026-02-24T07:42:12.729 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph osd set-allow-crimson --yes-i-really-mean-it 2026-02-24T07:42:12.843 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:42:12.865 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:12 trial035 ceph-mon[39284]: from='mgr.14154 10.20.193.35:0/1087484775' entity='mgr.x' 2026-02-24T07:42:12.865 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:12 trial035 ceph-mon[39284]: from='mgr.14154 10.20.193.35:0/1087484775' entity='mgr.x' 2026-02-24T07:42:12.865 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:12 trial035 ceph-mon[39284]: from='mgr.14154 10.20.193.35:0/1087484775' entity='mgr.x' 2026-02-24T07:42:12.865 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:12 trial035 ceph-mon[39284]: from='mgr.14154 10.20.193.35:0/1087484775' entity='mgr.x' 2026-02-24T07:42:12.865 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:12 trial035 ceph-mon[39284]: from='mgr.14154 10.20.193.35:0/1087484775' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd/host:trial091", "name": "osd_memory_target"} : dispatch 2026-02-24T07:42:12.865 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:12 trial035 ceph-mon[39284]: from='mgr.14154 10.20.193.35:0/1087484775' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-24T07:42:12.865 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:12 trial035 ceph-mon[39284]: from='mgr.14154 10.20.193.35:0/1087484775' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2026-02-24T07:42:12.865 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:12 trial035 ceph-mon[39284]: Updating trial091:/etc/ceph/ceph.conf 2026-02-24T07:42:12.865 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:12 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/3388594668' entity='client.admin' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-24T07:42:12.865 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:12 trial035 ceph-mon[39284]: Updating trial091:/var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/config/ceph.conf 2026-02-24T07:42:12.865 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:12 trial035 ceph-mon[39284]: Updating trial091:/etc/ceph/ceph.client.admin.keyring 2026-02-24T07:42:12.866 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:12 trial035 ceph-mon[39284]: from='mgr.14154 10.20.193.35:0/1087484775' entity='mgr.x' 2026-02-24T07:42:12.866 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:12 trial035 ceph-mon[39284]: from='mgr.14154 10.20.193.35:0/1087484775' entity='mgr.x' 2026-02-24T07:42:12.866 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:12 trial035 ceph-mon[39284]: from='mgr.14154 10.20.193.35:0/1087484775' entity='mgr.x' 2026-02-24T07:42:12.866 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:12 trial035 ceph-mon[39284]: from='mgr.14154 10.20.193.35:0/1087484775' entity='mgr.x' 2026-02-24T07:42:12.866 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:12 trial035 ceph-mon[39284]: from='mgr.14154 10.20.193.35:0/1087484775' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2026-02-24T07:42:12.866 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:12 trial035 ceph-mon[39284]: from='mgr.14154 10.20.193.35:0/1087484775' entity='mgr.x' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2026-02-24T07:42:12.866 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:12 trial035 ceph-mon[39284]: from='mgr.14154 10.20.193.35:0/1087484775' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-24T07:42:12.866 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:12 trial035 ceph-mon[39284]: from='mgr.14154 10.20.193.35:0/1087484775' entity='mgr.x' 2026-02-24T07:42:12.866 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:12 trial035 ceph-mon[39284]: from='mgr.14154 10.20.193.35:0/1087484775' entity='mgr.x' 2026-02-24T07:42:12.866 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:12 trial035 ceph-mon[39284]: from='mgr.14154 10.20.193.35:0/1087484775' entity='mgr.x' 2026-02-24T07:42:12.866 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:12 trial035 ceph-mon[39284]: from='mgr.14154 10.20.193.35:0/1087484775' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]} : dispatch 2026-02-24T07:42:12.866 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:12 trial035 ceph-mon[39284]: from='mgr.14154 10.20.193.35:0/1087484775' entity='mgr.x' cmd={"prefix": "mgr services"} : dispatch 2026-02-24T07:42:12.866 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:12 trial035 ceph-mon[39284]: from='mgr.14154 10.20.193.35:0/1087484775' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-24T07:42:12.942 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:42:12.939+0000 7f15cd50c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:42:12.942 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:42:12.940+0000 7f15cd50c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:42:13.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:13 trial035 ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-mon-a[39280]: 2026-02-24T07:42:13.057+0000 7f45ff681640 -1 WARNING: experimental feature 'crimson' is enabled 2026-02-24T07:42:13.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:13 trial035 ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-mon-a[39280]: Please be aware that this feature is experimental, untested, 2026-02-24T07:42:13.205 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:13 trial035 ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-mon-a[39280]: unsupported, and may result in data corruption, data loss, 2026-02-24T07:42:13.205 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:13 trial035 ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-mon-a[39280]: and/or irreparable damage to your cluster. Do not use 2026-02-24T07:42:13.205 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:13 trial035 ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-mon-a[39280]: feature with important data. 2026-02-24T07:42:13.205 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:13 trial035 ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-mon-a[39280]: 2026-02-24T07:42:13.689 INFO:tasks.cephadm:Deploying OSDs... 2026-02-24T07:42:13.690 DEBUG:teuthology.orchestra.run.trial035:> set -ex 2026-02-24T07:42:13.690 DEBUG:teuthology.orchestra.run.trial035:> dd if=/scratch_devs of=/dev/stdout 2026-02-24T07:42:13.708 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2026-02-24T07:42:13.708 DEBUG:teuthology.orchestra.run.trial035:> stat /dev/vg_nvme/lv_1 2026-02-24T07:42:13.764 INFO:teuthology.orchestra.run.trial035.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2026-02-24T07:42:13.764 INFO:teuthology.orchestra.run.trial035.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2026-02-24T07:42:13.764 INFO:teuthology.orchestra.run.trial035.stdout:Device: 6h/6d Inode: 1307 Links: 1 2026-02-24T07:42:13.764 INFO:teuthology.orchestra.run.trial035.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2026-02-24T07:42:13.764 INFO:teuthology.orchestra.run.trial035.stdout:Context: system_u:object_r:device_t:s0 2026-02-24T07:42:13.764 INFO:teuthology.orchestra.run.trial035.stdout:Access: 2026-02-24 07:42:07.588041023 +0000 2026-02-24T07:42:13.765 INFO:teuthology.orchestra.run.trial035.stdout:Modify: 2026-02-24 07:29:29.292185418 +0000 2026-02-24T07:42:13.765 INFO:teuthology.orchestra.run.trial035.stdout:Change: 2026-02-24 07:29:29.292185418 +0000 2026-02-24T07:42:13.765 INFO:teuthology.orchestra.run.trial035.stdout: Birth: 2026-02-24 07:29:29.292185418 +0000 2026-02-24T07:42:13.765 DEBUG:teuthology.orchestra.run.trial035:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2026-02-24T07:42:13.830 INFO:teuthology.orchestra.run.trial035.stderr:1+0 records in 2026-02-24T07:42:13.830 INFO:teuthology.orchestra.run.trial035.stderr:1+0 records out 2026-02-24T07:42:13.830 INFO:teuthology.orchestra.run.trial035.stderr:512 bytes copied, 0.000149297 s, 3.4 MB/s 2026-02-24T07:42:13.831 DEBUG:teuthology.orchestra.run.trial035:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2026-02-24T07:42:13.890 DEBUG:teuthology.orchestra.run.trial035:> stat /dev/vg_nvme/lv_2 2026-02-24T07:42:13.946 INFO:teuthology.orchestra.run.trial035.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2026-02-24T07:42:13.946 INFO:teuthology.orchestra.run.trial035.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2026-02-24T07:42:13.946 INFO:teuthology.orchestra.run.trial035.stdout:Device: 6h/6d Inode: 1300 Links: 1 2026-02-24T07:42:13.946 INFO:teuthology.orchestra.run.trial035.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2026-02-24T07:42:13.946 INFO:teuthology.orchestra.run.trial035.stdout:Context: system_u:object_r:device_t:s0 2026-02-24T07:42:13.946 INFO:teuthology.orchestra.run.trial035.stdout:Access: 2026-02-24 07:42:07.622039892 +0000 2026-02-24T07:42:13.947 INFO:teuthology.orchestra.run.trial035.stdout:Modify: 2026-02-24 07:29:29.292185418 +0000 2026-02-24T07:42:13.947 INFO:teuthology.orchestra.run.trial035.stdout:Change: 2026-02-24 07:29:29.292185418 +0000 2026-02-24T07:42:13.947 INFO:teuthology.orchestra.run.trial035.stdout: Birth: 2026-02-24 07:29:29.292185418 +0000 2026-02-24T07:42:13.947 DEBUG:teuthology.orchestra.run.trial035:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2026-02-24T07:42:13.954 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:13 trial035 ceph-mon[39284]: Updating trial091:/var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/config/ceph.client.admin.keyring 2026-02-24T07:42:13.954 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:13 trial035 ceph-mon[39284]: Reconfiguring mon.a (unknown last config time)... 2026-02-24T07:42:13.954 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:13 trial035 ceph-mon[39284]: Reconfiguring daemon mon.a on trial035 2026-02-24T07:42:13.954 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:13 trial035 ceph-mon[39284]: from='client.14194 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "1;trial035=x", "target": ["mon-mgr", ""]}]: dispatch 2026-02-24T07:42:13.954 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:13 trial035 ceph-mon[39284]: Saving service mgr spec with placement trial035=x;count:1 2026-02-24T07:42:13.954 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:13 trial035 ceph-mon[39284]: Reconfiguring mgr.x (unknown last config time)... 2026-02-24T07:42:13.954 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:13 trial035 ceph-mon[39284]: Reconfiguring daemon mgr.x on trial035 2026-02-24T07:42:13.954 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:13 trial035 ceph-mon[39284]: from='mgr.14154 10.20.193.35:0/1087484775' entity='mgr.x' 2026-02-24T07:42:13.954 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:13 trial035 ceph-mon[39284]: from='mgr.14154 10.20.193.35:0/1087484775' entity='mgr.x' 2026-02-24T07:42:13.954 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:13 trial035 ceph-mon[39284]: from='mgr.14154 10.20.193.35:0/1087484775' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2026-02-24T07:42:13.954 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:13 trial035 ceph-mon[39284]: from='mgr.14154 10.20.193.35:0/1087484775' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-24T07:42:13.954 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:13 trial035 ceph-mon[39284]: from='mgr.14154 10.20.193.35:0/1087484775' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2026-02-24T07:42:13.954 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:13 trial035 ceph-mon[39284]: from='mgr.14154 10.20.193.35:0/1087484775' entity='mgr.x' 2026-02-24T07:42:13.954 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:13 trial035 ceph-mon[39284]: from='mgr.14154 10.20.193.35:0/1087484775' entity='mgr.x' 2026-02-24T07:42:13.955 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:13 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/3484384930' entity='client.admin' cmd={"prefix": "config dump"} : dispatch 2026-02-24T07:42:13.955 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:13 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/1328142674' entity='client.admin' cmd={"prefix": "osd set-allow-crimson", "yes_i_really_mean_it": true} : dispatch 2026-02-24T07:42:13.977 INFO:teuthology.orchestra.run.trial035.stderr:1+0 records in 2026-02-24T07:42:13.977 INFO:teuthology.orchestra.run.trial035.stderr:1+0 records out 2026-02-24T07:42:13.977 INFO:teuthology.orchestra.run.trial035.stderr:512 bytes copied, 0.000193249 s, 2.6 MB/s 2026-02-24T07:42:13.978 DEBUG:teuthology.orchestra.run.trial035:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2026-02-24T07:42:14.037 DEBUG:teuthology.orchestra.run.trial035:> stat /dev/vg_nvme/lv_3 2026-02-24T07:42:14.093 INFO:teuthology.orchestra.run.trial035.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2026-02-24T07:42:14.094 INFO:teuthology.orchestra.run.trial035.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2026-02-24T07:42:14.094 INFO:teuthology.orchestra.run.trial035.stdout:Device: 6h/6d Inode: 1295 Links: 1 2026-02-24T07:42:14.094 INFO:teuthology.orchestra.run.trial035.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2026-02-24T07:42:14.094 INFO:teuthology.orchestra.run.trial035.stdout:Context: system_u:object_r:device_t:s0 2026-02-24T07:42:14.094 INFO:teuthology.orchestra.run.trial035.stdout:Access: 2026-02-24 07:42:07.657038727 +0000 2026-02-24T07:42:14.094 INFO:teuthology.orchestra.run.trial035.stdout:Modify: 2026-02-24 07:29:29.292185418 +0000 2026-02-24T07:42:14.095 INFO:teuthology.orchestra.run.trial035.stdout:Change: 2026-02-24 07:29:29.292185418 +0000 2026-02-24T07:42:14.095 INFO:teuthology.orchestra.run.trial035.stdout: Birth: 2026-02-24 07:29:29.292185418 +0000 2026-02-24T07:42:14.095 DEBUG:teuthology.orchestra.run.trial035:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2026-02-24T07:42:14.157 INFO:teuthology.orchestra.run.trial035.stderr:1+0 records in 2026-02-24T07:42:14.157 INFO:teuthology.orchestra.run.trial035.stderr:1+0 records out 2026-02-24T07:42:14.157 INFO:teuthology.orchestra.run.trial035.stderr:512 bytes copied, 0.000150149 s, 3.4 MB/s 2026-02-24T07:42:14.158 DEBUG:teuthology.orchestra.run.trial035:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2026-02-24T07:42:14.217 DEBUG:teuthology.orchestra.run.trial035:> stat /dev/vg_nvme/lv_4 2026-02-24T07:42:14.274 INFO:teuthology.orchestra.run.trial035.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2026-02-24T07:42:14.274 INFO:teuthology.orchestra.run.trial035.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2026-02-24T07:42:14.274 INFO:teuthology.orchestra.run.trial035.stdout:Device: 6h/6d Inode: 1289 Links: 1 2026-02-24T07:42:14.274 INFO:teuthology.orchestra.run.trial035.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2026-02-24T07:42:14.275 INFO:teuthology.orchestra.run.trial035.stdout:Context: system_u:object_r:device_t:s0 2026-02-24T07:42:14.275 INFO:teuthology.orchestra.run.trial035.stdout:Access: 2026-02-24 07:42:07.691037596 +0000 2026-02-24T07:42:14.275 INFO:teuthology.orchestra.run.trial035.stdout:Modify: 2026-02-24 07:29:29.291185451 +0000 2026-02-24T07:42:14.275 INFO:teuthology.orchestra.run.trial035.stdout:Change: 2026-02-24 07:29:29.291185451 +0000 2026-02-24T07:42:14.275 INFO:teuthology.orchestra.run.trial035.stdout: Birth: 2026-02-24 07:29:29.291185451 +0000 2026-02-24T07:42:14.275 DEBUG:teuthology.orchestra.run.trial035:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2026-02-24T07:42:14.339 INFO:teuthology.orchestra.run.trial035.stderr:1+0 records in 2026-02-24T07:42:14.339 INFO:teuthology.orchestra.run.trial035.stderr:1+0 records out 2026-02-24T07:42:14.339 INFO:teuthology.orchestra.run.trial035.stderr:512 bytes copied, 0.000149207 s, 3.4 MB/s 2026-02-24T07:42:14.340 DEBUG:teuthology.orchestra.run.trial035:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2026-02-24T07:42:14.397 DEBUG:teuthology.orchestra.run.trial091:> set -ex 2026-02-24T07:42:14.397 DEBUG:teuthology.orchestra.run.trial091:> dd if=/scratch_devs of=/dev/stdout 2026-02-24T07:42:14.417 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2026-02-24T07:42:14.418 DEBUG:teuthology.orchestra.run.trial091:> stat /dev/vg_nvme/lv_1 2026-02-24T07:42:14.474 INFO:teuthology.orchestra.run.trial091.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2026-02-24T07:42:14.474 INFO:teuthology.orchestra.run.trial091.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2026-02-24T07:42:14.474 INFO:teuthology.orchestra.run.trial091.stdout:Device: 6h/6d Inode: 1310 Links: 1 2026-02-24T07:42:14.474 INFO:teuthology.orchestra.run.trial091.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2026-02-24T07:42:14.475 INFO:teuthology.orchestra.run.trial091.stdout:Context: system_u:object_r:device_t:s0 2026-02-24T07:42:14.475 INFO:teuthology.orchestra.run.trial091.stdout:Access: 2026-02-24 07:42:11.319586717 +0000 2026-02-24T07:42:14.475 INFO:teuthology.orchestra.run.trial091.stdout:Modify: 2026-02-24 07:39:09.470665644 +0000 2026-02-24T07:42:14.475 INFO:teuthology.orchestra.run.trial091.stdout:Change: 2026-02-24 07:39:09.470665644 +0000 2026-02-24T07:42:14.475 INFO:teuthology.orchestra.run.trial091.stdout: Birth: 2026-02-24 07:39:09.470665644 +0000 2026-02-24T07:42:14.475 DEBUG:teuthology.orchestra.run.trial091:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2026-02-24T07:42:14.539 INFO:teuthology.orchestra.run.trial091.stderr:1+0 records in 2026-02-24T07:42:14.539 INFO:teuthology.orchestra.run.trial091.stderr:1+0 records out 2026-02-24T07:42:14.539 INFO:teuthology.orchestra.run.trial091.stderr:512 bytes copied, 0.000162211 s, 3.2 MB/s 2026-02-24T07:42:14.541 DEBUG:teuthology.orchestra.run.trial091:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2026-02-24T07:42:14.597 DEBUG:teuthology.orchestra.run.trial091:> stat /dev/vg_nvme/lv_2 2026-02-24T07:42:14.653 INFO:teuthology.orchestra.run.trial091.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2026-02-24T07:42:14.653 INFO:teuthology.orchestra.run.trial091.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2026-02-24T07:42:14.653 INFO:teuthology.orchestra.run.trial091.stdout:Device: 6h/6d Inode: 1305 Links: 1 2026-02-24T07:42:14.653 INFO:teuthology.orchestra.run.trial091.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2026-02-24T07:42:14.653 INFO:teuthology.orchestra.run.trial091.stdout:Context: system_u:object_r:device_t:s0 2026-02-24T07:42:14.653 INFO:teuthology.orchestra.run.trial091.stdout:Access: 2026-02-24 07:42:11.355585320 +0000 2026-02-24T07:42:14.653 INFO:teuthology.orchestra.run.trial091.stdout:Modify: 2026-02-24 07:39:09.470665644 +0000 2026-02-24T07:42:14.653 INFO:teuthology.orchestra.run.trial091.stdout:Change: 2026-02-24 07:39:09.470665644 +0000 2026-02-24T07:42:14.653 INFO:teuthology.orchestra.run.trial091.stdout: Birth: 2026-02-24 07:39:09.470665644 +0000 2026-02-24T07:42:14.654 DEBUG:teuthology.orchestra.run.trial091:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2026-02-24T07:42:14.715 INFO:teuthology.orchestra.run.trial091.stderr:1+0 records in 2026-02-24T07:42:14.715 INFO:teuthology.orchestra.run.trial091.stderr:1+0 records out 2026-02-24T07:42:14.715 INFO:teuthology.orchestra.run.trial091.stderr:512 bytes copied, 0.000144028 s, 3.6 MB/s 2026-02-24T07:42:14.716 DEBUG:teuthology.orchestra.run.trial091:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2026-02-24T07:42:14.773 DEBUG:teuthology.orchestra.run.trial091:> stat /dev/vg_nvme/lv_3 2026-02-24T07:42:14.830 INFO:teuthology.orchestra.run.trial091.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2026-02-24T07:42:14.830 INFO:teuthology.orchestra.run.trial091.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2026-02-24T07:42:14.831 INFO:teuthology.orchestra.run.trial091.stdout:Device: 6h/6d Inode: 1293 Links: 1 2026-02-24T07:42:14.831 INFO:teuthology.orchestra.run.trial091.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2026-02-24T07:42:14.831 INFO:teuthology.orchestra.run.trial091.stdout:Context: system_u:object_r:device_t:s0 2026-02-24T07:42:14.831 INFO:teuthology.orchestra.run.trial091.stdout:Access: 2026-02-24 07:42:11.389583999 +0000 2026-02-24T07:42:14.831 INFO:teuthology.orchestra.run.trial091.stdout:Modify: 2026-02-24 07:39:09.470665644 +0000 2026-02-24T07:42:14.832 INFO:teuthology.orchestra.run.trial091.stdout:Change: 2026-02-24 07:39:09.470665644 +0000 2026-02-24T07:42:14.832 INFO:teuthology.orchestra.run.trial091.stdout: Birth: 2026-02-24 07:39:09.470665644 +0000 2026-02-24T07:42:14.832 DEBUG:teuthology.orchestra.run.trial091:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2026-02-24T07:42:14.894 INFO:teuthology.orchestra.run.trial091.stderr:1+0 records in 2026-02-24T07:42:14.894 INFO:teuthology.orchestra.run.trial091.stderr:1+0 records out 2026-02-24T07:42:14.894 INFO:teuthology.orchestra.run.trial091.stderr:512 bytes copied, 0.000154968 s, 3.3 MB/s 2026-02-24T07:42:14.895 DEBUG:teuthology.orchestra.run.trial091:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2026-02-24T07:42:14.952 DEBUG:teuthology.orchestra.run.trial091:> stat /dev/vg_nvme/lv_4 2026-02-24T07:42:14.954 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:14 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/1328142674' entity='client.admin' cmd='[{"prefix": "osd set-allow-crimson", "yes_i_really_mean_it": true}]': finished 2026-02-24T07:42:14.954 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:14 trial035 ceph-mon[39284]: osdmap e5: 0 total, 0 up, 0 in 2026-02-24T07:42:15.009 INFO:teuthology.orchestra.run.trial091.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2026-02-24T07:42:15.009 INFO:teuthology.orchestra.run.trial091.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2026-02-24T07:42:15.009 INFO:teuthology.orchestra.run.trial091.stdout:Device: 6h/6d Inode: 1297 Links: 1 2026-02-24T07:42:15.010 INFO:teuthology.orchestra.run.trial091.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2026-02-24T07:42:15.010 INFO:teuthology.orchestra.run.trial091.stdout:Context: system_u:object_r:device_t:s0 2026-02-24T07:42:15.010 INFO:teuthology.orchestra.run.trial091.stdout:Access: 2026-02-24 07:42:11.421582756 +0000 2026-02-24T07:42:15.010 INFO:teuthology.orchestra.run.trial091.stdout:Modify: 2026-02-24 07:39:09.470665644 +0000 2026-02-24T07:42:15.010 INFO:teuthology.orchestra.run.trial091.stdout:Change: 2026-02-24 07:39:09.470665644 +0000 2026-02-24T07:42:15.010 INFO:teuthology.orchestra.run.trial091.stdout: Birth: 2026-02-24 07:39:09.470665644 +0000 2026-02-24T07:42:15.010 DEBUG:teuthology.orchestra.run.trial091:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2026-02-24T07:42:15.073 INFO:teuthology.orchestra.run.trial091.stderr:1+0 records in 2026-02-24T07:42:15.073 INFO:teuthology.orchestra.run.trial091.stderr:1+0 records out 2026-02-24T07:42:15.073 INFO:teuthology.orchestra.run.trial091.stderr:512 bytes copied, 0.000142324 s, 3.6 MB/s 2026-02-24T07:42:15.074 DEBUG:teuthology.orchestra.run.trial091:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2026-02-24T07:42:15.132 INFO:tasks.cephadm:Deploying osd.0 on trial035 with /dev/vg_nvme/lv_4... 2026-02-24T07:42:15.132 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- lvm zap /dev/vg_nvme/lv_4 2026-02-24T07:42:15.251 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:42:15.704 INFO:teuthology.orchestra.run.trial035.stdout: 2026-02-24T07:42:15.721 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph orch daemon add osd trial035:vg_nvme/lv_4 --skip-validation 2026-02-24T07:42:15.838 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:42:15.948 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:42:15.946+0000 7f0012def640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:42:15.948 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:42:15.946+0000 7f0012def640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:42:17.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:17 trial035 ceph-mon[39284]: from='client.14200 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "trial035:vg_nvme/lv_4", "skip_validation": true, "target": ["mon-mgr", ""]}]: dispatch 2026-02-24T07:42:17.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:17 trial035 ceph-mon[39284]: osd.default does not exist. Creating it now. 2026-02-24T07:42:17.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:17 trial035 ceph-mon[39284]: Creating OSDs with service ID: default on trial035:['vg_nvme/lv_4'] 2026-02-24T07:42:17.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:17 trial035 ceph-mon[39284]: from='mgr.14154 10.20.193.35:0/1087484775' entity='mgr.x' 2026-02-24T07:42:17.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:17 trial035 ceph-mon[39284]: Marking host: trial035 for OSDSpec preview refresh. 2026-02-24T07:42:17.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:17 trial035 ceph-mon[39284]: Saving service osd.default spec with placement trial035 2026-02-24T07:42:17.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:17 trial035 ceph-mon[39284]: from='mgr.14154 10.20.193.35:0/1087484775' entity='mgr.x' 2026-02-24T07:42:17.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:17 trial035 ceph-mon[39284]: from='mgr.14154 10.20.193.35:0/1087484775' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2026-02-24T07:42:17.205 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:17 trial035 ceph-mon[39284]: from='mgr.14154 10.20.193.35:0/1087484775' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2026-02-24T07:42:17.205 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:17 trial035 ceph-mon[39284]: from='mgr.14154 10.20.193.35:0/1087484775' entity='mgr.x' 2026-02-24T07:42:17.205 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:17 trial035 ceph-mon[39284]: from='mgr.14154 10.20.193.35:0/1087484775' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2026-02-24T07:42:17.205 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:17 trial035 ceph-mon[39284]: from='mgr.14154 10.20.193.35:0/1087484775' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-24T07:42:17.205 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:17 trial035 ceph-mon[39284]: from='mgr.14154 10.20.193.35:0/1087484775' entity='mgr.x' 2026-02-24T07:42:17.205 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:17 trial035 ceph-mon[39284]: from='mgr.14154 10.20.193.35:0/1087484775' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-24T07:42:17.205 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:17 trial035 ceph-mon[39284]: from='mgr.14154 10.20.193.35:0/1087484775' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2026-02-24T07:42:17.205 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:17 trial035 ceph-mon[39284]: from='mgr.14154 10.20.193.35:0/1087484775' entity='mgr.x' 2026-02-24T07:42:17.205 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:17 trial035 ceph-mon[39284]: from='mgr.14154 10.20.193.35:0/1087484775' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2026-02-24T07:42:17.205 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:17 trial035 ceph-mon[39284]: from='mgr.14154 10.20.193.35:0/1087484775' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2026-02-24T07:42:17.205 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:17 trial035 ceph-mon[39284]: from='mgr.14154 10.20.193.35:0/1087484775' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-24T07:42:17.205 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:17 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/477780258' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "8d5b0adf-0e02-40d4-8780-e64988c10e92"} : dispatch 2026-02-24T07:42:17.205 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:17 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/477780258' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8d5b0adf-0e02-40d4-8780-e64988c10e92"}]': finished 2026-02-24T07:42:17.205 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:17 trial035 ceph-mon[39284]: osdmap e6: 1 total, 0 up, 1 in 2026-02-24T07:42:17.205 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:17 trial035 ceph-mon[39284]: from='mgr.14154 10.20.193.35:0/1087484775' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2026-02-24T07:42:17.205 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:17 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/3736278995' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2026-02-24T07:42:19.149 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:18 trial035 ceph-mon[39284]: from='mgr.14154 10.20.193.35:0/1087484775' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.0"} : dispatch 2026-02-24T07:42:19.149 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:18 trial035 ceph-mon[39284]: from='mgr.14154 10.20.193.35:0/1087484775' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-24T07:42:20.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:19 trial035 ceph-mon[39284]: Deploying daemon osd.0 on trial035 2026-02-24T07:42:20.205 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:19 trial035 ceph-mon[39284]: from='mgr.14154 10.20.193.35:0/1087484775' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.0"} : dispatch 2026-02-24T07:42:20.205 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:19 trial035 ceph-mon[39284]: from='mgr.14154 10.20.193.35:0/1087484775' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-24T07:42:21.094 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:20 trial035 ceph-mon[39284]: Deploying daemon osd.0 on trial035 2026-02-24T07:42:21.094 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:20 trial035 ceph-mon[39284]: from='mgr.14154 10.20.193.35:0/1087484775' entity='mgr.x' 2026-02-24T07:42:21.094 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:20 trial035 ceph-mon[39284]: from='mgr.14154 10.20.193.35:0/1087484775' entity='mgr.x' 2026-02-24T07:42:23.454 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:23 trial035 ceph-mon[39284]: from='mgr.14154 10.20.193.35:0/1087484775' entity='mgr.x' 2026-02-24T07:42:23.454 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:23 trial035 ceph-mon[39284]: from='mgr.14154 10.20.193.35:0/1087484775' entity='mgr.x' 2026-02-24T07:42:23.454 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:23 trial035 ceph-mon[39284]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2026-02-24T07:42:23.455 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:23 trial035 ceph-mon[39284]: from='mgr.14154 10.20.193.35:0/1087484775' entity='mgr.x' 2026-02-24T07:42:23.455 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:23 trial035 ceph-mon[39284]: from='mgr.14154 10.20.193.35:0/1087484775' entity='mgr.x' 2026-02-24T07:42:23.455 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:23 trial035 ceph-mon[39284]: from='mgr.14154 10.20.193.35:0/1087484775' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2026-02-24T07:42:23.455 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:23 trial035 ceph-mon[39284]: from='osd.0 [v2:10.20.193.35:6802/2333513103,v1:10.20.193.35:6803/2333513103]' entity='osd.0' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]} : dispatch 2026-02-24T07:42:23.455 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:23 trial035 ceph-mon[39284]: from='mgr.14154 10.20.193.35:0/1087484775' entity='mgr.x' 2026-02-24T07:42:23.455 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:23 trial035 ceph-mon[39284]: from='mgr.14154 10.20.193.35:0/1087484775' entity='mgr.x' 2026-02-24T07:42:23.531 INFO:teuthology.orchestra.run.trial035.stdout:Created osd(s) 0 on host 'trial035' 2026-02-24T07:42:23.573 DEBUG:teuthology.orchestra.run.trial035:osd.0> sudo journalctl -f -n 0 -u ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460@osd.0.service 2026-02-24T07:42:23.575 INFO:tasks.cephadm:Deploying osd.1 on trial035 with /dev/vg_nvme/lv_3... 2026-02-24T07:42:23.576 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- lvm zap /dev/vg_nvme/lv_3 2026-02-24T07:42:23.731 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:42:24.533 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:24 trial035 ceph-mon[39284]: from='mgr.14154 10.20.193.35:0/1087484775' entity='mgr.x' 2026-02-24T07:42:24.533 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:24 trial035 ceph-mon[39284]: from='mgr.14154 10.20.193.35:0/1087484775' entity='mgr.x' 2026-02-24T07:42:24.533 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:24 trial035 ceph-mon[39284]: from='osd.0 [v2:10.20.193.35:6802/2333513103,v1:10.20.193.35:6803/2333513103]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2026-02-24T07:42:24.533 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:24 trial035 ceph-mon[39284]: osdmap e7: 1 total, 0 up, 1 in 2026-02-24T07:42:24.533 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:24 trial035 ceph-mon[39284]: from='mgr.14154 10.20.193.35:0/1087484775' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2026-02-24T07:42:24.533 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:24 trial035 ceph-mon[39284]: from='osd.0 [v2:10.20.193.35:6802/2333513103,v1:10.20.193.35:6803/2333513103]' entity='osd.0' cmd={"prefix": "osd crush create-or-move", "id": 0, "weight":0.6836, "args": ["host=trial035", "root=default"]} : dispatch 2026-02-24T07:42:24.533 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:24 trial035 ceph-mon[39284]: from='mgr.14154 10.20.193.35:0/1087484775' entity='mgr.x' 2026-02-24T07:42:24.533 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:24 trial035 ceph-mon[39284]: from='mgr.14154 10.20.193.35:0/1087484775' entity='mgr.x' 2026-02-24T07:42:24.533 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:24 trial035 ceph-mon[39284]: from='mgr.14154 10.20.193.35:0/1087484775' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2026-02-24T07:42:24.533 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:24 trial035 ceph-mon[39284]: from='mgr.14154 10.20.193.35:0/1087484775' entity='mgr.x' 2026-02-24T07:42:24.533 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:24 trial035 ceph-mon[39284]: from='mgr.14154 10.20.193.35:0/1087484775' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-24T07:42:24.534 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:24 trial035 ceph-mon[39284]: from='mgr.14154 10.20.193.35:0/1087484775' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2026-02-24T07:42:24.534 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:24 trial035 ceph-mon[39284]: from='mgr.14154 10.20.193.35:0/1087484775' entity='mgr.x' 2026-02-24T07:42:24.534 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:24 trial035 ceph-mon[39284]: from='mgr.14154 10.20.193.35:0/1087484775' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2026-02-24T07:42:24.534 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:24 trial035 ceph-mon[39284]: from='mgr.14154 10.20.193.35:0/1087484775' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2026-02-24T07:42:24.534 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:24 trial035 ceph-mon[39284]: from='mgr.14154 10.20.193.35:0/1087484775' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-24T07:42:24.759 INFO:teuthology.orchestra.run.trial035.stdout: 2026-02-24T07:42:24.776 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph orch daemon add osd trial035:vg_nvme/lv_3 --skip-validation 2026-02-24T07:42:24.892 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:42:24.994 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:42:24.992+0000 7fdb670d7640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:42:24.994 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:42:24.992+0000 7fdb670d7640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:42:25.054 INFO:journalctl@ceph.osd.0.trial035.stdout:Feb 24 07:42:24 trial035 ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-osd-0[55839]: 2026-02-24T07:42:24.915+0000 7fd8d59c3640 -1 osd.0 0 waiting for initial osdmap 2026-02-24T07:42:25.054 INFO:journalctl@ceph.osd.0.trial035.stdout:Feb 24 07:42:24 trial035 ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-osd-0[55839]: 2026-02-24T07:42:24.917+0000 7fd8cd882640 -1 osd.0 8 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2026-02-24T07:42:26.161 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:25 trial035 ceph-mon[39284]: Detected new or changed devices on trial035 2026-02-24T07:42:26.161 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:25 trial035 ceph-mon[39284]: Adjusting osd_memory_target on trial035 to 84047M 2026-02-24T07:42:26.161 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:25 trial035 ceph-mon[39284]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2026-02-24T07:42:26.161 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:25 trial035 ceph-mon[39284]: from='osd.0 [v2:10.20.193.35:6802/2333513103,v1:10.20.193.35:6803/2333513103]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.6836, "args": ["host=trial035", "root=default"]}]': finished 2026-02-24T07:42:26.161 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:25 trial035 ceph-mon[39284]: osdmap e8: 1 total, 0 up, 1 in 2026-02-24T07:42:26.161 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:25 trial035 ceph-mon[39284]: from='mgr.14154 10.20.193.35:0/1087484775' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2026-02-24T07:42:26.161 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:25 trial035 ceph-mon[39284]: from='mgr.14154 10.20.193.35:0/1087484775' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2026-02-24T07:42:26.161 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:25 trial035 ceph-mon[39284]: from='mgr.14154 10.20.193.35:0/1087484775' entity='mgr.x' 2026-02-24T07:42:26.162 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:25 trial035 ceph-mon[39284]: from='mgr.14154 10.20.193.35:0/1087484775' entity='mgr.x' 2026-02-24T07:42:26.162 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:25 trial035 ceph-mon[39284]: from='mgr.14154 10.20.193.35:0/1087484775' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2026-02-24T07:42:26.162 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:25 trial035 ceph-mon[39284]: from='mgr.14154 10.20.193.35:0/1087484775' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2026-02-24T07:42:26.162 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:25 trial035 ceph-mon[39284]: from='mgr.14154 10.20.193.35:0/1087484775' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-24T07:42:27.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:26 trial035 ceph-mon[39284]: purged_snaps scrub starts 2026-02-24T07:42:27.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:26 trial035 ceph-mon[39284]: purged_snaps scrub ok 2026-02-24T07:42:27.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:26 trial035 ceph-mon[39284]: from='client.14210 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "trial035:vg_nvme/lv_3", "skip_validation": true, "target": ["mon-mgr", ""]}]: dispatch 2026-02-24T07:42:27.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:26 trial035 ceph-mon[39284]: osd.default does not exist. Creating it now. 2026-02-24T07:42:27.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:26 trial035 ceph-mon[39284]: Creating OSDs with service ID: default on trial035:['vg_nvme/lv_3'] 2026-02-24T07:42:27.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:26 trial035 ceph-mon[39284]: Marking host: trial035 for OSDSpec preview refresh. 2026-02-24T07:42:27.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:26 trial035 ceph-mon[39284]: Saving service osd.default spec with placement trial035 2026-02-24T07:42:27.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:26 trial035 ceph-mon[39284]: osd.0 [v2:10.20.193.35:6802/2333513103,v1:10.20.193.35:6803/2333513103] boot 2026-02-24T07:42:27.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:26 trial035 ceph-mon[39284]: osdmap e9: 1 total, 1 up, 1 in 2026-02-24T07:42:27.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:26 trial035 ceph-mon[39284]: from='mgr.14154 10.20.193.35:0/1087484775' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2026-02-24T07:42:27.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:26 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/3813908311' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "b9b77957-7aa8-49b2-95a7-b50dcfc6bc3e"} : dispatch 2026-02-24T07:42:27.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:26 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/3813908311' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b9b77957-7aa8-49b2-95a7-b50dcfc6bc3e"}]': finished 2026-02-24T07:42:27.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:26 trial035 ceph-mon[39284]: osdmap e10: 2 total, 1 up, 2 in 2026-02-24T07:42:27.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:26 trial035 ceph-mon[39284]: from='mgr.14154 10.20.193.35:0/1087484775' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2026-02-24T07:42:27.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:26 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/2216261793' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2026-02-24T07:42:27.997 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:27 trial035 ceph-mon[39284]: pgmap v12: 0 pgs: ; 0 B data, 26 MiB used, 700 GiB / 700 GiB avail 2026-02-24T07:42:27.997 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:27 trial035 ceph-mon[39284]: osdmap e11: 2 total, 1 up, 2 in 2026-02-24T07:42:27.997 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:27 trial035 ceph-mon[39284]: from='mgr.14154 10.20.193.35:0/1087484775' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2026-02-24T07:42:29.099 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:29 trial035 ceph-mon[39284]: from='mgr.14154 10.20.193.35:0/1087484775' entity='mgr.x' 2026-02-24T07:42:29.099 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:29 trial035 ceph-mon[39284]: from='mgr.14154 10.20.193.35:0/1087484775' entity='mgr.x' 2026-02-24T07:42:29.099 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:29 trial035 ceph-mon[39284]: from='mgr.14154 10.20.193.35:0/1087484775' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2026-02-24T07:42:29.099 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:29 trial035 ceph-mon[39284]: from='mgr.14154 10.20.193.35:0/1087484775' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.1"} : dispatch 2026-02-24T07:42:29.099 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:29 trial035 ceph-mon[39284]: from='mgr.14154 10.20.193.35:0/1087484775' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-24T07:42:29.099 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:29 trial035 ceph-mon[39284]: Deploying daemon osd.1 on trial035 2026-02-24T07:42:29.099 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:29 trial035 ceph-mon[39284]: from='mgr.14154 10.20.193.35:0/1087484775' entity='mgr.x' 2026-02-24T07:42:29.099 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:29 trial035 ceph-mon[39284]: from='mgr.14154 10.20.193.35:0/1087484775' entity='mgr.x' 2026-02-24T07:42:29.099 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:29 trial035 ceph-mon[39284]: pgmap v14: 0 pgs: ; 0 B data, 26 MiB used, 700 GiB / 700 GiB avail 2026-02-24T07:42:31.031 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:30 trial035 ceph-mon[39284]: from='mgr.14154 10.20.193.35:0/1087484775' entity='mgr.x' 2026-02-24T07:42:31.031 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:30 trial035 ceph-mon[39284]: from='mgr.14154 10.20.193.35:0/1087484775' entity='mgr.x' 2026-02-24T07:42:31.031 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:30 trial035 ceph-mon[39284]: from='osd.1 [v2:10.20.193.35:6810/4095255721,v1:10.20.193.35:6811/4095255721]' entity='osd.1' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]} : dispatch 2026-02-24T07:42:31.032 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:30 trial035 ceph-mon[39284]: pgmap v15: 0 pgs: ; 0 B data, 26 MiB used, 700 GiB / 700 GiB avail 2026-02-24T07:42:31.032 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:30 trial035 ceph-mon[39284]: Detected new or changed devices on trial035 2026-02-24T07:42:31.032 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:30 trial035 ceph-mon[39284]: from='mgr.14154 10.20.193.35:0/1087484775' entity='mgr.x' 2026-02-24T07:42:31.032 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:30 trial035 ceph-mon[39284]: from='mgr.14154 10.20.193.35:0/1087484775' entity='mgr.x' 2026-02-24T07:42:31.032 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:30 trial035 ceph-mon[39284]: from='mgr.14154 10.20.193.35:0/1087484775' entity='mgr.x' 2026-02-24T07:42:31.032 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:30 trial035 ceph-mon[39284]: from='mgr.14154 10.20.193.35:0/1087484775' entity='mgr.x' 2026-02-24T07:42:31.032 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:30 trial035 ceph-mon[39284]: from='mgr.14154 10.20.193.35:0/1087484775' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-24T07:42:31.032 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:30 trial035 ceph-mon[39284]: from='mgr.14154 10.20.193.35:0/1087484775' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2026-02-24T07:42:31.032 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:30 trial035 ceph-mon[39284]: from='mgr.14154 10.20.193.35:0/1087484775' entity='mgr.x' 2026-02-24T07:42:31.032 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:30 trial035 ceph-mon[39284]: from='mgr.14154 10.20.193.35:0/1087484775' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2026-02-24T07:42:31.032 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:30 trial035 ceph-mon[39284]: from='mgr.14154 10.20.193.35:0/1087484775' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2026-02-24T07:42:31.033 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:30 trial035 ceph-mon[39284]: from='mgr.14154 10.20.193.35:0/1087484775' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-24T07:42:31.342 INFO:teuthology.orchestra.run.trial035.stdout:Created osd(s) 1 on host 'trial035' 2026-02-24T07:42:31.381 DEBUG:teuthology.orchestra.run.trial035:osd.1> sudo journalctl -f -n 0 -u ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460@osd.1.service 2026-02-24T07:42:31.384 INFO:tasks.cephadm:Deploying osd.2 on trial091 with /dev/vg_nvme/lv_4... 2026-02-24T07:42:31.384 DEBUG:teuthology.orchestra.run.trial091:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- lvm zap /dev/vg_nvme/lv_4 2026-02-24T07:42:31.499 INFO:teuthology.orchestra.run.trial091.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/config/ceph.conf 2026-02-24T07:42:32.027 INFO:teuthology.orchestra.run.trial091.stdout: 2026-02-24T07:42:32.044 DEBUG:teuthology.orchestra.run.trial091:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph orch daemon add osd trial091:vg_nvme/lv_4 --skip-validation 2026-02-24T07:42:32.200 INFO:teuthology.orchestra.run.trial091.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/config/ceph.conf 2026-02-24T07:42:32.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:31 trial035 ceph-mon[39284]: from='osd.1 [v2:10.20.193.35:6810/4095255721,v1:10.20.193.35:6811/4095255721]' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2026-02-24T07:42:32.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:31 trial035 ceph-mon[39284]: osdmap e12: 2 total, 1 up, 2 in 2026-02-24T07:42:32.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:31 trial035 ceph-mon[39284]: from='mgr.14154 10.20.193.35:0/1087484775' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2026-02-24T07:42:32.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:31 trial035 ceph-mon[39284]: from='osd.1 [v2:10.20.193.35:6810/4095255721,v1:10.20.193.35:6811/4095255721]' entity='osd.1' cmd={"prefix": "osd crush create-or-move", "id": 1, "weight":0.6836, "args": ["host=trial035", "root=default"]} : dispatch 2026-02-24T07:42:32.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:31 trial035 ceph-mon[39284]: from='mgr.14154 10.20.193.35:0/1087484775' entity='mgr.x' 2026-02-24T07:42:32.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:31 trial035 ceph-mon[39284]: from='mgr.14154 10.20.193.35:0/1087484775' entity='mgr.x' 2026-02-24T07:42:32.205 INFO:journalctl@ceph.osd.1.trial035.stdout:Feb 24 07:42:31 trial035 ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-osd-1[62992]: 2026-02-24T07:42:31.995+0000 7f9e1c090640 -1 osd.1 0 waiting for initial osdmap 2026-02-24T07:42:32.205 INFO:journalctl@ceph.osd.1.trial035.stdout:Feb 24 07:42:31 trial035 ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-osd-1[62992]: 2026-02-24T07:42:31.997+0000 7f9e17f57640 -1 osd.1 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2026-02-24T07:42:32.316 INFO:teuthology.orchestra.run.trial091.stderr:2026-02-24T07:42:32.314+0000 7f84b79ed640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:42:32.317 INFO:teuthology.orchestra.run.trial091.stderr:2026-02-24T07:42:32.315+0000 7f84b79ed640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:42:33.348 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:32 trial035 ceph-mon[39284]: from='osd.1 [v2:10.20.193.35:6810/4095255721,v1:10.20.193.35:6811/4095255721]' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.6836, "args": ["host=trial035", "root=default"]}]': finished 2026-02-24T07:42:33.348 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:32 trial035 ceph-mon[39284]: osdmap e13: 2 total, 1 up, 2 in 2026-02-24T07:42:33.348 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:32 trial035 ceph-mon[39284]: from='mgr.14154 10.20.193.35:0/1087484775' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2026-02-24T07:42:33.348 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:32 trial035 ceph-mon[39284]: from='mgr.14154 10.20.193.35:0/1087484775' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2026-02-24T07:42:33.348 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:32 trial035 ceph-mon[39284]: from='client.14219 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "trial091:vg_nvme/lv_4", "skip_validation": true, "target": ["mon-mgr", ""]}]: dispatch 2026-02-24T07:42:33.348 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:32 trial035 ceph-mon[39284]: osd.default does not exist. Creating it now. 2026-02-24T07:42:33.348 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:32 trial035 ceph-mon[39284]: Creating OSDs with service ID: default on trial091:['vg_nvme/lv_4'] 2026-02-24T07:42:33.348 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:32 trial035 ceph-mon[39284]: from='mgr.14154 10.20.193.35:0/1087484775' entity='mgr.x' 2026-02-24T07:42:33.348 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:32 trial035 ceph-mon[39284]: Marking host: trial091 for OSDSpec preview refresh. 2026-02-24T07:42:33.348 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:32 trial035 ceph-mon[39284]: Saving service osd.default spec with placement trial091 2026-02-24T07:42:33.348 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:32 trial035 ceph-mon[39284]: from='mgr.14154 10.20.193.35:0/1087484775' entity='mgr.x' 2026-02-24T07:42:33.348 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:32 trial035 ceph-mon[39284]: from='mgr.14154 10.20.193.35:0/1087484775' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2026-02-24T07:42:33.348 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:32 trial035 ceph-mon[39284]: from='mgr.14154 10.20.193.35:0/1087484775' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2026-02-24T07:42:33.348 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:32 trial035 ceph-mon[39284]: from='mgr.14154 10.20.193.35:0/1087484775' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-24T07:42:33.348 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:32 trial035 ceph-mon[39284]: pgmap v18: 0 pgs: ; 0 B data, 26 MiB used, 700 GiB / 700 GiB avail 2026-02-24T07:42:33.348 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:32 trial035 ceph-mon[39284]: from='mgr.14154 10.20.193.35:0/1087484775' entity='mgr.x' 2026-02-24T07:42:33.348 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:32 trial035 ceph-mon[39284]: from='mgr.14154 10.20.193.35:0/1087484775' entity='mgr.x' 2026-02-24T07:42:33.348 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:32 trial035 ceph-mon[39284]: from='mgr.14154 10.20.193.35:0/1087484775' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2026-02-24T07:42:33.348 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:32 trial035 ceph-mon[39284]: from='mgr.14154 10.20.193.35:0/1087484775' entity='mgr.x' 2026-02-24T07:42:33.348 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:32 trial035 ceph-mon[39284]: from='mgr.14154 10.20.193.35:0/1087484775' entity='mgr.x' 2026-02-24T07:42:34.454 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:33 trial035 ceph-mon[39284]: purged_snaps scrub starts 2026-02-24T07:42:34.454 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:33 trial035 ceph-mon[39284]: purged_snaps scrub ok 2026-02-24T07:42:34.454 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:33 trial035 ceph-mon[39284]: osd.1 [v2:10.20.193.35:6810/4095255721,v1:10.20.193.35:6811/4095255721] boot 2026-02-24T07:42:34.454 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:33 trial035 ceph-mon[39284]: osdmap e14: 2 total, 2 up, 2 in 2026-02-24T07:42:34.454 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:33 trial035 ceph-mon[39284]: from='mgr.14154 10.20.193.35:0/1087484775' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2026-02-24T07:42:34.454 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:33 trial035 ceph-mon[39284]: from='client.? 10.20.193.91:0/152158877' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "883dcd38-0b52-4889-90fe-44a624ba23cd"} : dispatch 2026-02-24T07:42:34.454 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:33 trial035 ceph-mon[39284]: from='client.? 10.20.193.91:0/152158877' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "883dcd38-0b52-4889-90fe-44a624ba23cd"}]': finished 2026-02-24T07:42:34.454 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:33 trial035 ceph-mon[39284]: osdmap e15: 3 total, 2 up, 3 in 2026-02-24T07:42:34.454 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:33 trial035 ceph-mon[39284]: from='mgr.14154 10.20.193.35:0/1087484775' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2026-02-24T07:42:34.454 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:33 trial035 ceph-mon[39284]: from='mgr.14154 10.20.193.35:0/1087484775' entity='mgr.x' 2026-02-24T07:42:34.454 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:33 trial035 ceph-mon[39284]: from='mgr.14154 10.20.193.35:0/1087484775' entity='mgr.x' 2026-02-24T07:42:34.454 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:33 trial035 ceph-mon[39284]: from='client.? 10.20.193.91:0/531613752' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2026-02-24T07:42:35.454 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:35 trial035 ceph-mon[39284]: from='mgr.14154 10.20.193.35:0/1087484775' entity='mgr.x' 2026-02-24T07:42:35.454 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:35 trial035 ceph-mon[39284]: from='mgr.14154 10.20.193.35:0/1087484775' entity='mgr.x' 2026-02-24T07:42:35.455 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:35 trial035 ceph-mon[39284]: from='mgr.14154 10.20.193.35:0/1087484775' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2026-02-24T07:42:35.455 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:35 trial035 ceph-mon[39284]: from='mgr.14154 10.20.193.35:0/1087484775' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2026-02-24T07:42:35.455 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:35 trial035 ceph-mon[39284]: Adjusting osd_memory_target on trial035 to 42023M 2026-02-24T07:42:35.455 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:35 trial035 ceph-mon[39284]: from='mgr.14154 10.20.193.35:0/1087484775' entity='mgr.x' 2026-02-24T07:42:35.455 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:35 trial035 ceph-mon[39284]: from='mgr.14154 10.20.193.35:0/1087484775' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-24T07:42:35.455 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:35 trial035 ceph-mon[39284]: from='mgr.14154 10.20.193.35:0/1087484775' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2026-02-24T07:42:35.455 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:35 trial035 ceph-mon[39284]: from='mgr.14154 10.20.193.35:0/1087484775' entity='mgr.x' 2026-02-24T07:42:35.455 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:35 trial035 ceph-mon[39284]: from='mgr.14154 10.20.193.35:0/1087484775' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2026-02-24T07:42:35.455 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:35 trial035 ceph-mon[39284]: from='mgr.14154 10.20.193.35:0/1087484775' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2026-02-24T07:42:35.455 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:35 trial035 ceph-mon[39284]: from='mgr.14154 10.20.193.35:0/1087484775' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-24T07:42:35.455 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:35 trial035 ceph-mon[39284]: osdmap e16: 3 total, 2 up, 3 in 2026-02-24T07:42:35.456 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:35 trial035 ceph-mon[39284]: from='mgr.14154 10.20.193.35:0/1087484775' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2026-02-24T07:42:35.456 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:35 trial035 ceph-mon[39284]: pgmap v22: 0 pgs: ; 0 B data, 53 MiB used, 1.4 TiB / 1.4 TiB avail 2026-02-24T07:42:36.454 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:36 trial035 ceph-mon[39284]: from='mgr.14154 10.20.193.35:0/1087484775' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.2"} : dispatch 2026-02-24T07:42:36.454 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:36 trial035 ceph-mon[39284]: from='mgr.14154 10.20.193.35:0/1087484775' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-24T07:42:36.454 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:36 trial035 ceph-mon[39284]: Deploying daemon osd.2 on trial091 2026-02-24T07:42:36.455 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:36 trial035 ceph-mon[39284]: from='mgr.14154 10.20.193.35:0/1087484775' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.2"} : dispatch 2026-02-24T07:42:36.455 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:36 trial035 ceph-mon[39284]: from='mgr.14154 10.20.193.35:0/1087484775' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-24T07:42:36.455 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:36 trial035 ceph-mon[39284]: Deploying daemon osd.2 on trial091 2026-02-24T07:42:37.454 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:37 trial035 ceph-mon[39284]: pgmap v23: 0 pgs: ; 0 B data, 53 MiB used, 1.4 TiB / 1.4 TiB avail 2026-02-24T07:42:38.704 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:38 trial035 ceph-mon[39284]: from='mgr.14154 10.20.193.35:0/1087484775' entity='mgr.x' 2026-02-24T07:42:38.704 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:38 trial035 ceph-mon[39284]: from='mgr.14154 10.20.193.35:0/1087484775' entity='mgr.x' 2026-02-24T07:42:39.640 INFO:teuthology.orchestra.run.trial091.stdout:Created osd(s) 2 on host 'trial091' 2026-02-24T07:42:39.681 DEBUG:teuthology.orchestra.run.trial091:osd.2> sudo journalctl -f -n 0 -u ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460@osd.2.service 2026-02-24T07:42:39.683 INFO:tasks.cephadm:Deploying osd.3 on trial091 with /dev/vg_nvme/lv_3... 2026-02-24T07:42:39.683 DEBUG:teuthology.orchestra.run.trial091:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- lvm zap /dev/vg_nvme/lv_3 2026-02-24T07:42:39.704 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:39 trial035 ceph-mon[39284]: pgmap v24: 0 pgs: ; 0 B data, 53 MiB used, 1.4 TiB / 1.4 TiB avail 2026-02-24T07:42:39.704 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:39 trial035 ceph-mon[39284]: from='mgr.14154 10.20.193.35:0/1087484775' entity='mgr.x' 2026-02-24T07:42:39.704 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:39 trial035 ceph-mon[39284]: from='mgr.14154 10.20.193.35:0/1087484775' entity='mgr.x' 2026-02-24T07:42:39.845 INFO:teuthology.orchestra.run.trial091.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/config/ceph.conf 2026-02-24T07:42:39.856 INFO:journalctl@ceph.osd.2.trial091.stdout:Feb 24 07:42:39 trial091 ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-osd-2[46763]: 2026-02-24T07:42:39.746+0000 7f698e600780 -1 osd.2 0 log_to_monitors true 2026-02-24T07:42:40.566 INFO:teuthology.orchestra.run.trial091.stdout: 2026-02-24T07:42:40.583 DEBUG:teuthology.orchestra.run.trial091:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph orch daemon add osd trial091:vg_nvme/lv_3 --skip-validation 2026-02-24T07:42:40.700 INFO:teuthology.orchestra.run.trial091.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/config/ceph.conf 2026-02-24T07:42:40.808 INFO:teuthology.orchestra.run.trial091.stderr:2026-02-24T07:42:40.806+0000 7f1d4aa04640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:42:40.809 INFO:teuthology.orchestra.run.trial091.stderr:2026-02-24T07:42:40.807+0000 7f1d4aa04640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:42:40.954 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:40 trial035 ceph-mon[39284]: from='mgr.14154 10.20.193.35:0/1087484775' entity='mgr.x' 2026-02-24T07:42:40.954 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:40 trial035 ceph-mon[39284]: from='mgr.14154 10.20.193.35:0/1087484775' entity='mgr.x' 2026-02-24T07:42:40.955 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:40 trial035 ceph-mon[39284]: from='osd.2 [v2:10.20.193.91:6800/1390012919,v1:10.20.193.91:6801/1390012919]' entity='osd.2' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]} : dispatch 2026-02-24T07:42:40.955 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:40 trial035 ceph-mon[39284]: from='mgr.14154 10.20.193.35:0/1087484775' entity='mgr.x' 2026-02-24T07:42:40.955 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:40 trial035 ceph-mon[39284]: from='mgr.14154 10.20.193.35:0/1087484775' entity='mgr.x' 2026-02-24T07:42:40.955 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:40 trial035 ceph-mon[39284]: from='mgr.14154 10.20.193.35:0/1087484775' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2026-02-24T07:42:40.955 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:40 trial035 ceph-mon[39284]: from='mgr.14154 10.20.193.35:0/1087484775' entity='mgr.x' 2026-02-24T07:42:40.955 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:40 trial035 ceph-mon[39284]: from='mgr.14154 10.20.193.35:0/1087484775' entity='mgr.x' 2026-02-24T07:42:41.715 INFO:journalctl@ceph.osd.2.trial091.stdout:Feb 24 07:42:41 trial091 ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-osd-2[46763]: 2026-02-24T07:42:41.641+0000 7f698a582640 -1 osd.2 0 waiting for initial osdmap 2026-02-24T07:42:41.715 INFO:journalctl@ceph.osd.2.trial091.stdout:Feb 24 07:42:41 trial091 ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-osd-2[46763]: 2026-02-24T07:42:41.643+0000 7f6986449640 -1 osd.2 18 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2026-02-24T07:42:41.954 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:41 trial035 ceph-mon[39284]: from='osd.2 [v2:10.20.193.91:6800/1390012919,v1:10.20.193.91:6801/1390012919]' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2026-02-24T07:42:41.955 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:41 trial035 ceph-mon[39284]: osdmap e17: 3 total, 2 up, 3 in 2026-02-24T07:42:41.955 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:41 trial035 ceph-mon[39284]: from='mgr.14154 10.20.193.35:0/1087484775' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2026-02-24T07:42:41.955 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:41 trial035 ceph-mon[39284]: from='osd.2 [v2:10.20.193.91:6800/1390012919,v1:10.20.193.91:6801/1390012919]' entity='osd.2' cmd={"prefix": "osd crush create-or-move", "id": 2, "weight":0.6836, "args": ["host=trial091", "root=default"]} : dispatch 2026-02-24T07:42:41.955 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:41 trial035 ceph-mon[39284]: pgmap v26: 0 pgs: ; 0 B data, 53 MiB used, 1.4 TiB / 1.4 TiB avail 2026-02-24T07:42:41.955 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:41 trial035 ceph-mon[39284]: from='client.14229 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "trial091:vg_nvme/lv_3", "skip_validation": true, "target": ["mon-mgr", ""]}]: dispatch 2026-02-24T07:42:41.955 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:41 trial035 ceph-mon[39284]: osd.default does not exist. Creating it now. 2026-02-24T07:42:41.955 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:41 trial035 ceph-mon[39284]: Creating OSDs with service ID: default on trial091:['vg_nvme/lv_3'] 2026-02-24T07:42:41.955 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:41 trial035 ceph-mon[39284]: from='mgr.14154 10.20.193.35:0/1087484775' entity='mgr.x' 2026-02-24T07:42:41.956 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:41 trial035 ceph-mon[39284]: Marking host: trial091 for OSDSpec preview refresh. 2026-02-24T07:42:41.956 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:41 trial035 ceph-mon[39284]: Saving service osd.default spec with placement trial091 2026-02-24T07:42:41.956 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:41 trial035 ceph-mon[39284]: from='mgr.14154 10.20.193.35:0/1087484775' entity='mgr.x' 2026-02-24T07:42:41.956 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:41 trial035 ceph-mon[39284]: from='mgr.14154 10.20.193.35:0/1087484775' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2026-02-24T07:42:41.956 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:41 trial035 ceph-mon[39284]: from='mgr.14154 10.20.193.35:0/1087484775' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2026-02-24T07:42:41.956 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:41 trial035 ceph-mon[39284]: from='mgr.14154 10.20.193.35:0/1087484775' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-24T07:42:41.956 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:41 trial035 ceph-mon[39284]: from='mgr.14154 10.20.193.35:0/1087484775' entity='mgr.x' 2026-02-24T07:42:41.956 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:41 trial035 ceph-mon[39284]: from='mgr.14154 10.20.193.35:0/1087484775' entity='mgr.x' 2026-02-24T07:42:41.956 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:41 trial035 ceph-mon[39284]: from='mgr.14154 10.20.193.35:0/1087484775' entity='mgr.x' 2026-02-24T07:42:41.956 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:41 trial035 ceph-mon[39284]: from='mgr.14154 10.20.193.35:0/1087484775' entity='mgr.x' 2026-02-24T07:42:41.957 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:41 trial035 ceph-mon[39284]: from='mgr.14154 10.20.193.35:0/1087484775' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2026-02-24T07:42:41.957 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:41 trial035 ceph-mon[39284]: from='mgr.14154 10.20.193.35:0/1087484775' entity='mgr.x' 2026-02-24T07:42:41.957 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:41 trial035 ceph-mon[39284]: from='mgr.14154 10.20.193.35:0/1087484775' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-24T07:42:41.957 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:41 trial035 ceph-mon[39284]: from='mgr.14154 10.20.193.35:0/1087484775' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2026-02-24T07:42:41.957 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:41 trial035 ceph-mon[39284]: from='mgr.14154 10.20.193.35:0/1087484775' entity='mgr.x' 2026-02-24T07:42:41.957 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:41 trial035 ceph-mon[39284]: from='mgr.14154 10.20.193.35:0/1087484775' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2026-02-24T07:42:41.957 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:41 trial035 ceph-mon[39284]: from='mgr.14154 10.20.193.35:0/1087484775' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2026-02-24T07:42:41.957 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:41 trial035 ceph-mon[39284]: from='mgr.14154 10.20.193.35:0/1087484775' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-24T07:42:42.954 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:42 trial035 ceph-mon[39284]: Detected new or changed devices on trial091 2026-02-24T07:42:42.954 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:42 trial035 ceph-mon[39284]: Adjusting osd_memory_target on trial091 to 89167M 2026-02-24T07:42:42.955 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:42 trial035 ceph-mon[39284]: from='osd.2 [v2:10.20.193.91:6800/1390012919,v1:10.20.193.91:6801/1390012919]' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.6836, "args": ["host=trial091", "root=default"]}]': finished 2026-02-24T07:42:42.955 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:42 trial035 ceph-mon[39284]: osdmap e18: 3 total, 2 up, 3 in 2026-02-24T07:42:42.955 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:42 trial035 ceph-mon[39284]: from='mgr.14154 10.20.193.35:0/1087484775' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2026-02-24T07:42:42.955 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:42 trial035 ceph-mon[39284]: from='mgr.14154 10.20.193.35:0/1087484775' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2026-02-24T07:42:42.955 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:42 trial035 ceph-mon[39284]: from='client.? 10.20.193.91:0/1678709436' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "2582e7bb-0265-44e6-bf18-c68070be0dc7"} : dispatch 2026-02-24T07:42:42.955 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:42 trial035 ceph-mon[39284]: from='client.? 10.20.193.91:0/1678709436' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2582e7bb-0265-44e6-bf18-c68070be0dc7"}]': finished 2026-02-24T07:42:42.955 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:42 trial035 ceph-mon[39284]: osd.2 [v2:10.20.193.91:6800/1390012919,v1:10.20.193.91:6801/1390012919] boot 2026-02-24T07:42:42.955 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:42 trial035 ceph-mon[39284]: osdmap e19: 4 total, 3 up, 4 in 2026-02-24T07:42:42.955 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:42 trial035 ceph-mon[39284]: from='mgr.14154 10.20.193.35:0/1087484775' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2026-02-24T07:42:42.955 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:42 trial035 ceph-mon[39284]: from='mgr.14154 10.20.193.35:0/1087484775' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2026-02-24T07:42:42.956 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:42 trial035 ceph-mon[39284]: from='client.? 10.20.193.91:0/1876352033' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2026-02-24T07:42:43.954 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:43 trial035 ceph-mon[39284]: purged_snaps scrub starts 2026-02-24T07:42:43.954 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:43 trial035 ceph-mon[39284]: purged_snaps scrub ok 2026-02-24T07:42:43.954 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:43 trial035 ceph-mon[39284]: pgmap v29: 0 pgs: ; 0 B data, 79 MiB used, 2.1 TiB / 2.1 TiB avail 2026-02-24T07:42:43.955 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:43 trial035 ceph-mon[39284]: from='mgr.14154 10.20.193.35:0/1087484775' entity='mgr.x' cmd={"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true} : dispatch 2026-02-24T07:42:44.954 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:44 trial035 ceph-mon[39284]: from='mgr.14154 10.20.193.35:0/1087484775' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2026-02-24T07:42:44.954 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:44 trial035 ceph-mon[39284]: osdmap e20: 4 total, 3 up, 4 in 2026-02-24T07:42:44.954 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:44 trial035 ceph-mon[39284]: from='mgr.14154 10.20.193.35:0/1087484775' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2026-02-24T07:42:44.954 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:44 trial035 ceph-mon[39284]: from='mgr.14154 10.20.193.35:0/1087484775' entity='mgr.x' cmd={"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true} : dispatch 2026-02-24T07:42:44.954 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:44 trial035 ceph-mon[39284]: from='mgr.14154 10.20.193.35:0/1087484775' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.3"} : dispatch 2026-02-24T07:42:44.954 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:44 trial035 ceph-mon[39284]: from='mgr.14154 10.20.193.35:0/1087484775' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-24T07:42:44.954 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:44 trial035 ceph-mon[39284]: from='mgr.14154 10.20.193.35:0/1087484775' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.3"} : dispatch 2026-02-24T07:42:44.954 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:44 trial035 ceph-mon[39284]: from='mgr.14154 10.20.193.35:0/1087484775' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-24T07:42:45.357 INFO:journalctl@ceph.osd.2.trial091.stdout:Feb 24 07:42:45 trial091 sudo[53130]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme2n1 2026-02-24T07:42:45.357 INFO:journalctl@ceph.osd.2.trial091.stdout:Feb 24 07:42:45 trial091 sudo[53130]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2026-02-24T07:42:45.357 INFO:journalctl@ceph.osd.2.trial091.stdout:Feb 24 07:42:45 trial091 sudo[53130]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2026-02-24T07:42:45.357 INFO:journalctl@ceph.osd.2.trial091.stdout:Feb 24 07:42:45 trial091 sudo[53130]: pam_unix(sudo:session): session closed for user root 2026-02-24T07:42:45.357 INFO:journalctl@ceph.osd.2.trial091.stdout:Feb 24 07:42:45 trial091 sudo[53138]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme mtfdkcc1t9tgp-1bk1jabyy smart-log-add --json /dev/nvme2n1 2026-02-24T07:42:45.357 INFO:journalctl@ceph.osd.2.trial091.stdout:Feb 24 07:42:45 trial091 sudo[53138]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2026-02-24T07:42:45.357 INFO:journalctl@ceph.osd.2.trial091.stdout:Feb 24 07:42:45 trial091 sudo[53138]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2026-02-24T07:42:45.358 INFO:journalctl@ceph.osd.2.trial091.stdout:Feb 24 07:42:45 trial091 sudo[53138]: pam_unix(sudo:session): session closed for user root 2026-02-24T07:42:45.454 INFO:journalctl@ceph.osd.0.trial035.stdout:Feb 24 07:42:45 trial035 sudo[71400]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme2n1 2026-02-24T07:42:45.454 INFO:journalctl@ceph.osd.0.trial035.stdout:Feb 24 07:42:45 trial035 sudo[71400]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2026-02-24T07:42:45.454 INFO:journalctl@ceph.osd.0.trial035.stdout:Feb 24 07:42:45 trial035 sudo[71400]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2026-02-24T07:42:45.454 INFO:journalctl@ceph.osd.0.trial035.stdout:Feb 24 07:42:45 trial035 sudo[71400]: pam_unix(sudo:session): session closed for user root 2026-02-24T07:42:45.454 INFO:journalctl@ceph.osd.0.trial035.stdout:Feb 24 07:42:45 trial035 sudo[71404]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme mtfdkcc1t9tgp-1bk1jabyy smart-log-add --json /dev/nvme2n1 2026-02-24T07:42:45.454 INFO:journalctl@ceph.osd.0.trial035.stdout:Feb 24 07:42:45 trial035 sudo[71404]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2026-02-24T07:42:45.454 INFO:journalctl@ceph.osd.0.trial035.stdout:Feb 24 07:42:45 trial035 sudo[71404]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2026-02-24T07:42:45.454 INFO:journalctl@ceph.osd.0.trial035.stdout:Feb 24 07:42:45 trial035 sudo[71404]: pam_unix(sudo:session): session closed for user root 2026-02-24T07:42:45.454 INFO:journalctl@ceph.osd.1.trial035.stdout:Feb 24 07:42:45 trial035 sudo[71408]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme2n1 2026-02-24T07:42:45.454 INFO:journalctl@ceph.osd.1.trial035.stdout:Feb 24 07:42:45 trial035 sudo[71408]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2026-02-24T07:42:45.455 INFO:journalctl@ceph.osd.1.trial035.stdout:Feb 24 07:42:45 trial035 sudo[71408]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2026-02-24T07:42:45.455 INFO:journalctl@ceph.osd.1.trial035.stdout:Feb 24 07:42:45 trial035 sudo[71408]: pam_unix(sudo:session): session closed for user root 2026-02-24T07:42:45.455 INFO:journalctl@ceph.osd.1.trial035.stdout:Feb 24 07:42:45 trial035 sudo[71412]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme mtfdkcc1t9tgp-1bk1jabyy smart-log-add --json /dev/nvme2n1 2026-02-24T07:42:45.455 INFO:journalctl@ceph.osd.1.trial035.stdout:Feb 24 07:42:45 trial035 sudo[71412]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2026-02-24T07:42:45.455 INFO:journalctl@ceph.osd.1.trial035.stdout:Feb 24 07:42:45 trial035 sudo[71412]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2026-02-24T07:42:45.455 INFO:journalctl@ceph.osd.1.trial035.stdout:Feb 24 07:42:45 trial035 sudo[71412]: pam_unix(sudo:session): session closed for user root 2026-02-24T07:42:45.455 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:45 trial035 sudo[71416]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2026-02-24T07:42:45.455 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:45 trial035 sudo[71416]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2026-02-24T07:42:45.455 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:45 trial035 sudo[71416]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2026-02-24T07:42:45.455 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:45 trial035 sudo[71416]: pam_unix(sudo:session): session closed for user root 2026-02-24T07:42:45.455 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:45 trial035 sudo[71420]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme mtfdkcc1t9tgp-1bk1jabyy smart-log-add --json /dev/nvme0n1 2026-02-24T07:42:45.455 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:45 trial035 sudo[71420]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2026-02-24T07:42:45.455 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:45 trial035 sudo[71420]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2026-02-24T07:42:45.455 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:45 trial035 sudo[71420]: pam_unix(sudo:session): session closed for user root 2026-02-24T07:42:45.954 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:45 trial035 ceph-mon[39284]: Deploying daemon osd.3 on trial091 2026-02-24T07:42:45.955 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:45 trial035 ceph-mon[39284]: Deploying daemon osd.3 on trial091 2026-02-24T07:42:45.955 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:45 trial035 ceph-mon[39284]: from='mgr.14154 10.20.193.35:0/1087484775' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2026-02-24T07:42:45.955 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:45 trial035 ceph-mon[39284]: osdmap e21: 4 total, 3 up, 4 in 2026-02-24T07:42:45.955 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:45 trial035 ceph-mon[39284]: from='mgr.14154 10.20.193.35:0/1087484775' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2026-02-24T07:42:45.955 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:45 trial035 ceph-mon[39284]: pgmap v32: 1 pgs: 1 unknown; 0 B data, 79 MiB used, 2.1 TiB / 2.1 TiB avail 2026-02-24T07:42:45.955 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:45 trial035 ceph-mon[39284]: from='mgr.14154 10.20.193.35:0/1087484775' entity='mgr.x' 2026-02-24T07:42:45.955 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:45 trial035 ceph-mon[39284]: from='mgr.14154 10.20.193.35:0/1087484775' entity='mgr.x' 2026-02-24T07:42:45.955 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:45 trial035 ceph-mon[39284]: from='mgr.14154 10.20.193.35:0/1087484775' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2026-02-24T07:42:45.955 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:45 trial035 ceph-mon[39284]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2026-02-24T07:42:45.955 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:45 trial035 ceph-mon[39284]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2026-02-24T07:42:45.956 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:45 trial035 ceph-mon[39284]: from='mgr.14154 10.20.193.35:0/1087484775' entity='mgr.x' 2026-02-24T07:42:46.954 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:46 trial035 ceph-mon[39284]: osdmap e22: 4 total, 3 up, 4 in 2026-02-24T07:42:46.954 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:46 trial035 ceph-mon[39284]: from='mgr.14154 10.20.193.35:0/1087484775' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2026-02-24T07:42:46.954 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:46 trial035 ceph-mon[39284]: from='mgr.14154 10.20.193.35:0/1087484775' entity='mgr.x' 2026-02-24T07:42:46.954 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:46 trial035 ceph-mon[39284]: from='mgr.14154 10.20.193.35:0/1087484775' entity='mgr.x' 2026-02-24T07:42:47.954 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:47 trial035 ceph-mon[39284]: pgmap v34: 1 pgs: 1 unknown; 0 B data, 79 MiB used, 2.1 TiB / 2.1 TiB avail 2026-02-24T07:42:48.181 INFO:teuthology.orchestra.run.trial091.stdout:Created osd(s) 3 on host 'trial091' 2026-02-24T07:42:48.224 DEBUG:teuthology.orchestra.run.trial091:osd.3> sudo journalctl -f -n 0 -u ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460@osd.3.service 2026-02-24T07:42:48.226 INFO:tasks.cephadm:Waiting for 4 OSDs to come up... 2026-02-24T07:42:48.226 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph osd stat -f json 2026-02-24T07:42:48.345 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:42:48.449 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:42:48.446+0000 7f6f1d128640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:42:48.449 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:42:48.447+0000 7f6f1d128640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:42:48.556 INFO:teuthology.orchestra.run.trial035.stdout: 2026-02-24T07:42:48.606 INFO:journalctl@ceph.osd.3.trial091.stdout:Feb 24 07:42:48 trial091 ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-osd-3[54462]: 2026-02-24T07:42:48.338+0000 7f4e037c4780 -1 osd.3 0 log_to_monitors true 2026-02-24T07:42:48.631 INFO:teuthology.orchestra.run.trial035.stdout:{"epoch":22,"num_osds":4,"num_up_osds":3,"osd_up_since":1771918962,"num_in_osds":4,"osd_in_since":1771918962,"num_remapped_pgs":0} 2026-02-24T07:42:48.954 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:48 trial035 ceph-mon[39284]: mgrmap e14: x(active, since 44s) 2026-02-24T07:42:48.954 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:48 trial035 ceph-mon[39284]: from='mgr.14154 10.20.193.35:0/1087484775' entity='mgr.x' 2026-02-24T07:42:48.955 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:48 trial035 ceph-mon[39284]: from='mgr.14154 10.20.193.35:0/1087484775' entity='mgr.x' 2026-02-24T07:42:48.955 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:48 trial035 ceph-mon[39284]: from='mgr.14154 10.20.193.35:0/1087484775' entity='mgr.x' 2026-02-24T07:42:48.955 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:48 trial035 ceph-mon[39284]: from='mgr.14154 10.20.193.35:0/1087484775' entity='mgr.x' 2026-02-24T07:42:48.955 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:48 trial035 ceph-mon[39284]: from='osd.3 [v2:10.20.193.91:6808/940596907,v1:10.20.193.91:6809/940596907]' entity='osd.3' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]} : dispatch 2026-02-24T07:42:48.955 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:48 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/3151595940' entity='client.admin' cmd={"prefix": "osd stat", "format": "json"} : dispatch 2026-02-24T07:42:49.632 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph osd stat -f json 2026-02-24T07:42:49.751 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:42:49.854 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:42:49.852+0000 7f63673e8640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:42:49.854 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:42:49.853+0000 7f63673e8640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:42:49.954 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:49 trial035 ceph-mon[39284]: from='mgr.14154 10.20.193.35:0/1087484775' entity='mgr.x' 2026-02-24T07:42:49.954 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:49 trial035 ceph-mon[39284]: from='mgr.14154 10.20.193.35:0/1087484775' entity='mgr.x' 2026-02-24T07:42:49.955 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:49 trial035 ceph-mon[39284]: pgmap v35: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 2.1 TiB / 2.1 TiB avail 2026-02-24T07:42:49.955 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:49 trial035 ceph-mon[39284]: from='mgr.14154 10.20.193.35:0/1087484775' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2026-02-24T07:42:49.955 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:49 trial035 ceph-mon[39284]: from='mgr.14154 10.20.193.35:0/1087484775' entity='mgr.x' 2026-02-24T07:42:49.955 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:49 trial035 ceph-mon[39284]: from='mgr.14154 10.20.193.35:0/1087484775' entity='mgr.x' 2026-02-24T07:42:49.955 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:49 trial035 ceph-mon[39284]: from='osd.3 [v2:10.20.193.91:6808/940596907,v1:10.20.193.91:6809/940596907]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2026-02-24T07:42:49.955 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:49 trial035 ceph-mon[39284]: osdmap e23: 4 total, 3 up, 4 in 2026-02-24T07:42:49.955 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:49 trial035 ceph-mon[39284]: from='mgr.14154 10.20.193.35:0/1087484775' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2026-02-24T07:42:49.955 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:49 trial035 ceph-mon[39284]: from='osd.3 [v2:10.20.193.91:6808/940596907,v1:10.20.193.91:6809/940596907]' entity='osd.3' cmd={"prefix": "osd crush create-or-move", "id": 3, "weight":0.6836, "args": ["host=trial091", "root=default"]} : dispatch 2026-02-24T07:42:49.964 INFO:teuthology.orchestra.run.trial035.stdout: 2026-02-24T07:42:50.002 INFO:teuthology.orchestra.run.trial035.stdout:{"epoch":23,"num_osds":4,"num_up_osds":3,"osd_up_since":1771918962,"num_in_osds":4,"osd_in_since":1771918962,"num_remapped_pgs":0} 2026-02-24T07:42:50.198 INFO:journalctl@ceph.osd.3.trial091.stdout:Feb 24 07:42:50 trial091 ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-osd-3[54462]: 2026-02-24T07:42:50.183+0000 7f4dff746640 -1 osd.3 0 waiting for initial osdmap 2026-02-24T07:42:50.199 INFO:journalctl@ceph.osd.3.trial091.stdout:Feb 24 07:42:50 trial091 ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-osd-3[54462]: 2026-02-24T07:42:50.186+0000 7f4df7605640 -1 osd.3 24 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2026-02-24T07:42:50.954 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:50 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/3978943608' entity='client.admin' cmd={"prefix": "osd stat", "format": "json"} : dispatch 2026-02-24T07:42:50.955 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:50 trial035 ceph-mon[39284]: from='mgr.14154 10.20.193.35:0/1087484775' entity='mgr.x' 2026-02-24T07:42:50.955 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:50 trial035 ceph-mon[39284]: from='mgr.14154 10.20.193.35:0/1087484775' entity='mgr.x' 2026-02-24T07:42:50.955 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:50 trial035 ceph-mon[39284]: from='mgr.14154 10.20.193.35:0/1087484775' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2026-02-24T07:42:50.955 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:50 trial035 ceph-mon[39284]: from='mgr.14154 10.20.193.35:0/1087484775' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2026-02-24T07:42:50.955 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:50 trial035 ceph-mon[39284]: from='mgr.14154 10.20.193.35:0/1087484775' entity='mgr.x' 2026-02-24T07:42:50.955 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:50 trial035 ceph-mon[39284]: from='mgr.14154 10.20.193.35:0/1087484775' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-24T07:42:50.955 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:50 trial035 ceph-mon[39284]: from='mgr.14154 10.20.193.35:0/1087484775' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2026-02-24T07:42:50.955 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:50 trial035 ceph-mon[39284]: from='mgr.14154 10.20.193.35:0/1087484775' entity='mgr.x' 2026-02-24T07:42:50.955 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:50 trial035 ceph-mon[39284]: from='mgr.14154 10.20.193.35:0/1087484775' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2026-02-24T07:42:50.956 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:50 trial035 ceph-mon[39284]: from='mgr.14154 10.20.193.35:0/1087484775' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2026-02-24T07:42:50.956 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:50 trial035 ceph-mon[39284]: from='mgr.14154 10.20.193.35:0/1087484775' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-24T07:42:50.956 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:50 trial035 ceph-mon[39284]: from='osd.3 [v2:10.20.193.91:6808/940596907,v1:10.20.193.91:6809/940596907]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.6836, "args": ["host=trial091", "root=default"]}]': finished 2026-02-24T07:42:50.956 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:50 trial035 ceph-mon[39284]: osdmap e24: 4 total, 3 up, 4 in 2026-02-24T07:42:50.956 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:50 trial035 ceph-mon[39284]: from='mgr.14154 10.20.193.35:0/1087484775' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2026-02-24T07:42:50.956 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:50 trial035 ceph-mon[39284]: from='mgr.14154 10.20.193.35:0/1087484775' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2026-02-24T07:42:51.004 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph osd stat -f json 2026-02-24T07:42:51.120 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:42:51.225 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:42:51.223+0000 7fab85de6640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:42:51.225 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:42:51.223+0000 7fab85de6640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:42:51.333 INFO:teuthology.orchestra.run.trial035.stdout: 2026-02-24T07:42:51.371 INFO:teuthology.orchestra.run.trial035.stdout:{"epoch":25,"num_osds":4,"num_up_osds":4,"osd_up_since":1771918971,"num_in_osds":4,"osd_in_since":1771918962,"num_remapped_pgs":1} 2026-02-24T07:42:51.372 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph osd dump --format=json 2026-02-24T07:42:51.489 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:42:51.582 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:42:51.579+0000 7f5da3c37640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:42:51.582 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:42:51.580+0000 7f5da3c37640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:42:51.694 INFO:teuthology.orchestra.run.trial035.stdout: 2026-02-24T07:42:51.694 INFO:teuthology.orchestra.run.trial035.stdout:{"epoch":25,"fsid":"3a5983d2-1154-11f1-9c05-d404e6e7d460","created":"2026-02-24T07:41:42.805703+0000","modified":"2026-02-24T07:42:51.180844+0000","last_up_change":"2026-02-24T07:42:51.180844+0000","last_in_change":"2026-02-24T07:42:42.005822+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":10,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":4,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"tentacle","allow_crimson":true,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2026-02-24T07:42:42.898873+0000","flags":262145,"flags_names":"hashpspool,crimson","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"nonprimary_shards":"{}","options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":4,"score_stable":4,"optimal_score":0.75,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"8d5b0adf-0e02-40d4-8780-e64988c10e92","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.35:6802","nonce":2333513103},{"type":"v1","addr":"10.20.193.35:6803","nonce":2333513103}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.35:6804","nonce":2333513103},{"type":"v1","addr":"10.20.193.35:6805","nonce":2333513103}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.35:6808","nonce":2333513103},{"type":"v1","addr":"10.20.193.35:6809","nonce":2333513103}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.35:6806","nonce":2333513103},{"type":"v1","addr":"10.20.193.35:6807","nonce":2333513103}]},"public_addr":"10.20.193.35:6803/2333513103","cluster_addr":"10.20.193.35:6805/2333513103","heartbeat_back_addr":"10.20.193.35:6809/2333513103","heartbeat_front_addr":"10.20.193.35:6807/2333513103","state":["exists","up"]},{"osd":1,"uuid":"b9b77957-7aa8-49b2-95a7-b50dcfc6bc3e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.35:6810","nonce":4095255721},{"type":"v1","addr":"10.20.193.35:6811","nonce":4095255721}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.35:6812","nonce":4095255721},{"type":"v1","addr":"10.20.193.35:6813","nonce":4095255721}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.35:6816","nonce":4095255721},{"type":"v1","addr":"10.20.193.35:6817","nonce":4095255721}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.35:6814","nonce":4095255721},{"type":"v1","addr":"10.20.193.35:6815","nonce":4095255721}]},"public_addr":"10.20.193.35:6811/4095255721","cluster_addr":"10.20.193.35:6813/4095255721","heartbeat_back_addr":"10.20.193.35:6817/4095255721","heartbeat_front_addr":"10.20.193.35:6815/4095255721","state":["exists","up"]},{"osd":2,"uuid":"883dcd38-0b52-4889-90fe-44a624ba23cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.91:6800","nonce":1390012919},{"type":"v1","addr":"10.20.193.91:6801","nonce":1390012919}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.91:6802","nonce":1390012919},{"type":"v1","addr":"10.20.193.91:6803","nonce":1390012919}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.91:6806","nonce":1390012919},{"type":"v1","addr":"10.20.193.91:6807","nonce":1390012919}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.91:6804","nonce":1390012919},{"type":"v1","addr":"10.20.193.91:6805","nonce":1390012919}]},"public_addr":"10.20.193.91:6801/1390012919","cluster_addr":"10.20.193.91:6803/1390012919","heartbeat_back_addr":"10.20.193.91:6807/1390012919","heartbeat_front_addr":"10.20.193.91:6805/1390012919","state":["exists","up"]},{"osd":3,"uuid":"2582e7bb-0265-44e6-bf18-c68070be0dc7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.91:6808","nonce":940596907},{"type":"v1","addr":"10.20.193.91:6809","nonce":940596907}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.91:6810","nonce":940596907},{"type":"v1","addr":"10.20.193.91:6811","nonce":940596907}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.91:6814","nonce":940596907},{"type":"v1","addr":"10.20.193.91:6815","nonce":940596907}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.91:6812","nonce":940596907},{"type":"v1","addr":"10.20.193.91:6813","nonce":940596907}]},"public_addr":"10.20.193.91:6809/940596907","cluster_addr":"10.20.193.91:6811/940596907","heartbeat_back_addr":"10.20.193.91:6815/940596907","heartbeat_front_addr":"10.20.193.91:6813/940596907","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-24T07:42:24.073399+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-24T07:42:31.659670+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-24T07:42:40.760086+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[{"pgid":"1.0","osds":[0,2,1]}],"primary_temp":[],"blocklist":{"10.20.193.35:6801/432269861":"2026-02-25T07:42:02.689708+0000","10.20.193.35:6800/432269861":"2026-02-25T07:42:02.689708+0000","10.20.193.35:0/2958395575":"2026-02-25T07:42:02.689708+0000","10.20.193.35:0/3517194207":"2026-02-25T07:41:53.007692+0000","10.20.193.35:0/2848450259":"2026-02-25T07:41:53.007692+0000","10.20.193.35:0/2628287835":"2026-02-25T07:42:02.689708+0000","10.20.193.35:0/1697559475":"2026-02-25T07:41:53.007692+0000","10.20.193.35:0/1760415898":"2026-02-25T07:42:02.689708+0000","10.20.193.35:6801/1865836941":"2026-02-25T07:41:53.007692+0000","10.20.193.35:6800/1865836941":"2026-02-25T07:41:53.007692+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"isa","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2026-02-24T07:42:51.732 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2026-02-24T07:42:42.898873+0000', 'flags': 262145, 'flags_names': 'hashpspool,crimson', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'nonprimary_shards': '{}', 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 4, 'score_stable': 4, 'optimal_score': 0.75, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2026-02-24T07:42:51.732 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph osd pool get .mgr pg_num 2026-02-24T07:42:51.847 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:42:51.868 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:51 trial035 ceph-mon[39284]: purged_snaps scrub starts 2026-02-24T07:42:51.868 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:51 trial035 ceph-mon[39284]: purged_snaps scrub ok 2026-02-24T07:42:51.868 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:51 trial035 ceph-mon[39284]: Detected new or changed devices on trial091 2026-02-24T07:42:51.868 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:51 trial035 ceph-mon[39284]: Adjusting osd_memory_target on trial091 to 44583M 2026-02-24T07:42:51.868 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:51 trial035 ceph-mon[39284]: pgmap v38: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 2.1 TiB / 2.1 TiB avail 2026-02-24T07:42:51.869 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:51 trial035 ceph-mon[39284]: osd.3 [v2:10.20.193.91:6808/940596907,v1:10.20.193.91:6809/940596907] boot 2026-02-24T07:42:51.869 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:51 trial035 ceph-mon[39284]: osdmap e25: 4 total, 4 up, 4 in 2026-02-24T07:42:51.869 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:51 trial035 ceph-mon[39284]: from='mgr.14154 10.20.193.35:0/1087484775' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2026-02-24T07:42:51.869 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:51 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/3512360760' entity='client.admin' cmd={"prefix": "osd stat", "format": "json"} : dispatch 2026-02-24T07:42:51.869 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:51 trial035 ceph-mon[39284]: from='mgr.14154 10.20.193.35:0/1087484775' entity='mgr.x' 2026-02-24T07:42:51.869 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:51 trial035 ceph-mon[39284]: from='mgr.14154 10.20.193.35:0/1087484775' entity='mgr.x' 2026-02-24T07:42:51.953 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:42:51.951+0000 7f9d70dd8640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:42:51.953 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:42:51.951+0000 7f9d70dd8640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:42:52.065 INFO:teuthology.orchestra.run.trial035.stdout:pg_num: 1 2026-02-24T07:42:52.103 INFO:tasks.cephadm:Adding prometheus.a on trial091 2026-02-24T07:42:52.104 DEBUG:teuthology.orchestra.run.trial091:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph orch apply prometheus '1;trial091=a' 2026-02-24T07:42:52.223 INFO:teuthology.orchestra.run.trial091.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/config/ceph.conf 2026-02-24T07:42:52.328 INFO:teuthology.orchestra.run.trial091.stderr:2026-02-24T07:42:52.325+0000 7f48ee185640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:42:52.328 INFO:teuthology.orchestra.run.trial091.stderr:2026-02-24T07:42:52.326+0000 7f48ee185640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:42:52.446 INFO:teuthology.orchestra.run.trial091.stdout:Scheduled prometheus update... 2026-02-24T07:42:52.484 DEBUG:teuthology.orchestra.run.trial091:prometheus.a> sudo journalctl -f -n 0 -u ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460@prometheus.a.service 2026-02-24T07:42:52.486 INFO:tasks.cephadm:Adding node-exporter.a on trial035 2026-02-24T07:42:52.486 INFO:tasks.cephadm:Adding node-exporter.b on trial091 2026-02-24T07:42:52.486 DEBUG:teuthology.orchestra.run.trial091:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph orch apply node-exporter '2;trial035=a;trial091=b' 2026-02-24T07:42:52.642 INFO:teuthology.orchestra.run.trial091.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/config/ceph.conf 2026-02-24T07:42:52.751 INFO:teuthology.orchestra.run.trial091.stderr:2026-02-24T07:42:52.748+0000 7f8cbec8c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:42:52.751 INFO:teuthology.orchestra.run.trial091.stderr:2026-02-24T07:42:52.748+0000 7f8cbec8c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:42:52.871 INFO:teuthology.orchestra.run.trial091.stdout:Scheduled node-exporter update... 2026-02-24T07:42:52.910 DEBUG:teuthology.orchestra.run.trial035:node-exporter.a> sudo journalctl -f -n 0 -u ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460@node-exporter.a.service 2026-02-24T07:42:52.913 DEBUG:teuthology.orchestra.run.trial091:node-exporter.b> sudo journalctl -f -n 0 -u ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460@node-exporter.b.service 2026-02-24T07:42:52.916 INFO:tasks.cephadm:Setting up client nodes... 2026-02-24T07:42:52.917 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2026-02-24T07:42:52.976 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:52 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/3005596621' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2026-02-24T07:42:52.976 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:52 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/3666510014' entity='client.admin' cmd={"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} : dispatch 2026-02-24T07:42:52.976 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:52 trial035 ceph-mon[39284]: osdmap e26: 4 total, 4 up, 4 in 2026-02-24T07:42:52.976 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:52 trial035 ceph-mon[39284]: from='mgr.14154 10.20.193.35:0/1087484775' entity='mgr.x' 2026-02-24T07:42:52.976 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:52 trial035 ceph-mon[39284]: from='mgr.14154 10.20.193.35:0/1087484775' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2026-02-24T07:42:53.089 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:42:53.198 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:42:53.196+0000 7f05195af640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:42:53.198 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:42:53.196+0000 7f05195af640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:42:53.347 INFO:teuthology.orchestra.run.trial035.stdout:[client.0] 2026-02-24T07:42:53.348 INFO:teuthology.orchestra.run.trial035.stdout: key = AQB9Vp1pcGynFBAAOKSy3wc0Zp5Plb9ll33gWw== 2026-02-24T07:42:53.386 DEBUG:teuthology.orchestra.run.trial035:> set -ex 2026-02-24T07:42:53.387 DEBUG:teuthology.orchestra.run.trial035:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2026-02-24T07:42:53.387 DEBUG:teuthology.orchestra.run.trial035:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2026-02-24T07:42:53.423 DEBUG:teuthology.orchestra.run.trial091:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph auth get-or-create client.1 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2026-02-24T07:42:53.540 INFO:teuthology.orchestra.run.trial091.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/config/ceph.conf 2026-02-24T07:42:53.655 INFO:teuthology.orchestra.run.trial091.stderr:2026-02-24T07:42:53.653+0000 7f4aa4a3c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:42:53.656 INFO:teuthology.orchestra.run.trial091.stderr:2026-02-24T07:42:53.653+0000 7f4aa4a3c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:42:53.809 INFO:teuthology.orchestra.run.trial091.stdout:[client.1] 2026-02-24T07:42:53.809 INFO:teuthology.orchestra.run.trial091.stdout: key = AQB9Vp1pWCgrMBAAyqvyqdiSbD6E7ve50W5H9g== 2026-02-24T07:42:53.848 DEBUG:teuthology.orchestra.run.trial091:> set -ex 2026-02-24T07:42:53.848 DEBUG:teuthology.orchestra.run.trial091:> sudo dd of=/etc/ceph/ceph.client.1.keyring 2026-02-24T07:42:53.848 DEBUG:teuthology.orchestra.run.trial091:> sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2026-02-24T07:42:53.884 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph config log 1 --format=json 2026-02-24T07:42:54.002 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:42:54.043 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:53 trial035 ceph-mon[39284]: 1.0 starting backfill to osd.3 from (0'0,0'0] MAX to 21'92 2026-02-24T07:42:54.043 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:53 trial035 ceph-mon[39284]: from='client.14252 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "prometheus", "placement": "1;trial091=a", "target": ["mon-mgr", ""]}]: dispatch 2026-02-24T07:42:54.043 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:53 trial035 ceph-mon[39284]: Saving service prometheus spec with placement trial091=a;count:1 2026-02-24T07:42:54.043 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:53 trial035 ceph-mon[39284]: pgmap v41: 1 pgs: 1 unknown; 577 KiB data, 108 MiB used, 2.7 TiB / 2.7 TiB avail 2026-02-24T07:42:54.044 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:53 trial035 ceph-mon[39284]: from='mgr.14154 10.20.193.35:0/1087484775' entity='mgr.x' 2026-02-24T07:42:54.044 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:53 trial035 ceph-mon[39284]: from='mgr.14154 10.20.193.35:0/1087484775' entity='mgr.x' 2026-02-24T07:42:54.044 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:53 trial035 ceph-mon[39284]: from='client.14254 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "node-exporter", "placement": "2;trial035=a;trial091=b", "target": ["mon-mgr", ""]}]: dispatch 2026-02-24T07:42:54.044 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:53 trial035 ceph-mon[39284]: Saving service node-exporter spec with placement trial035=a;trial091=b;count:2 2026-02-24T07:42:54.044 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:53 trial035 ceph-mon[39284]: from='mgr.14154 10.20.193.35:0/1087484775' entity='mgr.x' 2026-02-24T07:42:54.044 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:53 trial035 ceph-mon[39284]: osdmap e27: 4 total, 4 up, 4 in 2026-02-24T07:42:54.044 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:53 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/2383213872' entity='client.admin' cmd={"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} : dispatch 2026-02-24T07:42:54.044 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:53 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/2383213872' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2026-02-24T07:42:54.044 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:53 trial035 ceph-mon[39284]: from='mgr.14154 10.20.193.35:0/1087484775' entity='mgr.x' 2026-02-24T07:42:54.044 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:53 trial035 ceph-mon[39284]: from='mgr.14154 10.20.193.35:0/1087484775' entity='mgr.x' 2026-02-24T07:42:54.044 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:53 trial035 ceph-mon[39284]: from='mgr.14154 10.20.193.35:0/1087484775' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-24T07:42:54.045 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:53 trial035 ceph-mon[39284]: from='mgr.14154 10.20.193.35:0/1087484775' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2026-02-24T07:42:54.045 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:53 trial035 ceph-mon[39284]: from='mgr.14154 10.20.193.35:0/1087484775' entity='mgr.x' 2026-02-24T07:42:54.045 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:53 trial035 ceph-mon[39284]: from='mgr.14154 10.20.193.35:0/1087484775' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2026-02-24T07:42:54.045 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:53 trial035 ceph-mon[39284]: from='mgr.14154 10.20.193.35:0/1087484775' entity='mgr.x' 2026-02-24T07:42:54.045 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:53 trial035 ceph-mon[39284]: from='mgr.14154 10.20.193.35:0/1087484775' entity='mgr.x' 2026-02-24T07:42:54.045 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:53 trial035 ceph-mon[39284]: from='mgr.14154 10.20.193.35:0/1087484775' entity='mgr.x' 2026-02-24T07:42:54.045 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:53 trial035 ceph-mon[39284]: from='mgr.14154 10.20.193.35:0/1087484775' entity='mgr.x' 2026-02-24T07:42:54.100 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:42:54.098+0000 7f667b8fb640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:42:54.100 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:42:54.098+0000 7f667b8fb640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:42:54.208 INFO:teuthology.orchestra.run.trial035.stdout: 2026-02-24T07:42:54.246 INFO:teuthology.orchestra.run.trial035.stdout:[{"version":13,"timestamp":"2026-02-24T07:42:49.994146+0000","name":"","changes":[{"name":"osd/host:trial091/osd_memory_target","previous_value":"93498623590","new_value":"46749311795"}]}] 2026-02-24T07:42:54.247 INFO:tasks.ceph_manager:config epoch is 13 2026-02-24T07:42:54.247 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2026-02-24T07:42:54.247 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2026-02-24T07:42:54.247 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph mgr dump --format=json 2026-02-24T07:42:54.361 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:42:54.461 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:42:54.459+0000 7feefe0d6640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:42:54.461 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:42:54.459+0000 7feefe0d6640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:42:54.603 INFO:teuthology.orchestra.run.trial035.stdout: 2026-02-24T07:42:54.638 INFO:teuthology.orchestra.run.trial035.stdout:{"epoch":14,"flags":0,"active_gid":14154,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.35:6800","nonce":3632488220},{"type":"v1","addr":"10.20.193.35:6801","nonce":3632488220}]},"active_addr":"10.20.193.35:6801/3632488220","active_change":"2026-02-24T07:42:02.689789+0000","active_mgr_features":4544132024016699391,"available":true,"standbys":[],"modules":["cephadm","dashboard","iostat","nfs"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to, use commas to separate multiple","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_pg_upmap_activity":{"name":"update_pg_upmap_activity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Updates pg_upmap activity stats to be used in `balancer status detail`","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often the agent on each host will gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"The first TCP port the agent will try to bind to. Up to 1000 subsequent ports will be attempted if this port cannot be bound","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this option can facilitate debugging daemons that encounter runtime problems.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"How frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"Fraction of total system memory to divide among autotuned daemons. Converged systems hosting both Ceph and compute should set a lower value; systems with more than 10 GiB of memory per OSD may benefit from a slightly larger value so that the abundance of RAM can be fully utilized","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command persistent logging","long_desc":"","tags":[],"see_also":[]},"certificate_automated_rotation_enabled":{"name":"certificate_automated_rotation_enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"This flag controls whether cephadm automatically rotates certificates upon expiration.","long_desc":"","tags":[],"see_also":[]},"certificate_check_debug_mode":{"name":"certificate_check_debug_mode","type":"bool","level":"dev","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"FOR TESTING ONLY: This flag forces the certificate check instead of waiting for certificate_check_period.","long_desc":"","tags":[],"see_also":[]},"certificate_check_period":{"name":"certificate_check_period","type":"int","level":"advanced","flags":0,"default_value":"1","min":"0","max":"30","enum_allowed":[],"desc":"Specifies how often (in days) the certificate should be checked for validity.","long_desc":"","tags":[],"see_also":[]},"certificate_duration_days":{"name":"certificate_duration_days","type":"int","level":"advanced","flags":0,"default_value":"1095","min":"90","max":"3650","enum_allowed":[],"desc":"Specifies the duration of self-signed certificates generated and signed by the cephadm root CA","long_desc":"","tags":[],"see_also":[]},"certificate_renewal_threshold_days":{"name":"certificate_renewal_threshold_days","type":"int","level":"advanced","flags":0,"default_value":"30","min":"10","max":"90","enum_allowed":[],"desc":"Specifies the lead time in days to initiate certificate renewal before expiration.","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently Podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.28.1","min":"","max":"","enum_allowed":[],"desc":"Alertmanager container image","long_desc":"","tags":[],"see_also":[]},"container_image_alloy":{"name":"container_image_alloy","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/alloy:latest","min":"","max":"","enum_allowed":[],"desc":"Alloy container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"Elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:12.3.1","min":"","max":"","enum_allowed":[],"desc":"Grafana container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"Haproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_nginx":{"name":"container_image_nginx","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nginx:sclorg-nginx-126","min":"","max":"","enum_allowed":[],"desc":"Nginx container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.9.1","min":"","max":"","enum_allowed":[],"desc":"Node exporter container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.6","min":"","max":"","enum_allowed":[],"desc":"Nvmeof container image","long_desc":"","tags":[],"see_also":[]},"container_image_oauth2_proxy":{"name":"container_image_oauth2_proxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/oauth2-proxy/oauth2-proxy:v7.6.0","min":"","max":"","enum_allowed":[],"desc":"Oauth2 proxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v3.6.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba_metrics":{"name":"container_image_samba_metrics","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-metrics:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba metrics container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"Snmp gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"Seconds for which to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"quay.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"Seconds for which to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"Seconds for which to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"Location of dashboards to include in Grafana","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"How frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether the hardware monitoring daemon be deployed on every host?","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at the debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"Max number of daemons per service per host: guard against mistakes and runaways","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"Max number of OSDs that will be drained simultaneously when OSDs are removed","long_desc":"","tags":[],"see_also":[]},"max_parallel_osd_upgrades":{"name":"max_parallel_osd_upgrades","type":"int","level":"advanced","flags":0,"default_value":"16","min":"","max":"","enum_allowed":[],"desc":"Maximum number of OSD daemons upgraded in parallel.","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"Mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default IP address for RedFish API (OOB management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"Location of alerts to include in Prometheus Alertmanager","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry URL for login purposes. This is not the default registry.","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"Cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Customized SSH config file to distribute to managed hosts","long_desc":"","tags":[],"see_also":[]},"ssh_keepalive_count_max":{"name":"ssh_keepalive_count_max","type":"int","level":"advanced","flags":0,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"How many sequential SSH connection liveness check failures before the host is marked offline","long_desc":"","tags":[],"see_also":[]},"ssh_keepalive_interval":{"name":"ssh_keepalive_interval","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"How often SSH connections are checked for liveness","long_desc":"","tags":[],"see_also":[]},"stray_daemon_check_interval":{"name":"stray_daemon_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"How frequently cephadm should check for the presence of stray daemons","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use the cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest to ensure that all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROM_ALERT_CREDENTIAL_CACHE_TTL":{"name":"PROM_ALERT_CREDENTIAL_CACHE_TTL","type":"int","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_HOSTNAME_PER_DAEMON":{"name":"RGW_HOSTNAME_PER_DAEMON","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crypto_caller":{"name":"crypto_caller","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sso_oauth2":{"name":"sso_oauth2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"healthcheck_history_max_entries":{"name":"healthcheck_history_max_entries","type":"int","level":"advanced","flags":1,"default_value":"1000","min":"","max":"","enum_allowed":[],"desc":"Maximum number of health check history entries to keep","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"secondary_zone_period_retry_limit":{"name":"secondary_zone_period_retry_limit","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"RGW module period update retry limit for secondary site","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"prometheus_tls_secret_name":{"name":"prometheus_tls_secret_name","type":"str","level":"advanced","flags":0,"default_value":"rook-ceph-prometheus-server-tls","min":"","max":"","enum_allowed":[],"desc":"name of tls secret in k8s for prometheus","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"smb","can_run":true,"error_string":"","module_options":{"internal_store_backend":{"name":"internal_store_backend","type":"str","level":"dev","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"set internal store backend. for develoment and testing only","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_orchestration":{"name":"update_orchestration","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically update orchestration when smb resources are changed","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"pause_cloning":{"name":"pause_cloning","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Pause asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"pause_purging":{"name":"pause_purging","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Pause asynchronous subvolume purge threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://10.20.193.35:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"tentacle":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"force_disabled_modules":{},"last_failure_osd_epoch":3,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"10.20.193.35:0","nonce":4081776025}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"10.20.193.35:0","nonce":794704272}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"10.20.193.35:0","nonce":1753799146}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"10.20.193.35:0","nonce":1250927865}]}]} 2026-02-24T07:42:54.639 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2026-02-24T07:42:54.639 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2026-02-24T07:42:54.640 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph osd dump --format=json 2026-02-24T07:42:54.755 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:42:54.852 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:42:54.849+0000 7f5c69ae2640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:42:54.852 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:42:54.849+0000 7f5c69ae2640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:42:54.954 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:54 trial035 ceph-mon[39284]: Deploying daemon prometheus.a on trial091 2026-02-24T07:42:54.954 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:54 trial035 ceph-mon[39284]: from='client.? 10.20.193.91:0/4132067698' entity='client.admin' cmd={"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} : dispatch 2026-02-24T07:42:54.955 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:54 trial035 ceph-mon[39284]: from='client.? 10.20.193.91:0/4132067698' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2026-02-24T07:42:54.955 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:54 trial035 ceph-mon[39284]: osdmap e28: 4 total, 4 up, 4 in 2026-02-24T07:42:54.955 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:54 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/2576278922' entity='client.admin' cmd={"prefix": "config log", "num": 1, "format": "json"} : dispatch 2026-02-24T07:42:54.955 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:54 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/1465171233' entity='client.admin' cmd={"prefix": "mgr dump", "format": "json"} : dispatch 2026-02-24T07:42:54.962 INFO:teuthology.orchestra.run.trial035.stdout: 2026-02-24T07:42:54.962 INFO:teuthology.orchestra.run.trial035.stdout:{"epoch":28,"fsid":"3a5983d2-1154-11f1-9c05-d404e6e7d460","created":"2026-02-24T07:41:42.805703+0000","modified":"2026-02-24T07:42:54.182669+0000","last_up_change":"2026-02-24T07:42:51.180844+0000","last_in_change":"2026-02-24T07:42:42.005822+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":10,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":4,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"tentacle","allow_crimson":true,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2026-02-24T07:42:42.898873+0000","flags":262145,"flags_names":"hashpspool,crimson","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"nonprimary_shards":"{}","options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":4,"score_stable":4,"optimal_score":0.75,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"8d5b0adf-0e02-40d4-8780-e64988c10e92","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.35:6802","nonce":2333513103},{"type":"v1","addr":"10.20.193.35:6803","nonce":2333513103}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.35:6804","nonce":2333513103},{"type":"v1","addr":"10.20.193.35:6805","nonce":2333513103}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.35:6808","nonce":2333513103},{"type":"v1","addr":"10.20.193.35:6809","nonce":2333513103}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.35:6806","nonce":2333513103},{"type":"v1","addr":"10.20.193.35:6807","nonce":2333513103}]},"public_addr":"10.20.193.35:6803/2333513103","cluster_addr":"10.20.193.35:6805/2333513103","heartbeat_back_addr":"10.20.193.35:6809/2333513103","heartbeat_front_addr":"10.20.193.35:6807/2333513103","state":["exists","up"]},{"osd":1,"uuid":"b9b77957-7aa8-49b2-95a7-b50dcfc6bc3e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.35:6810","nonce":4095255721},{"type":"v1","addr":"10.20.193.35:6811","nonce":4095255721}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.35:6812","nonce":4095255721},{"type":"v1","addr":"10.20.193.35:6813","nonce":4095255721}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.35:6816","nonce":4095255721},{"type":"v1","addr":"10.20.193.35:6817","nonce":4095255721}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.35:6814","nonce":4095255721},{"type":"v1","addr":"10.20.193.35:6815","nonce":4095255721}]},"public_addr":"10.20.193.35:6811/4095255721","cluster_addr":"10.20.193.35:6813/4095255721","heartbeat_back_addr":"10.20.193.35:6817/4095255721","heartbeat_front_addr":"10.20.193.35:6815/4095255721","state":["exists","up"]},{"osd":2,"uuid":"883dcd38-0b52-4889-90fe-44a624ba23cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.91:6800","nonce":1390012919},{"type":"v1","addr":"10.20.193.91:6801","nonce":1390012919}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.91:6802","nonce":1390012919},{"type":"v1","addr":"10.20.193.91:6803","nonce":1390012919}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.91:6806","nonce":1390012919},{"type":"v1","addr":"10.20.193.91:6807","nonce":1390012919}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.91:6804","nonce":1390012919},{"type":"v1","addr":"10.20.193.91:6805","nonce":1390012919}]},"public_addr":"10.20.193.91:6801/1390012919","cluster_addr":"10.20.193.91:6803/1390012919","heartbeat_back_addr":"10.20.193.91:6807/1390012919","heartbeat_front_addr":"10.20.193.91:6805/1390012919","state":["exists","up"]},{"osd":3,"uuid":"2582e7bb-0265-44e6-bf18-c68070be0dc7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.91:6808","nonce":940596907},{"type":"v1","addr":"10.20.193.91:6809","nonce":940596907}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.91:6810","nonce":940596907},{"type":"v1","addr":"10.20.193.91:6811","nonce":940596907}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.91:6814","nonce":940596907},{"type":"v1","addr":"10.20.193.91:6815","nonce":940596907}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.91:6812","nonce":940596907},{"type":"v1","addr":"10.20.193.91:6813","nonce":940596907}]},"public_addr":"10.20.193.91:6809/940596907","cluster_addr":"10.20.193.91:6811/940596907","heartbeat_back_addr":"10.20.193.91:6815/940596907","heartbeat_front_addr":"10.20.193.91:6813/940596907","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-24T07:42:24.073399+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-24T07:42:31.659670+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-24T07:42:40.760086+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-24T07:42:49.343383+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"10.20.193.35:6801/432269861":"2026-02-25T07:42:02.689708+0000","10.20.193.35:6800/432269861":"2026-02-25T07:42:02.689708+0000","10.20.193.35:0/2958395575":"2026-02-25T07:42:02.689708+0000","10.20.193.35:0/3517194207":"2026-02-25T07:41:53.007692+0000","10.20.193.35:0/2848450259":"2026-02-25T07:41:53.007692+0000","10.20.193.35:0/2628287835":"2026-02-25T07:42:02.689708+0000","10.20.193.35:0/1697559475":"2026-02-25T07:41:53.007692+0000","10.20.193.35:0/1760415898":"2026-02-25T07:42:02.689708+0000","10.20.193.35:6801/1865836941":"2026-02-25T07:41:53.007692+0000","10.20.193.35:6800/1865836941":"2026-02-25T07:41:53.007692+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"isa","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2026-02-24T07:42:54.997 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2026-02-24T07:42:54.997 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph osd dump --format=json 2026-02-24T07:42:55.112 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:42:55.210 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:42:55.208+0000 7f017aab4640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:42:55.210 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:42:55.208+0000 7f017aab4640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:42:55.322 INFO:teuthology.orchestra.run.trial035.stdout: 2026-02-24T07:42:55.322 INFO:teuthology.orchestra.run.trial035.stdout:{"epoch":28,"fsid":"3a5983d2-1154-11f1-9c05-d404e6e7d460","created":"2026-02-24T07:41:42.805703+0000","modified":"2026-02-24T07:42:54.182669+0000","last_up_change":"2026-02-24T07:42:51.180844+0000","last_in_change":"2026-02-24T07:42:42.005822+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":10,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":4,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"tentacle","allow_crimson":true,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2026-02-24T07:42:42.898873+0000","flags":262145,"flags_names":"hashpspool,crimson","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"nonprimary_shards":"{}","options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":4,"score_stable":4,"optimal_score":0.75,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"8d5b0adf-0e02-40d4-8780-e64988c10e92","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.35:6802","nonce":2333513103},{"type":"v1","addr":"10.20.193.35:6803","nonce":2333513103}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.35:6804","nonce":2333513103},{"type":"v1","addr":"10.20.193.35:6805","nonce":2333513103}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.35:6808","nonce":2333513103},{"type":"v1","addr":"10.20.193.35:6809","nonce":2333513103}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.35:6806","nonce":2333513103},{"type":"v1","addr":"10.20.193.35:6807","nonce":2333513103}]},"public_addr":"10.20.193.35:6803/2333513103","cluster_addr":"10.20.193.35:6805/2333513103","heartbeat_back_addr":"10.20.193.35:6809/2333513103","heartbeat_front_addr":"10.20.193.35:6807/2333513103","state":["exists","up"]},{"osd":1,"uuid":"b9b77957-7aa8-49b2-95a7-b50dcfc6bc3e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.35:6810","nonce":4095255721},{"type":"v1","addr":"10.20.193.35:6811","nonce":4095255721}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.35:6812","nonce":4095255721},{"type":"v1","addr":"10.20.193.35:6813","nonce":4095255721}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.35:6816","nonce":4095255721},{"type":"v1","addr":"10.20.193.35:6817","nonce":4095255721}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.35:6814","nonce":4095255721},{"type":"v1","addr":"10.20.193.35:6815","nonce":4095255721}]},"public_addr":"10.20.193.35:6811/4095255721","cluster_addr":"10.20.193.35:6813/4095255721","heartbeat_back_addr":"10.20.193.35:6817/4095255721","heartbeat_front_addr":"10.20.193.35:6815/4095255721","state":["exists","up"]},{"osd":2,"uuid":"883dcd38-0b52-4889-90fe-44a624ba23cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.91:6800","nonce":1390012919},{"type":"v1","addr":"10.20.193.91:6801","nonce":1390012919}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.91:6802","nonce":1390012919},{"type":"v1","addr":"10.20.193.91:6803","nonce":1390012919}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.91:6806","nonce":1390012919},{"type":"v1","addr":"10.20.193.91:6807","nonce":1390012919}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.91:6804","nonce":1390012919},{"type":"v1","addr":"10.20.193.91:6805","nonce":1390012919}]},"public_addr":"10.20.193.91:6801/1390012919","cluster_addr":"10.20.193.91:6803/1390012919","heartbeat_back_addr":"10.20.193.91:6807/1390012919","heartbeat_front_addr":"10.20.193.91:6805/1390012919","state":["exists","up"]},{"osd":3,"uuid":"2582e7bb-0265-44e6-bf18-c68070be0dc7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.91:6808","nonce":940596907},{"type":"v1","addr":"10.20.193.91:6809","nonce":940596907}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.91:6810","nonce":940596907},{"type":"v1","addr":"10.20.193.91:6811","nonce":940596907}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.91:6814","nonce":940596907},{"type":"v1","addr":"10.20.193.91:6815","nonce":940596907}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.91:6812","nonce":940596907},{"type":"v1","addr":"10.20.193.91:6813","nonce":940596907}]},"public_addr":"10.20.193.91:6809/940596907","cluster_addr":"10.20.193.91:6811/940596907","heartbeat_back_addr":"10.20.193.91:6815/940596907","heartbeat_front_addr":"10.20.193.91:6813/940596907","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-24T07:42:24.073399+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-24T07:42:31.659670+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-24T07:42:40.760086+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-24T07:42:49.343383+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"10.20.193.35:6801/432269861":"2026-02-25T07:42:02.689708+0000","10.20.193.35:6800/432269861":"2026-02-25T07:42:02.689708+0000","10.20.193.35:0/2958395575":"2026-02-25T07:42:02.689708+0000","10.20.193.35:0/3517194207":"2026-02-25T07:41:53.007692+0000","10.20.193.35:0/2848450259":"2026-02-25T07:41:53.007692+0000","10.20.193.35:0/2628287835":"2026-02-25T07:42:02.689708+0000","10.20.193.35:0/1697559475":"2026-02-25T07:41:53.007692+0000","10.20.193.35:0/1760415898":"2026-02-25T07:42:02.689708+0000","10.20.193.35:6801/1865836941":"2026-02-25T07:41:53.007692+0000","10.20.193.35:6800/1865836941":"2026-02-25T07:41:53.007692+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"isa","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2026-02-24T07:42:55.362 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph tell osd.0 flush_pg_stats 2026-02-24T07:42:55.362 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph tell osd.1 flush_pg_stats 2026-02-24T07:42:55.362 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph tell osd.2 flush_pg_stats 2026-02-24T07:42:55.362 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph tell osd.3 flush_pg_stats 2026-02-24T07:42:55.483 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:42:55.483 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:42:55.489 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:42:55.492 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:42:55.598 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:42:55.596+0000 7fcdebe44640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:42:55.598 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:42:55.596+0000 7fcdebe44640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:42:55.598 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:42:55.596+0000 7fe6df0ad640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:42:55.598 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:42:55.596+0000 7fe6df0ad640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:42:55.599 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:42:55.598+0000 7f0ee2b91640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:42:55.599 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:42:55.598+0000 7f86ec176640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:42:55.600 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:42:55.598+0000 7f0ee2b91640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:42:55.600 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:42:55.598+0000 7f86ec176640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:42:55.658 INFO:teuthology.orchestra.run.trial035.stdout:81604378629 2026-02-24T07:42:55.659 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph osd last-stat-seq osd.2 2026-02-24T07:42:55.660 INFO:teuthology.orchestra.run.trial035.stdout:107374182402 2026-02-24T07:42:55.660 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph osd last-stat-seq osd.3 2026-02-24T07:42:55.664 INFO:teuthology.orchestra.run.trial035.stdout:38654705672 2026-02-24T07:42:55.664 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph osd last-stat-seq osd.0 2026-02-24T07:42:55.672 INFO:teuthology.orchestra.run.trial035.stdout:60129542150 2026-02-24T07:42:55.673 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph osd last-stat-seq osd.1 2026-02-24T07:42:55.779 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:42:55.785 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:42:55.787 INFO:journalctl@ceph.prometheus.a.trial091.stdout:Feb 24 07:42:55 trial091 systemd[1]: Started Ceph prometheus.a for 3a5983d2-1154-11f1-9c05-d404e6e7d460. 2026-02-24T07:42:55.827 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:42:55.829 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:42:55.883 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:42:55.881+0000 7f11b8e17640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:42:55.883 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:42:55.881+0000 7f11b8e17640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:42:55.884 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:42:55.882+0000 7ff66d551640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:42:55.885 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:42:55.883+0000 7ff66d551640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:42:55.935 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:42:55.932+0000 7f50d9e2f640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:42:55.935 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:42:55.932+0000 7f50d9e2f640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:42:55.936 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:42:55.933+0000 7fb62a6c8640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:42:55.936 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:42:55.933+0000 7fb62a6c8640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:42:55.995 INFO:teuthology.orchestra.run.trial035.stdout:107374182401 2026-02-24T07:42:56.000 INFO:teuthology.orchestra.run.trial035.stdout:81604378628 2026-02-24T07:42:56.036 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:55 trial035 ceph-mon[39284]: pgmap v44: 1 pgs: 1 unknown; 577 KiB data, 108 MiB used, 2.7 TiB / 2.7 TiB avail 2026-02-24T07:42:56.036 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:55 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/2279207008' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2026-02-24T07:42:56.036 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:55 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/3705635402' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2026-02-24T07:42:56.037 INFO:tasks.cephadm.ceph_manager.ceph:need seq 107374182402 got 107374182401 for osd.3 2026-02-24T07:42:56.044 INFO:tasks.cephadm.ceph_manager.ceph:need seq 81604378629 got 81604378628 for osd.2 2026-02-24T07:42:56.046 INFO:teuthology.orchestra.run.trial035.stdout:38654705671 2026-02-24T07:42:56.050 INFO:teuthology.orchestra.run.trial035.stdout:60129542149 2026-02-24T07:42:56.084 INFO:tasks.cephadm.ceph_manager.ceph:need seq 38654705672 got 38654705671 for osd.0 2026-02-24T07:42:56.097 INFO:tasks.cephadm.ceph_manager.ceph:need seq 60129542150 got 60129542149 for osd.1 2026-02-24T07:42:56.106 INFO:journalctl@ceph.prometheus.a.trial091.stdout:Feb 24 07:42:55 trial091 ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-prometheus-a[63777]: time=2026-02-24T07:42:55.786Z level=INFO source=main.go:1545 msg="updated GOGC" old=100 new=75 2026-02-24T07:42:56.107 INFO:journalctl@ceph.prometheus.a.trial091.stdout:Feb 24 07:42:55 trial091 ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-prometheus-a[63777]: time=2026-02-24T07:42:55.787Z level=INFO source=main.go:677 msg="Leaving GOMAXPROCS=24: CPU quota undefined" component=automaxprocs 2026-02-24T07:42:56.107 INFO:journalctl@ceph.prometheus.a.trial091.stdout:Feb 24 07:42:55 trial091 ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-prometheus-a[63777]: time=2026-02-24T07:42:55.787Z level=INFO source=memlimit.go:198 msg="GOMEMLIMIT is updated" component=automemlimit package=github.com/KimMachineGun/automemlimit/memlimit GOMEMLIMIT=120212516044 previous=9223372036854775807 2026-02-24T07:42:56.107 INFO:journalctl@ceph.prometheus.a.trial091.stdout:Feb 24 07:42:55 trial091 ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-prometheus-a[63777]: time=2026-02-24T07:42:55.787Z level=INFO source=main.go:770 msg="Starting Prometheus Server" mode=server version="(version=3.6.0, branch=HEAD, revision=491734606591ef8c188f5489c70380805bc868c0)" 2026-02-24T07:42:56.107 INFO:journalctl@ceph.prometheus.a.trial091.stdout:Feb 24 07:42:55 trial091 ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-prometheus-a[63777]: time=2026-02-24T07:42:55.787Z level=INFO source=main.go:775 msg="operational information" build_context="(go=go1.24.7, platform=linux/amd64, user=root@bfdf1aa26e4d, date=20250921-19:51:16, tags=netgo,builtinassets)" host_details="(Linux 5.14.0-686.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Thu Feb 19 10:49:27 UTC 2026 x86_64 trial091 (none))" fd_limits="(soft=1048576, hard=1048576)" vm_limits="(soft=unlimited, hard=unlimited)" 2026-02-24T07:42:56.107 INFO:journalctl@ceph.prometheus.a.trial091.stdout:Feb 24 07:42:55 trial091 ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-prometheus-a[63777]: time=2026-02-24T07:42:55.788Z level=INFO source=web.go:659 msg="Start listening for connections" component=web address=:9095 2026-02-24T07:42:56.107 INFO:journalctl@ceph.prometheus.a.trial091.stdout:Feb 24 07:42:55 trial091 ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-prometheus-a[63777]: time=2026-02-24T07:42:55.789Z level=INFO source=main.go:1289 msg="Starting TSDB ..." 2026-02-24T07:42:56.107 INFO:journalctl@ceph.prometheus.a.trial091.stdout:Feb 24 07:42:55 trial091 ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-prometheus-a[63777]: time=2026-02-24T07:42:55.790Z level=INFO source=tls_config.go:347 msg="Listening on" component=web address=[::]:9095 2026-02-24T07:42:56.107 INFO:journalctl@ceph.prometheus.a.trial091.stdout:Feb 24 07:42:55 trial091 ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-prometheus-a[63777]: time=2026-02-24T07:42:55.790Z level=INFO source=tls_config.go:350 msg="TLS is disabled." component=web http2=false address=[::]:9095 2026-02-24T07:42:56.107 INFO:journalctl@ceph.prometheus.a.trial091.stdout:Feb 24 07:42:55 trial091 ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-prometheus-a[63777]: time=2026-02-24T07:42:55.791Z level=INFO source=head.go:667 msg="Replaying on-disk memory mappable chunks if any" component=tsdb 2026-02-24T07:42:56.107 INFO:journalctl@ceph.prometheus.a.trial091.stdout:Feb 24 07:42:55 trial091 ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-prometheus-a[63777]: time=2026-02-24T07:42:55.791Z level=INFO source=head.go:754 msg="On-disk memory mappable chunks replay completed" component=tsdb duration=1.082µs 2026-02-24T07:42:56.107 INFO:journalctl@ceph.prometheus.a.trial091.stdout:Feb 24 07:42:55 trial091 ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-prometheus-a[63777]: time=2026-02-24T07:42:55.791Z level=INFO source=head.go:762 msg="Replaying WAL, this may take a while" component=tsdb 2026-02-24T07:42:56.107 INFO:journalctl@ceph.prometheus.a.trial091.stdout:Feb 24 07:42:55 trial091 ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-prometheus-a[63777]: time=2026-02-24T07:42:55.792Z level=INFO source=head.go:835 msg="WAL segment loaded" component=tsdb segment=0 maxSegment=0 duration=785.479µs 2026-02-24T07:42:56.107 INFO:journalctl@ceph.prometheus.a.trial091.stdout:Feb 24 07:42:55 trial091 ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-prometheus-a[63777]: time=2026-02-24T07:42:55.792Z level=INFO source=head.go:872 msg="WAL replay completed" component=tsdb checkpoint_replay_duration=15.138µs wal_replay_duration=800.426µs wbl_replay_duration=151ns chunk_snapshot_load_duration=0s mmap_chunk_replay_duration=1.082µs total_replay_duration=843.396µs 2026-02-24T07:42:56.107 INFO:journalctl@ceph.prometheus.a.trial091.stdout:Feb 24 07:42:55 trial091 ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-prometheus-a[63777]: time=2026-02-24T07:42:55.793Z level=INFO source=main.go:1310 msg="filesystem information" fs_type=XFS_SUPER_MAGIC 2026-02-24T07:42:56.107 INFO:journalctl@ceph.prometheus.a.trial091.stdout:Feb 24 07:42:55 trial091 ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-prometheus-a[63777]: time=2026-02-24T07:42:55.793Z level=INFO source=main.go:1313 msg="TSDB started" 2026-02-24T07:42:56.107 INFO:journalctl@ceph.prometheus.a.trial091.stdout:Feb 24 07:42:55 trial091 ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-prometheus-a[63777]: time=2026-02-24T07:42:55.793Z level=INFO source=main.go:1498 msg="Loading configuration file" filename=/etc/prometheus/prometheus.yml 2026-02-24T07:42:56.107 INFO:journalctl@ceph.prometheus.a.trial091.stdout:Feb 24 07:42:55 trial091 ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-prometheus-a[63777]: time=2026-02-24T07:42:55.804Z level=INFO source=main.go:1538 msg="Completed loading of configuration file" db_storage=651ns remote_storage=1.172µs web_handler=340ns query_engine=641ns scrape=350.591µs scrape_sd=46.857µs notify=742ns notify_sd=551ns rules=10.040883ms tracing=4.218µs filename=/etc/prometheus/prometheus.yml totalDuration=10.630587ms 2026-02-24T07:42:56.107 INFO:journalctl@ceph.prometheus.a.trial091.stdout:Feb 24 07:42:55 trial091 ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-prometheus-a[63777]: time=2026-02-24T07:42:55.804Z level=INFO source=main.go:1274 msg="Server is ready to receive web requests." 2026-02-24T07:42:56.107 INFO:journalctl@ceph.prometheus.a.trial091.stdout:Feb 24 07:42:55 trial091 ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-prometheus-a[63777]: time=2026-02-24T07:42:55.804Z level=INFO source=manager.go:176 msg="Starting rule manager..." component="rule manager" 2026-02-24T07:42:57.038 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph osd last-stat-seq osd.3 2026-02-24T07:42:57.045 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph osd last-stat-seq osd.2 2026-02-24T07:42:57.061 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:56 trial035 ceph-mon[39284]: from='mgr.14154 10.20.193.35:0/1087484775' entity='mgr.x' 2026-02-24T07:42:57.061 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:56 trial035 ceph-mon[39284]: from='mgr.14154 10.20.193.35:0/1087484775' entity='mgr.x' 2026-02-24T07:42:57.061 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:56 trial035 ceph-mon[39284]: from='mgr.14154 10.20.193.35:0/1087484775' entity='mgr.x' 2026-02-24T07:42:57.061 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:56 trial035 ceph-mon[39284]: from='mgr.14154 10.20.193.35:0/1087484775' entity='mgr.x' cmd={"prefix": "mgr module enable", "module": "prometheus"} : dispatch 2026-02-24T07:42:57.061 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:56 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/4243226743' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 3} : dispatch 2026-02-24T07:42:57.061 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:56 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/891719737' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 2} : dispatch 2026-02-24T07:42:57.061 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:56 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/3437571804' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 0} : dispatch 2026-02-24T07:42:57.061 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:56 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/2090338403' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 1} : dispatch 2026-02-24T07:42:57.061 INFO:journalctl@ceph.mgr.x.trial035.stdout:Feb 24 07:42:56 trial035 ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-mgr-x[39621]: ignoring --setuser ceph since I am not root 2026-02-24T07:42:57.061 INFO:journalctl@ceph.mgr.x.trial035.stdout:Feb 24 07:42:56 trial035 ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-mgr-x[39621]: ignoring --setgroup ceph since I am not root 2026-02-24T07:42:57.061 INFO:journalctl@ceph.mgr.x.trial035.stdout:Feb 24 07:42:56 trial035 ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-mgr-x[39621]: 2026-02-24T07:42:56.845+0000 7f228dcbdfc0 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:42:57.061 INFO:journalctl@ceph.mgr.x.trial035.stdout:Feb 24 07:42:56 trial035 ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-mgr-x[39621]: 2026-02-24T07:42:56.847+0000 7f228dcbdfc0 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:42:57.061 INFO:journalctl@ceph.mgr.x.trial035.stdout:Feb 24 07:42:56 trial035 ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-mgr-x[39621]: 2026-02-24T07:42:56.848+0000 7f228dcbdfc0 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:42:57.085 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph osd last-stat-seq osd.0 2026-02-24T07:42:57.097 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph osd last-stat-seq osd.1 2026-02-24T07:42:57.157 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:42:57.176 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:42:57.220 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:42:57.223 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:42:57.270 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:42:57.268+0000 7f654fffc640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:42:57.270 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:42:57.268+0000 7f654fffc640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:42:57.274 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:42:57.273+0000 7f76c182e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:42:57.274 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:42:57.273+0000 7f76c182e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:42:57.326 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:42:57.324+0000 7fd7e21ba640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:42:57.327 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:42:57.324+0000 7f025f92f640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:42:57.327 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:42:57.324+0000 7fd7e21ba640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:42:57.327 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:42:57.324+0000 7f025f92f640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:42:57.385 INFO:teuthology.orchestra.run.trial035.stdout:107374182403 2026-02-24T07:42:57.392 INFO:teuthology.orchestra.run.trial035.stdout:81604378629 2026-02-24T07:42:57.425 INFO:tasks.cephadm.ceph_manager.ceph:need seq 107374182402 got 107374182403 for osd.3 2026-02-24T07:42:57.425 DEBUG:teuthology.parallel:result is None 2026-02-24T07:42:57.434 INFO:tasks.cephadm.ceph_manager.ceph:need seq 81604378629 got 81604378629 for osd.2 2026-02-24T07:42:57.435 DEBUG:teuthology.parallel:result is None 2026-02-24T07:42:57.442 INFO:teuthology.orchestra.run.trial035.stdout:38654705673 2026-02-24T07:42:57.442 INFO:teuthology.orchestra.run.trial035.stdout:60129542150 2026-02-24T07:42:57.486 INFO:tasks.cephadm.ceph_manager.ceph:need seq 38654705672 got 38654705673 for osd.0 2026-02-24T07:42:57.486 DEBUG:teuthology.parallel:result is None 2026-02-24T07:42:57.488 INFO:tasks.cephadm.ceph_manager.ceph:need seq 60129542150 got 60129542150 for osd.1 2026-02-24T07:42:57.488 DEBUG:teuthology.parallel:result is None 2026-02-24T07:42:57.488 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2026-02-24T07:42:57.489 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph pg dump --format=json 2026-02-24T07:42:57.647 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:42:57.748 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:42:57.747+0000 7f17eeb40640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:42:57.749 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:42:57.747+0000 7f17eeb40640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:42:57.954 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:57 trial035 ceph-mon[39284]: from='mgr.14154 10.20.193.35:0/1087484775' entity='mgr.x' cmd='[{"prefix": "mgr module enable", "module": "prometheus"}]': finished 2026-02-24T07:42:57.954 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:57 trial035 ceph-mon[39284]: mgrmap e15: x(active, since 54s) 2026-02-24T07:42:57.954 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:57 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/3825915011' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 3} : dispatch 2026-02-24T07:42:57.954 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:57 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/3076418037' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 2} : dispatch 2026-02-24T07:42:57.954 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:57 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/2344560305' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 0} : dispatch 2026-02-24T07:42:57.954 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:57 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/749210253' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 1} : dispatch 2026-02-24T07:42:57.955 INFO:journalctl@ceph.mgr.x.trial035.stdout:Feb 24 07:42:57 trial035 ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-mgr-x[39621]: /lib64/python3.9/site-packages/scipy/__init__.py:73: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2026-02-24T07:42:57.955 INFO:journalctl@ceph.mgr.x.trial035.stdout:Feb 24 07:42:57 trial035 ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-mgr-x[39621]: Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2026-02-24T07:42:57.955 INFO:journalctl@ceph.mgr.x.trial035.stdout:Feb 24 07:42:57 trial035 ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-mgr-x[39621]: from numpy import show_config as show_numpy_config 2026-02-24T07:43:00.180 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:59 trial035 ceph-mon[39284]: Active manager daemon x restarted 2026-02-24T07:43:00.180 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:59 trial035 ceph-mon[39284]: Activating manager daemon x 2026-02-24T07:43:00.180 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:59 trial035 ceph-mon[39284]: osdmap e29: 4 total, 4 up, 4 in 2026-02-24T07:43:00.180 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:59 trial035 ceph-mon[39284]: mgrmap e16: x(active, starting, since 0.00310601s) 2026-02-24T07:43:00.180 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:59 trial035 ceph-mon[39284]: from='mgr.14284 10.20.193.35:0/3681373222' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2026-02-24T07:43:00.180 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:59 trial035 ceph-mon[39284]: from='mgr.14284 10.20.193.35:0/3681373222' entity='mgr.x' cmd={"prefix": "mgr metadata", "who": "x", "id": "x"} : dispatch 2026-02-24T07:43:00.180 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:59 trial035 ceph-mon[39284]: from='mgr.14284 10.20.193.35:0/3681373222' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2026-02-24T07:43:00.180 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:59 trial035 ceph-mon[39284]: from='mgr.14284 10.20.193.35:0/3681373222' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2026-02-24T07:43:00.180 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:59 trial035 ceph-mon[39284]: from='mgr.14284 10.20.193.35:0/3681373222' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2026-02-24T07:43:00.180 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:59 trial035 ceph-mon[39284]: from='mgr.14284 10.20.193.35:0/3681373222' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2026-02-24T07:43:00.180 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:59 trial035 ceph-mon[39284]: from='mgr.14284 10.20.193.35:0/3681373222' entity='mgr.x' cmd={"prefix": "mds metadata"} : dispatch 2026-02-24T07:43:00.180 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:59 trial035 ceph-mon[39284]: from='mgr.14284 10.20.193.35:0/3681373222' entity='mgr.x' cmd={"prefix": "osd metadata"} : dispatch 2026-02-24T07:43:00.180 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:59 trial035 ceph-mon[39284]: from='mgr.14284 10.20.193.35:0/3681373222' entity='mgr.x' cmd={"prefix": "mon metadata"} : dispatch 2026-02-24T07:43:00.180 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:42:59 trial035 ceph-mon[39284]: Manager daemon x is now available 2026-02-24T07:43:00.660 INFO:teuthology.orchestra.run.trial035.stdout: 2026-02-24T07:43:00.661 INFO:teuthology.orchestra.run.trial035.stderr:Warning: due to ceph-mgr restart, some PG states may not be up to date 2026-02-24T07:43:00.661 INFO:teuthology.orchestra.run.trial035.stderr:dumped all 2026-02-24T07:43:00.701 INFO:teuthology.orchestra.run.trial035.stdout:{"pg_ready":false,"pg_map":{"version":2,"stamp":"2026-02-24T07:42:59.659746+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.000000"},"pg_stats":[{"pgid":"1.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2026-02-24T07:42:59.654328+0000","last_change":"2026-02-24T07:42:59.654328+0000","last_active":"2026-02-24T07:42:59.654328+0000","last_peered":"2026-02-24T07:42:59.654328+0000","last_clean":"2026-02-24T07:42:59.654328+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2026-02-24T07:42:59.654328+0000","last_undegraded":"2026-02-24T07:42:59.654328+0000","last_fullsized":"2026-02-24T07:42:59.654328+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:42:59.654328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:42:59.654328+0000","last_clean_scrub_stamp":"2026-02-24T07:42:59.654328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0}],"osd_stats":[],"pool_statfs":[]}} 2026-02-24T07:43:00.702 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph pg dump --format=json 2026-02-24T07:43:00.822 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:43:00.935 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:43:00.933+0000 7fded8991640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:43:00.935 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:43:00.934+0000 7fded8991640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:43:00.935 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:43:00 trial035 ceph-mon[39284]: from='mgr.14284 10.20.193.35:0/3681373222' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2026-02-24T07:43:00.935 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:43:00 trial035 ceph-mon[39284]: from='mgr.14284 10.20.193.35:0/3681373222' entity='mgr.x' 2026-02-24T07:43:00.935 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:43:00 trial035 ceph-mon[39284]: from='mgr.14284 10.20.193.35:0/3681373222' entity='mgr.x' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2026-02-24T07:43:00.935 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:43:00 trial035 ceph-mon[39284]: from='mgr.14284 10.20.193.35:0/3681373222' entity='mgr.x' cmd={"prefix": "orch get-security-config"} : dispatch 2026-02-24T07:43:00.935 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:43:00 trial035 ceph-mon[39284]: from='mgr.14284 10.20.193.35:0/3681373222' entity='mgr.x' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"} : dispatch 2026-02-24T07:43:00.935 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:43:00 trial035 ceph-mon[39284]: from='mgr.14284 10.20.193.35:0/3681373222' entity='mgr.x' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"} : dispatch 2026-02-24T07:43:00.935 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:43:00 trial035 ceph-mon[39284]: from='mgr.14284 10.20.193.35:0/3681373222' entity='mgr.x' 2026-02-24T07:43:00.935 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:43:00 trial035 ceph-mon[39284]: from='mgr.14284 10.20.193.35:0/3681373222' entity='mgr.x' 2026-02-24T07:43:00.935 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:43:00 trial035 ceph-mon[39284]: from='mgr.14284 10.20.193.35:0/3681373222' entity='mgr.x' 2026-02-24T07:43:00.935 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:43:00 trial035 ceph-mon[39284]: from='mgr.14284 10.20.193.35:0/3681373222' entity='mgr.x' 2026-02-24T07:43:00.935 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:43:00 trial035 ceph-mon[39284]: mgrmap e17: x(active, since 1.00548s) 2026-02-24T07:43:00.935 INFO:journalctl@ceph.mgr.x.trial035.stdout:Feb 24 07:43:00 trial035 ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-mgr-x[39621]: [24/Feb/2026:07:43:00] ENGINE Bus STARTING 2026-02-24T07:43:00.936 INFO:journalctl@ceph.mgr.x.trial035.stdout:Feb 24 07:43:00 trial035 ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-mgr-x[39621]: CherryPy Checker: 2026-02-24T07:43:00.936 INFO:journalctl@ceph.mgr.x.trial035.stdout:Feb 24 07:43:00 trial035 ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-mgr-x[39621]: The Application mounted at '' has an empty config. 2026-02-24T07:43:00.936 INFO:journalctl@ceph.mgr.x.trial035.stdout:Feb 24 07:43:00 trial035 ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-mgr-x[39621]: 2026-02-24T07:43:00.936 INFO:journalctl@ceph.mgr.x.trial035.stdout:Feb 24 07:43:00 trial035 ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-mgr-x[39621]: [24/Feb/2026:07:43:00] ENGINE Serving on http://:::9283 2026-02-24T07:43:00.936 INFO:journalctl@ceph.mgr.x.trial035.stdout:Feb 24 07:43:00 trial035 ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-mgr-x[39621]: [24/Feb/2026:07:43:00] ENGINE Bus STARTED 2026-02-24T07:43:01.043 INFO:teuthology.orchestra.run.trial035.stdout: 2026-02-24T07:43:01.043 INFO:teuthology.orchestra.run.trial035.stderr:dumped all 2026-02-24T07:43:01.085 INFO:teuthology.orchestra.run.trial035.stdout:{"pg_ready":true,"pg_map":{"version":3,"stamp":"2026-02-24T07:43:00.661220+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":3,"acting":3,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":3,"num_osds":4,"num_per_pool_osds":4,"num_per_pool_omap_osds":3,"kb":2936012800,"kb_used":110556,"kb_used_data":2700,"kb_used_omap":45,"kb_used_meta":107730,"kb_avail":2935902244,"statfs":{"total":3006477107200,"available":3006363897856,"internally_reserved":0,"allocated":2764800,"data_stored":2108276,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":46187,"internal_metadata":110316437},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.000000"},"pg_stats":[{"pgid":"1.0","version":"21'92","reported_seq":154,"reported_epoch":29,"state":"active+clean","last_fresh":"2026-02-24T07:42:59.656402+0000","last_change":"2026-02-24T07:42:54.186862+0000","last_active":"2026-02-24T07:42:59.656402+0000","last_peered":"2026-02-24T07:42:59.656402+0000","last_clean":"2026-02-24T07:42:59.656402+0000","last_became_active":"2026-02-24T07:42:54.186719+0000","last_became_peered":"2026-02-24T07:42:54.186719+0000","last_unstale":"2026-02-24T07:42:59.656402+0000","last_undegraded":"2026-02-24T07:42:59.656402+0000","last_fullsized":"2026-02-24T07:42:59.656402+0000","mapping_epoch":27,"log_start":"21'90","ondisk_log_start":"21'90","created":20,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:42:43.640776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:42:43.640776+0000","last_clean_scrub_stamp":"2026-02-24T07:42:43.640776+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":90,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:43:57.783028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":3,"up_from":25,"seq":107374182404,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":27784,"kb_used_data":820,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":733975416,"statfs":{"total":751619276800,"available":751590825984,"internally_reserved":0,"allocated":839680,"data_stored":674661,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6823,"internal_metadata":27583833},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":19,"seq":81604378630,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":27784,"kb_used_data":820,"kb_used_omap":10,"kb_used_meta":26933,"kb_avail":733975416,"statfs":{"total":751619276800,"available":751590825984,"internally_reserved":0,"allocated":839680,"data_stored":674661,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":10725,"internal_metadata":27579931},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542151,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":734003200,"kb_used":27204,"kb_used_data":240,"kb_used_omap":11,"kb_used_meta":26932,"kb_avail":733975996,"statfs":{"total":751619276800,"available":751591419904,"internally_reserved":0,"allocated":245760,"data_stored":84293,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12041,"internal_metadata":27578615},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705674,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":27784,"kb_used_data":820,"kb_used_omap":16,"kb_used_meta":26927,"kb_avail":733975416,"statfs":{"total":751619276800,"available":751590825984,"internally_reserved":0,"allocated":839680,"data_stored":674661,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16598,"internal_metadata":27574058},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-24T07:43:01.086 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2026-02-24T07:43:01.086 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2026-02-24T07:43:01.086 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2026-02-24T07:43:01.086 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph health --format=json 2026-02-24T07:43:01.201 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:43:01.312 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:43:01.311+0000 7f79ef0c6640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:43:01.312 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:43:01.311+0000 7f79ef0c6640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:43:01.450 INFO:teuthology.orchestra.run.trial035.stdout: 2026-02-24T07:43:01.451 INFO:teuthology.orchestra.run.trial035.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2026-02-24T07:43:01.592 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2026-02-24T07:43:01.593 INFO:tasks.cephadm:Setup complete, yielding 2026-02-24T07:43:01.593 INFO:teuthology.run_tasks:Running task cephadm.shell... 2026-02-24T07:43:01.603 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@trial035.front.sepia.ceph.com 2026-02-24T07:43:01.603 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- bash -c 'ceph orch status' 2026-02-24T07:43:01.801 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:43:01.908 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:43:01.906+0000 7ffab9d34640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:43:01.908 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:43:01.906+0000 7ffab9d34640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:43:02.025 INFO:teuthology.orchestra.run.trial035.stdout:Backend: cephadm 2026-02-24T07:43:02.025 INFO:teuthology.orchestra.run.trial035.stdout:Available: Yes 2026-02-24T07:43:02.025 INFO:teuthology.orchestra.run.trial035.stdout:Paused: No 2026-02-24T07:43:02.025 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:43:01 trial035 ceph-mon[39284]: [24/Feb/2026:07:43:00] ENGINE Bus STARTING 2026-02-24T07:43:02.025 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:43:01 trial035 ceph-mon[39284]: [24/Feb/2026:07:43:00] ENGINE Serving on http://10.20.193.35:8765 2026-02-24T07:43:02.026 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:43:01 trial035 ceph-mon[39284]: [24/Feb/2026:07:43:00] ENGINE Serving on https://10.20.193.35:7150 2026-02-24T07:43:02.026 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:43:01 trial035 ceph-mon[39284]: [24/Feb/2026:07:43:00] ENGINE Bus STARTED 2026-02-24T07:43:02.026 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:43:01 trial035 ceph-mon[39284]: [24/Feb/2026:07:43:00] ENGINE Client ('10.20.193.35', 59540) lost — peer dropped the TLS connection suddenly, during handshake: (6, 'TLS/SSL connection has been closed (EOF) (_ssl.c:1147)') 2026-02-24T07:43:02.026 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:43:01 trial035 ceph-mon[39284]: from='client.14294 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2026-02-24T07:43:02.026 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:43:01 trial035 ceph-mon[39284]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "orch get-security-config"}]: dispatch 2026-02-24T07:43:02.026 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:43:01 trial035 ceph-mon[39284]: pgmap v3: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 2.7 TiB / 2.7 TiB avail 2026-02-24T07:43:02.026 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:43:01 trial035 ceph-mon[39284]: from='mgr.14284 10.20.193.35:0/3681373222' entity='mgr.x' 2026-02-24T07:43:02.026 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:43:01 trial035 ceph-mon[39284]: from='mgr.14284 10.20.193.35:0/3681373222' entity='mgr.x' 2026-02-24T07:43:02.026 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:43:01 trial035 ceph-mon[39284]: from='mgr.14284 10.20.193.35:0/3681373222' entity='mgr.x' 2026-02-24T07:43:02.026 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:43:01 trial035 ceph-mon[39284]: from='mgr.14284 10.20.193.35:0/3681373222' entity='mgr.x' 2026-02-24T07:43:02.026 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:43:01 trial035 ceph-mon[39284]: from='mgr.14284 10.20.193.35:0/3681373222' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2026-02-24T07:43:02.027 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:43:01 trial035 ceph-mon[39284]: from='mgr.14284 10.20.193.35:0/3681373222' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2026-02-24T07:43:02.027 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:43:01 trial035 ceph-mon[39284]: Adjusting osd_memory_target on trial091 to 44071M 2026-02-24T07:43:02.027 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:43:01 trial035 ceph-mon[39284]: from='mgr.14284 10.20.193.35:0/3681373222' entity='mgr.x' 2026-02-24T07:43:02.027 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:43:01 trial035 ceph-mon[39284]: from='mgr.14284 10.20.193.35:0/3681373222' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-24T07:43:02.027 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:43:01 trial035 ceph-mon[39284]: from='mgr.14284 10.20.193.35:0/3681373222' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2026-02-24T07:43:02.027 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:43:01 trial035 ceph-mon[39284]: Updating trial035:/etc/ceph/ceph.conf 2026-02-24T07:43:02.027 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:43:01 trial035 ceph-mon[39284]: Updating trial091:/etc/ceph/ceph.conf 2026-02-24T07:43:02.027 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:43:01 trial035 ceph-mon[39284]: from='mgr.14284 10.20.193.35:0/3681373222' entity='mgr.x' 2026-02-24T07:43:02.027 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:43:01 trial035 ceph-mon[39284]: from='mgr.14284 10.20.193.35:0/3681373222' entity='mgr.x' 2026-02-24T07:43:02.027 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:43:01 trial035 ceph-mon[39284]: from='mgr.14284 10.20.193.35:0/3681373222' entity='mgr.x' 2026-02-24T07:43:02.027 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:43:01 trial035 ceph-mon[39284]: from='mgr.14284 10.20.193.35:0/3681373222' entity='mgr.x' 2026-02-24T07:43:02.028 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:43:01 trial035 ceph-mon[39284]: from='mgr.14284 10.20.193.35:0/3681373222' entity='mgr.x' 2026-02-24T07:43:02.028 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:43:01 trial035 ceph-mon[39284]: from='mgr.14284 10.20.193.35:0/3681373222' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2026-02-24T07:43:02.028 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:43:01 trial035 ceph-mon[39284]: from='mgr.14284 10.20.193.35:0/3681373222' entity='mgr.x' 2026-02-24T07:43:02.028 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:43:01 trial035 ceph-mon[39284]: from='mgr.14284 10.20.193.35:0/3681373222' entity='mgr.x' 2026-02-24T07:43:02.028 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:43:01 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/1593745698' entity='client.admin' cmd={"prefix": "health", "format": "json"} : dispatch 2026-02-24T07:43:02.061 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- bash -c 'ceph orch ps' 2026-02-24T07:43:02.177 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:43:02.294 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:43:02.293+0000 7fbb1bfff640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:43:02.294 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:43:02.293+0000 7fbb1bfff640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:43:02.415 INFO:teuthology.orchestra.run.trial035.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2026-02-24T07:43:02.415 INFO:teuthology.orchestra.run.trial035.stdout:mgr.x trial035 *:9283,8765 running (77s) 2s ago 77s 509M - 20.3.0-5440-gff63f3d9 3992f70354a3 0241ef9fa2ef 2026-02-24T07:43:02.415 INFO:teuthology.orchestra.run.trial035.stdout:mon.a trial035 running (78s) 2s ago 79s 47.1M 2048M 20.3.0-5440-gff63f3d9 3992f70354a3 ce32af800c61 2026-02-24T07:43:02.415 INFO:teuthology.orchestra.run.trial035.stdout:osd.0 trial035 running (40s) 2s ago 41s 41.0M 41.0G 20.3.0-5440-gff63f3d9 3992f70354a3 1a909b31430d 2026-02-24T07:43:02.416 INFO:teuthology.orchestra.run.trial035.stdout:osd.1 trial035 running (32s) 2s ago 32s 40.0M 41.0G 20.3.0-5440-gff63f3d9 3992f70354a3 24045a3a2d0a 2026-02-24T07:43:02.416 INFO:teuthology.orchestra.run.trial035.stdout:osd.2 trial091 running (23s) 2s ago 25s 41.4M 43.0G 20.3.0-5440-gff63f3d9 3992f70354a3 8a0910f7b6fa 2026-02-24T07:43:02.416 INFO:teuthology.orchestra.run.trial035.stdout:osd.3 trial091 running (14s) 2s ago 16s 38.8M 43.0G 20.3.0-5440-gff63f3d9 3992f70354a3 4150509624bf 2026-02-24T07:43:02.416 INFO:teuthology.orchestra.run.trial035.stdout:prometheus.a trial091 *:9095 running (6s) 2s ago 6s 29.9M - 3.6.0 4fcecf061b74 06e50658ff3d 2026-02-24T07:43:02.456 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- bash -c 'ceph orch ls' 2026-02-24T07:43:02.575 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:43:02.680 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:43:02.679+0000 7fd98118c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:43:02.680 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:43:02.679+0000 7fd98118c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:43:02.794 INFO:teuthology.orchestra.run.trial035.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2026-02-24T07:43:02.795 INFO:teuthology.orchestra.run.trial035.stdout:mgr 1/1 2s ago 50s trial035=x;count:1 2026-02-24T07:43:02.795 INFO:teuthology.orchestra.run.trial035.stdout:mon 1/1 2s ago 51s trial035:10.20.193.35=a;count:1 2026-02-24T07:43:02.795 INFO:teuthology.orchestra.run.trial035.stdout:node-exporter ?:9100 0/2 - 9s trial035=a;trial091=b;count:2 2026-02-24T07:43:02.795 INFO:teuthology.orchestra.run.trial035.stdout:osd.default 4 2s ago 21s trial091 2026-02-24T07:43:02.795 INFO:teuthology.orchestra.run.trial035.stdout:prometheus ?:9095 1/1 2s ago 10s trial091=a;count:1 2026-02-24T07:43:02.830 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- bash -c 'ceph orch host ls' 2026-02-24T07:43:02.851 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:43:02 trial035 ceph-mon[39284]: Updating trial091:/var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/config/ceph.conf 2026-02-24T07:43:02.852 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:43:02 trial035 ceph-mon[39284]: Updating trial035:/var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/config/ceph.conf 2026-02-24T07:43:02.852 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:43:02 trial035 ceph-mon[39284]: Updating trial091:/etc/ceph/ceph.client.admin.keyring 2026-02-24T07:43:02.852 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:43:02 trial035 ceph-mon[39284]: Updating trial035:/etc/ceph/ceph.client.admin.keyring 2026-02-24T07:43:02.852 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:43:02 trial035 ceph-mon[39284]: from='client.14302 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2026-02-24T07:43:02.852 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:43:02 trial035 ceph-mon[39284]: Updating trial091:/var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/config/ceph.client.admin.keyring 2026-02-24T07:43:02.852 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:43:02 trial035 ceph-mon[39284]: Updating trial035:/var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/config/ceph.client.admin.keyring 2026-02-24T07:43:02.852 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:43:02 trial035 ceph-mon[39284]: Deploying daemon node-exporter.a on trial035 2026-02-24T07:43:02.852 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:43:02 trial035 ceph-mon[39284]: pgmap v4: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 2.7 TiB / 2.7 TiB avail 2026-02-24T07:43:02.852 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:43:02 trial035 ceph-mon[39284]: mgrmap e18: x(active, since 2s) 2026-02-24T07:43:02.853 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:43:02 trial035 ceph-mon[39284]: from='mgr.14284 10.20.193.35:0/3681373222' entity='mgr.x' 2026-02-24T07:43:02.853 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:43:02 trial035 ceph-mon[39284]: from='mgr.14284 10.20.193.35:0/3681373222' entity='mgr.x' 2026-02-24T07:43:02.853 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:43:02 trial035 ceph-mon[39284]: from='mgr.14284 10.20.193.35:0/3681373222' entity='mgr.x' 2026-02-24T07:43:02.945 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:43:03.049 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:43:03.048+0000 7fa3f1280640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:43:03.049 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:43:03.048+0000 7fa3f1280640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:43:03.163 INFO:teuthology.orchestra.run.trial035.stdout:HOST ADDR LABELS STATUS 2026-02-24T07:43:03.163 INFO:teuthology.orchestra.run.trial035.stdout:trial035 10.20.193.35 2026-02-24T07:43:03.163 INFO:teuthology.orchestra.run.trial035.stdout:trial091 10.20.193.91 2026-02-24T07:43:03.163 INFO:teuthology.orchestra.run.trial035.stdout:2 hosts in cluster 2026-02-24T07:43:03.200 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- bash -c 'ceph orch device ls' 2026-02-24T07:43:03.355 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:43:03.356 INFO:journalctl@ceph.node-exporter.b.trial091.stdout:Feb 24 07:43:02 trial091 bash[65153]: Trying to pull quay.io/prometheus/node-exporter:v1.9.1... 2026-02-24T07:43:03.451 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:43:03.450+0000 7f49f7ae8640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:43:03.451 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:43:03.450+0000 7f49f7ae8640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:43:03.563 INFO:teuthology.orchestra.run.trial035.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2026-02-24T07:43:03.563 INFO:teuthology.orchestra.run.trial035.stdout:trial035 /dev/nvme0n1 ssd MTFDKCC1T9TGP-1BK1JABYY_092510AB6BD1 1788G No 29s ago Has a FileSystem, LVM detected 2026-02-24T07:43:03.563 INFO:teuthology.orchestra.run.trial035.stdout:trial035 /dev/nvme2n1 ssd MTFDKCC1T9TGP-1BK1JABYY_092510AB6C00 1788G No 29s ago Has a FileSystem, LVM detected 2026-02-24T07:43:03.563 INFO:teuthology.orchestra.run.trial035.stdout:trial091 /dev/nvme1n1 ssd MTFDKCC1T9TGP-1BK1JABYY_092510AB6CCA 1788G No 9s ago Has a FileSystem, LVM detected 2026-02-24T07:43:03.563 INFO:teuthology.orchestra.run.trial035.stdout:trial091 /dev/nvme2n1 ssd MTFDKCC1T9TGP-1BK1JABYY_012510B0022B 1788G No 9s ago Has a FileSystem, LVM detected 2026-02-24T07:43:03.601 INFO:teuthology.run_tasks:Running task thrashosds... 2026-02-24T07:43:03.616 INFO:tasks.thrashosds:config is {'ceph_objectstore_tool': False, 'chance_down': 0, 'chance_inject_pause_short': 0, 'chance_pgnum_grow': 2, 'chance_pgpnum_fix': 1, 'chance_reset_purged_snaps_last': 0, 'chance_thrash_cluster_full': 0, 'chance_thrash_pg_upmap': 0, 'dump_ops_enable': False, 'min_in': 2, 'noscrub_toggle_delay': 0, 'reweight_osd': 0, 'sighup_delay': 0, 'thrash_primary_affinity': False, 'timeout': 2400, 'optrack_toggle_delay': 2.0, 'random_eio': 0.0} 2026-02-24T07:43:03.617 INFO:tasks.thrashosds:overrides is {'admin_socket': {'branch': 'main'}, 'ceph': {'conf': {'client.0': {'debug ms': 1, 'debug objecter': 20, 'debug rados': 20}, 'global': {'crimson_allow_pg_split': False, 'enable experimental unrecoverable data corrupting features': 'crimson', 'ms client mode': 'crc', 'ms cluster mode': 'crc', 'ms mon client mode': 'crc', 'ms mon cluster mode': 'crc', 'ms mon service mode': 'crc', 'ms service mode': 'crc', 'osd_max_pg_log_entries': 2, 'osd_min_pg_log_entries': 1, 'osd_pg_log_trim_min': 0, 'osd_pool_default_min_size': 2, 'osd_pool_default_size': 3}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon min osdmap epochs': 50, 'mon osdmap full prune interval': 2, 'mon osdmap full prune min': 15, 'mon osdmap full prune txsize': 2, 'osd pool default crimson': True, 'paxos service trim min': 10}, 'osd': {'crimson cpu num': 4, 'crimson osd obc lru size': 10, 'debug ms': 20, 'debug osd': 20, 'debug seastore': 20, 'debug seastore backref': 20, 'debug seastore cache': 20, 'debug seastore cleaner': 20, 'debug seastore device': 20, 'debug seastore epm': 20, 'debug seastore fixedkv tree': 20, 'debug seastore journal': 20, 'debug seastore lba': 20, 'debug seastore odata': 20, 'debug seastore omap': 20, 'debug seastore onode': 20, 'debug seastore t': 20, 'debug seastore tm': 20, 'osd blocked scrub grace period': 3600, 'osd debug reject backfill probability': 0.3, 'osd delete sleep': 1, 'osd max backfills': 3, 'osd max markdown count': 1000, 'osd objectstore': 'seastore', 'osd scrub max interval': 120, 'osd scrub min interval': 60, 'osd shutdown pgref assert': True, 'osd snap trim sleep': 2, 'seastore cachepin size pershard': '64M', 'seastore main device type': 'RANDOM_BLOCK_SSD', 'seastore max concurrent transactions': 8}}, 'crimson_compat': True, 'flavor': 'debug', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'but it is still running', 'objects unfound and apparently lost', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', 'SLOW_OPS', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'timeout on replica', 'late reservation from', 'MON_DOWN', 'OSDMAP_FLAGS', 'OSD_DOWN', 'PG_DEGRADED', 'PG_AVAILABILITY', 'POOL_APP_NOT_ENABLED', 'mons down', 'mon down', 'out of quorum', 'noscrub', 'nodeep-scrub', 'Degraded data redundancy', 'is down', 'osds down', 'pg .*? is .*?degraded.*?, acting', 'pg .*? is stuck', 'pg degraded', 'PG_BACKFILL_FULL', 'Low space hindering backfill .*? backfill_toofull', 'OSD_ROOT_DOWN', 'pgs degraded', 'pgs undersized', 'is active.*backfill_toofull.*'], 'sha1': 'ff63f3d997e38f69382fd23ac0206f07f7b05017'}, 'ceph-deploy': {'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}}}, 'install': {'ceph': {'flavor': 'debug', 'sha1': 'ff63f3d997e38f69382fd23ac0206f07f7b05017'}}, 'thrashosds': {'min_in': 4}, 'workunit': {'branch': 'wip-shraddhaag-enable-cephadm-tests', 'env': {'CRIMSON_COMPAT': '1'}, 'sha1': 'a78040ddef3ba5a489697ca2fe95df59a7d8bd08'}} 2026-02-24T07:43:03.617 INFO:tasks.thrashosds:config is {'ceph_objectstore_tool': False, 'chance_down': 0, 'chance_inject_pause_short': 0, 'chance_pgnum_grow': 2, 'chance_pgpnum_fix': 1, 'chance_reset_purged_snaps_last': 0, 'chance_thrash_cluster_full': 0, 'chance_thrash_pg_upmap': 0, 'dump_ops_enable': False, 'min_in': 4, 'noscrub_toggle_delay': 0, 'reweight_osd': 0, 'sighup_delay': 0, 'thrash_primary_affinity': False, 'timeout': 2400, 'optrack_toggle_delay': 2.0, 'random_eio': 0.0} 2026-02-24T07:43:03.617 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph config set mgr mgr_debug_aggressive_pg_num_changes true 2026-02-24T07:43:03.732 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:43:03.848 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:43:03.847+0000 7f2af4b1a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:43:03.849 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:43:03.847+0000 7f2af4b1a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:43:03.857 INFO:journalctl@ceph.node-exporter.b.trial091.stdout:Feb 24 07:43:03 trial091 bash[65153]: Getting image source signatures 2026-02-24T07:43:03.857 INFO:journalctl@ceph.node-exporter.b.trial091.stdout:Feb 24 07:43:03 trial091 bash[65153]: Copying blob sha256:c6e37428e3b36731b2b7a99979e0dbb27ce0849df0c23fb2b73e99bdbcd7ca33 2026-02-24T07:43:03.857 INFO:journalctl@ceph.node-exporter.b.trial091.stdout:Feb 24 07:43:03 trial091 bash[65153]: Copying blob sha256:1617e25568b2231fdd0d5caff63b06f6f7738d8d961f031c80e47d35aaec9733 2026-02-24T07:43:03.857 INFO:journalctl@ceph.node-exporter.b.trial091.stdout:Feb 24 07:43:03 trial091 bash[65153]: Copying blob sha256:9fa9226be034e47923c0457d916aa68474cdfb23af8d4525e9baeebc4760977a 2026-02-24T07:43:03.857 INFO:journalctl@ceph.node-exporter.b.trial091.stdout:Feb 24 07:43:03 trial091 bash[65153]: Copying config sha256:255ec253085fb105696533b31864a5e3b64f1f1e2147b5ec368100cbf3ec9b80 2026-02-24T07:43:03.857 INFO:journalctl@ceph.node-exporter.b.trial091.stdout:Feb 24 07:43:03 trial091 bash[65153]: Writing manifest to image destination 2026-02-24T07:43:03.857 INFO:journalctl@ceph.node-exporter.b.trial091.stdout:Feb 24 07:43:03 trial091 podman[65153]: 2026-02-24 07:43:03.569864658 +0000 UTC m=+0.617147661 container create 42595c8ca7ba5feb96b0769ff36383d32706295000e421cf541c9d6afb4eefaa (image=quay.io/prometheus/node-exporter:v1.9.1, name=ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-node-exporter-b, maintainer=The Prometheus Authors ) 2026-02-24T07:43:03.858 INFO:journalctl@ceph.node-exporter.b.trial091.stdout:Feb 24 07:43:03 trial091 podman[65153]: 2026-02-24 07:43:03.594222051 +0000 UTC m=+0.641505054 container init 42595c8ca7ba5feb96b0769ff36383d32706295000e421cf541c9d6afb4eefaa (image=quay.io/prometheus/node-exporter:v1.9.1, name=ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-node-exporter-b, maintainer=The Prometheus Authors ) 2026-02-24T07:43:03.858 INFO:journalctl@ceph.node-exporter.b.trial091.stdout:Feb 24 07:43:03 trial091 podman[65153]: 2026-02-24 07:43:03.596447934 +0000 UTC m=+0.643730937 container start 42595c8ca7ba5feb96b0769ff36383d32706295000e421cf541c9d6afb4eefaa (image=quay.io/prometheus/node-exporter:v1.9.1, name=ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-node-exporter-b, maintainer=The Prometheus Authors ) 2026-02-24T07:43:03.858 INFO:journalctl@ceph.node-exporter.b.trial091.stdout:Feb 24 07:43:03 trial091 bash[65153]: 42595c8ca7ba5feb96b0769ff36383d32706295000e421cf541c9d6afb4eefaa 2026-02-24T07:43:03.858 INFO:journalctl@ceph.node-exporter.b.trial091.stdout:Feb 24 07:43:03 trial091 podman[65153]: 2026-02-24 07:43:03.564641541 +0000 UTC m=+0.611924554 image pull 255ec253085fb105696533b31864a5e3b64f1f1e2147b5ec368100cbf3ec9b80 quay.io/prometheus/node-exporter:v1.9.1 2026-02-24T07:43:03.858 INFO:journalctl@ceph.node-exporter.b.trial091.stdout:Feb 24 07:43:03 trial091 ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-node-exporter-b[65204]: time=2026-02-24T07:43:03.599Z level=INFO source=node_exporter.go:216 msg="Starting node_exporter" version="(version=1.9.1, branch=HEAD, revision=f2ec547b49af53815038a50265aa2adcd1275959)" 2026-02-24T07:43:03.858 INFO:journalctl@ceph.node-exporter.b.trial091.stdout:Feb 24 07:43:03 trial091 ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-node-exporter-b[65204]: time=2026-02-24T07:43:03.599Z level=INFO source=node_exporter.go:217 msg="Build context" build_context="(go=go1.23.7, platform=linux/amd64, user=root@7023beaa563a, date=20250401-15:19:01, tags=unknown)" 2026-02-24T07:43:03.858 INFO:journalctl@ceph.node-exporter.b.trial091.stdout:Feb 24 07:43:03 trial091 systemd[1]: Started Ceph node-exporter.b for 3a5983d2-1154-11f1-9c05-d404e6e7d460. 2026-02-24T07:43:03.858 INFO:journalctl@ceph.node-exporter.b.trial091.stdout:Feb 24 07:43:03 trial091 ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-node-exporter-b[65204]: time=2026-02-24T07:43:03.600Z level=INFO source=diskstats_common.go:110 msg="Parsed flag --collector.diskstats.device-exclude" collector=diskstats flag=^(z?ram|loop|fd|(h|s|v|xv)d[a-z]|nvme\d+n\d+p)\d+$ 2026-02-24T07:43:03.858 INFO:journalctl@ceph.node-exporter.b.trial091.stdout:Feb 24 07:43:03 trial091 ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-node-exporter-b[65204]: time=2026-02-24T07:43:03.600Z level=ERROR source=diskstats_linux.go:264 msg="Failed to open directory, disabling udev device properties" collector=diskstats path=/run/udev/data 2026-02-24T07:43:03.858 INFO:journalctl@ceph.node-exporter.b.trial091.stdout:Feb 24 07:43:03 trial091 ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-node-exporter-b[65204]: time=2026-02-24T07:43:03.600Z level=INFO source=filesystem_common.go:265 msg="Parsed flag --collector.filesystem.mount-points-exclude" collector=filesystem flag=^/(dev|proc|run/credentials/.+|sys|var/lib/docker/.+|var/lib/containers/storage/.+)($|/) 2026-02-24T07:43:03.859 INFO:journalctl@ceph.node-exporter.b.trial091.stdout:Feb 24 07:43:03 trial091 ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-node-exporter-b[65204]: time=2026-02-24T07:43:03.600Z level=INFO source=filesystem_common.go:294 msg="Parsed flag --collector.filesystem.fs-types-exclude" collector=filesystem flag=^(autofs|binfmt_misc|bpf|cgroup2?|configfs|debugfs|devpts|devtmpfs|fusectl|hugetlbfs|iso9660|mqueue|nsfs|overlay|proc|procfs|pstore|rpc_pipefs|securityfs|selinuxfs|squashfs|sysfs|tracefs)$ 2026-02-24T07:43:03.859 INFO:journalctl@ceph.node-exporter.b.trial091.stdout:Feb 24 07:43:03 trial091 ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-node-exporter-b[65204]: time=2026-02-24T07:43:03.600Z level=INFO source=node_exporter.go:135 msg="Enabled collectors" 2026-02-24T07:43:03.859 INFO:journalctl@ceph.node-exporter.b.trial091.stdout:Feb 24 07:43:03 trial091 ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-node-exporter-b[65204]: time=2026-02-24T07:43:03.600Z level=INFO source=node_exporter.go:141 msg=arp 2026-02-24T07:43:03.859 INFO:journalctl@ceph.node-exporter.b.trial091.stdout:Feb 24 07:43:03 trial091 ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-node-exporter-b[65204]: time=2026-02-24T07:43:03.600Z level=INFO source=node_exporter.go:141 msg=bcache 2026-02-24T07:43:03.859 INFO:journalctl@ceph.node-exporter.b.trial091.stdout:Feb 24 07:43:03 trial091 ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-node-exporter-b[65204]: time=2026-02-24T07:43:03.600Z level=INFO source=node_exporter.go:141 msg=bonding 2026-02-24T07:43:03.859 INFO:journalctl@ceph.node-exporter.b.trial091.stdout:Feb 24 07:43:03 trial091 ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-node-exporter-b[65204]: time=2026-02-24T07:43:03.600Z level=INFO source=node_exporter.go:141 msg=btrfs 2026-02-24T07:43:03.859 INFO:journalctl@ceph.node-exporter.b.trial091.stdout:Feb 24 07:43:03 trial091 ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-node-exporter-b[65204]: time=2026-02-24T07:43:03.600Z level=INFO source=node_exporter.go:141 msg=conntrack 2026-02-24T07:43:03.859 INFO:journalctl@ceph.node-exporter.b.trial091.stdout:Feb 24 07:43:03 trial091 ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-node-exporter-b[65204]: time=2026-02-24T07:43:03.600Z level=INFO source=node_exporter.go:141 msg=cpu 2026-02-24T07:43:03.859 INFO:journalctl@ceph.node-exporter.b.trial091.stdout:Feb 24 07:43:03 trial091 ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-node-exporter-b[65204]: time=2026-02-24T07:43:03.600Z level=INFO source=node_exporter.go:141 msg=cpufreq 2026-02-24T07:43:03.859 INFO:journalctl@ceph.node-exporter.b.trial091.stdout:Feb 24 07:43:03 trial091 ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-node-exporter-b[65204]: time=2026-02-24T07:43:03.600Z level=INFO source=node_exporter.go:141 msg=diskstats 2026-02-24T07:43:03.859 INFO:journalctl@ceph.node-exporter.b.trial091.stdout:Feb 24 07:43:03 trial091 ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-node-exporter-b[65204]: time=2026-02-24T07:43:03.600Z level=INFO source=node_exporter.go:141 msg=dmi 2026-02-24T07:43:03.859 INFO:journalctl@ceph.node-exporter.b.trial091.stdout:Feb 24 07:43:03 trial091 ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-node-exporter-b[65204]: time=2026-02-24T07:43:03.600Z level=INFO source=node_exporter.go:141 msg=edac 2026-02-24T07:43:03.860 INFO:journalctl@ceph.node-exporter.b.trial091.stdout:Feb 24 07:43:03 trial091 ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-node-exporter-b[65204]: time=2026-02-24T07:43:03.600Z level=INFO source=node_exporter.go:141 msg=entropy 2026-02-24T07:43:03.860 INFO:journalctl@ceph.node-exporter.b.trial091.stdout:Feb 24 07:43:03 trial091 ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-node-exporter-b[65204]: time=2026-02-24T07:43:03.600Z level=INFO source=node_exporter.go:141 msg=fibrechannel 2026-02-24T07:43:03.860 INFO:journalctl@ceph.node-exporter.b.trial091.stdout:Feb 24 07:43:03 trial091 ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-node-exporter-b[65204]: time=2026-02-24T07:43:03.600Z level=INFO source=node_exporter.go:141 msg=filefd 2026-02-24T07:43:03.860 INFO:journalctl@ceph.node-exporter.b.trial091.stdout:Feb 24 07:43:03 trial091 ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-node-exporter-b[65204]: time=2026-02-24T07:43:03.600Z level=INFO source=node_exporter.go:141 msg=filesystem 2026-02-24T07:43:03.860 INFO:journalctl@ceph.node-exporter.b.trial091.stdout:Feb 24 07:43:03 trial091 ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-node-exporter-b[65204]: time=2026-02-24T07:43:03.600Z level=INFO source=node_exporter.go:141 msg=hwmon 2026-02-24T07:43:03.860 INFO:journalctl@ceph.node-exporter.b.trial091.stdout:Feb 24 07:43:03 trial091 ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-node-exporter-b[65204]: time=2026-02-24T07:43:03.600Z level=INFO source=node_exporter.go:141 msg=infiniband 2026-02-24T07:43:03.860 INFO:journalctl@ceph.node-exporter.b.trial091.stdout:Feb 24 07:43:03 trial091 ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-node-exporter-b[65204]: time=2026-02-24T07:43:03.601Z level=INFO source=node_exporter.go:141 msg=ipvs 2026-02-24T07:43:03.860 INFO:journalctl@ceph.node-exporter.b.trial091.stdout:Feb 24 07:43:03 trial091 ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-node-exporter-b[65204]: time=2026-02-24T07:43:03.601Z level=INFO source=node_exporter.go:141 msg=loadavg 2026-02-24T07:43:03.860 INFO:journalctl@ceph.node-exporter.b.trial091.stdout:Feb 24 07:43:03 trial091 ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-node-exporter-b[65204]: time=2026-02-24T07:43:03.601Z level=INFO source=node_exporter.go:141 msg=mdadm 2026-02-24T07:43:03.860 INFO:journalctl@ceph.node-exporter.b.trial091.stdout:Feb 24 07:43:03 trial091 ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-node-exporter-b[65204]: time=2026-02-24T07:43:03.601Z level=INFO source=node_exporter.go:141 msg=meminfo 2026-02-24T07:43:03.860 INFO:journalctl@ceph.node-exporter.b.trial091.stdout:Feb 24 07:43:03 trial091 ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-node-exporter-b[65204]: time=2026-02-24T07:43:03.601Z level=INFO source=node_exporter.go:141 msg=netclass 2026-02-24T07:43:03.860 INFO:journalctl@ceph.node-exporter.b.trial091.stdout:Feb 24 07:43:03 trial091 ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-node-exporter-b[65204]: time=2026-02-24T07:43:03.601Z level=INFO source=node_exporter.go:141 msg=netdev 2026-02-24T07:43:03.860 INFO:journalctl@ceph.node-exporter.b.trial091.stdout:Feb 24 07:43:03 trial091 ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-node-exporter-b[65204]: time=2026-02-24T07:43:03.601Z level=INFO source=node_exporter.go:141 msg=netstat 2026-02-24T07:43:03.861 INFO:journalctl@ceph.node-exporter.b.trial091.stdout:Feb 24 07:43:03 trial091 ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-node-exporter-b[65204]: time=2026-02-24T07:43:03.601Z level=INFO source=node_exporter.go:141 msg=nfs 2026-02-24T07:43:03.861 INFO:journalctl@ceph.node-exporter.b.trial091.stdout:Feb 24 07:43:03 trial091 ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-node-exporter-b[65204]: time=2026-02-24T07:43:03.601Z level=INFO source=node_exporter.go:141 msg=nfsd 2026-02-24T07:43:03.861 INFO:journalctl@ceph.node-exporter.b.trial091.stdout:Feb 24 07:43:03 trial091 ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-node-exporter-b[65204]: time=2026-02-24T07:43:03.601Z level=INFO source=node_exporter.go:141 msg=nvme 2026-02-24T07:43:03.861 INFO:journalctl@ceph.node-exporter.b.trial091.stdout:Feb 24 07:43:03 trial091 ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-node-exporter-b[65204]: time=2026-02-24T07:43:03.601Z level=INFO source=node_exporter.go:141 msg=os 2026-02-24T07:43:03.861 INFO:journalctl@ceph.node-exporter.b.trial091.stdout:Feb 24 07:43:03 trial091 ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-node-exporter-b[65204]: time=2026-02-24T07:43:03.601Z level=INFO source=node_exporter.go:141 msg=powersupplyclass 2026-02-24T07:43:03.861 INFO:journalctl@ceph.node-exporter.b.trial091.stdout:Feb 24 07:43:03 trial091 ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-node-exporter-b[65204]: time=2026-02-24T07:43:03.601Z level=INFO source=node_exporter.go:141 msg=pressure 2026-02-24T07:43:03.861 INFO:journalctl@ceph.node-exporter.b.trial091.stdout:Feb 24 07:43:03 trial091 ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-node-exporter-b[65204]: time=2026-02-24T07:43:03.601Z level=INFO source=node_exporter.go:141 msg=rapl 2026-02-24T07:43:03.861 INFO:journalctl@ceph.node-exporter.b.trial091.stdout:Feb 24 07:43:03 trial091 ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-node-exporter-b[65204]: time=2026-02-24T07:43:03.601Z level=INFO source=node_exporter.go:141 msg=schedstat 2026-02-24T07:43:03.861 INFO:journalctl@ceph.node-exporter.b.trial091.stdout:Feb 24 07:43:03 trial091 ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-node-exporter-b[65204]: time=2026-02-24T07:43:03.601Z level=INFO source=node_exporter.go:141 msg=selinux 2026-02-24T07:43:03.861 INFO:journalctl@ceph.node-exporter.b.trial091.stdout:Feb 24 07:43:03 trial091 ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-node-exporter-b[65204]: time=2026-02-24T07:43:03.601Z level=INFO source=node_exporter.go:141 msg=sockstat 2026-02-24T07:43:03.861 INFO:journalctl@ceph.node-exporter.b.trial091.stdout:Feb 24 07:43:03 trial091 ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-node-exporter-b[65204]: time=2026-02-24T07:43:03.601Z level=INFO source=node_exporter.go:141 msg=softnet 2026-02-24T07:43:03.862 INFO:journalctl@ceph.node-exporter.b.trial091.stdout:Feb 24 07:43:03 trial091 ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-node-exporter-b[65204]: time=2026-02-24T07:43:03.601Z level=INFO source=node_exporter.go:141 msg=stat 2026-02-24T07:43:03.862 INFO:journalctl@ceph.node-exporter.b.trial091.stdout:Feb 24 07:43:03 trial091 ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-node-exporter-b[65204]: time=2026-02-24T07:43:03.601Z level=INFO source=node_exporter.go:141 msg=tapestats 2026-02-24T07:43:03.862 INFO:journalctl@ceph.node-exporter.b.trial091.stdout:Feb 24 07:43:03 trial091 ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-node-exporter-b[65204]: time=2026-02-24T07:43:03.601Z level=INFO source=node_exporter.go:141 msg=textfile 2026-02-24T07:43:03.862 INFO:journalctl@ceph.node-exporter.b.trial091.stdout:Feb 24 07:43:03 trial091 ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-node-exporter-b[65204]: time=2026-02-24T07:43:03.601Z level=INFO source=node_exporter.go:141 msg=thermal_zone 2026-02-24T07:43:03.862 INFO:journalctl@ceph.node-exporter.b.trial091.stdout:Feb 24 07:43:03 trial091 ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-node-exporter-b[65204]: time=2026-02-24T07:43:03.601Z level=INFO source=node_exporter.go:141 msg=time 2026-02-24T07:43:03.862 INFO:journalctl@ceph.node-exporter.b.trial091.stdout:Feb 24 07:43:03 trial091 ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-node-exporter-b[65204]: time=2026-02-24T07:43:03.601Z level=INFO source=node_exporter.go:141 msg=udp_queues 2026-02-24T07:43:03.862 INFO:journalctl@ceph.node-exporter.b.trial091.stdout:Feb 24 07:43:03 trial091 ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-node-exporter-b[65204]: time=2026-02-24T07:43:03.601Z level=INFO source=node_exporter.go:141 msg=uname 2026-02-24T07:43:03.862 INFO:journalctl@ceph.node-exporter.b.trial091.stdout:Feb 24 07:43:03 trial091 ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-node-exporter-b[65204]: time=2026-02-24T07:43:03.601Z level=INFO source=node_exporter.go:141 msg=vmstat 2026-02-24T07:43:03.862 INFO:journalctl@ceph.node-exporter.b.trial091.stdout:Feb 24 07:43:03 trial091 ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-node-exporter-b[65204]: time=2026-02-24T07:43:03.601Z level=INFO source=node_exporter.go:141 msg=watchdog 2026-02-24T07:43:03.862 INFO:journalctl@ceph.node-exporter.b.trial091.stdout:Feb 24 07:43:03 trial091 ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-node-exporter-b[65204]: time=2026-02-24T07:43:03.601Z level=INFO source=node_exporter.go:141 msg=xfs 2026-02-24T07:43:03.862 INFO:journalctl@ceph.node-exporter.b.trial091.stdout:Feb 24 07:43:03 trial091 ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-node-exporter-b[65204]: time=2026-02-24T07:43:03.601Z level=INFO source=node_exporter.go:141 msg=zfs 2026-02-24T07:43:03.862 INFO:journalctl@ceph.node-exporter.b.trial091.stdout:Feb 24 07:43:03 trial091 ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-node-exporter-b[65204]: time=2026-02-24T07:43:03.601Z level=INFO source=tls_config.go:347 msg="Listening on" address=[::]:9100 2026-02-24T07:43:03.862 INFO:journalctl@ceph.node-exporter.b.trial091.stdout:Feb 24 07:43:03 trial091 ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-node-exporter-b[65204]: time=2026-02-24T07:43:03.601Z level=INFO source=tls_config.go:350 msg="TLS is disabled." http2=false address=[::]:9100 2026-02-24T07:43:04.002 INFO:tasks.thrashosds:Beginning thrashosds... 2026-02-24T07:43:04.002 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2026-02-24T07:43:04.003 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph pg dump --format=json 2026-02-24T07:43:04.025 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:43:03 trial035 ceph-mon[39284]: from='client.14308 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2026-02-24T07:43:04.025 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:43:03 trial035 ceph-mon[39284]: from='client.14310 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2026-02-24T07:43:04.025 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:43:03 trial035 ceph-mon[39284]: Deploying daemon node-exporter.b on trial091 2026-02-24T07:43:04.026 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:43:03 trial035 ceph-mon[39284]: from='client.14312 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2026-02-24T07:43:04.026 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:43:03 trial035 ceph-mon[39284]: from='mgr.14284 10.20.193.35:0/3681373222' entity='mgr.x' 2026-02-24T07:43:04.026 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:43:03 trial035 ceph-mon[39284]: from='mgr.14284 10.20.193.35:0/3681373222' entity='mgr.x' 2026-02-24T07:43:04.026 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:43:03 trial035 ceph-mon[39284]: from='mgr.14284 10.20.193.35:0/3681373222' entity='mgr.x' 2026-02-24T07:43:04.026 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:43:03 trial035 ceph-mon[39284]: from='mgr.14284 10.20.193.35:0/3681373222' entity='mgr.x' 2026-02-24T07:43:04.026 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:43:03 trial035 ceph-mon[39284]: from='mgr.14284 10.20.193.35:0/3681373222' entity='mgr.x' 2026-02-24T07:43:04.026 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:43:03 trial035 ceph-mon[39284]: from='mgr.14284 10.20.193.35:0/3681373222' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2026-02-24T07:43:04.121 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:43:04.238 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:43:04.237+0000 7fc64fbc8640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:43:04.238 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:43:04.237+0000 7fc64fbc8640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:43:04.277 INFO:journalctl@ceph.mgr.x.trial035.stdout:Feb 24 07:43:04 trial035 ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-mgr-x[39621]: [24/Feb/2026:07:43:04] ENGINE Bus STOPPING 2026-02-24T07:43:04.277 INFO:journalctl@ceph.mgr.x.trial035.stdout:Feb 24 07:43:04 trial035 ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-mgr-x[39621]: [24/Feb/2026:07:43:04] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283)) shut down 2026-02-24T07:43:04.278 INFO:journalctl@ceph.mgr.x.trial035.stdout:Feb 24 07:43:04 trial035 ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-mgr-x[39621]: [24/Feb/2026:07:43:04] ENGINE Bus STOPPED 2026-02-24T07:43:04.350 INFO:teuthology.orchestra.run.trial035.stdout: 2026-02-24T07:43:04.351 INFO:teuthology.orchestra.run.trial035.stderr:dumped all 2026-02-24T07:43:04.386 INFO:teuthology.orchestra.run.trial035.stdout:{"pg_ready":true,"pg_map":{"version":5,"stamp":"2026-02-24T07:43:03.659637+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":3,"acting":3,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":3,"num_osds":4,"num_per_pool_osds":4,"num_per_pool_omap_osds":3,"kb":2936012800,"kb_used":110556,"kb_used_data":2700,"kb_used_omap":45,"kb_used_meta":107730,"kb_avail":2935902244,"statfs":{"total":3006477107200,"available":3006363897856,"internally_reserved":0,"allocated":2764800,"data_stored":2108276,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":46187,"internal_metadata":110316437},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"2.998417"},"pg_stats":[{"pgid":"1.0","version":"21'92","reported_seq":154,"reported_epoch":29,"state":"active+clean","last_fresh":"2026-02-24T07:42:59.656402+0000","last_change":"2026-02-24T07:42:54.186862+0000","last_active":"2026-02-24T07:42:59.656402+0000","last_peered":"2026-02-24T07:42:59.656402+0000","last_clean":"2026-02-24T07:42:59.656402+0000","last_became_active":"2026-02-24T07:42:54.186719+0000","last_became_peered":"2026-02-24T07:42:54.186719+0000","last_unstale":"2026-02-24T07:42:59.656402+0000","last_undegraded":"2026-02-24T07:42:59.656402+0000","last_fullsized":"2026-02-24T07:42:59.656402+0000","mapping_epoch":27,"log_start":"21'90","ondisk_log_start":"21'90","created":20,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:42:43.640776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:42:43.640776+0000","last_clean_scrub_stamp":"2026-02-24T07:42:43.640776+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":90,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:43:57.783028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":3,"up_from":25,"seq":107374182404,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":27784,"kb_used_data":820,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":733975416,"statfs":{"total":751619276800,"available":751590825984,"internally_reserved":0,"allocated":839680,"data_stored":674661,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6823,"internal_metadata":27583833},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":19,"seq":81604378630,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":27784,"kb_used_data":820,"kb_used_omap":10,"kb_used_meta":26933,"kb_avail":733975416,"statfs":{"total":751619276800,"available":751590825984,"internally_reserved":0,"allocated":839680,"data_stored":674661,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":10725,"internal_metadata":27579931},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542151,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":734003200,"kb_used":27204,"kb_used_data":240,"kb_used_omap":11,"kb_used_meta":26932,"kb_avail":733975996,"statfs":{"total":751619276800,"available":751591419904,"internally_reserved":0,"allocated":245760,"data_stored":84293,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12041,"internal_metadata":27578615},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705674,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":27784,"kb_used_data":820,"kb_used_omap":16,"kb_used_meta":26927,"kb_avail":733975416,"statfs":{"total":751619276800,"available":751590825984,"internally_reserved":0,"allocated":839680,"data_stored":674661,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16598,"internal_metadata":27574058},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-24T07:43:04.387 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph pg dump --format=json 2026-02-24T07:43:04.542 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:43:04.564 INFO:journalctl@ceph.mgr.x.trial035.stdout:Feb 24 07:43:04 trial035 ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-mgr-x[39621]: [24/Feb/2026:07:43:04] ENGINE Bus STARTING 2026-02-24T07:43:04.564 INFO:journalctl@ceph.mgr.x.trial035.stdout:Feb 24 07:43:04 trial035 ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-mgr-x[39621]: [24/Feb/2026:07:43:04] ENGINE Serving on http://:::9283 2026-02-24T07:43:04.564 INFO:journalctl@ceph.mgr.x.trial035.stdout:Feb 24 07:43:04 trial035 ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-mgr-x[39621]: [24/Feb/2026:07:43:04] ENGINE Bus STARTED 2026-02-24T07:43:04.644 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:43:04.643+0000 7f7f81e79640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:43:04.644 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:43:04.643+0000 7f7f81e79640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:43:04.679 INFO:journalctl@ceph.prometheus.a.trial091.stdout:Feb 24 07:43:04 trial091 systemd[1]: Stopping Ceph prometheus.a for 3a5983d2-1154-11f1-9c05-d404e6e7d460... 2026-02-24T07:43:04.679 INFO:journalctl@ceph.prometheus.a.trial091.stdout:Feb 24 07:43:04 trial091 ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-prometheus-a[63777]: time=2026-02-24T07:43:04.605Z level=WARN source=main.go:1076 msg="Received an OS signal, exiting gracefully..." signal=terminated 2026-02-24T07:43:04.679 INFO:journalctl@ceph.prometheus.a.trial091.stdout:Feb 24 07:43:04 trial091 ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-prometheus-a[63777]: time=2026-02-24T07:43:04.605Z level=INFO source=main.go:1101 msg="Stopping scrape discovery manager..." 2026-02-24T07:43:04.679 INFO:journalctl@ceph.prometheus.a.trial091.stdout:Feb 24 07:43:04 trial091 ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-prometheus-a[63777]: time=2026-02-24T07:43:04.605Z level=INFO source=main.go:1115 msg="Stopping notify discovery manager..." 2026-02-24T07:43:04.679 INFO:journalctl@ceph.prometheus.a.trial091.stdout:Feb 24 07:43:04 trial091 ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-prometheus-a[63777]: time=2026-02-24T07:43:04.605Z level=INFO source=manager.go:190 msg="Stopping rule manager..." component="rule manager" 2026-02-24T07:43:04.679 INFO:journalctl@ceph.prometheus.a.trial091.stdout:Feb 24 07:43:04 trial091 ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-prometheus-a[63777]: time=2026-02-24T07:43:04.605Z level=INFO source=main.go:1097 msg="Scrape discovery manager stopped" 2026-02-24T07:43:04.679 INFO:journalctl@ceph.prometheus.a.trial091.stdout:Feb 24 07:43:04 trial091 ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-prometheus-a[63777]: time=2026-02-24T07:43:04.605Z level=INFO source=manager.go:206 msg="Rule manager stopped" component="rule manager" 2026-02-24T07:43:04.679 INFO:journalctl@ceph.prometheus.a.trial091.stdout:Feb 24 07:43:04 trial091 ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-prometheus-a[63777]: time=2026-02-24T07:43:04.605Z level=INFO source=main.go:1111 msg="Notify discovery manager stopped" 2026-02-24T07:43:04.679 INFO:journalctl@ceph.prometheus.a.trial091.stdout:Feb 24 07:43:04 trial091 ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-prometheus-a[63777]: time=2026-02-24T07:43:04.605Z level=INFO source=main.go:1152 msg="Stopping scrape manager..." 2026-02-24T07:43:04.680 INFO:journalctl@ceph.prometheus.a.trial091.stdout:Feb 24 07:43:04 trial091 ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-prometheus-a[63777]: time=2026-02-24T07:43:04.605Z level=INFO source=main.go:1144 msg="Scrape manager stopped" 2026-02-24T07:43:04.680 INFO:journalctl@ceph.prometheus.a.trial091.stdout:Feb 24 07:43:04 trial091 ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-prometheus-a[63777]: time=2026-02-24T07:43:04.606Z level=INFO source=manager.go:543 msg="Stopping notification manager..." component=notifier 2026-02-24T07:43:04.680 INFO:journalctl@ceph.prometheus.a.trial091.stdout:Feb 24 07:43:04 trial091 ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-prometheus-a[63777]: time=2026-02-24T07:43:04.606Z level=INFO source=manager.go:285 msg="Draining any remaining notifications..." component=notifier 2026-02-24T07:43:04.680 INFO:journalctl@ceph.prometheus.a.trial091.stdout:Feb 24 07:43:04 trial091 ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-prometheus-a[63777]: time=2026-02-24T07:43:04.606Z level=INFO source=manager.go:291 msg="Remaining notifications drained" component=notifier 2026-02-24T07:43:04.680 INFO:journalctl@ceph.prometheus.a.trial091.stdout:Feb 24 07:43:04 trial091 ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-prometheus-a[63777]: time=2026-02-24T07:43:04.606Z level=INFO source=manager.go:218 msg="Notification manager stopped" component=notifier 2026-02-24T07:43:04.680 INFO:journalctl@ceph.prometheus.a.trial091.stdout:Feb 24 07:43:04 trial091 ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-prometheus-a[63777]: time=2026-02-24T07:43:04.606Z level=INFO source=main.go:1422 msg="Notifier manager stopped" 2026-02-24T07:43:04.680 INFO:journalctl@ceph.prometheus.a.trial091.stdout:Feb 24 07:43:04 trial091 ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-prometheus-a[63777]: time=2026-02-24T07:43:04.606Z level=INFO source=main.go:1436 msg="See you next time!" 2026-02-24T07:43:04.680 INFO:journalctl@ceph.prometheus.a.trial091.stdout:Feb 24 07:43:04 trial091 podman[65853]: 2026-02-24 07:43:04.615493621 +0000 UTC m=+0.022969146 container died 06e50658ff3dbcbfa7026d8dbc9c032fa6b265ce9ca207aa5b95a83b17975643 (image=quay.io/prometheus/prometheus:v3.6.0, name=ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-prometheus-a, maintainer=The Prometheus Authors , org.opencontainers.image.source=https://github.com/prometheus/prometheus) 2026-02-24T07:43:04.680 INFO:journalctl@ceph.prometheus.a.trial091.stdout:Feb 24 07:43:04 trial091 podman[65853]: 2026-02-24 07:43:04.623095875 +0000 UTC m=+0.030571400 container remove 06e50658ff3dbcbfa7026d8dbc9c032fa6b265ce9ca207aa5b95a83b17975643 (image=quay.io/prometheus/prometheus:v3.6.0, name=ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-prometheus-a, maintainer=The Prometheus Authors , org.opencontainers.image.source=https://github.com/prometheus/prometheus) 2026-02-24T07:43:04.680 INFO:journalctl@ceph.prometheus.a.trial091.stdout:Feb 24 07:43:04 trial091 bash[65853]: ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-prometheus-a 2026-02-24T07:43:04.753 INFO:teuthology.orchestra.run.trial035.stdout: 2026-02-24T07:43:04.754 INFO:teuthology.orchestra.run.trial035.stderr:dumped all 2026-02-24T07:43:04.794 INFO:teuthology.orchestra.run.trial035.stdout:{"pg_ready":true,"pg_map":{"version":5,"stamp":"2026-02-24T07:43:03.659637+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":3,"acting":3,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":3,"num_osds":4,"num_per_pool_osds":4,"num_per_pool_omap_osds":3,"kb":2936012800,"kb_used":110556,"kb_used_data":2700,"kb_used_omap":45,"kb_used_meta":107730,"kb_avail":2935902244,"statfs":{"total":3006477107200,"available":3006363897856,"internally_reserved":0,"allocated":2764800,"data_stored":2108276,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":46187,"internal_metadata":110316437},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"2.998417"},"pg_stats":[{"pgid":"1.0","version":"21'92","reported_seq":154,"reported_epoch":29,"state":"active+clean","last_fresh":"2026-02-24T07:42:59.656402+0000","last_change":"2026-02-24T07:42:54.186862+0000","last_active":"2026-02-24T07:42:59.656402+0000","last_peered":"2026-02-24T07:42:59.656402+0000","last_clean":"2026-02-24T07:42:59.656402+0000","last_became_active":"2026-02-24T07:42:54.186719+0000","last_became_peered":"2026-02-24T07:42:54.186719+0000","last_unstale":"2026-02-24T07:42:59.656402+0000","last_undegraded":"2026-02-24T07:42:59.656402+0000","last_fullsized":"2026-02-24T07:42:59.656402+0000","mapping_epoch":27,"log_start":"21'90","ondisk_log_start":"21'90","created":20,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:42:43.640776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:42:43.640776+0000","last_clean_scrub_stamp":"2026-02-24T07:42:43.640776+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":90,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:43:57.783028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":3,"up_from":25,"seq":107374182404,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":27784,"kb_used_data":820,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":733975416,"statfs":{"total":751619276800,"available":751590825984,"internally_reserved":0,"allocated":839680,"data_stored":674661,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6823,"internal_metadata":27583833},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":19,"seq":81604378630,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":27784,"kb_used_data":820,"kb_used_omap":10,"kb_used_meta":26933,"kb_avail":733975416,"statfs":{"total":751619276800,"available":751590825984,"internally_reserved":0,"allocated":839680,"data_stored":674661,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":10725,"internal_metadata":27579931},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542151,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":734003200,"kb_used":27204,"kb_used_data":240,"kb_used_omap":11,"kb_used_meta":26932,"kb_avail":733975996,"statfs":{"total":751619276800,"available":751591419904,"internally_reserved":0,"allocated":245760,"data_stored":84293,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12041,"internal_metadata":27578615},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705674,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":27784,"kb_used_data":820,"kb_used_omap":16,"kb_used_meta":26927,"kb_avail":733975416,"statfs":{"total":751619276800,"available":751590825984,"internally_reserved":0,"allocated":839680,"data_stored":674661,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16598,"internal_metadata":27574058},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-24T07:43:04.794 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2026-02-24T07:43:04.794 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph osd dump 2026-02-24T07:43:04.911 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:43:05.015 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:43:05.014+0000 7f0db3ecb640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:43:05.015 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:43:05.014+0000 7f0db3ecb640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:43:05.107 INFO:journalctl@ceph.prometheus.a.trial091.stdout:Feb 24 07:43:04 trial091 systemd[1]: ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460@prometheus.a.service: Deactivated successfully. 2026-02-24T07:43:05.107 INFO:journalctl@ceph.prometheus.a.trial091.stdout:Feb 24 07:43:04 trial091 systemd[1]: Stopped Ceph prometheus.a for 3a5983d2-1154-11f1-9c05-d404e6e7d460. 2026-02-24T07:43:05.107 INFO:journalctl@ceph.prometheus.a.trial091.stdout:Feb 24 07:43:04 trial091 systemd[1]: Starting Ceph prometheus.a for 3a5983d2-1154-11f1-9c05-d404e6e7d460... 2026-02-24T07:43:05.107 INFO:journalctl@ceph.prometheus.a.trial091.stdout:Feb 24 07:43:04 trial091 podman[65983]: 2026-02-24 07:43:04.784053135 +0000 UTC m=+0.018777635 container create bbd51718b6d93b57c3fef297bdf0f64ac87c522c6f61881faddb3d62aab49563 (image=quay.io/prometheus/prometheus:v3.6.0, name=ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-prometheus-a, maintainer=The Prometheus Authors , org.opencontainers.image.source=https://github.com/prometheus/prometheus) 2026-02-24T07:43:05.107 INFO:journalctl@ceph.prometheus.a.trial091.stdout:Feb 24 07:43:04 trial091 podman[65983]: 2026-02-24 07:43:04.807040544 +0000 UTC m=+0.041765034 container init bbd51718b6d93b57c3fef297bdf0f64ac87c522c6f61881faddb3d62aab49563 (image=quay.io/prometheus/prometheus:v3.6.0, name=ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-prometheus-a, maintainer=The Prometheus Authors , org.opencontainers.image.source=https://github.com/prometheus/prometheus) 2026-02-24T07:43:05.107 INFO:journalctl@ceph.prometheus.a.trial091.stdout:Feb 24 07:43:04 trial091 podman[65983]: 2026-02-24 07:43:04.809134391 +0000 UTC m=+0.043858881 container start bbd51718b6d93b57c3fef297bdf0f64ac87c522c6f61881faddb3d62aab49563 (image=quay.io/prometheus/prometheus:v3.6.0, name=ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-prometheus-a, maintainer=The Prometheus Authors , org.opencontainers.image.source=https://github.com/prometheus/prometheus) 2026-02-24T07:43:05.107 INFO:journalctl@ceph.prometheus.a.trial091.stdout:Feb 24 07:43:04 trial091 bash[65983]: bbd51718b6d93b57c3fef297bdf0f64ac87c522c6f61881faddb3d62aab49563 2026-02-24T07:43:05.107 INFO:journalctl@ceph.prometheus.a.trial091.stdout:Feb 24 07:43:04 trial091 podman[65983]: 2026-02-24 07:43:04.776483791 +0000 UTC m=+0.011208301 image pull 4fcecf061b747d0b5dc55bc7efdba26f87768d058d403b5bcc3f0904cf49f973 quay.io/prometheus/prometheus:v3.6.0 2026-02-24T07:43:05.107 INFO:journalctl@ceph.prometheus.a.trial091.stdout:Feb 24 07:43:04 trial091 systemd[1]: Started Ceph prometheus.a for 3a5983d2-1154-11f1-9c05-d404e6e7d460. 2026-02-24T07:43:05.107 INFO:journalctl@ceph.prometheus.a.trial091.stdout:Feb 24 07:43:04 trial091 ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-prometheus-a[66007]: time=2026-02-24T07:43:04.833Z level=INFO source=main.go:1545 msg="updated GOGC" old=100 new=75 2026-02-24T07:43:05.108 INFO:journalctl@ceph.prometheus.a.trial091.stdout:Feb 24 07:43:04 trial091 ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-prometheus-a[66007]: time=2026-02-24T07:43:04.833Z level=INFO source=main.go:677 msg="Leaving GOMAXPROCS=24: CPU quota undefined" component=automaxprocs 2026-02-24T07:43:05.108 INFO:journalctl@ceph.prometheus.a.trial091.stdout:Feb 24 07:43:04 trial091 ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-prometheus-a[66007]: time=2026-02-24T07:43:04.833Z level=INFO source=memlimit.go:198 msg="GOMEMLIMIT is updated" component=automemlimit package=github.com/KimMachineGun/automemlimit/memlimit GOMEMLIMIT=120212516044 previous=9223372036854775807 2026-02-24T07:43:05.108 INFO:journalctl@ceph.prometheus.a.trial091.stdout:Feb 24 07:43:04 trial091 ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-prometheus-a[66007]: time=2026-02-24T07:43:04.833Z level=INFO source=main.go:770 msg="Starting Prometheus Server" mode=server version="(version=3.6.0, branch=HEAD, revision=491734606591ef8c188f5489c70380805bc868c0)" 2026-02-24T07:43:05.108 INFO:journalctl@ceph.prometheus.a.trial091.stdout:Feb 24 07:43:04 trial091 ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-prometheus-a[66007]: time=2026-02-24T07:43:04.834Z level=INFO source=main.go:775 msg="operational information" build_context="(go=go1.24.7, platform=linux/amd64, user=root@bfdf1aa26e4d, date=20250921-19:51:16, tags=netgo,builtinassets)" host_details="(Linux 5.14.0-686.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Thu Feb 19 10:49:27 UTC 2026 x86_64 trial091 (none))" fd_limits="(soft=1048576, hard=1048576)" vm_limits="(soft=unlimited, hard=unlimited)" 2026-02-24T07:43:05.108 INFO:journalctl@ceph.prometheus.a.trial091.stdout:Feb 24 07:43:04 trial091 ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-prometheus-a[66007]: time=2026-02-24T07:43:04.835Z level=INFO source=web.go:659 msg="Start listening for connections" component=web address=:9095 2026-02-24T07:43:05.108 INFO:journalctl@ceph.prometheus.a.trial091.stdout:Feb 24 07:43:04 trial091 ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-prometheus-a[66007]: time=2026-02-24T07:43:04.835Z level=INFO source=main.go:1289 msg="Starting TSDB ..." 2026-02-24T07:43:05.108 INFO:journalctl@ceph.prometheus.a.trial091.stdout:Feb 24 07:43:04 trial091 ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-prometheus-a[66007]: time=2026-02-24T07:43:04.836Z level=INFO source=tls_config.go:347 msg="Listening on" component=web address=[::]:9095 2026-02-24T07:43:05.108 INFO:journalctl@ceph.prometheus.a.trial091.stdout:Feb 24 07:43:04 trial091 ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-prometheus-a[66007]: time=2026-02-24T07:43:04.836Z level=INFO source=tls_config.go:350 msg="TLS is disabled." component=web http2=false address=[::]:9095 2026-02-24T07:43:05.108 INFO:journalctl@ceph.prometheus.a.trial091.stdout:Feb 24 07:43:04 trial091 ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-prometheus-a[66007]: time=2026-02-24T07:43:04.838Z level=INFO source=head.go:667 msg="Replaying on-disk memory mappable chunks if any" component=tsdb 2026-02-24T07:43:05.108 INFO:journalctl@ceph.prometheus.a.trial091.stdout:Feb 24 07:43:04 trial091 ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-prometheus-a[66007]: time=2026-02-24T07:43:04.838Z level=INFO source=head.go:754 msg="On-disk memory mappable chunks replay completed" component=tsdb duration=812ns 2026-02-24T07:43:05.108 INFO:journalctl@ceph.prometheus.a.trial091.stdout:Feb 24 07:43:04 trial091 ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-prometheus-a[66007]: time=2026-02-24T07:43:04.838Z level=INFO source=head.go:762 msg="Replaying WAL, this may take a while" component=tsdb 2026-02-24T07:43:05.108 INFO:journalctl@ceph.prometheus.a.trial091.stdout:Feb 24 07:43:04 trial091 ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-prometheus-a[66007]: time=2026-02-24T07:43:04.838Z level=INFO source=head.go:835 msg="WAL segment loaded" component=tsdb segment=0 maxSegment=1 duration=368.875µs 2026-02-24T07:43:05.108 INFO:journalctl@ceph.prometheus.a.trial091.stdout:Feb 24 07:43:04 trial091 ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-prometheus-a[66007]: time=2026-02-24T07:43:04.839Z level=INFO source=head.go:835 msg="WAL segment loaded" component=tsdb segment=1 maxSegment=1 duration=592.43µs 2026-02-24T07:43:05.108 INFO:journalctl@ceph.prometheus.a.trial091.stdout:Feb 24 07:43:04 trial091 ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-prometheus-a[66007]: time=2026-02-24T07:43:04.839Z level=INFO source=head.go:872 msg="WAL replay completed" component=tsdb checkpoint_replay_duration=12.233µs wal_replay_duration=994.987µs wbl_replay_duration=140ns chunk_snapshot_load_duration=0s mmap_chunk_replay_duration=812ns total_replay_duration=1.029902ms 2026-02-24T07:43:05.109 INFO:journalctl@ceph.prometheus.a.trial091.stdout:Feb 24 07:43:04 trial091 ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-prometheus-a[66007]: time=2026-02-24T07:43:04.840Z level=INFO source=main.go:1310 msg="filesystem information" fs_type=XFS_SUPER_MAGIC 2026-02-24T07:43:05.109 INFO:journalctl@ceph.prometheus.a.trial091.stdout:Feb 24 07:43:04 trial091 ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-prometheus-a[66007]: time=2026-02-24T07:43:04.840Z level=INFO source=main.go:1313 msg="TSDB started" 2026-02-24T07:43:05.109 INFO:journalctl@ceph.prometheus.a.trial091.stdout:Feb 24 07:43:04 trial091 ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-prometheus-a[66007]: time=2026-02-24T07:43:04.840Z level=INFO source=main.go:1498 msg="Loading configuration file" filename=/etc/prometheus/prometheus.yml 2026-02-24T07:43:05.109 INFO:journalctl@ceph.prometheus.a.trial091.stdout:Feb 24 07:43:04 trial091 ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-prometheus-a[66007]: time=2026-02-24T07:43:04.851Z level=INFO source=main.go:1538 msg="Completed loading of configuration file" db_storage=541ns remote_storage=1.202µs web_handler=231ns query_engine=571ns scrape=333.59µs scrape_sd=44.874µs notify=761ns notify_sd=370ns rules=10.995985ms tracing=7.504µs filename=/etc/prometheus/prometheus.yml totalDuration=11.562989ms 2026-02-24T07:43:05.109 INFO:journalctl@ceph.prometheus.a.trial091.stdout:Feb 24 07:43:04 trial091 ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-prometheus-a[66007]: time=2026-02-24T07:43:04.851Z level=INFO source=main.go:1274 msg="Server is ready to receive web requests." 2026-02-24T07:43:05.109 INFO:journalctl@ceph.prometheus.a.trial091.stdout:Feb 24 07:43:04 trial091 ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-prometheus-a[66007]: time=2026-02-24T07:43:04.851Z level=INFO source=manager.go:176 msg="Starting rule manager..." component="rule manager" 2026-02-24T07:43:05.123 INFO:journalctl@ceph.mgr.x.trial035.stdout:Feb 24 07:43:04 trial035 ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-mgr-x[39621]: [24/Feb/2026:07:43:04] ENGINE Bus STOPPING 2026-02-24T07:43:05.124 INFO:journalctl@ceph.mgr.x.trial035.stdout:Feb 24 07:43:04 trial035 ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-mgr-x[39621]: [24/Feb/2026:07:43:04] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283)) shut down 2026-02-24T07:43:05.124 INFO:journalctl@ceph.mgr.x.trial035.stdout:Feb 24 07:43:04 trial035 ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-mgr-x[39621]: [24/Feb/2026:07:43:04] ENGINE Bus STOPPED 2026-02-24T07:43:05.124 INFO:journalctl@ceph.mgr.x.trial035.stdout:Feb 24 07:43:04 trial035 ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-mgr-x[39621]: [24/Feb/2026:07:43:04] ENGINE Bus STARTING 2026-02-24T07:43:05.124 INFO:journalctl@ceph.mgr.x.trial035.stdout:Feb 24 07:43:04 trial035 ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-mgr-x[39621]: [24/Feb/2026:07:43:04] ENGINE Serving on http://:::9283 2026-02-24T07:43:05.124 INFO:journalctl@ceph.mgr.x.trial035.stdout:Feb 24 07:43:04 trial035 ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-mgr-x[39621]: [24/Feb/2026:07:43:04] ENGINE Bus STARTED 2026-02-24T07:43:05.124 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:43:04 trial035 ceph-mon[39284]: from='client.14314 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2026-02-24T07:43:05.124 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:43:04 trial035 ceph-mon[39284]: from='client.14316 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2026-02-24T07:43:05.124 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:43:04 trial035 ceph-mon[39284]: pgmap v5: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 2.7 TiB / 2.7 TiB avail 2026-02-24T07:43:05.124 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:43:04 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/1046918953' entity='client.admin' 2026-02-24T07:43:05.124 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:43:04 trial035 ceph-mon[39284]: from='mgr.14284 10.20.193.35:0/3681373222' entity='mgr.x' 2026-02-24T07:43:05.124 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:43:04 trial035 ceph-mon[39284]: from='mgr.14284 10.20.193.35:0/3681373222' entity='mgr.x' 2026-02-24T07:43:05.124 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:43:04 trial035 ceph-mon[39284]: from='mgr.14284 10.20.193.35:0/3681373222' entity='mgr.x' 2026-02-24T07:43:05.124 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:43:04 trial035 ceph-mon[39284]: from='mgr.14284 10.20.193.35:0/3681373222' entity='mgr.x' 2026-02-24T07:43:05.124 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:43:04 trial035 ceph-mon[39284]: from='mgr.14284 10.20.193.35:0/3681373222' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-24T07:43:05.124 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:43:04 trial035 ceph-mon[39284]: from='mgr.14284 10.20.193.35:0/3681373222' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2026-02-24T07:43:05.124 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:43:04 trial035 ceph-mon[39284]: from='mgr.14284 10.20.193.35:0/3681373222' entity='mgr.x' 2026-02-24T07:43:05.124 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:43:04 trial035 ceph-mon[39284]: from='mgr.14284 10.20.193.35:0/3681373222' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2026-02-24T07:43:05.124 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:43:04 trial035 ceph-mon[39284]: from='mgr.14284 10.20.193.35:0/3681373222' entity='mgr.x' cmd={"prefix": "orch get-security-config"} : dispatch 2026-02-24T07:43:05.124 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:43:04 trial035 ceph-mon[39284]: mgrmap e19: x(active, since 4s) 2026-02-24T07:43:05.124 INFO:teuthology.orchestra.run.trial035.stdout:epoch 29 2026-02-24T07:43:05.124 INFO:teuthology.orchestra.run.trial035.stdout:fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 2026-02-24T07:43:05.124 INFO:teuthology.orchestra.run.trial035.stdout:created 2026-02-24T07:41:42.805703+0000 2026-02-24T07:43:05.125 INFO:teuthology.orchestra.run.trial035.stdout:modified 2026-02-24T07:42:59.654328+0000 2026-02-24T07:43:05.125 INFO:teuthology.orchestra.run.trial035.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2026-02-24T07:43:05.125 INFO:teuthology.orchestra.run.trial035.stdout:crush_version 10 2026-02-24T07:43:05.125 INFO:teuthology.orchestra.run.trial035.stdout:full_ratio 0.95 2026-02-24T07:43:05.125 INFO:teuthology.orchestra.run.trial035.stdout:backfillfull_ratio 0.9 2026-02-24T07:43:05.125 INFO:teuthology.orchestra.run.trial035.stdout:nearfull_ratio 0.85 2026-02-24T07:43:05.125 INFO:teuthology.orchestra.run.trial035.stdout:require_min_compat_client luminous 2026-02-24T07:43:05.125 INFO:teuthology.orchestra.run.trial035.stdout:min_compat_client jewel 2026-02-24T07:43:05.125 INFO:teuthology.orchestra.run.trial035.stdout:require_osd_release tentacle 2026-02-24T07:43:05.125 INFO:teuthology.orchestra.run.trial035.stdout:stretch_mode_enabled false 2026-02-24T07:43:05.125 INFO:teuthology.orchestra.run.trial035.stdout:allow_crimson=true 2026-02-24T07:43:05.125 INFO:teuthology.orchestra.run.trial035.stdout:pool 1 '.mgr' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 22 flags hashpspool,crimson stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 4.00 2026-02-24T07:43:05.125 INFO:teuthology.orchestra.run.trial035.stdout:max_osd 4 2026-02-24T07:43:05.125 INFO:teuthology.orchestra.run.trial035.stdout:osd.0 up in weight 1 up_from 9 up_thru 26 down_at 0 last_clean_interval [0,0) [v2:10.20.193.35:6802/2333513103,v1:10.20.193.35:6803/2333513103] [v2:10.20.193.35:6804/2333513103,v1:10.20.193.35:6805/2333513103] exists,up 8d5b0adf-0e02-40d4-8780-e64988c10e92 2026-02-24T07:43:05.125 INFO:teuthology.orchestra.run.trial035.stdout:osd.1 up in weight 1 up_from 14 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:10.20.193.35:6810/4095255721,v1:10.20.193.35:6811/4095255721] [v2:10.20.193.35:6812/4095255721,v1:10.20.193.35:6813/4095255721] exists,up b9b77957-7aa8-49b2-95a7-b50dcfc6bc3e 2026-02-24T07:43:05.125 INFO:teuthology.orchestra.run.trial035.stdout:osd.2 up in weight 1 up_from 19 up_thru 20 down_at 0 last_clean_interval [0,0) [v2:10.20.193.91:6800/1390012919,v1:10.20.193.91:6801/1390012919] [v2:10.20.193.91:6802/1390012919,v1:10.20.193.91:6803/1390012919] exists,up 883dcd38-0b52-4889-90fe-44a624ba23cd 2026-02-24T07:43:05.125 INFO:teuthology.orchestra.run.trial035.stdout:osd.3 up in weight 1 up_from 25 up_thru 27 down_at 0 last_clean_interval [0,0) [v2:10.20.193.91:6808/940596907,v1:10.20.193.91:6809/940596907] [v2:10.20.193.91:6810/940596907,v1:10.20.193.91:6811/940596907] exists,up 2582e7bb-0265-44e6-bf18-c68070be0dc7 2026-02-24T07:43:05.125 INFO:teuthology.orchestra.run.trial035.stdout:blocklist 10.20.193.35:0/1250927865 expires 2026-02-25T07:42:59.654301+0000 2026-02-24T07:43:05.125 INFO:teuthology.orchestra.run.trial035.stdout:blocklist 10.20.193.35:0/4081776025 expires 2026-02-25T07:42:59.654301+0000 2026-02-24T07:43:05.125 INFO:teuthology.orchestra.run.trial035.stdout:blocklist 10.20.193.35:6800/1865836941 expires 2026-02-25T07:41:53.007692+0000 2026-02-24T07:43:05.125 INFO:teuthology.orchestra.run.trial035.stdout:blocklist 10.20.193.35:0/1753799146 expires 2026-02-25T07:42:59.654301+0000 2026-02-24T07:43:05.125 INFO:teuthology.orchestra.run.trial035.stdout:blocklist 10.20.193.35:6800/3632488220 expires 2026-02-25T07:42:59.654301+0000 2026-02-24T07:43:05.125 INFO:teuthology.orchestra.run.trial035.stdout:blocklist 10.20.193.35:0/1760415898 expires 2026-02-25T07:42:02.689708+0000 2026-02-24T07:43:05.125 INFO:teuthology.orchestra.run.trial035.stdout:blocklist 10.20.193.35:0/1697559475 expires 2026-02-25T07:41:53.007692+0000 2026-02-24T07:43:05.125 INFO:teuthology.orchestra.run.trial035.stdout:blocklist 10.20.193.35:0/2628287835 expires 2026-02-25T07:42:02.689708+0000 2026-02-24T07:43:05.125 INFO:teuthology.orchestra.run.trial035.stdout:blocklist 10.20.193.35:0/2848450259 expires 2026-02-25T07:41:53.007692+0000 2026-02-24T07:43:05.125 INFO:teuthology.orchestra.run.trial035.stdout:blocklist 10.20.193.35:6801/1865836941 expires 2026-02-25T07:41:53.007692+0000 2026-02-24T07:43:05.125 INFO:teuthology.orchestra.run.trial035.stdout:blocklist 10.20.193.35:0/3517194207 expires 2026-02-25T07:41:53.007692+0000 2026-02-24T07:43:05.125 INFO:teuthology.orchestra.run.trial035.stdout:blocklist 10.20.193.35:0/2958395575 expires 2026-02-25T07:42:02.689708+0000 2026-02-24T07:43:05.125 INFO:teuthology.orchestra.run.trial035.stdout:blocklist 10.20.193.35:6800/432269861 expires 2026-02-25T07:42:02.689708+0000 2026-02-24T07:43:05.125 INFO:teuthology.orchestra.run.trial035.stdout:blocklist 10.20.193.35:6801/432269861 expires 2026-02-25T07:42:02.689708+0000 2026-02-24T07:43:05.125 INFO:teuthology.orchestra.run.trial035.stdout:blocklist 10.20.193.35:0/794704272 expires 2026-02-25T07:42:59.654301+0000 2026-02-24T07:43:05.125 INFO:teuthology.orchestra.run.trial035.stdout:blocklist 10.20.193.35:6801/3632488220 expires 2026-02-25T07:42:59.654301+0000 2026-02-24T07:43:05.161 INFO:tasks.cephadm.ceph_manager.ceph:['osd.0 up in weight 1 up_from 9 up_thru 26 down_at 0 last_clean_interval [0,0) [v2:10.20.193.35:6802/2333513103,v1:10.20.193.35:6803/2333513103] [v2:10.20.193.35:6804/2333513103,v1:10.20.193.35:6805/2333513103] exists,up 8d5b0adf-0e02-40d4-8780-e64988c10e92', 'osd.1 up in weight 1 up_from 14 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:10.20.193.35:6810/4095255721,v1:10.20.193.35:6811/4095255721] [v2:10.20.193.35:6812/4095255721,v1:10.20.193.35:6813/4095255721] exists,up b9b77957-7aa8-49b2-95a7-b50dcfc6bc3e', 'osd.2 up in weight 1 up_from 19 up_thru 20 down_at 0 last_clean_interval [0,0) [v2:10.20.193.91:6800/1390012919,v1:10.20.193.91:6801/1390012919] [v2:10.20.193.91:6802/1390012919,v1:10.20.193.91:6803/1390012919] exists,up 883dcd38-0b52-4889-90fe-44a624ba23cd', 'osd.3 up in weight 1 up_from 25 up_thru 27 down_at 0 last_clean_interval [0,0) [v2:10.20.193.91:6808/940596907,v1:10.20.193.91:6809/940596907] [v2:10.20.193.91:6810/940596907,v1:10.20.193.91:6811/940596907] exists,up 2582e7bb-0265-44e6-bf18-c68070be0dc7'] 2026-02-24T07:43:05.161 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph daemon mon.a config show 2026-02-24T07:43:05.277 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:43:05.386 INFO:teuthology.orchestra.run.trial035.stdout:{ 2026-02-24T07:43:05.386 INFO:teuthology.orchestra.run.trial035.stdout: "name": "mon.a", 2026-02-24T07:43:05.386 INFO:teuthology.orchestra.run.trial035.stdout: "cluster": "ceph", 2026-02-24T07:43:05.386 INFO:teuthology.orchestra.run.trial035.stdout: "admin_socket": "/var/run/ceph/ceph-mon.a.asok", 2026-02-24T07:43:05.387 INFO:teuthology.orchestra.run.trial035.stdout: "admin_socket_mode": "", 2026-02-24T07:43:05.387 INFO:teuthology.orchestra.run.trial035.stdout: "auth_allow_insecure_global_id_reclaim": "false", 2026-02-24T07:43:05.387 INFO:teuthology.orchestra.run.trial035.stdout: "auth_client_required": "cephx, none", 2026-02-24T07:43:05.387 INFO:teuthology.orchestra.run.trial035.stdout: "auth_cluster_required": "cephx", 2026-02-24T07:43:05.387 INFO:teuthology.orchestra.run.trial035.stdout: "auth_debug": "true", 2026-02-24T07:43:05.387 INFO:teuthology.orchestra.run.trial035.stdout: "auth_expose_insecure_global_id_reclaim": "true", 2026-02-24T07:43:05.387 INFO:teuthology.orchestra.run.trial035.stdout: "auth_mon_ticket_ttl": "660.000000", 2026-02-24T07:43:05.387 INFO:teuthology.orchestra.run.trial035.stdout: "auth_service_required": "cephx", 2026-02-24T07:43:05.387 INFO:teuthology.orchestra.run.trial035.stdout: "auth_service_ticket_ttl": "240.000000", 2026-02-24T07:43:05.387 INFO:teuthology.orchestra.run.trial035.stdout: "auth_supported": "", 2026-02-24T07:43:05.387 INFO:teuthology.orchestra.run.trial035.stdout: "bdev_aio": "true", 2026-02-24T07:43:05.387 INFO:teuthology.orchestra.run.trial035.stdout: "bdev_aio_max_queue_depth": "1024", 2026-02-24T07:43:05.388 INFO:teuthology.orchestra.run.trial035.stdout: "bdev_aio_poll_ms": "250", 2026-02-24T07:43:05.388 INFO:teuthology.orchestra.run.trial035.stdout: "bdev_aio_reap_max": "16", 2026-02-24T07:43:05.388 INFO:teuthology.orchestra.run.trial035.stdout: "bdev_aio_submit_retry_initial_delay_us": "125", 2026-02-24T07:43:05.388 INFO:teuthology.orchestra.run.trial035.stdout: "bdev_aio_submit_retry_max": "16", 2026-02-24T07:43:05.388 INFO:teuthology.orchestra.run.trial035.stdout: "bdev_async_discard": "false", 2026-02-24T07:43:05.388 INFO:teuthology.orchestra.run.trial035.stdout: "bdev_async_discard_max_pending": "1000000", 2026-02-24T07:43:05.388 INFO:teuthology.orchestra.run.trial035.stdout: "bdev_async_discard_threads": "0", 2026-02-24T07:43:05.388 INFO:teuthology.orchestra.run.trial035.stdout: "bdev_block_size": "4096", 2026-02-24T07:43:05.388 INFO:teuthology.orchestra.run.trial035.stdout: "bdev_debug_aio": "false", 2026-02-24T07:43:05.388 INFO:teuthology.orchestra.run.trial035.stdout: "bdev_debug_aio_log_age": "5.000000", 2026-02-24T07:43:05.388 INFO:teuthology.orchestra.run.trial035.stdout: "bdev_debug_aio_suicide_timeout": "60.000000", 2026-02-24T07:43:05.388 INFO:teuthology.orchestra.run.trial035.stdout: "bdev_debug_discard_sleep": "0", 2026-02-24T07:43:05.389 INFO:teuthology.orchestra.run.trial035.stdout: "bdev_debug_inflight_ios": "false", 2026-02-24T07:43:05.389 INFO:teuthology.orchestra.run.trial035.stdout: "bdev_discard_max_bytes": "10737418240", 2026-02-24T07:43:05.389 INFO:teuthology.orchestra.run.trial035.stdout: "bdev_enable_discard": "false", 2026-02-24T07:43:05.389 INFO:teuthology.orchestra.run.trial035.stdout: "bdev_flock_retry": "3", 2026-02-24T07:43:05.389 INFO:teuthology.orchestra.run.trial035.stdout: "bdev_flock_retry_interval": "0.100000", 2026-02-24T07:43:05.389 INFO:teuthology.orchestra.run.trial035.stdout: "bdev_inject_crash": "0", 2026-02-24T07:43:05.389 INFO:teuthology.orchestra.run.trial035.stdout: "bdev_inject_crash_flush_delay": "2", 2026-02-24T07:43:05.389 INFO:teuthology.orchestra.run.trial035.stdout: "bdev_ioring": "false", 2026-02-24T07:43:05.389 INFO:teuthology.orchestra.run.trial035.stdout: "bdev_ioring_hipri": "false", 2026-02-24T07:43:05.389 INFO:teuthology.orchestra.run.trial035.stdout: "bdev_ioring_sqthread_poll": "false", 2026-02-24T07:43:05.389 INFO:teuthology.orchestra.run.trial035.stdout: "bdev_max_discard_length": "2147483648", 2026-02-24T07:43:05.390 INFO:teuthology.orchestra.run.trial035.stdout: "bdev_nvme_unbind_from_kernel": "false", 2026-02-24T07:43:05.390 INFO:teuthology.orchestra.run.trial035.stdout: "bdev_read_buffer_alignment": "4096", 2026-02-24T07:43:05.390 INFO:teuthology.orchestra.run.trial035.stdout: "bdev_read_preallocated_huge_buffers": "", 2026-02-24T07:43:05.390 INFO:teuthology.orchestra.run.trial035.stdout: "bdev_stalled_read_warn_lifetime": "86400", 2026-02-24T07:43:05.390 INFO:teuthology.orchestra.run.trial035.stdout: "bdev_stalled_read_warn_threshold": "1", 2026-02-24T07:43:05.390 INFO:teuthology.orchestra.run.trial035.stdout: "bdev_type": "", 2026-02-24T07:43:05.390 INFO:teuthology.orchestra.run.trial035.stdout: "bluefs_alloc_size": "1048576", 2026-02-24T07:43:05.390 INFO:teuthology.orchestra.run.trial035.stdout: "bluefs_allocator": "hybrid", 2026-02-24T07:43:05.390 INFO:teuthology.orchestra.run.trial035.stdout: "bluefs_buffered_io": "true", 2026-02-24T07:43:05.390 INFO:teuthology.orchestra.run.trial035.stdout: "bluefs_check_for_zeros": "false", 2026-02-24T07:43:05.390 INFO:teuthology.orchestra.run.trial035.stdout: "bluefs_check_volume_selector_often": "false", 2026-02-24T07:43:05.390 INFO:teuthology.orchestra.run.trial035.stdout: "bluefs_check_volume_selector_on_umount": "false", 2026-02-24T07:43:05.391 INFO:teuthology.orchestra.run.trial035.stdout: "bluefs_compact_log_sync": "false", 2026-02-24T07:43:05.391 INFO:teuthology.orchestra.run.trial035.stdout: "bluefs_failed_shared_alloc_cooldown": "600.000000", 2026-02-24T07:43:05.391 INFO:teuthology.orchestra.run.trial035.stdout: "bluefs_log_compact_min_ratio": "5.000000", 2026-02-24T07:43:05.391 INFO:teuthology.orchestra.run.trial035.stdout: "bluefs_log_compact_min_size": "16777216", 2026-02-24T07:43:05.391 INFO:teuthology.orchestra.run.trial035.stdout: "bluefs_log_replay_check_allocations": "true", 2026-02-24T07:43:05.391 INFO:teuthology.orchestra.run.trial035.stdout: "bluefs_max_log_runway": "4194304", 2026-02-24T07:43:05.391 INFO:teuthology.orchestra.run.trial035.stdout: "bluefs_max_prefetch": "1048576", 2026-02-24T07:43:05.391 INFO:teuthology.orchestra.run.trial035.stdout: "bluefs_min_flush_size": "524288", 2026-02-24T07:43:05.391 INFO:teuthology.orchestra.run.trial035.stdout: "bluefs_min_log_runway": "1048576", 2026-02-24T07:43:05.391 INFO:teuthology.orchestra.run.trial035.stdout: "bluefs_replay_recovery": "false", 2026-02-24T07:43:05.391 INFO:teuthology.orchestra.run.trial035.stdout: "bluefs_replay_recovery_disable_compact": "false", 2026-02-24T07:43:05.392 INFO:teuthology.orchestra.run.trial035.stdout: "bluefs_shared_alloc_size": "65536", 2026-02-24T07:43:05.392 INFO:teuthology.orchestra.run.trial035.stdout: "bluefs_sync_write": "false", 2026-02-24T07:43:05.392 INFO:teuthology.orchestra.run.trial035.stdout: "bluefs_wal_envelope_mode": "true", 2026-02-24T07:43:05.392 INFO:teuthology.orchestra.run.trial035.stdout: "bluestore_2q_cache_kin_ratio": "0.500000", 2026-02-24T07:43:05.392 INFO:teuthology.orchestra.run.trial035.stdout: "bluestore_2q_cache_kout_ratio": "0.500000", 2026-02-24T07:43:05.392 INFO:teuthology.orchestra.run.trial035.stdout: "bluestore_alloc_stats_dump_interval": "86400.000000", 2026-02-24T07:43:05.392 INFO:teuthology.orchestra.run.trial035.stdout: "bluestore_allocation_from_file": "true", 2026-02-24T07:43:05.392 INFO:teuthology.orchestra.run.trial035.stdout: "bluestore_allocator": "hybrid", 2026-02-24T07:43:05.392 INFO:teuthology.orchestra.run.trial035.stdout: "bluestore_allocator_lookup_policy": "auto", 2026-02-24T07:43:05.392 INFO:teuthology.orchestra.run.trial035.stdout: "bluestore_async_db_compaction": "true", 2026-02-24T07:43:05.392 INFO:teuthology.orchestra.run.trial035.stdout: "bluestore_avl_alloc_bf_free_pct": "4", 2026-02-24T07:43:05.392 INFO:teuthology.orchestra.run.trial035.stdout: "bluestore_avl_alloc_bf_threshold": "131072", 2026-02-24T07:43:05.393 INFO:teuthology.orchestra.run.trial035.stdout: "bluestore_avl_alloc_ff_max_search_bytes": "16777216", 2026-02-24T07:43:05.393 INFO:teuthology.orchestra.run.trial035.stdout: "bluestore_avl_alloc_ff_max_search_count": "100", 2026-02-24T07:43:05.393 INFO:teuthology.orchestra.run.trial035.stdout: "bluestore_bdev_label_multi": "true", 2026-02-24T07:43:05.393 INFO:teuthology.orchestra.run.trial035.stdout: "bluestore_bdev_label_multi_upgrade": "false", 2026-02-24T07:43:05.393 INFO:teuthology.orchestra.run.trial035.stdout: "bluestore_bdev_label_require_all": "true", 2026-02-24T07:43:05.393 INFO:teuthology.orchestra.run.trial035.stdout: "bluestore_bitmapallocator_blocks_per_zone": "1024", 2026-02-24T07:43:05.393 INFO:teuthology.orchestra.run.trial035.stdout: "bluestore_bitmapallocator_span_size": "1024", 2026-02-24T07:43:05.393 INFO:teuthology.orchestra.run.trial035.stdout: "bluestore_blobid_prealloc": "10240", 2026-02-24T07:43:05.393 INFO:teuthology.orchestra.run.trial035.stdout: "bluestore_block_create": "true", 2026-02-24T07:43:05.393 INFO:teuthology.orchestra.run.trial035.stdout: "bluestore_block_db_create": "false", 2026-02-24T07:43:05.393 INFO:teuthology.orchestra.run.trial035.stdout: "bluestore_block_db_path": "", 2026-02-24T07:43:05.393 INFO:teuthology.orchestra.run.trial035.stdout: "bluestore_block_db_size": "0", 2026-02-24T07:43:05.394 INFO:teuthology.orchestra.run.trial035.stdout: "bluestore_block_path": "", 2026-02-24T07:43:05.394 INFO:teuthology.orchestra.run.trial035.stdout: "bluestore_block_preallocate_file": "false", 2026-02-24T07:43:05.394 INFO:teuthology.orchestra.run.trial035.stdout: "bluestore_block_size": "107374182400", 2026-02-24T07:43:05.394 INFO:teuthology.orchestra.run.trial035.stdout: "bluestore_block_wal_create": "false", 2026-02-24T07:43:05.394 INFO:teuthology.orchestra.run.trial035.stdout: "bluestore_block_wal_path": "", 2026-02-24T07:43:05.394 INFO:teuthology.orchestra.run.trial035.stdout: "bluestore_block_wal_size": "100663296", 2026-02-24T07:43:05.394 INFO:teuthology.orchestra.run.trial035.stdout: "bluestore_bluefs": "true", 2026-02-24T07:43:05.394 INFO:teuthology.orchestra.run.trial035.stdout: "bluestore_bluefs_alloc_failure_dump_interval": "0.000000", 2026-02-24T07:43:05.394 INFO:teuthology.orchestra.run.trial035.stdout: "bluestore_bluefs_env_mirror": "false", 2026-02-24T07:43:05.394 INFO:teuthology.orchestra.run.trial035.stdout: "bluestore_bluefs_max_free": "10737418240", 2026-02-24T07:43:05.394 INFO:teuthology.orchestra.run.trial035.stdout: "bluestore_btree2_alloc_weight_factor": "2.000000", 2026-02-24T07:43:05.395 INFO:teuthology.orchestra.run.trial035.stdout: "bluestore_cache_age_bin_interval": "1.000000", 2026-02-24T07:43:05.395 INFO:teuthology.orchestra.run.trial035.stdout: "bluestore_cache_age_bins_data": "1 2 6 24 120 720 0 0 0 0", 2026-02-24T07:43:05.395 INFO:teuthology.orchestra.run.trial035.stdout: "bluestore_cache_age_bins_kv": "1 2 6 24 120 720 0 0 0 0", 2026-02-24T07:43:05.395 INFO:teuthology.orchestra.run.trial035.stdout: "bluestore_cache_age_bins_kv_onode": "0 0 0 0 0 0 0 0 0 720", 2026-02-24T07:43:05.395 INFO:teuthology.orchestra.run.trial035.stdout: "bluestore_cache_age_bins_meta": "1 2 6 24 120 720 0 0 0 0", 2026-02-24T07:43:05.395 INFO:teuthology.orchestra.run.trial035.stdout: "bluestore_cache_autotune": "true", 2026-02-24T07:43:05.395 INFO:teuthology.orchestra.run.trial035.stdout: "bluestore_cache_autotune_interval": "5.000000", 2026-02-24T07:43:05.395 INFO:teuthology.orchestra.run.trial035.stdout: "bluestore_cache_kv_onode_ratio": "0.040000", 2026-02-24T07:43:05.395 INFO:teuthology.orchestra.run.trial035.stdout: "bluestore_cache_kv_ratio": "0.450000", 2026-02-24T07:43:05.395 INFO:teuthology.orchestra.run.trial035.stdout: "bluestore_cache_meta_ratio": "0.450000", 2026-02-24T07:43:05.395 INFO:teuthology.orchestra.run.trial035.stdout: "bluestore_cache_size": "0", 2026-02-24T07:43:05.395 INFO:teuthology.orchestra.run.trial035.stdout: "bluestore_cache_size_hdd": "1073741824", 2026-02-24T07:43:05.396 INFO:teuthology.orchestra.run.trial035.stdout: "bluestore_cache_size_ssd": "3221225472", 2026-02-24T07:43:05.396 INFO:teuthology.orchestra.run.trial035.stdout: "bluestore_cache_trim_interval": "0.050000", 2026-02-24T07:43:05.396 INFO:teuthology.orchestra.run.trial035.stdout: "bluestore_cache_trim_max_skip_pinned": "1000", 2026-02-24T07:43:05.396 INFO:teuthology.orchestra.run.trial035.stdout: "bluestore_cache_type": "2q", 2026-02-24T07:43:05.396 INFO:teuthology.orchestra.run.trial035.stdout: "bluestore_cleaner_sleep_interval": "5.000000", 2026-02-24T07:43:05.396 INFO:teuthology.orchestra.run.trial035.stdout: "bluestore_clone_cow": "true", 2026-02-24T07:43:05.396 INFO:teuthology.orchestra.run.trial035.stdout: "bluestore_compression_algorithm": "snappy", 2026-02-24T07:43:05.396 INFO:teuthology.orchestra.run.trial035.stdout: "bluestore_compression_max_blob_size": "0", 2026-02-24T07:43:05.396 INFO:teuthology.orchestra.run.trial035.stdout: "bluestore_compression_max_blob_size_hdd": "65536", 2026-02-24T07:43:05.396 INFO:teuthology.orchestra.run.trial035.stdout: "bluestore_compression_max_blob_size_ssd": "65536", 2026-02-24T07:43:05.396 INFO:teuthology.orchestra.run.trial035.stdout: "bluestore_compression_min_blob_size": "0", 2026-02-24T07:43:05.396 INFO:teuthology.orchestra.run.trial035.stdout: "bluestore_compression_min_blob_size_hdd": "8192", 2026-02-24T07:43:05.397 INFO:teuthology.orchestra.run.trial035.stdout: "bluestore_compression_min_blob_size_ssd": "65536", 2026-02-24T07:43:05.397 INFO:teuthology.orchestra.run.trial035.stdout: "bluestore_compression_mode": "none", 2026-02-24T07:43:05.397 INFO:teuthology.orchestra.run.trial035.stdout: "bluestore_compression_required_ratio": "0.875000", 2026-02-24T07:43:05.397 INFO:teuthology.orchestra.run.trial035.stdout: "bluestore_csum_type": "crc32c", 2026-02-24T07:43:05.397 INFO:teuthology.orchestra.run.trial035.stdout: "bluestore_debug_enforce_min_alloc_size": "0", 2026-02-24T07:43:05.397 INFO:teuthology.orchestra.run.trial035.stdout: "bluestore_debug_enforce_settings": "default", 2026-02-24T07:43:05.397 INFO:teuthology.orchestra.run.trial035.stdout: "bluestore_debug_extent_map_encode_check": "false", 2026-02-24T07:43:05.397 INFO:teuthology.orchestra.run.trial035.stdout: "bluestore_debug_freelist": "false", 2026-02-24T07:43:05.397 INFO:teuthology.orchestra.run.trial035.stdout: "bluestore_debug_fsck_abort": "false", 2026-02-24T07:43:05.397 INFO:teuthology.orchestra.run.trial035.stdout: "bluestore_debug_inject_allocation_from_file_failure": "0.000000", 2026-02-24T07:43:05.397 INFO:teuthology.orchestra.run.trial035.stdout: "bluestore_debug_inject_csum_err_probability": "0.000000", 2026-02-24T07:43:05.397 INFO:teuthology.orchestra.run.trial035.stdout: "bluestore_debug_inject_read_err": "false", 2026-02-24T07:43:05.397 INFO:teuthology.orchestra.run.trial035.stdout: "bluestore_debug_legacy_omap": "false", 2026-02-24T07:43:05.398 INFO:teuthology.orchestra.run.trial035.stdout: "bluestore_debug_no_reuse_blocks": "false", 2026-02-24T07:43:05.398 INFO:teuthology.orchestra.run.trial035.stdout: "bluestore_debug_omit_block_device_write": "false", 2026-02-24T07:43:05.398 INFO:teuthology.orchestra.run.trial035.stdout: "bluestore_debug_omit_kv_commit": "false", 2026-02-24T07:43:05.398 INFO:teuthology.orchestra.run.trial035.stdout: "bluestore_debug_onode_segmentation_random": "false", 2026-02-24T07:43:05.398 INFO:teuthology.orchestra.run.trial035.stdout: "bluestore_debug_permit_any_bdev_label": "false", 2026-02-24T07:43:05.398 INFO:teuthology.orchestra.run.trial035.stdout: "bluestore_debug_prefragment_max": "1048576", 2026-02-24T07:43:05.398 INFO:teuthology.orchestra.run.trial035.stdout: "bluestore_debug_random_read_err": "0.000000", 2026-02-24T07:43:05.398 INFO:teuthology.orchestra.run.trial035.stdout: "bluestore_debug_randomize_serial_transaction": "0", 2026-02-24T07:43:05.398 INFO:teuthology.orchestra.run.trial035.stdout: "bluestore_debug_small_allocations": "0", 2026-02-24T07:43:05.398 INFO:teuthology.orchestra.run.trial035.stdout: "bluestore_debug_too_many_blobs_threshold": "24576", 2026-02-24T07:43:05.398 INFO:teuthology.orchestra.run.trial035.stdout: "bluestore_default_buffered_read": "true", 2026-02-24T07:43:05.398 INFO:teuthology.orchestra.run.trial035.stdout: "bluestore_default_buffered_write": "false", 2026-02-24T07:43:05.398 INFO:teuthology.orchestra.run.trial035.stdout: "bluestore_deferred_batch_ops": "0", 2026-02-24T07:43:05.399 INFO:teuthology.orchestra.run.trial035.stdout: "bluestore_deferred_batch_ops_hdd": "64", 2026-02-24T07:43:05.399 INFO:teuthology.orchestra.run.trial035.stdout: "bluestore_deferred_batch_ops_ssd": "16", 2026-02-24T07:43:05.399 INFO:teuthology.orchestra.run.trial035.stdout: "bluestore_discard_on_mkfs": "true", 2026-02-24T07:43:05.399 INFO:teuthology.orchestra.run.trial035.stdout: "bluestore_elastic_shared_blobs": "true", 2026-02-24T07:43:05.399 INFO:teuthology.orchestra.run.trial035.stdout: "bluestore_extent_map_inline_shard_prealloc_size": "256", 2026-02-24T07:43:05.399 INFO:teuthology.orchestra.run.trial035.stdout: "bluestore_extent_map_shard_max_size": "1200", 2026-02-24T07:43:05.399 INFO:teuthology.orchestra.run.trial035.stdout: "bluestore_extent_map_shard_min_size": "150", 2026-02-24T07:43:05.399 INFO:teuthology.orchestra.run.trial035.stdout: "bluestore_extent_map_shard_target_size": "500", 2026-02-24T07:43:05.399 INFO:teuthology.orchestra.run.trial035.stdout: "bluestore_extent_map_shard_target_size_slop": "0.200000", 2026-02-24T07:43:05.399 INFO:teuthology.orchestra.run.trial035.stdout: "bluestore_fail_eio": "false", 2026-02-24T07:43:05.399 INFO:teuthology.orchestra.run.trial035.stdout: "bluestore_fragmentation_check_period": "3600", 2026-02-24T07:43:05.399 INFO:teuthology.orchestra.run.trial035.stdout: "bluestore_freelist_blocks_per_key": "128", 2026-02-24T07:43:05.399 INFO:teuthology.orchestra.run.trial035.stdout: "bluestore_fsck_error_on_no_per_pg_omap": "false", 2026-02-24T07:43:05.400 INFO:teuthology.orchestra.run.trial035.stdout: "bluestore_fsck_error_on_no_per_pool_omap": "false", 2026-02-24T07:43:05.400 INFO:teuthology.orchestra.run.trial035.stdout: "bluestore_fsck_error_on_no_per_pool_stats": "false", 2026-02-24T07:43:05.400 INFO:teuthology.orchestra.run.trial035.stdout: "bluestore_fsck_on_mkfs": "true", 2026-02-24T07:43:05.400 INFO:teuthology.orchestra.run.trial035.stdout: "bluestore_fsck_on_mkfs_deep": "false", 2026-02-24T07:43:05.400 INFO:teuthology.orchestra.run.trial035.stdout: "bluestore_fsck_on_mount": "false", 2026-02-24T07:43:05.400 INFO:teuthology.orchestra.run.trial035.stdout: "bluestore_fsck_on_mount_deep": "false", 2026-02-24T07:43:05.400 INFO:teuthology.orchestra.run.trial035.stdout: "bluestore_fsck_on_umount": "false", 2026-02-24T07:43:05.400 INFO:teuthology.orchestra.run.trial035.stdout: "bluestore_fsck_on_umount_deep": "false", 2026-02-24T07:43:05.400 INFO:teuthology.orchestra.run.trial035.stdout: "bluestore_fsck_quick_fix_on_mount": "false", 2026-02-24T07:43:05.400 INFO:teuthology.orchestra.run.trial035.stdout: "bluestore_fsck_quick_fix_threads": "2", 2026-02-24T07:43:05.400 INFO:teuthology.orchestra.run.trial035.stdout: "bluestore_fsck_read_bytes_cap": "67108864", 2026-02-24T07:43:05.400 INFO:teuthology.orchestra.run.trial035.stdout: "bluestore_fsck_shared_blob_tracker_size": "0.031250", 2026-02-24T07:43:05.400 INFO:teuthology.orchestra.run.trial035.stdout: "bluestore_gc_enable_blob_threshold": "0", 2026-02-24T07:43:05.401 INFO:teuthology.orchestra.run.trial035.stdout: "bluestore_gc_enable_total_threshold": "0", 2026-02-24T07:43:05.401 INFO:teuthology.orchestra.run.trial035.stdout: "bluestore_hybrid_alloc_mem_cap": "67108864", 2026-02-24T07:43:05.401 INFO:teuthology.orchestra.run.trial035.stdout: "bluestore_ignore_data_csum": "false", 2026-02-24T07:43:05.401 INFO:teuthology.orchestra.run.trial035.stdout: "bluestore_kv_sync_util_logging_s": "10.000000", 2026-02-24T07:43:05.401 INFO:teuthology.orchestra.run.trial035.stdout: "bluestore_kvbackend": "rocksdb", 2026-02-24T07:43:05.401 INFO:teuthology.orchestra.run.trial035.stdout: "bluestore_log_collection_list_age": "60.000000", 2026-02-24T07:43:05.401 INFO:teuthology.orchestra.run.trial035.stdout: "bluestore_log_omap_iterator_age": "5.000000", 2026-02-24T07:43:05.401 INFO:teuthology.orchestra.run.trial035.stdout: "bluestore_log_op_age": "5.000000", 2026-02-24T07:43:05.403 INFO:teuthology.orchestra.run.trial035.stdout: "bluestore_max_alloc_size": "0", 2026-02-24T07:43:05.403 INFO:teuthology.orchestra.run.trial035.stdout: "bluestore_max_blob_size": "0", 2026-02-24T07:43:05.403 INFO:teuthology.orchestra.run.trial035.stdout: "bluestore_max_blob_size_hdd": "65536", 2026-02-24T07:43:05.403 INFO:teuthology.orchestra.run.trial035.stdout: "bluestore_max_blob_size_ssd": "65536", 2026-02-24T07:43:05.403 INFO:teuthology.orchestra.run.trial035.stdout: "bluestore_max_defer_interval": "3.000000", 2026-02-24T07:43:05.403 INFO:teuthology.orchestra.run.trial035.stdout: "bluestore_max_deferred_txc": "32", 2026-02-24T07:43:05.403 INFO:teuthology.orchestra.run.trial035.stdout: "bluestore_min_alloc_size": "0", 2026-02-24T07:43:05.403 INFO:teuthology.orchestra.run.trial035.stdout: "bluestore_min_alloc_size_hdd": "4096", 2026-02-24T07:43:05.403 INFO:teuthology.orchestra.run.trial035.stdout: "bluestore_min_alloc_size_ssd": "4096", 2026-02-24T07:43:05.403 INFO:teuthology.orchestra.run.trial035.stdout: "bluestore_nid_prealloc": "1024", 2026-02-24T07:43:05.403 INFO:teuthology.orchestra.run.trial035.stdout: "bluestore_onode_segment_size": "0", 2026-02-24T07:43:05.404 INFO:teuthology.orchestra.run.trial035.stdout: "bluestore_prefer_deferred_size": "0", 2026-02-24T07:43:05.404 INFO:teuthology.orchestra.run.trial035.stdout: "bluestore_prefer_deferred_size_hdd": "65536", 2026-02-24T07:43:05.404 INFO:teuthology.orchestra.run.trial035.stdout: "bluestore_prefer_deferred_size_ssd": "0", 2026-02-24T07:43:05.404 INFO:teuthology.orchestra.run.trial035.stdout: "bluestore_qfsck_on_mount": "true", 2026-02-24T07:43:05.404 INFO:teuthology.orchestra.run.trial035.stdout: "bluestore_recompression_min_gain": "1.200000", 2026-02-24T07:43:05.404 INFO:teuthology.orchestra.run.trial035.stdout: "bluestore_retry_disk_reads": "3", 2026-02-24T07:43:05.404 INFO:teuthology.orchestra.run.trial035.stdout: "bluestore_rocksdb_cf": "true", 2026-02-24T07:43:05.404 INFO:teuthology.orchestra.run.trial035.stdout: "bluestore_rocksdb_cfs": "m(3) p(3,0-12) O(3,0-13)=block_cache={type=binned_lru} L=min_write_buffer_number_to_merge=32 P=min_write_buffer_number_to_merge=32", 2026-02-24T07:43:05.404 INFO:teuthology.orchestra.run.trial035.stdout: "bluestore_rocksdb_options": "compression=kLZ4Compression,max_write_buffer_number=64,min_write_buffer_number_to_merge=6,compaction_style=kCompactionStyleLevel,write_buffer_size=16777216,max_background_jobs=4,level0_file_num_compaction_trigger=8,max_bytes_for_level_base=1073741824,max_bytes_for_level_multiplier=8,compaction_readahead_size=2MB,max_total_wal_size=1073741824,writable_file_max_buffer_size=0", 2026-02-24T07:43:05.404 INFO:teuthology.orchestra.run.trial035.stdout: "bluestore_rocksdb_options_annex": "", 2026-02-24T07:43:05.404 INFO:teuthology.orchestra.run.trial035.stdout: "bluestore_slow_ops_warn_lifetime": "86400", 2026-02-24T07:43:05.405 INFO:teuthology.orchestra.run.trial035.stdout: "bluestore_slow_ops_warn_threshold": "1", 2026-02-24T07:43:05.405 INFO:teuthology.orchestra.run.trial035.stdout: "bluestore_spdk_coremask": "0x1", 2026-02-24T07:43:05.405 INFO:teuthology.orchestra.run.trial035.stdout: "bluestore_spdk_io_sleep": "5", 2026-02-24T07:43:05.405 INFO:teuthology.orchestra.run.trial035.stdout: "bluestore_spdk_max_io_completion": "0", 2026-02-24T07:43:05.405 INFO:teuthology.orchestra.run.trial035.stdout: "bluestore_spdk_mem": "512", 2026-02-24T07:43:05.405 INFO:teuthology.orchestra.run.trial035.stdout: "bluestore_sync_submit_transaction": "false", 2026-02-24T07:43:05.405 INFO:teuthology.orchestra.run.trial035.stdout: "bluestore_throttle_bytes": "67108864", 2026-02-24T07:43:05.405 INFO:teuthology.orchestra.run.trial035.stdout: "bluestore_throttle_cost_per_io": "0", 2026-02-24T07:43:05.405 INFO:teuthology.orchestra.run.trial035.stdout: "bluestore_throttle_cost_per_io_hdd": "670000", 2026-02-24T07:43:05.405 INFO:teuthology.orchestra.run.trial035.stdout: "bluestore_throttle_cost_per_io_ssd": "4000", 2026-02-24T07:43:05.405 INFO:teuthology.orchestra.run.trial035.stdout: "bluestore_throttle_deferred_bytes": "134217728", 2026-02-24T07:43:05.405 INFO:teuthology.orchestra.run.trial035.stdout: "bluestore_throttle_trace_rate": "0.000000", 2026-02-24T07:43:05.405 INFO:teuthology.orchestra.run.trial035.stdout: "bluestore_tracing": "false", 2026-02-24T07:43:05.406 INFO:teuthology.orchestra.run.trial035.stdout: "bluestore_use_optimal_io_size_for_min_alloc_size": "false", 2026-02-24T07:43:05.406 INFO:teuthology.orchestra.run.trial035.stdout: "bluestore_volume_selection_policy": "use_some_extra", 2026-02-24T07:43:05.406 INFO:teuthology.orchestra.run.trial035.stdout: "bluestore_volume_selection_reserved": "0", 2026-02-24T07:43:05.406 INFO:teuthology.orchestra.run.trial035.stdout: "bluestore_volume_selection_reserved_factor": "2.000000", 2026-02-24T07:43:05.406 INFO:teuthology.orchestra.run.trial035.stdout: "bluestore_warn_on_bluefs_spillover": "true", 2026-02-24T07:43:05.406 INFO:teuthology.orchestra.run.trial035.stdout: "bluestore_warn_on_free_fragmentation": "0.800000", 2026-02-24T07:43:05.406 INFO:teuthology.orchestra.run.trial035.stdout: "bluestore_warn_on_legacy_statfs": "true", 2026-02-24T07:43:05.406 INFO:teuthology.orchestra.run.trial035.stdout: "bluestore_warn_on_no_per_pg_omap": "false", 2026-02-24T07:43:05.406 INFO:teuthology.orchestra.run.trial035.stdout: "bluestore_warn_on_no_per_pool_omap": "true", 2026-02-24T07:43:05.406 INFO:teuthology.orchestra.run.trial035.stdout: "bluestore_warn_on_spurious_read_errors": "true", 2026-02-24T07:43:05.406 INFO:teuthology.orchestra.run.trial035.stdout: "bluestore_write_v2": "false", 2026-02-24T07:43:05.406 INFO:teuthology.orchestra.run.trial035.stdout: "bluestore_write_v2_random": "false", 2026-02-24T07:43:05.407 INFO:teuthology.orchestra.run.trial035.stdout: "bluestore_zero_block_detection": "false", 2026-02-24T07:43:05.407 INFO:teuthology.orchestra.run.trial035.stdout: "breakpad": "false", 2026-02-24T07:43:05.407 INFO:teuthology.orchestra.run.trial035.stdout: "ceph_assert_supresssions": "", 2026-02-24T07:43:05.407 INFO:teuthology.orchestra.run.trial035.stdout: "cephadm_path": "/usr/sbin/cephadm", 2026-02-24T07:43:05.407 INFO:teuthology.orchestra.run.trial035.stdout: "cephfs_mirror_action_update_interval": "2", 2026-02-24T07:43:05.407 INFO:teuthology.orchestra.run.trial035.stdout: "cephfs_mirror_directory_scan_interval": "10", 2026-02-24T07:43:05.407 INFO:teuthology.orchestra.run.trial035.stdout: "cephfs_mirror_max_concurrent_directory_syncs": "3", 2026-02-24T07:43:05.407 INFO:teuthology.orchestra.run.trial035.stdout: "cephfs_mirror_max_consecutive_failures_per_directory": "10", 2026-02-24T07:43:05.407 INFO:teuthology.orchestra.run.trial035.stdout: "cephfs_mirror_max_snapshot_sync_per_cycle": "3", 2026-02-24T07:43:05.407 INFO:teuthology.orchestra.run.trial035.stdout: "cephfs_mirror_mount_timeout": "10", 2026-02-24T07:43:05.407 INFO:teuthology.orchestra.run.trial035.stdout: "cephfs_mirror_perf_stats_prio": "5", 2026-02-24T07:43:05.407 INFO:teuthology.orchestra.run.trial035.stdout: "cephfs_mirror_restart_mirror_on_blocklist_interval": "30", 2026-02-24T07:43:05.407 INFO:teuthology.orchestra.run.trial035.stdout: "cephfs_mirror_restart_mirror_on_failure_interval": "20", 2026-02-24T07:43:05.408 INFO:teuthology.orchestra.run.trial035.stdout: "cephfs_mirror_retry_failed_directories_interval": "60", 2026-02-24T07:43:05.408 INFO:teuthology.orchestra.run.trial035.stdout: "cephsqlite_blocklist_dead_locker": "true", 2026-02-24T07:43:05.408 INFO:teuthology.orchestra.run.trial035.stdout: "cephsqlite_lock_renewal_interval": "2000", 2026-02-24T07:43:05.408 INFO:teuthology.orchestra.run.trial035.stdout: "cephsqlite_lock_renewal_timeout": "30000", 2026-02-24T07:43:05.408 INFO:teuthology.orchestra.run.trial035.stdout: "cephx_cluster_require_signatures": "false", 2026-02-24T07:43:05.408 INFO:teuthology.orchestra.run.trial035.stdout: "cephx_cluster_require_version": "2", 2026-02-24T07:43:05.408 INFO:teuthology.orchestra.run.trial035.stdout: "cephx_require_signatures": "false", 2026-02-24T07:43:05.408 INFO:teuthology.orchestra.run.trial035.stdout: "cephx_require_version": "2", 2026-02-24T07:43:05.408 INFO:teuthology.orchestra.run.trial035.stdout: "cephx_service_require_signatures": "false", 2026-02-24T07:43:05.408 INFO:teuthology.orchestra.run.trial035.stdout: "cephx_service_require_version": "2", 2026-02-24T07:43:05.408 INFO:teuthology.orchestra.run.trial035.stdout: "cephx_sign_messages": "true", 2026-02-24T07:43:05.408 INFO:teuthology.orchestra.run.trial035.stdout: "chdir": "", 2026-02-24T07:43:05.408 INFO:teuthology.orchestra.run.trial035.stdout: "client_acl_type": "", 2026-02-24T07:43:05.409 INFO:teuthology.orchestra.run.trial035.stdout: "client_asio_thread_count": "2", 2026-02-24T07:43:05.409 INFO:teuthology.orchestra.run.trial035.stdout: "client_cache_mid": "0.750000", 2026-02-24T07:43:05.409 INFO:teuthology.orchestra.run.trial035.stdout: "client_cache_size": "16384", 2026-02-24T07:43:05.409 INFO:teuthology.orchestra.run.trial035.stdout: "client_caps_release_delay": "5", 2026-02-24T07:43:05.409 INFO:teuthology.orchestra.run.trial035.stdout: "client_check_pool_perm": "true", 2026-02-24T07:43:05.409 INFO:teuthology.orchestra.run.trial035.stdout: "client_collect_and_send_global_metrics": "false", 2026-02-24T07:43:05.409 INFO:teuthology.orchestra.run.trial035.stdout: "client_debug_force_sync_read": "false", 2026-02-24T07:43:05.409 INFO:teuthology.orchestra.run.trial035.stdout: "client_debug_getattr_caps": "false", 2026-02-24T07:43:05.409 INFO:teuthology.orchestra.run.trial035.stdout: "client_debug_inject_features": "", 2026-02-24T07:43:05.409 INFO:teuthology.orchestra.run.trial035.stdout: "client_debug_inject_tick_delay": "0", 2026-02-24T07:43:05.409 INFO:teuthology.orchestra.run.trial035.stdout: "client_die_on_failed_dentry_invalidate": "true", 2026-02-24T07:43:05.409 INFO:teuthology.orchestra.run.trial035.stdout: "client_die_on_failed_remount": "false", 2026-02-24T07:43:05.410 INFO:teuthology.orchestra.run.trial035.stdout: "client_dirsize_rbytes": "true", 2026-02-24T07:43:05.410 INFO:teuthology.orchestra.run.trial035.stdout: "client_file_blockdiff_max_concurrent_object_scans": "16", 2026-02-24T07:43:05.410 INFO:teuthology.orchestra.run.trial035.stdout: "client_force_lazyio": "false", 2026-02-24T07:43:05.410 INFO:teuthology.orchestra.run.trial035.stdout: "client_fs": "", 2026-02-24T07:43:05.410 INFO:teuthology.orchestra.run.trial035.stdout: "client_fscrypt_as": "true", 2026-02-24T07:43:05.410 INFO:teuthology.orchestra.run.trial035.stdout: "client_fscrypt_dummy_encryption": "false", 2026-02-24T07:43:05.410 INFO:teuthology.orchestra.run.trial035.stdout: "client_inject_fixed_oldest_tid": "false", 2026-02-24T07:43:05.410 INFO:teuthology.orchestra.run.trial035.stdout: "client_inject_release_failure": "false", 2026-02-24T07:43:05.410 INFO:teuthology.orchestra.run.trial035.stdout: "client_inject_write_delay_secs": "0", 2026-02-24T07:43:05.410 INFO:teuthology.orchestra.run.trial035.stdout: "client_max_inline_size": "4096", 2026-02-24T07:43:05.410 INFO:teuthology.orchestra.run.trial035.stdout: "client_max_retries_on_remount_failure": "5", 2026-02-24T07:43:05.410 INFO:teuthology.orchestra.run.trial035.stdout: "client_mds_namespace": "", 2026-02-24T07:43:05.411 INFO:teuthology.orchestra.run.trial035.stdout: "client_metadata": "", 2026-02-24T07:43:05.411 INFO:teuthology.orchestra.run.trial035.stdout: "client_mount_gid": "-1", 2026-02-24T07:43:05.411 INFO:teuthology.orchestra.run.trial035.stdout: "client_mount_timeout": "300", 2026-02-24T07:43:05.411 INFO:teuthology.orchestra.run.trial035.stdout: "client_mount_uid": "-1", 2026-02-24T07:43:05.411 INFO:teuthology.orchestra.run.trial035.stdout: "client_mountpoint": "/", 2026-02-24T07:43:05.411 INFO:teuthology.orchestra.run.trial035.stdout: "client_notify_timeout": "10", 2026-02-24T07:43:05.411 INFO:teuthology.orchestra.run.trial035.stdout: "client_oc": "true", 2026-02-24T07:43:05.411 INFO:teuthology.orchestra.run.trial035.stdout: "client_oc_max_dirty": "104857600", 2026-02-24T07:43:05.411 INFO:teuthology.orchestra.run.trial035.stdout: "client_oc_max_dirty_age": "5.000000", 2026-02-24T07:43:05.411 INFO:teuthology.orchestra.run.trial035.stdout: "client_oc_max_objects": "1000", 2026-02-24T07:43:05.411 INFO:teuthology.orchestra.run.trial035.stdout: "client_oc_size": "209715200", 2026-02-24T07:43:05.411 INFO:teuthology.orchestra.run.trial035.stdout: "client_oc_target_dirty": "8388608", 2026-02-24T07:43:05.411 INFO:teuthology.orchestra.run.trial035.stdout: "client_permissions": "true", 2026-02-24T07:43:05.412 INFO:teuthology.orchestra.run.trial035.stdout: "client_quota": "true", 2026-02-24T07:43:05.412 INFO:teuthology.orchestra.run.trial035.stdout: "client_quota_df": "true", 2026-02-24T07:43:05.412 INFO:teuthology.orchestra.run.trial035.stdout: "client_readahead_max_bytes": "0", 2026-02-24T07:43:05.412 INFO:teuthology.orchestra.run.trial035.stdout: "client_readahead_max_periods": "4", 2026-02-24T07:43:05.412 INFO:teuthology.orchestra.run.trial035.stdout: "client_readahead_min": "131072", 2026-02-24T07:43:05.412 INFO:teuthology.orchestra.run.trial035.stdout: "client_reconnect_stale": "false", 2026-02-24T07:43:05.412 INFO:teuthology.orchestra.run.trial035.stdout: "client_respect_subvolume_snapshot_visibility": "false", 2026-02-24T07:43:05.412 INFO:teuthology.orchestra.run.trial035.stdout: "client_shutdown_timeout": "30", 2026-02-24T07:43:05.412 INFO:teuthology.orchestra.run.trial035.stdout: "client_snapdir": ".snap", 2026-02-24T07:43:05.412 INFO:teuthology.orchestra.run.trial035.stdout: "client_tick_interval": "1", 2026-02-24T07:43:05.412 INFO:teuthology.orchestra.run.trial035.stdout: "client_trace": "", 2026-02-24T07:43:05.412 INFO:teuthology.orchestra.run.trial035.stdout: "client_try_dentry_invalidate": "false", 2026-02-24T07:43:05.412 INFO:teuthology.orchestra.run.trial035.stdout: "client_use_faked_inos": "false", 2026-02-24T07:43:05.413 INFO:teuthology.orchestra.run.trial035.stdout: "client_use_random_mds": "false", 2026-02-24T07:43:05.413 INFO:teuthology.orchestra.run.trial035.stdout: "clog_to_graylog": "false", 2026-02-24T07:43:05.413 INFO:teuthology.orchestra.run.trial035.stdout: "clog_to_graylog_host": "127.0.0.1", 2026-02-24T07:43:05.413 INFO:teuthology.orchestra.run.trial035.stdout: "clog_to_graylog_port": "12201", 2026-02-24T07:43:05.413 INFO:teuthology.orchestra.run.trial035.stdout: "clog_to_monitors": "default=true", 2026-02-24T07:43:05.413 INFO:teuthology.orchestra.run.trial035.stdout: "clog_to_syslog": "false", 2026-02-24T07:43:05.413 INFO:teuthology.orchestra.run.trial035.stdout: "clog_to_syslog_facility": "default=daemon audit=local0", 2026-02-24T07:43:05.413 INFO:teuthology.orchestra.run.trial035.stdout: "clog_to_syslog_level": "info", 2026-02-24T07:43:05.413 INFO:teuthology.orchestra.run.trial035.stdout: "cluster_addr": "-", 2026-02-24T07:43:05.413 INFO:teuthology.orchestra.run.trial035.stdout: "cluster_network": "", 2026-02-24T07:43:05.413 INFO:teuthology.orchestra.run.trial035.stdout: "cluster_network_interface": "", 2026-02-24T07:43:05.413 INFO:teuthology.orchestra.run.trial035.stdout: "compressor_zlib_isal": "false", 2026-02-24T07:43:05.413 INFO:teuthology.orchestra.run.trial035.stdout: "compressor_zlib_level": "5", 2026-02-24T07:43:05.414 INFO:teuthology.orchestra.run.trial035.stdout: "compressor_zlib_winsize": "-15", 2026-02-24T07:43:05.414 INFO:teuthology.orchestra.run.trial035.stdout: "compressor_zstd_level": "1", 2026-02-24T07:43:05.414 INFO:teuthology.orchestra.run.trial035.stdout: "container_image": "quay.ceph.io/ceph-ci/ceph@sha256:9a5db51bca6d139691bb8c7054be8834f3ba065d1c0aaaecbafb190e389fb148", 2026-02-24T07:43:05.414 INFO:teuthology.orchestra.run.trial035.stdout: "crash_dir": "/var/lib/ceph/crash", 2026-02-24T07:43:05.414 INFO:teuthology.orchestra.run.trial035.stdout: "crimson_allow_pg_split": "false", 2026-02-24T07:43:05.414 INFO:teuthology.orchestra.run.trial035.stdout: "crimson_bluestore_cpu_set": "", 2026-02-24T07:43:05.414 INFO:teuthology.orchestra.run.trial035.stdout: "crimson_bluestore_num_threads": "6", 2026-02-24T07:43:05.414 INFO:teuthology.orchestra.run.trial035.stdout: "crimson_cpu_num": "0", 2026-02-24T07:43:05.414 INFO:teuthology.orchestra.run.trial035.stdout: "crimson_cpu_set": "", 2026-02-24T07:43:05.414 INFO:teuthology.orchestra.run.trial035.stdout: "crimson_osd_obc_lru_size": "512", 2026-02-24T07:43:05.414 INFO:teuthology.orchestra.run.trial035.stdout: "crimson_osd_scheduler_concurrency": "0", 2026-02-24T07:43:05.415 INFO:teuthology.orchestra.run.trial035.stdout: "crimson_osd_stat_interval": "0", 2026-02-24T07:43:05.415 INFO:teuthology.orchestra.run.trial035.stdout: "crimson_poll_mode": "false", 2026-02-24T07:43:05.415 INFO:teuthology.orchestra.run.trial035.stdout: "crimson_reactor_backend": "", 2026-02-24T07:43:05.415 INFO:teuthology.orchestra.run.trial035.stdout: "crimson_reactor_idle_poll_time_us": "200", 2026-02-24T07:43:05.415 INFO:teuthology.orchestra.run.trial035.stdout: "crimson_reactor_io_latency_goal_ms": "0.000000", 2026-02-24T07:43:05.415 INFO:teuthology.orchestra.run.trial035.stdout: "crimson_reactor_task_quota_ms": "0.500000", 2026-02-24T07:43:05.415 INFO:teuthology.orchestra.run.trial035.stdout: "crush_location": "", 2026-02-24T07:43:05.415 INFO:teuthology.orchestra.run.trial035.stdout: "crush_location_hook": "", 2026-02-24T07:43:05.415 INFO:teuthology.orchestra.run.trial035.stdout: "crush_location_hook_timeout": "10", 2026-02-24T07:43:05.415 INFO:teuthology.orchestra.run.trial035.stdout: "d4n_writecache_enabled": "false", 2026-02-24T07:43:05.415 INFO:teuthology.orchestra.run.trial035.stdout: "daemonize": "false", 2026-02-24T07:43:05.415 INFO:teuthology.orchestra.run.trial035.stdout: "daos_pool": "tank", 2026-02-24T07:43:05.416 INFO:teuthology.orchestra.run.trial035.stdout: "dbstore_config_uri": "file:/var/lib/ceph/radosgw/dbstore-config.db", 2026-02-24T07:43:05.416 INFO:teuthology.orchestra.run.trial035.stdout: "dbstore_db_dir": "/var/lib/ceph/radosgw", 2026-02-24T07:43:05.416 INFO:teuthology.orchestra.run.trial035.stdout: "dbstore_db_name_prefix": "dbstore", 2026-02-24T07:43:05.416 INFO:teuthology.orchestra.run.trial035.stdout: "debug_alienstore": "0/5", 2026-02-24T07:43:05.416 INFO:teuthology.orchestra.run.trial035.stdout: "debug_allow_any_pool_priority": "false", 2026-02-24T07:43:05.416 INFO:teuthology.orchestra.run.trial035.stdout: "debug_asok": "1/5", 2026-02-24T07:43:05.416 INFO:teuthology.orchestra.run.trial035.stdout: "debug_asok_assert_abort": "false", 2026-02-24T07:43:05.416 INFO:teuthology.orchestra.run.trial035.stdout: "debug_asserts_on_shutdown": "true", 2026-02-24T07:43:05.416 INFO:teuthology.orchestra.run.trial035.stdout: "debug_auth": "1/5", 2026-02-24T07:43:05.416 INFO:teuthology.orchestra.run.trial035.stdout: "debug_bdev": "1/3", 2026-02-24T07:43:05.416 INFO:teuthology.orchestra.run.trial035.stdout: "debug_bluefs": "1/5", 2026-02-24T07:43:05.416 INFO:teuthology.orchestra.run.trial035.stdout: "debug_bluestore": "1/5", 2026-02-24T07:43:05.417 INFO:teuthology.orchestra.run.trial035.stdout: "debug_bluestore_compression": "1/5", 2026-02-24T07:43:05.417 INFO:teuthology.orchestra.run.trial035.stdout: "debug_buffer": "0/1", 2026-02-24T07:43:05.417 INFO:teuthology.orchestra.run.trial035.stdout: "debug_ceph_dedup": "0/5", 2026-02-24T07:43:05.417 INFO:teuthology.orchestra.run.trial035.stdout: "debug_ceph_exporter": "1/5", 2026-02-24T07:43:05.417 INFO:teuthology.orchestra.run.trial035.stdout: "debug_cephfs_mirror": "0/5", 2026-02-24T07:43:05.417 INFO:teuthology.orchestra.run.trial035.stdout: "debug_cephsqlite": "0/5", 2026-02-24T07:43:05.417 INFO:teuthology.orchestra.run.trial035.stdout: "debug_client": "0/5", 2026-02-24T07:43:05.417 INFO:teuthology.orchestra.run.trial035.stdout: "debug_compressor": "1/5", 2026-02-24T07:43:05.417 INFO:teuthology.orchestra.run.trial035.stdout: "debug_context": "0/1", 2026-02-24T07:43:05.417 INFO:teuthology.orchestra.run.trial035.stdout: "debug_crimson_interrupt": "0/5", 2026-02-24T07:43:05.417 INFO:teuthology.orchestra.run.trial035.stdout: "debug_crush": "1/1", 2026-02-24T07:43:05.417 INFO:teuthology.orchestra.run.trial035.stdout: "debug_crypto": "1/5", 2026-02-24T07:43:05.417 INFO:teuthology.orchestra.run.trial035.stdout: "debug_cyanstore": "0/5", 2026-02-24T07:43:05.418 INFO:teuthology.orchestra.run.trial035.stdout: "debug_deliberately_leak_memory": "false", 2026-02-24T07:43:05.418 INFO:teuthology.orchestra.run.trial035.stdout: "debug_disable_randomized_ping": "false", 2026-02-24T07:43:05.418 INFO:teuthology.orchestra.run.trial035.stdout: "debug_dpdk": "1/5", 2026-02-24T07:43:05.418 INFO:teuthology.orchestra.run.trial035.stdout: "debug_eventtrace": "1/5", 2026-02-24T07:43:05.418 INFO:teuthology.orchestra.run.trial035.stdout: "debug_filer": "0/1", 2026-02-24T07:43:05.418 INFO:teuthology.orchestra.run.trial035.stdout: "debug_filestore": "1/3", 2026-02-24T07:43:05.418 INFO:teuthology.orchestra.run.trial035.stdout: "debug_finisher": "1/1", 2026-02-24T07:43:05.418 INFO:teuthology.orchestra.run.trial035.stdout: "debug_fuse": "1/5", 2026-02-24T07:43:05.418 INFO:teuthology.orchestra.run.trial035.stdout: "debug_heartbeat_testing_span": "0", 2026-02-24T07:43:05.418 INFO:teuthology.orchestra.run.trial035.stdout: "debug_heartbeatmap": "1/5", 2026-02-24T07:43:05.418 INFO:teuthology.orchestra.run.trial035.stdout: "debug_immutable_obj_cache": "0/5", 2026-02-24T07:43:05.419 INFO:teuthology.orchestra.run.trial035.stdout: "debug_javaclient": "1/5", 2026-02-24T07:43:05.419 INFO:teuthology.orchestra.run.trial035.stdout: "debug_journal": "1/3", 2026-02-24T07:43:05.419 INFO:teuthology.orchestra.run.trial035.stdout: "debug_journaler": "0/5", 2026-02-24T07:43:05.419 INFO:teuthology.orchestra.run.trial035.stdout: "debug_kstore": "1/5", 2026-02-24T07:43:05.419 INFO:teuthology.orchestra.run.trial035.stdout: "debug_lockdep": "0/1", 2026-02-24T07:43:05.419 INFO:teuthology.orchestra.run.trial035.stdout: "debug_mclock": "1/5", 2026-02-24T07:43:05.419 INFO:teuthology.orchestra.run.trial035.stdout: "debug_mds": "1/5", 2026-02-24T07:43:05.419 INFO:teuthology.orchestra.run.trial035.stdout: "debug_mds_balancer": "1/5", 2026-02-24T07:43:05.419 INFO:teuthology.orchestra.run.trial035.stdout: "debug_mds_locker": "1/5", 2026-02-24T07:43:05.419 INFO:teuthology.orchestra.run.trial035.stdout: "debug_mds_log": "1/5", 2026-02-24T07:43:05.419 INFO:teuthology.orchestra.run.trial035.stdout: "debug_mds_log_expire": "1/5", 2026-02-24T07:43:05.419 INFO:teuthology.orchestra.run.trial035.stdout: "debug_mds_migrator": "1/5", 2026-02-24T07:43:05.420 INFO:teuthology.orchestra.run.trial035.stdout: "debug_mds_quiesce": "3/5", 2026-02-24T07:43:05.420 INFO:teuthology.orchestra.run.trial035.stdout: "debug_memstore": "1/5", 2026-02-24T07:43:05.420 INFO:teuthology.orchestra.run.trial035.stdout: "debug_mgr": "2/5", 2026-02-24T07:43:05.420 INFO:teuthology.orchestra.run.trial035.stdout: "debug_mgrc": "1/5", 2026-02-24T07:43:05.420 INFO:teuthology.orchestra.run.trial035.stdout: "debug_mon": "20/20", 2026-02-24T07:43:05.420 INFO:teuthology.orchestra.run.trial035.stdout: "debug_monc": "0/10", 2026-02-24T07:43:05.420 INFO:teuthology.orchestra.run.trial035.stdout: "debug_ms": "1/1", 2026-02-24T07:43:05.420 INFO:teuthology.orchestra.run.trial035.stdout: "debug_none": "0/5", 2026-02-24T07:43:05.420 INFO:teuthology.orchestra.run.trial035.stdout: "debug_objclass": "0/5", 2026-02-24T07:43:05.420 INFO:teuthology.orchestra.run.trial035.stdout: "debug_objectcacher": "0/5", 2026-02-24T07:43:05.420 INFO:teuthology.orchestra.run.trial035.stdout: "debug_objecter": "0/1", 2026-02-24T07:43:05.420 INFO:teuthology.orchestra.run.trial035.stdout: "debug_optracker": "0/5", 2026-02-24T07:43:05.420 INFO:teuthology.orchestra.run.trial035.stdout: "debug_osd": "1/5", 2026-02-24T07:43:05.421 INFO:teuthology.orchestra.run.trial035.stdout: "debug_paxos": "20/20", 2026-02-24T07:43:05.421 INFO:teuthology.orchestra.run.trial035.stdout: "debug_perfcounter": "1/5", 2026-02-24T07:43:05.421 INFO:teuthology.orchestra.run.trial035.stdout: "debug_prioritycache": "1/5", 2026-02-24T07:43:05.421 INFO:teuthology.orchestra.run.trial035.stdout: "debug_rados": "0/5", 2026-02-24T07:43:05.421 INFO:teuthology.orchestra.run.trial035.stdout: "debug_rbd": "0/5", 2026-02-24T07:43:05.421 INFO:teuthology.orchestra.run.trial035.stdout: "debug_rbd_mirror": "0/5", 2026-02-24T07:43:05.421 INFO:teuthology.orchestra.run.trial035.stdout: "debug_rbd_pwl": "0/5", 2026-02-24T07:43:05.421 INFO:teuthology.orchestra.run.trial035.stdout: "debug_rbd_replay": "0/5", 2026-02-24T07:43:05.421 INFO:teuthology.orchestra.run.trial035.stdout: "debug_refs": "0/0", 2026-02-24T07:43:05.421 INFO:teuthology.orchestra.run.trial035.stdout: "debug_reserver": "1/1", 2026-02-24T07:43:05.421 INFO:teuthology.orchestra.run.trial035.stdout: "debug_rgw": "1/5", 2026-02-24T07:43:05.421 INFO:teuthology.orchestra.run.trial035.stdout: "debug_rgw_access": "1/5", 2026-02-24T07:43:05.421 INFO:teuthology.orchestra.run.trial035.stdout: "debug_rgw_bucket_logging": "1/5", 2026-02-24T07:43:05.422 INFO:teuthology.orchestra.run.trial035.stdout: "debug_rgw_datacache": "1/5", 2026-02-24T07:43:05.422 INFO:teuthology.orchestra.run.trial035.stdout: "debug_rgw_dbstore": "1/5", 2026-02-24T07:43:05.422 INFO:teuthology.orchestra.run.trial035.stdout: "debug_rgw_dedup": "1/5", 2026-02-24T07:43:05.422 INFO:teuthology.orchestra.run.trial035.stdout: "debug_rgw_flight": "1/5", 2026-02-24T07:43:05.422 INFO:teuthology.orchestra.run.trial035.stdout: "debug_rgw_lifecycle": "1/5", 2026-02-24T07:43:05.422 INFO:teuthology.orchestra.run.trial035.stdout: "debug_rgw_notification": "1/5", 2026-02-24T07:43:05.422 INFO:teuthology.orchestra.run.trial035.stdout: "debug_rgw_restore": "1/5", 2026-02-24T07:43:05.422 INFO:teuthology.orchestra.run.trial035.stdout: "debug_rgw_sync": "1/5", 2026-02-24T07:43:05.422 INFO:teuthology.orchestra.run.trial035.stdout: "debug_rocksdb": "4/5", 2026-02-24T07:43:05.422 INFO:teuthology.orchestra.run.trial035.stdout: "debug_seastore": "0/5", 2026-02-24T07:43:05.422 INFO:teuthology.orchestra.run.trial035.stdout: "debug_seastore_backref": "0/5", 2026-02-24T07:43:05.422 INFO:teuthology.orchestra.run.trial035.stdout: "debug_seastore_cache": "0/5", 2026-02-24T07:43:05.422 INFO:teuthology.orchestra.run.trial035.stdout: "debug_seastore_cleaner": "0/5", 2026-02-24T07:43:05.422 INFO:teuthology.orchestra.run.trial035.stdout: "debug_seastore_device": "0/5", 2026-02-24T07:43:05.423 INFO:teuthology.orchestra.run.trial035.stdout: "debug_seastore_epm": "0/5", 2026-02-24T07:43:05.423 INFO:teuthology.orchestra.run.trial035.stdout: "debug_seastore_fixedkv_tree": "0/5", 2026-02-24T07:43:05.423 INFO:teuthology.orchestra.run.trial035.stdout: "debug_seastore_journal": "0/5", 2026-02-24T07:43:05.423 INFO:teuthology.orchestra.run.trial035.stdout: "debug_seastore_lba": "0/5", 2026-02-24T07:43:05.423 INFO:teuthology.orchestra.run.trial035.stdout: "debug_seastore_odata": "0/5", 2026-02-24T07:43:05.423 INFO:teuthology.orchestra.run.trial035.stdout: "debug_seastore_omap": "0/5", 2026-02-24T07:43:05.423 INFO:teuthology.orchestra.run.trial035.stdout: "debug_seastore_onode": "0/5", 2026-02-24T07:43:05.423 INFO:teuthology.orchestra.run.trial035.stdout: "debug_seastore_t": "0/5", 2026-02-24T07:43:05.423 INFO:teuthology.orchestra.run.trial035.stdout: "debug_seastore_tm": "0/5", 2026-02-24T07:43:05.423 INFO:teuthology.orchestra.run.trial035.stdout: "debug_striper": "0/1", 2026-02-24T07:43:05.423 INFO:teuthology.orchestra.run.trial035.stdout: "debug_test": "0/5", 2026-02-24T07:43:05.423 INFO:teuthology.orchestra.run.trial035.stdout: "debug_throttle": "1/1", 2026-02-24T07:43:05.423 INFO:teuthology.orchestra.run.trial035.stdout: "debug_timer": "0/1", 2026-02-24T07:43:05.423 INFO:teuthology.orchestra.run.trial035.stdout: "debug_tp": "0/5", 2026-02-24T07:43:05.423 INFO:teuthology.orchestra.run.trial035.stdout: "debug_trace": "1/5", 2026-02-24T07:43:05.424 INFO:teuthology.orchestra.run.trial035.stdout: "defer_client_eviction_on_laggy_osds": "false", 2026-02-24T07:43:05.424 INFO:teuthology.orchestra.run.trial035.stdout: "device_failure_prediction_mode": "none", 2026-02-24T07:43:05.424 INFO:teuthology.orchestra.run.trial035.stdout: "ec_extent_cache_size": "10485760", 2026-02-24T07:43:05.424 INFO:teuthology.orchestra.run.trial035.stdout: "ec_pdw_write_mode": "0", 2026-02-24T07:43:05.424 INFO:teuthology.orchestra.run.trial035.stdout: "enable_availability_tracking": "false", 2026-02-24T07:43:05.424 INFO:teuthology.orchestra.run.trial035.stdout: "enable_experimental_unrecoverable_data_corrupting_features": "crimson", 2026-02-24T07:43:05.424 INFO:teuthology.orchestra.run.trial035.stdout: "erasure_code_dir": "/usr/lib64/ceph/erasure-code", 2026-02-24T07:43:05.424 INFO:teuthology.orchestra.run.trial035.stdout: "err_to_graylog": "false", 2026-02-24T07:43:05.424 INFO:teuthology.orchestra.run.trial035.stdout: "err_to_journald": "false", 2026-02-24T07:43:05.424 INFO:teuthology.orchestra.run.trial035.stdout: "err_to_stderr": "true", 2026-02-24T07:43:05.424 INFO:teuthology.orchestra.run.trial035.stdout: "err_to_syslog": "false", 2026-02-24T07:43:05.424 INFO:teuthology.orchestra.run.trial035.stdout: "event_tracing": "false", 2026-02-24T07:43:05.424 INFO:teuthology.orchestra.run.trial035.stdout: "exporter_addr": "0.0.0.0", 2026-02-24T07:43:05.424 INFO:teuthology.orchestra.run.trial035.stdout: "exporter_cert_file": "", 2026-02-24T07:43:05.425 INFO:teuthology.orchestra.run.trial035.stdout: "exporter_http_port": "9926", 2026-02-24T07:43:05.425 INFO:teuthology.orchestra.run.trial035.stdout: "exporter_key_file": "", 2026-02-24T07:43:05.425 INFO:teuthology.orchestra.run.trial035.stdout: "exporter_prio_limit": "5", 2026-02-24T07:43:05.425 INFO:teuthology.orchestra.run.trial035.stdout: "exporter_sock_dir": "/var/run/ceph/", 2026-02-24T07:43:05.425 INFO:teuthology.orchestra.run.trial035.stdout: "exporter_sort_metrics": "true", 2026-02-24T07:43:05.425 INFO:teuthology.orchestra.run.trial035.stdout: "exporter_stats_period": "5", 2026-02-24T07:43:05.425 INFO:teuthology.orchestra.run.trial035.stdout: "fake_statfs_for_testing": "0", 2026-02-24T07:43:05.425 INFO:teuthology.orchestra.run.trial035.stdout: "fatal_signal_handlers": "true", 2026-02-24T07:43:05.425 INFO:teuthology.orchestra.run.trial035.stdout: "filer_max_purge_ops": "10", 2026-02-24T07:43:05.425 INFO:teuthology.orchestra.run.trial035.stdout: "filer_max_truncate_ops": "128", 2026-02-24T07:43:05.425 INFO:teuthology.orchestra.run.trial035.stdout: "filestore_apply_finisher_threads": "1", 2026-02-24T07:43:05.425 INFO:teuthology.orchestra.run.trial035.stdout: "filestore_blackhole": "false", 2026-02-24T07:43:05.425 INFO:teuthology.orchestra.run.trial035.stdout: "filestore_btrfs_clone_range": "true", 2026-02-24T07:43:05.425 INFO:teuthology.orchestra.run.trial035.stdout: "filestore_btrfs_snap": "true", 2026-02-24T07:43:05.425 INFO:teuthology.orchestra.run.trial035.stdout: "filestore_caller_concurrency": "10", 2026-02-24T07:43:05.426 INFO:teuthology.orchestra.run.trial035.stdout: "filestore_collect_device_partition_information": "true", 2026-02-24T07:43:05.426 INFO:teuthology.orchestra.run.trial035.stdout: "filestore_commit_timeout": "600.000000", 2026-02-24T07:43:05.426 INFO:teuthology.orchestra.run.trial035.stdout: "filestore_debug_inject_read_err": "false", 2026-02-24T07:43:05.426 INFO:teuthology.orchestra.run.trial035.stdout: "filestore_debug_omap_check": "false", 2026-02-24T07:43:05.426 INFO:teuthology.orchestra.run.trial035.stdout: "filestore_debug_verify_split": "false", 2026-02-24T07:43:05.426 INFO:teuthology.orchestra.run.trial035.stdout: "filestore_dump_file": "", 2026-02-24T07:43:05.426 INFO:teuthology.orchestra.run.trial035.stdout: "filestore_expected_throughput_bytes": "209715200.000000", 2026-02-24T07:43:05.426 INFO:teuthology.orchestra.run.trial035.stdout: "filestore_expected_throughput_ops": "200.000000", 2026-02-24T07:43:05.426 INFO:teuthology.orchestra.run.trial035.stdout: "filestore_fadvise": "true", 2026-02-24T07:43:05.426 INFO:teuthology.orchestra.run.trial035.stdout: "filestore_fail_eio": "true", 2026-02-24T07:43:05.426 INFO:teuthology.orchestra.run.trial035.stdout: "filestore_fd_cache_shards": "16", 2026-02-24T07:43:05.426 INFO:teuthology.orchestra.run.trial035.stdout: "filestore_fd_cache_size": "128", 2026-02-24T07:43:05.426 INFO:teuthology.orchestra.run.trial035.stdout: "filestore_fiemap": "false", 2026-02-24T07:43:05.426 INFO:teuthology.orchestra.run.trial035.stdout: "filestore_fiemap_threshold": "4096", 2026-02-24T07:43:05.426 INFO:teuthology.orchestra.run.trial035.stdout: "filestore_fsync_flushes_journal_data": "false", 2026-02-24T07:43:05.427 INFO:teuthology.orchestra.run.trial035.stdout: "filestore_index_retry_probability": "0.000000", 2026-02-24T07:43:05.427 INFO:teuthology.orchestra.run.trial035.stdout: "filestore_inject_stall": "0", 2026-02-24T07:43:05.427 INFO:teuthology.orchestra.run.trial035.stdout: "filestore_journal_parallel": "false", 2026-02-24T07:43:05.427 INFO:teuthology.orchestra.run.trial035.stdout: "filestore_journal_trailing": "false", 2026-02-24T07:43:05.427 INFO:teuthology.orchestra.run.trial035.stdout: "filestore_journal_writeahead": "false", 2026-02-24T07:43:05.427 INFO:teuthology.orchestra.run.trial035.stdout: "filestore_kill_at": "0", 2026-02-24T07:43:05.427 INFO:teuthology.orchestra.run.trial035.stdout: "filestore_max_alloc_hint_size": "1048576", 2026-02-24T07:43:05.427 INFO:teuthology.orchestra.run.trial035.stdout: "filestore_max_inline_xattr_size": "0", 2026-02-24T07:43:05.427 INFO:teuthology.orchestra.run.trial035.stdout: "filestore_max_inline_xattr_size_btrfs": "2048", 2026-02-24T07:43:05.427 INFO:teuthology.orchestra.run.trial035.stdout: "filestore_max_inline_xattr_size_other": "512", 2026-02-24T07:43:05.427 INFO:teuthology.orchestra.run.trial035.stdout: "filestore_max_inline_xattr_size_xfs": "65536", 2026-02-24T07:43:05.427 INFO:teuthology.orchestra.run.trial035.stdout: "filestore_max_inline_xattrs": "0", 2026-02-24T07:43:05.427 INFO:teuthology.orchestra.run.trial035.stdout: "filestore_max_inline_xattrs_btrfs": "10", 2026-02-24T07:43:05.427 INFO:teuthology.orchestra.run.trial035.stdout: "filestore_max_inline_xattrs_other": "2", 2026-02-24T07:43:05.427 INFO:teuthology.orchestra.run.trial035.stdout: "filestore_max_inline_xattrs_xfs": "10", 2026-02-24T07:43:05.427 INFO:teuthology.orchestra.run.trial035.stdout: "filestore_max_sync_interval": "5.000000", 2026-02-24T07:43:05.428 INFO:teuthology.orchestra.run.trial035.stdout: "filestore_max_xattr_value_size": "0", 2026-02-24T07:43:05.428 INFO:teuthology.orchestra.run.trial035.stdout: "filestore_max_xattr_value_size_btrfs": "65536", 2026-02-24T07:43:05.428 INFO:teuthology.orchestra.run.trial035.stdout: "filestore_max_xattr_value_size_other": "1024", 2026-02-24T07:43:05.428 INFO:teuthology.orchestra.run.trial035.stdout: "filestore_max_xattr_value_size_xfs": "65536", 2026-02-24T07:43:05.428 INFO:teuthology.orchestra.run.trial035.stdout: "filestore_merge_threshold": "-10", 2026-02-24T07:43:05.428 INFO:teuthology.orchestra.run.trial035.stdout: "filestore_min_sync_interval": "0.010000", 2026-02-24T07:43:05.428 INFO:teuthology.orchestra.run.trial035.stdout: "filestore_odsync_write": "false", 2026-02-24T07:43:05.428 INFO:teuthology.orchestra.run.trial035.stdout: "filestore_omap_backend": "rocksdb", 2026-02-24T07:43:05.428 INFO:teuthology.orchestra.run.trial035.stdout: "filestore_omap_backend_path": "", 2026-02-24T07:43:05.428 INFO:teuthology.orchestra.run.trial035.stdout: "filestore_omap_header_cache_size": "1024", 2026-02-24T07:43:05.428 INFO:teuthology.orchestra.run.trial035.stdout: "filestore_ondisk_finisher_threads": "1", 2026-02-24T07:43:05.428 INFO:teuthology.orchestra.run.trial035.stdout: "filestore_op_thread_suicide_timeout": "180", 2026-02-24T07:43:05.428 INFO:teuthology.orchestra.run.trial035.stdout: "filestore_op_thread_timeout": "60", 2026-02-24T07:43:05.428 INFO:teuthology.orchestra.run.trial035.stdout: "filestore_op_threads": "2", 2026-02-24T07:43:05.428 INFO:teuthology.orchestra.run.trial035.stdout: "filestore_punch_hole": "false", 2026-02-24T07:43:05.428 INFO:teuthology.orchestra.run.trial035.stdout: "filestore_queue_high_delay_multiple": "0.000000", 2026-02-24T07:43:05.428 INFO:teuthology.orchestra.run.trial035.stdout: "filestore_queue_high_delay_multiple_bytes": "0.000000", 2026-02-24T07:43:05.429 INFO:teuthology.orchestra.run.trial035.stdout: "filestore_queue_high_delay_multiple_ops": "0.000000", 2026-02-24T07:43:05.429 INFO:teuthology.orchestra.run.trial035.stdout: "filestore_queue_high_threshhold": "0.900000", 2026-02-24T07:43:05.429 INFO:teuthology.orchestra.run.trial035.stdout: "filestore_queue_low_threshhold": "0.300000", 2026-02-24T07:43:05.429 INFO:teuthology.orchestra.run.trial035.stdout: "filestore_queue_max_bytes": "104857600", 2026-02-24T07:43:05.429 INFO:teuthology.orchestra.run.trial035.stdout: "filestore_queue_max_delay_multiple": "0.000000", 2026-02-24T07:43:05.429 INFO:teuthology.orchestra.run.trial035.stdout: "filestore_queue_max_delay_multiple_bytes": "0.000000", 2026-02-24T07:43:05.429 INFO:teuthology.orchestra.run.trial035.stdout: "filestore_queue_max_delay_multiple_ops": "0.000000", 2026-02-24T07:43:05.429 INFO:teuthology.orchestra.run.trial035.stdout: "filestore_queue_max_ops": "50", 2026-02-24T07:43:05.429 INFO:teuthology.orchestra.run.trial035.stdout: "filestore_rocksdb_options": "max_background_jobs=10,compaction_readahead_size=2097152,compression=kNoCompression", 2026-02-24T07:43:05.429 INFO:teuthology.orchestra.run.trial035.stdout: "filestore_seek_data_hole": "false", 2026-02-24T07:43:05.429 INFO:teuthology.orchestra.run.trial035.stdout: "filestore_sloppy_crc": "false", 2026-02-24T07:43:05.429 INFO:teuthology.orchestra.run.trial035.stdout: "filestore_sloppy_crc_block_size": "65536", 2026-02-24T07:43:05.429 INFO:teuthology.orchestra.run.trial035.stdout: "filestore_splice": "false", 2026-02-24T07:43:05.429 INFO:teuthology.orchestra.run.trial035.stdout: "filestore_split_multiple": "2", 2026-02-24T07:43:05.429 INFO:teuthology.orchestra.run.trial035.stdout: "filestore_split_rand_factor": "20", 2026-02-24T07:43:05.429 INFO:teuthology.orchestra.run.trial035.stdout: "filestore_update_to": "1000", 2026-02-24T07:43:05.430 INFO:teuthology.orchestra.run.trial035.stdout: "filestore_wbthrottle_btrfs_bytes_hard_limit": "419430400", 2026-02-24T07:43:05.430 INFO:teuthology.orchestra.run.trial035.stdout: "filestore_wbthrottle_btrfs_bytes_start_flusher": "41943040", 2026-02-24T07:43:05.430 INFO:teuthology.orchestra.run.trial035.stdout: "filestore_wbthrottle_btrfs_inodes_hard_limit": "5000", 2026-02-24T07:43:05.430 INFO:teuthology.orchestra.run.trial035.stdout: "filestore_wbthrottle_btrfs_inodes_start_flusher": "500", 2026-02-24T07:43:05.430 INFO:teuthology.orchestra.run.trial035.stdout: "filestore_wbthrottle_btrfs_ios_hard_limit": "5000", 2026-02-24T07:43:05.430 INFO:teuthology.orchestra.run.trial035.stdout: "filestore_wbthrottle_btrfs_ios_start_flusher": "500", 2026-02-24T07:43:05.430 INFO:teuthology.orchestra.run.trial035.stdout: "filestore_wbthrottle_enable": "true", 2026-02-24T07:43:05.430 INFO:teuthology.orchestra.run.trial035.stdout: "filestore_wbthrottle_xfs_bytes_hard_limit": "419430400", 2026-02-24T07:43:05.430 INFO:teuthology.orchestra.run.trial035.stdout: "filestore_wbthrottle_xfs_bytes_start_flusher": "41943040", 2026-02-24T07:43:05.430 INFO:teuthology.orchestra.run.trial035.stdout: "filestore_wbthrottle_xfs_inodes_hard_limit": "5000", 2026-02-24T07:43:05.430 INFO:teuthology.orchestra.run.trial035.stdout: "filestore_wbthrottle_xfs_inodes_start_flusher": "500", 2026-02-24T07:43:05.430 INFO:teuthology.orchestra.run.trial035.stdout: "filestore_wbthrottle_xfs_ios_hard_limit": "5000", 2026-02-24T07:43:05.430 INFO:teuthology.orchestra.run.trial035.stdout: "filestore_wbthrottle_xfs_ios_start_flusher": "500", 2026-02-24T07:43:05.430 INFO:teuthology.orchestra.run.trial035.stdout: "filestore_xfs_extsize": "false", 2026-02-24T07:43:05.430 INFO:teuthology.orchestra.run.trial035.stdout: "filestore_zfs_snap": "false", 2026-02-24T07:43:05.430 INFO:teuthology.orchestra.run.trial035.stdout: "fio_dir": "/tmp/fio", 2026-02-24T07:43:05.431 INFO:teuthology.orchestra.run.trial035.stdout: "fsid": "3a5983d2-1154-11f1-9c05-d404e6e7d460", 2026-02-24T07:43:05.431 INFO:teuthology.orchestra.run.trial035.stdout: "fuse_allow_other": "true", 2026-02-24T07:43:05.431 INFO:teuthology.orchestra.run.trial035.stdout: "fuse_atomic_o_trunc": "true", 2026-02-24T07:43:05.431 INFO:teuthology.orchestra.run.trial035.stdout: "fuse_big_writes": "true", 2026-02-24T07:43:05.431 INFO:teuthology.orchestra.run.trial035.stdout: "fuse_debug": "false", 2026-02-24T07:43:05.431 INFO:teuthology.orchestra.run.trial035.stdout: "fuse_default_permissions": "false", 2026-02-24T07:43:05.431 INFO:teuthology.orchestra.run.trial035.stdout: "fuse_disable_pagecache": "false", 2026-02-24T07:43:05.431 INFO:teuthology.orchestra.run.trial035.stdout: "fuse_max_write": "0", 2026-02-24T07:43:05.431 INFO:teuthology.orchestra.run.trial035.stdout: "fuse_multithreaded": "true", 2026-02-24T07:43:05.431 INFO:teuthology.orchestra.run.trial035.stdout: "fuse_require_active_mds": "true", 2026-02-24T07:43:05.431 INFO:teuthology.orchestra.run.trial035.stdout: "fuse_set_user_groups": "true", 2026-02-24T07:43:05.431 INFO:teuthology.orchestra.run.trial035.stdout: "fuse_splice_move": "true", 2026-02-24T07:43:05.431 INFO:teuthology.orchestra.run.trial035.stdout: "fuse_splice_read": "true", 2026-02-24T07:43:05.431 INFO:teuthology.orchestra.run.trial035.stdout: "fuse_splice_write": "true", 2026-02-24T07:43:05.431 INFO:teuthology.orchestra.run.trial035.stdout: "fuse_syncfs_on_mksnap": "true", 2026-02-24T07:43:05.431 INFO:teuthology.orchestra.run.trial035.stdout: "fuse_use_invalidate_cb": "true", 2026-02-24T07:43:05.432 INFO:teuthology.orchestra.run.trial035.stdout: "gss_ktab_client_file": "/var/lib/ceph/mon.a/gss_client_mon.a.ktab", 2026-02-24T07:43:05.432 INFO:teuthology.orchestra.run.trial035.stdout: "gss_target_name": "ceph", 2026-02-24T07:43:05.432 INFO:teuthology.orchestra.run.trial035.stdout: "heartbeat_file": "", 2026-02-24T07:43:05.432 INFO:teuthology.orchestra.run.trial035.stdout: "heartbeat_inject_failure": "0", 2026-02-24T07:43:05.432 INFO:teuthology.orchestra.run.trial035.stdout: "heartbeat_interval": "5", 2026-02-24T07:43:05.432 INFO:teuthology.orchestra.run.trial035.stdout: "host": "trial035", 2026-02-24T07:43:05.432 INFO:teuthology.orchestra.run.trial035.stdout: "immutable_object_cache_client_dedicated_thread_num": "2", 2026-02-24T07:43:05.432 INFO:teuthology.orchestra.run.trial035.stdout: "immutable_object_cache_max_inflight_ops": "128", 2026-02-24T07:43:05.432 INFO:teuthology.orchestra.run.trial035.stdout: "immutable_object_cache_max_size": "1073741824", 2026-02-24T07:43:05.432 INFO:teuthology.orchestra.run.trial035.stdout: "immutable_object_cache_path": "/tmp/ceph_immutable_object_cache", 2026-02-24T07:43:05.432 INFO:teuthology.orchestra.run.trial035.stdout: "immutable_object_cache_qos_bps_burst": "0", 2026-02-24T07:43:05.432 INFO:teuthology.orchestra.run.trial035.stdout: "immutable_object_cache_qos_bps_burst_seconds": "1", 2026-02-24T07:43:05.432 INFO:teuthology.orchestra.run.trial035.stdout: "immutable_object_cache_qos_bps_limit": "0", 2026-02-24T07:43:05.432 INFO:teuthology.orchestra.run.trial035.stdout: "immutable_object_cache_qos_iops_burst": "0", 2026-02-24T07:43:05.432 INFO:teuthology.orchestra.run.trial035.stdout: "immutable_object_cache_qos_iops_burst_seconds": "1", 2026-02-24T07:43:05.433 INFO:teuthology.orchestra.run.trial035.stdout: "immutable_object_cache_qos_iops_limit": "0", 2026-02-24T07:43:05.433 INFO:teuthology.orchestra.run.trial035.stdout: "immutable_object_cache_qos_schedule_tick_min": "50", 2026-02-24T07:43:05.433 INFO:teuthology.orchestra.run.trial035.stdout: "immutable_object_cache_sock": "/var/run/ceph/immutable_object_cache_sock", 2026-02-24T07:43:05.433 INFO:teuthology.orchestra.run.trial035.stdout: "immutable_object_cache_watermark": "0.900000", 2026-02-24T07:43:05.433 INFO:teuthology.orchestra.run.trial035.stdout: "inject_early_sigterm": "false", 2026-02-24T07:43:05.433 INFO:teuthology.orchestra.run.trial035.stdout: "jaeger_agent_port": "6799", 2026-02-24T07:43:05.433 INFO:teuthology.orchestra.run.trial035.stdout: "jaeger_tracing_enable": "false", 2026-02-24T07:43:05.433 INFO:teuthology.orchestra.run.trial035.stdout: "journal_aio": "true", 2026-02-24T07:43:05.433 INFO:teuthology.orchestra.run.trial035.stdout: "journal_align_min_size": "65536", 2026-02-24T07:43:05.433 INFO:teuthology.orchestra.run.trial035.stdout: "journal_block_align": "true", 2026-02-24T07:43:05.433 INFO:teuthology.orchestra.run.trial035.stdout: "journal_block_size": "4096", 2026-02-24T07:43:05.433 INFO:teuthology.orchestra.run.trial035.stdout: "journal_dio": "true", 2026-02-24T07:43:05.433 INFO:teuthology.orchestra.run.trial035.stdout: "journal_discard": "false", 2026-02-24T07:43:05.433 INFO:teuthology.orchestra.run.trial035.stdout: "journal_force_aio": "false", 2026-02-24T07:43:05.433 INFO:teuthology.orchestra.run.trial035.stdout: "journal_ignore_corruption": "false", 2026-02-24T07:43:05.433 INFO:teuthology.orchestra.run.trial035.stdout: "journal_max_write_bytes": "10485760", 2026-02-24T07:43:05.433 INFO:teuthology.orchestra.run.trial035.stdout: "journal_max_write_entries": "100", 2026-02-24T07:43:05.434 INFO:teuthology.orchestra.run.trial035.stdout: "journal_replay_from": "0", 2026-02-24T07:43:05.434 INFO:teuthology.orchestra.run.trial035.stdout: "journal_throttle_high_multiple": "0.000000", 2026-02-24T07:43:05.434 INFO:teuthology.orchestra.run.trial035.stdout: "journal_throttle_high_threshhold": "0.900000", 2026-02-24T07:43:05.434 INFO:teuthology.orchestra.run.trial035.stdout: "journal_throttle_low_threshhold": "0.600000", 2026-02-24T07:43:05.434 INFO:teuthology.orchestra.run.trial035.stdout: "journal_throttle_max_multiple": "0.000000", 2026-02-24T07:43:05.434 INFO:teuthology.orchestra.run.trial035.stdout: "journal_write_header_frequency": "0", 2026-02-24T07:43:05.434 INFO:teuthology.orchestra.run.trial035.stdout: "journal_zero_on_create": "false", 2026-02-24T07:43:05.434 INFO:teuthology.orchestra.run.trial035.stdout: "journaler_prefetch_periods": "10", 2026-02-24T07:43:05.434 INFO:teuthology.orchestra.run.trial035.stdout: "journaler_prezero_periods": "5", 2026-02-24T07:43:05.434 INFO:teuthology.orchestra.run.trial035.stdout: "journaler_write_head_interval": "15", 2026-02-24T07:43:05.434 INFO:teuthology.orchestra.run.trial035.stdout: "key": "", 2026-02-24T07:43:05.434 INFO:teuthology.orchestra.run.trial035.stdout: "keyfile": "", 2026-02-24T07:43:05.434 INFO:teuthology.orchestra.run.trial035.stdout: "keyring": "/var/lib/ceph/mon/ceph-a/keyring", 2026-02-24T07:43:05.434 INFO:teuthology.orchestra.run.trial035.stdout: "librados_thread_count": "2", 2026-02-24T07:43:05.434 INFO:teuthology.orchestra.run.trial035.stdout: "lockdep": "false", 2026-02-24T07:43:05.434 INFO:teuthology.orchestra.run.trial035.stdout: "lockdep_force_backtrace": "false", 2026-02-24T07:43:05.434 INFO:teuthology.orchestra.run.trial035.stdout: "log_coarse_timestamps": "true", 2026-02-24T07:43:05.434 INFO:teuthology.orchestra.run.trial035.stdout: "log_file": "/var/log/ceph/ceph-mon.a.log", 2026-02-24T07:43:05.435 INFO:teuthology.orchestra.run.trial035.stdout: "log_flush_on_exit": "false", 2026-02-24T07:43:05.435 INFO:teuthology.orchestra.run.trial035.stdout: "log_graylog_host": "127.0.0.1", 2026-02-24T07:43:05.435 INFO:teuthology.orchestra.run.trial035.stdout: "log_graylog_port": "12201", 2026-02-24T07:43:05.435 INFO:teuthology.orchestra.run.trial035.stdout: "log_max_new": "1000", 2026-02-24T07:43:05.435 INFO:teuthology.orchestra.run.trial035.stdout: "log_max_recent": "10000", 2026-02-24T07:43:05.435 INFO:teuthology.orchestra.run.trial035.stdout: "log_stderr_prefix": "", 2026-02-24T07:43:05.435 INFO:teuthology.orchestra.run.trial035.stdout: "log_stop_at_utilization": "0.970000", 2026-02-24T07:43:05.435 INFO:teuthology.orchestra.run.trial035.stdout: "log_to_file": "true", 2026-02-24T07:43:05.435 INFO:teuthology.orchestra.run.trial035.stdout: "log_to_graylog": "false", 2026-02-24T07:43:05.435 INFO:teuthology.orchestra.run.trial035.stdout: "log_to_journald": "false", 2026-02-24T07:43:05.435 INFO:teuthology.orchestra.run.trial035.stdout: "log_to_stderr": "false", 2026-02-24T07:43:05.435 INFO:teuthology.orchestra.run.trial035.stdout: "log_to_syslog": "false", 2026-02-24T07:43:05.435 INFO:teuthology.orchestra.run.trial035.stdout: "max_rotating_auth_attempts": "10", 2026-02-24T07:43:05.435 INFO:teuthology.orchestra.run.trial035.stdout: "mds_abort_on_newly_corrupt_dentry": "true", 2026-02-24T07:43:05.435 INFO:teuthology.orchestra.run.trial035.stdout: "mds_action_on_write_error": "1", 2026-02-24T07:43:05.435 INFO:teuthology.orchestra.run.trial035.stdout: "mds_allow_async_dirops": "true", 2026-02-24T07:43:05.435 INFO:teuthology.orchestra.run.trial035.stdout: "mds_allow_batched_ops": "true", 2026-02-24T07:43:05.436 INFO:teuthology.orchestra.run.trial035.stdout: "mds_alternate_name_max": "8192", 2026-02-24T07:43:05.436 INFO:teuthology.orchestra.run.trial035.stdout: "mds_asio_thread_count": "2", 2026-02-24T07:43:05.436 INFO:teuthology.orchestra.run.trial035.stdout: "mds_bal_export_pin": "true", 2026-02-24T07:43:05.436 INFO:teuthology.orchestra.run.trial035.stdout: "mds_bal_fragment_dirs": "true", 2026-02-24T07:43:05.436 INFO:teuthology.orchestra.run.trial035.stdout: "mds_bal_fragment_fast_factor": "1.500000", 2026-02-24T07:43:05.436 INFO:teuthology.orchestra.run.trial035.stdout: "mds_bal_fragment_interval": "5", 2026-02-24T07:43:05.436 INFO:teuthology.orchestra.run.trial035.stdout: "mds_bal_fragment_size_max": "100000", 2026-02-24T07:43:05.436 INFO:teuthology.orchestra.run.trial035.stdout: "mds_bal_idle_threshold": "0.000000", 2026-02-24T07:43:05.436 INFO:teuthology.orchestra.run.trial035.stdout: "mds_bal_interval": "10", 2026-02-24T07:43:05.436 INFO:teuthology.orchestra.run.trial035.stdout: "mds_bal_max": "-1", 2026-02-24T07:43:05.436 INFO:teuthology.orchestra.run.trial035.stdout: "mds_bal_max_until": "-1", 2026-02-24T07:43:05.436 INFO:teuthology.orchestra.run.trial035.stdout: "mds_bal_merge_size": "50", 2026-02-24T07:43:05.436 INFO:teuthology.orchestra.run.trial035.stdout: "mds_bal_midchunk": "0.300000", 2026-02-24T07:43:05.436 INFO:teuthology.orchestra.run.trial035.stdout: "mds_bal_min_rebalance": "0.100000", 2026-02-24T07:43:05.436 INFO:teuthology.orchestra.run.trial035.stdout: "mds_bal_min_start": "0.200000", 2026-02-24T07:43:05.436 INFO:teuthology.orchestra.run.trial035.stdout: "mds_bal_minchunk": "0.001000", 2026-02-24T07:43:05.436 INFO:teuthology.orchestra.run.trial035.stdout: "mds_bal_mode": "0", 2026-02-24T07:43:05.436 INFO:teuthology.orchestra.run.trial035.stdout: "mds_bal_need_max": "1.200000", 2026-02-24T07:43:05.436 INFO:teuthology.orchestra.run.trial035.stdout: "mds_bal_need_min": "0.800000", 2026-02-24T07:43:05.436 INFO:teuthology.orchestra.run.trial035.stdout: "mds_bal_overload_epochs": "2", 2026-02-24T07:43:05.437 INFO:teuthology.orchestra.run.trial035.stdout: "mds_bal_replicate_threshold": "8000.000000", 2026-02-24T07:43:05.437 INFO:teuthology.orchestra.run.trial035.stdout: "mds_bal_sample_interval": "3.000000", 2026-02-24T07:43:05.437 INFO:teuthology.orchestra.run.trial035.stdout: "mds_bal_split_bits": "3", 2026-02-24T07:43:05.437 INFO:teuthology.orchestra.run.trial035.stdout: "mds_bal_split_rd": "25000.000000", 2026-02-24T07:43:05.437 INFO:teuthology.orchestra.run.trial035.stdout: "mds_bal_split_size": "10000", 2026-02-24T07:43:05.437 INFO:teuthology.orchestra.run.trial035.stdout: "mds_bal_split_wr": "10000.000000", 2026-02-24T07:43:05.437 INFO:teuthology.orchestra.run.trial035.stdout: "mds_bal_target_decay": "10.000000", 2026-02-24T07:43:05.437 INFO:teuthology.orchestra.run.trial035.stdout: "mds_bal_unreplicate_threshold": "0.000000", 2026-02-24T07:43:05.437 INFO:teuthology.orchestra.run.trial035.stdout: "mds_beacon_grace": "15.000000", 2026-02-24T07:43:05.437 INFO:teuthology.orchestra.run.trial035.stdout: "mds_beacon_interval": "4.000000", 2026-02-24T07:43:05.437 INFO:teuthology.orchestra.run.trial035.stdout: "mds_beacon_mon_down_grace": "60", 2026-02-24T07:43:05.437 INFO:teuthology.orchestra.run.trial035.stdout: "mds_cache_memory_limit": "4294967296", 2026-02-24T07:43:05.437 INFO:teuthology.orchestra.run.trial035.stdout: "mds_cache_mid": "0.700000", 2026-02-24T07:43:05.437 INFO:teuthology.orchestra.run.trial035.stdout: "mds_cache_quiesce_decay_rate": "1.000000", 2026-02-24T07:43:05.437 INFO:teuthology.orchestra.run.trial035.stdout: "mds_cache_quiesce_delay": "0", 2026-02-24T07:43:05.437 INFO:teuthology.orchestra.run.trial035.stdout: "mds_cache_quiesce_sleep": "200", 2026-02-24T07:43:05.437 INFO:teuthology.orchestra.run.trial035.stdout: "mds_cache_quiesce_splitauth": "true", 2026-02-24T07:43:05.437 INFO:teuthology.orchestra.run.trial035.stdout: "mds_cache_quiesce_threshold": "524288", 2026-02-24T07:43:05.438 INFO:teuthology.orchestra.run.trial035.stdout: "mds_cache_release_free_interval": "10", 2026-02-24T07:43:05.438 INFO:teuthology.orchestra.run.trial035.stdout: "mds_cache_reservation": "0.050000", 2026-02-24T07:43:05.438 INFO:teuthology.orchestra.run.trial035.stdout: "mds_cache_trim_decay_rate": "1.000000", 2026-02-24T07:43:05.438 INFO:teuthology.orchestra.run.trial035.stdout: "mds_cache_trim_interval": "1", 2026-02-24T07:43:05.438 INFO:teuthology.orchestra.run.trial035.stdout: "mds_cache_trim_threshold": "262144", 2026-02-24T07:43:05.438 INFO:teuthology.orchestra.run.trial035.stdout: "mds_cap_acquisition_throttle_retry_request_timeout": "0.500000", 2026-02-24T07:43:05.438 INFO:teuthology.orchestra.run.trial035.stdout: "mds_cap_revoke_eviction_timeout": "0.000000", 2026-02-24T07:43:05.438 INFO:teuthology.orchestra.run.trial035.stdout: "mds_client_delegate_inos_pct": "50", 2026-02-24T07:43:05.438 INFO:teuthology.orchestra.run.trial035.stdout: "mds_client_prealloc_inos": "1000", 2026-02-24T07:43:05.438 INFO:teuthology.orchestra.run.trial035.stdout: "mds_client_writeable_range_max_inc_objs": "1024", 2026-02-24T07:43:05.438 INFO:teuthology.orchestra.run.trial035.stdout: "mds_connect_bootstrapping": "false", 2026-02-24T07:43:05.438 INFO:teuthology.orchestra.run.trial035.stdout: "mds_damage_table_max_entries": "10000", 2026-02-24T07:43:05.438 INFO:teuthology.orchestra.run.trial035.stdout: "mds_data": "/var/lib/ceph/mds/ceph-a", 2026-02-24T07:43:05.438 INFO:teuthology.orchestra.run.trial035.stdout: "mds_debug_auth_pins": "false", 2026-02-24T07:43:05.438 INFO:teuthology.orchestra.run.trial035.stdout: "mds_debug_frag": "false", 2026-02-24T07:43:05.438 INFO:teuthology.orchestra.run.trial035.stdout: "mds_debug_scatterstat": "false", 2026-02-24T07:43:05.438 INFO:teuthology.orchestra.run.trial035.stdout: "mds_debug_subtrees": "false", 2026-02-24T07:43:05.438 INFO:teuthology.orchestra.run.trial035.stdout: "mds_decay_halflife": "5.000000", 2026-02-24T07:43:05.438 INFO:teuthology.orchestra.run.trial035.stdout: "mds_default_dir_hash": "2", 2026-02-24T07:43:05.439 INFO:teuthology.orchestra.run.trial035.stdout: "mds_defer_session_stale": "true", 2026-02-24T07:43:05.439 INFO:teuthology.orchestra.run.trial035.stdout: "mds_delay_journal_replay_for_testing": "0", 2026-02-24T07:43:05.439 INFO:teuthology.orchestra.run.trial035.stdout: "mds_deny_all_reconnect": "false", 2026-02-24T07:43:05.439 INFO:teuthology.orchestra.run.trial035.stdout: "mds_dir_keys_per_op": "16384", 2026-02-24T07:43:05.439 INFO:teuthology.orchestra.run.trial035.stdout: "mds_dir_max_commit_size": "10", 2026-02-24T07:43:05.439 INFO:teuthology.orchestra.run.trial035.stdout: "mds_dir_max_entries": "0", 2026-02-24T07:43:05.439 INFO:teuthology.orchestra.run.trial035.stdout: "mds_dir_prefetch": "true", 2026-02-24T07:43:05.439 INFO:teuthology.orchestra.run.trial035.stdout: "mds_dirstat_min_interval": "1.000000", 2026-02-24T07:43:05.439 INFO:teuthology.orchestra.run.trial035.stdout: "mds_dump_cache_after_rejoin": "false", 2026-02-24T07:43:05.439 INFO:teuthology.orchestra.run.trial035.stdout: "mds_dump_cache_on_map": "false", 2026-02-24T07:43:05.439 INFO:teuthology.orchestra.run.trial035.stdout: "mds_dump_cache_threshold_file": "0", 2026-02-24T07:43:05.439 INFO:teuthology.orchestra.run.trial035.stdout: "mds_dump_cache_threshold_formatter": "1073741824", 2026-02-24T07:43:05.439 INFO:teuthology.orchestra.run.trial035.stdout: "mds_early_reply": "true", 2026-02-24T07:43:05.439 INFO:teuthology.orchestra.run.trial035.stdout: "mds_enable_op_tracker": "true", 2026-02-24T07:43:05.439 INFO:teuthology.orchestra.run.trial035.stdout: "mds_enforce_unique_name": "true", 2026-02-24T07:43:05.439 INFO:teuthology.orchestra.run.trial035.stdout: "mds_export_ephemeral_distributed": "true", 2026-02-24T07:43:05.439 INFO:teuthology.orchestra.run.trial035.stdout: "mds_export_ephemeral_distributed_factor": "2.000000", 2026-02-24T07:43:05.439 INFO:teuthology.orchestra.run.trial035.stdout: "mds_export_ephemeral_random": "true", 2026-02-24T07:43:05.439 INFO:teuthology.orchestra.run.trial035.stdout: "mds_export_ephemeral_random_max": "0.010000", 2026-02-24T07:43:05.439 INFO:teuthology.orchestra.run.trial035.stdout: "mds_extraordinary_events_dump_interval": "0", 2026-02-24T07:43:05.439 INFO:teuthology.orchestra.run.trial035.stdout: "mds_file_blockdiff_max_concurrent_object_scans": "16", 2026-02-24T07:43:05.440 INFO:teuthology.orchestra.run.trial035.stdout: "mds_forward_all_requests_to_auth": "false", 2026-02-24T07:43:05.440 INFO:teuthology.orchestra.run.trial035.stdout: "mds_freeze_tree_timeout": "30.000000", 2026-02-24T07:43:05.440 INFO:teuthology.orchestra.run.trial035.stdout: "mds_fscrypt_last_block_max_size": "4096", 2026-02-24T07:43:05.440 INFO:teuthology.orchestra.run.trial035.stdout: "mds_go_bad_corrupt_dentry": "true", 2026-02-24T07:43:05.440 INFO:teuthology.orchestra.run.trial035.stdout: "mds_hack_allow_loading_invalid_metadata": "false", 2026-02-24T07:43:05.440 INFO:teuthology.orchestra.run.trial035.stdout: "mds_health_cache_threshold": "1.500000", 2026-02-24T07:43:05.440 INFO:teuthology.orchestra.run.trial035.stdout: "mds_health_summarize_threshold": "10", 2026-02-24T07:43:05.440 INFO:teuthology.orchestra.run.trial035.stdout: "mds_heartbeat_grace": "15.000000", 2026-02-24T07:43:05.440 INFO:teuthology.orchestra.run.trial035.stdout: "mds_heartbeat_reset_grace": "1000", 2026-02-24T07:43:05.440 INFO:teuthology.orchestra.run.trial035.stdout: "mds_inject_health_dummy": "false", 2026-02-24T07:43:05.440 INFO:teuthology.orchestra.run.trial035.stdout: "mds_inject_journal_corrupt_dentry_first": "0.000000", 2026-02-24T07:43:05.440 INFO:teuthology.orchestra.run.trial035.stdout: "mds_inject_migrator_session_race": "false", 2026-02-24T07:43:05.440 INFO:teuthology.orchestra.run.trial035.stdout: "mds_inject_rename_corrupt_dentry_first": "0.000000", 2026-02-24T07:43:05.440 INFO:teuthology.orchestra.run.trial035.stdout: "mds_inject_skip_replaying_inotable": "false", 2026-02-24T07:43:05.440 INFO:teuthology.orchestra.run.trial035.stdout: "mds_inject_traceless_reply_probability": "0.000000", 2026-02-24T07:43:05.440 INFO:teuthology.orchestra.run.trial035.stdout: "mds_join_fs": "", 2026-02-24T07:43:05.440 INFO:teuthology.orchestra.run.trial035.stdout: "mds_journal_format": "1", 2026-02-24T07:43:05.440 INFO:teuthology.orchestra.run.trial035.stdout: "mds_kill_after_journal_logs_flushed": "false", 2026-02-24T07:43:05.440 INFO:teuthology.orchestra.run.trial035.stdout: "mds_kill_create_at": "0", 2026-02-24T07:43:05.440 INFO:teuthology.orchestra.run.trial035.stdout: "mds_kill_dirfrag_at": "0", 2026-02-24T07:43:05.440 INFO:teuthology.orchestra.run.trial035.stdout: "mds_kill_export_at": "0", 2026-02-24T07:43:05.441 INFO:teuthology.orchestra.run.trial035.stdout: "mds_kill_import_at": "0", 2026-02-24T07:43:05.441 INFO:teuthology.orchestra.run.trial035.stdout: "mds_kill_journal_at": "0", 2026-02-24T07:43:05.441 INFO:teuthology.orchestra.run.trial035.stdout: "mds_kill_journal_expire_at": "0", 2026-02-24T07:43:05.441 INFO:teuthology.orchestra.run.trial035.stdout: "mds_kill_journal_replay_at": "0", 2026-02-24T07:43:05.441 INFO:teuthology.orchestra.run.trial035.stdout: "mds_kill_link_at": "0", 2026-02-24T07:43:05.441 INFO:teuthology.orchestra.run.trial035.stdout: "mds_kill_mdstable_at": "0", 2026-02-24T07:43:05.441 INFO:teuthology.orchestra.run.trial035.stdout: "mds_kill_openc_at": "0", 2026-02-24T07:43:05.441 INFO:teuthology.orchestra.run.trial035.stdout: "mds_kill_rename_at": "0", 2026-02-24T07:43:05.441 INFO:teuthology.orchestra.run.trial035.stdout: "mds_kill_shutdown_at": "0", 2026-02-24T07:43:05.441 INFO:teuthology.orchestra.run.trial035.stdout: "mds_log_event_large_threshold": "524288", 2026-02-24T07:43:05.441 INFO:teuthology.orchestra.run.trial035.stdout: "mds_log_events_per_segment": "1024", 2026-02-24T07:43:05.441 INFO:teuthology.orchestra.run.trial035.stdout: "mds_log_max_events": "-1", 2026-02-24T07:43:05.441 INFO:teuthology.orchestra.run.trial035.stdout: "mds_log_max_segments": "128", 2026-02-24T07:43:05.441 INFO:teuthology.orchestra.run.trial035.stdout: "mds_log_minor_segments_per_major_segment": "16", 2026-02-24T07:43:05.441 INFO:teuthology.orchestra.run.trial035.stdout: "mds_log_pause": "false", 2026-02-24T07:43:05.441 INFO:teuthology.orchestra.run.trial035.stdout: "mds_log_segment_size": "0", 2026-02-24T07:43:05.441 INFO:teuthology.orchestra.run.trial035.stdout: "mds_log_skip_corrupt_events": "false", 2026-02-24T07:43:05.441 INFO:teuthology.orchestra.run.trial035.stdout: "mds_log_skip_unbounded_events": "false", 2026-02-24T07:43:05.441 INFO:teuthology.orchestra.run.trial035.stdout: "mds_log_trim_decay_rate": "1.000000", 2026-02-24T07:43:05.441 INFO:teuthology.orchestra.run.trial035.stdout: "mds_log_trim_threshold": "128", 2026-02-24T07:43:05.441 INFO:teuthology.orchestra.run.trial035.stdout: "mds_log_trim_upkeep_interval": "1000", 2026-02-24T07:43:05.441 INFO:teuthology.orchestra.run.trial035.stdout: "mds_log_warn_factor": "2.000000", 2026-02-24T07:43:05.442 INFO:teuthology.orchestra.run.trial035.stdout: "mds_max_caps_per_client": "1048576", 2026-02-24T07:43:05.442 INFO:teuthology.orchestra.run.trial035.stdout: "mds_max_completed_flushes": "100000", 2026-02-24T07:43:05.442 INFO:teuthology.orchestra.run.trial035.stdout: "mds_max_completed_requests": "100000", 2026-02-24T07:43:05.442 INFO:teuthology.orchestra.run.trial035.stdout: "mds_max_export_size": "20971520", 2026-02-24T07:43:05.442 INFO:teuthology.orchestra.run.trial035.stdout: "mds_max_file_recover": "32", 2026-02-24T07:43:05.442 INFO:teuthology.orchestra.run.trial035.stdout: "mds_max_purge_files": "64", 2026-02-24T07:43:05.442 INFO:teuthology.orchestra.run.trial035.stdout: "mds_max_purge_ops": "8192", 2026-02-24T07:43:05.442 INFO:teuthology.orchestra.run.trial035.stdout: "mds_max_purge_ops_per_pg": "0.500000", 2026-02-24T07:43:05.442 INFO:teuthology.orchestra.run.trial035.stdout: "mds_max_scrub_ops_in_progress": "5", 2026-02-24T07:43:05.442 INFO:teuthology.orchestra.run.trial035.stdout: "mds_max_snaps_per_dir": "100", 2026-02-24T07:43:05.442 INFO:teuthology.orchestra.run.trial035.stdout: "mds_metrics_update_interval": "2", 2026-02-24T07:43:05.442 INFO:teuthology.orchestra.run.trial035.stdout: "mds_min_caps_per_client": "100", 2026-02-24T07:43:05.442 INFO:teuthology.orchestra.run.trial035.stdout: "mds_min_caps_working_set": "10000", 2026-02-24T07:43:05.442 INFO:teuthology.orchestra.run.trial035.stdout: "mds_mon_shutdown_timeout": "5.000000", 2026-02-24T07:43:05.442 INFO:teuthology.orchestra.run.trial035.stdout: "mds_numa_node": "-1", 2026-02-24T07:43:05.442 INFO:teuthology.orchestra.run.trial035.stdout: "mds_oft_prefetch_dirfrags": "false", 2026-02-24T07:43:05.442 INFO:teuthology.orchestra.run.trial035.stdout: "mds_op_complaint_time": "30.000000", 2026-02-24T07:43:05.442 INFO:teuthology.orchestra.run.trial035.stdout: "mds_op_history_duration": "600", 2026-02-24T07:43:05.442 INFO:teuthology.orchestra.run.trial035.stdout: "mds_op_history_size": "20", 2026-02-24T07:43:05.442 INFO:teuthology.orchestra.run.trial035.stdout: "mds_op_history_slow_op_size": "20", 2026-02-24T07:43:05.442 INFO:teuthology.orchestra.run.trial035.stdout: "mds_op_history_slow_op_threshold": "10", 2026-02-24T07:43:05.442 INFO:teuthology.orchestra.run.trial035.stdout: "mds_op_log_threshold": "5", 2026-02-24T07:43:05.442 INFO:teuthology.orchestra.run.trial035.stdout: "mds_ping_grace": "15", 2026-02-24T07:43:05.442 INFO:teuthology.orchestra.run.trial035.stdout: "mds_ping_interval": "5", 2026-02-24T07:43:05.443 INFO:teuthology.orchestra.run.trial035.stdout: "mds_purge_queue_busy_flush_period": "1.000000", 2026-02-24T07:43:05.443 INFO:teuthology.orchestra.run.trial035.stdout: "mds_recall_global_max_decay_threshold": "131072", 2026-02-24T07:43:05.443 INFO:teuthology.orchestra.run.trial035.stdout: "mds_recall_max_caps": "30000", 2026-02-24T07:43:05.443 INFO:teuthology.orchestra.run.trial035.stdout: "mds_recall_max_decay_rate": "1.500000", 2026-02-24T07:43:05.443 INFO:teuthology.orchestra.run.trial035.stdout: "mds_recall_max_decay_threshold": "131072", 2026-02-24T07:43:05.443 INFO:teuthology.orchestra.run.trial035.stdout: "mds_recall_warning_decay_rate": "60.000000", 2026-02-24T07:43:05.443 INFO:teuthology.orchestra.run.trial035.stdout: "mds_recall_warning_threshold": "262144", 2026-02-24T07:43:05.443 INFO:teuthology.orchestra.run.trial035.stdout: "mds_reconnect_timeout": "45.000000", 2026-02-24T07:43:05.443 INFO:teuthology.orchestra.run.trial035.stdout: "mds_replay_interval": "1.000000", 2026-02-24T07:43:05.443 INFO:teuthology.orchestra.run.trial035.stdout: "mds_replay_unsafe_with_closed_session": "false", 2026-02-24T07:43:05.443 INFO:teuthology.orchestra.run.trial035.stdout: "mds_request_load_average_decay_rate": "60.000000", 2026-02-24T07:43:05.443 INFO:teuthology.orchestra.run.trial035.stdout: "mds_root_ino_gid": "0", 2026-02-24T07:43:05.443 INFO:teuthology.orchestra.run.trial035.stdout: "mds_root_ino_uid": "0", 2026-02-24T07:43:05.443 INFO:teuthology.orchestra.run.trial035.stdout: "mds_scatter_nudge_interval": "5.000000", 2026-02-24T07:43:05.443 INFO:teuthology.orchestra.run.trial035.stdout: "mds_scrub_stats_review_period": "1", 2026-02-24T07:43:05.443 INFO:teuthology.orchestra.run.trial035.stdout: "mds_server_dispatch_client_request_delay": "0", 2026-02-24T07:43:05.443 INFO:teuthology.orchestra.run.trial035.stdout: "mds_server_dispatch_killpoint_random": "0.000000", 2026-02-24T07:43:05.443 INFO:teuthology.orchestra.run.trial035.stdout: "mds_session_blocklist_on_evict": "true", 2026-02-24T07:43:05.443 INFO:teuthology.orchestra.run.trial035.stdout: "mds_session_blocklist_on_timeout": "true", 2026-02-24T07:43:05.443 INFO:teuthology.orchestra.run.trial035.stdout: "mds_session_cache_liveness_decay_rate": "300.000000", 2026-02-24T07:43:05.443 INFO:teuthology.orchestra.run.trial035.stdout: "mds_session_cache_liveness_magnitude": "10", 2026-02-24T07:43:05.444 INFO:teuthology.orchestra.run.trial035.stdout: "mds_session_cap_acquisition_decay_rate": "30.000000", 2026-02-24T07:43:05.444 INFO:teuthology.orchestra.run.trial035.stdout: "mds_session_cap_acquisition_throttle": "100000", 2026-02-24T07:43:05.444 INFO:teuthology.orchestra.run.trial035.stdout: "mds_session_max_caps_throttle_ratio": "1.100000", 2026-02-24T07:43:05.444 INFO:teuthology.orchestra.run.trial035.stdout: "mds_session_metadata_threshold": "16777216", 2026-02-24T07:43:05.444 INFO:teuthology.orchestra.run.trial035.stdout: "mds_sessionmap_keys_per_op": "1024", 2026-02-24T07:43:05.444 INFO:teuthology.orchestra.run.trial035.stdout: "mds_shutdown_check": "0", 2026-02-24T07:43:05.444 INFO:teuthology.orchestra.run.trial035.stdout: "mds_skip_ino": "0", 2026-02-24T07:43:05.444 INFO:teuthology.orchestra.run.trial035.stdout: "mds_sleep_rank_change": "0.000000", 2026-02-24T07:43:05.444 INFO:teuthology.orchestra.run.trial035.stdout: "mds_snap_max_uid": "4294967294", 2026-02-24T07:43:05.444 INFO:teuthology.orchestra.run.trial035.stdout: "mds_snap_min_uid": "0", 2026-02-24T07:43:05.444 INFO:teuthology.orchestra.run.trial035.stdout: "mds_snap_rstat": "false", 2026-02-24T07:43:05.444 INFO:teuthology.orchestra.run.trial035.stdout: "mds_standby_replay_damaged": "false", 2026-02-24T07:43:05.444 INFO:teuthology.orchestra.run.trial035.stdout: "mds_symlink_recovery": "true", 2026-02-24T07:43:05.444 INFO:teuthology.orchestra.run.trial035.stdout: "mds_task_status_update_interval": "2.000000", 2026-02-24T07:43:05.444 INFO:teuthology.orchestra.run.trial035.stdout: "mds_thrash_exports": "0", 2026-02-24T07:43:05.444 INFO:teuthology.orchestra.run.trial035.stdout: "mds_thrash_fragments": "0", 2026-02-24T07:43:05.444 INFO:teuthology.orchestra.run.trial035.stdout: "mds_tick_interval": "5.000000", 2026-02-24T07:43:05.444 INFO:teuthology.orchestra.run.trial035.stdout: "mds_use_global_snaprealm_seq_for_subvol": "true", 2026-02-24T07:43:05.444 INFO:teuthology.orchestra.run.trial035.stdout: "mds_valgrind_exit": "false", 2026-02-24T07:43:05.444 INFO:teuthology.orchestra.run.trial035.stdout: "mds_verify_backtrace": "1", 2026-02-24T07:43:05.444 INFO:teuthology.orchestra.run.trial035.stdout: "mds_verify_scatter": "false", 2026-02-24T07:43:05.444 INFO:teuthology.orchestra.run.trial035.stdout: "mds_wipe_ino_prealloc": "false", 2026-02-24T07:43:05.444 INFO:teuthology.orchestra.run.trial035.stdout: "mds_wipe_sessions": "false", 2026-02-24T07:43:05.444 INFO:teuthology.orchestra.run.trial035.stdout: "mempool_debug": "false", 2026-02-24T07:43:05.445 INFO:teuthology.orchestra.run.trial035.stdout: "memstore_debug_omit_block_device_write": "false", 2026-02-24T07:43:05.445 INFO:teuthology.orchestra.run.trial035.stdout: "memstore_device_bytes": "1073741824", 2026-02-24T07:43:05.445 INFO:teuthology.orchestra.run.trial035.stdout: "memstore_page_set": "false", 2026-02-24T07:43:05.445 INFO:teuthology.orchestra.run.trial035.stdout: "memstore_page_size": "65536", 2026-02-24T07:43:05.445 INFO:teuthology.orchestra.run.trial035.stdout: "mgr_client_bytes": "134217728", 2026-02-24T07:43:05.445 INFO:teuthology.orchestra.run.trial035.stdout: "mgr_client_messages": "512", 2026-02-24T07:43:05.445 INFO:teuthology.orchestra.run.trial035.stdout: "mgr_client_service_daemon_unregister_timeout": "1.000000", 2026-02-24T07:43:05.445 INFO:teuthology.orchestra.run.trial035.stdout: "mgr_connect_retry_interval": "1.000000", 2026-02-24T07:43:05.445 INFO:teuthology.orchestra.run.trial035.stdout: "mgr_data": "/var/lib/ceph/mgr/ceph-a", 2026-02-24T07:43:05.445 INFO:teuthology.orchestra.run.trial035.stdout: "mgr_debug_aggressive_pg_num_changes": "false", 2026-02-24T07:43:05.445 INFO:teuthology.orchestra.run.trial035.stdout: "mgr_disabled_modules": "", 2026-02-24T07:43:05.445 INFO:teuthology.orchestra.run.trial035.stdout: "mgr_enable_op_tracker": "true", 2026-02-24T07:43:05.445 INFO:teuthology.orchestra.run.trial035.stdout: "mgr_initial_modules": "iostat nfs", 2026-02-24T07:43:05.445 INFO:teuthology.orchestra.run.trial035.stdout: "mgr_max_pg_creating": "1024", 2026-02-24T07:43:05.445 INFO:teuthology.orchestra.run.trial035.stdout: "mgr_max_pg_num_change": "128", 2026-02-24T07:43:05.445 INFO:teuthology.orchestra.run.trial035.stdout: "mgr_mds_bytes": "134217728", 2026-02-24T07:43:05.445 INFO:teuthology.orchestra.run.trial035.stdout: "mgr_mds_messages": "128", 2026-02-24T07:43:05.445 INFO:teuthology.orchestra.run.trial035.stdout: "mgr_module_path": "/usr/share/ceph/mgr", 2026-02-24T07:43:05.445 INFO:teuthology.orchestra.run.trial035.stdout: "mgr_mon_bytes": "134217728", 2026-02-24T07:43:05.445 INFO:teuthology.orchestra.run.trial035.stdout: "mgr_mon_messages": "128", 2026-02-24T07:43:05.445 INFO:teuthology.orchestra.run.trial035.stdout: "mgr_num_op_tracker_shard": "32", 2026-02-24T07:43:05.445 INFO:teuthology.orchestra.run.trial035.stdout: "mgr_op_complaint_time": "30.000000", 2026-02-24T07:43:05.445 INFO:teuthology.orchestra.run.trial035.stdout: "mgr_op_history_duration": "600", 2026-02-24T07:43:05.445 INFO:teuthology.orchestra.run.trial035.stdout: "mgr_op_history_size": "20", 2026-02-24T07:43:05.446 INFO:teuthology.orchestra.run.trial035.stdout: "mgr_op_history_slow_op_size": "20", 2026-02-24T07:43:05.446 INFO:teuthology.orchestra.run.trial035.stdout: "mgr_op_history_slow_op_threshold": "10.000000", 2026-02-24T07:43:05.446 INFO:teuthology.orchestra.run.trial035.stdout: "mgr_op_log_threshold": "5", 2026-02-24T07:43:05.446 INFO:teuthology.orchestra.run.trial035.stdout: "mgr_osd_bytes": "536870912", 2026-02-24T07:43:05.446 INFO:teuthology.orchestra.run.trial035.stdout: "mgr_osd_messages": "8192", 2026-02-24T07:43:05.446 INFO:teuthology.orchestra.run.trial035.stdout: "mgr_pool": "true", 2026-02-24T07:43:05.446 INFO:teuthology.orchestra.run.trial035.stdout: "mgr_service_beacon_grace": "60.000000", 2026-02-24T07:43:05.446 INFO:teuthology.orchestra.run.trial035.stdout: "mgr_standby_modules": "true", 2026-02-24T07:43:05.446 INFO:teuthology.orchestra.run.trial035.stdout: "mgr_stats_period": "5", 2026-02-24T07:43:05.446 INFO:teuthology.orchestra.run.trial035.stdout: "mgr_stats_threshold": "5", 2026-02-24T07:43:05.446 INFO:teuthology.orchestra.run.trial035.stdout: "mgr_tick_period": "2", 2026-02-24T07:43:05.446 INFO:teuthology.orchestra.run.trial035.stdout: "mgr_ttl_cache_expire_seconds": "0", 2026-02-24T07:43:05.446 INFO:teuthology.orchestra.run.trial035.stdout: "mon_accept_timeout_factor": "2.000000", 2026-02-24T07:43:05.446 INFO:teuthology.orchestra.run.trial035.stdout: "mon_allow_pool_delete": "true", 2026-02-24T07:43:05.446 INFO:teuthology.orchestra.run.trial035.stdout: "mon_allow_pool_size_one": "false", 2026-02-24T07:43:05.446 INFO:teuthology.orchestra.run.trial035.stdout: "mon_auth_validate_all_caps": "true", 2026-02-24T07:43:05.446 INFO:teuthology.orchestra.run.trial035.stdout: "mon_cache_target_full_warn_ratio": "0.660000", 2026-02-24T07:43:05.446 INFO:teuthology.orchestra.run.trial035.stdout: "mon_clean_pg_upmaps_per_chunk": "256", 2026-02-24T07:43:05.446 INFO:teuthology.orchestra.run.trial035.stdout: "mon_client_bytes": "104857600", 2026-02-24T07:43:05.446 INFO:teuthology.orchestra.run.trial035.stdout: "mon_client_directed_command_retry": "2", 2026-02-24T07:43:05.446 INFO:teuthology.orchestra.run.trial035.stdout: "mon_client_hunt_interval": "3.000000", 2026-02-24T07:43:05.446 INFO:teuthology.orchestra.run.trial035.stdout: "mon_client_hunt_interval_backoff": "1.500000", 2026-02-24T07:43:05.446 INFO:teuthology.orchestra.run.trial035.stdout: "mon_client_hunt_interval_max_multiple": "10.000000", 2026-02-24T07:43:05.446 INFO:teuthology.orchestra.run.trial035.stdout: "mon_client_hunt_interval_min_multiple": "1.000000", 2026-02-24T07:43:05.447 INFO:teuthology.orchestra.run.trial035.stdout: "mon_client_hunt_on_resend": "true", 2026-02-24T07:43:05.447 INFO:teuthology.orchestra.run.trial035.stdout: "mon_client_hunt_parallel": "3", 2026-02-24T07:43:05.447 INFO:teuthology.orchestra.run.trial035.stdout: "mon_client_log_interval": "1.000000", 2026-02-24T07:43:05.447 INFO:teuthology.orchestra.run.trial035.stdout: "mon_client_max_log_entries_per_message": "1000", 2026-02-24T07:43:05.447 INFO:teuthology.orchestra.run.trial035.stdout: "mon_client_ping_interval": "10.000000", 2026-02-24T07:43:05.447 INFO:teuthology.orchestra.run.trial035.stdout: "mon_client_ping_timeout": "30.000000", 2026-02-24T07:43:05.447 INFO:teuthology.orchestra.run.trial035.stdout: "mon_client_target_rank": "-1", 2026-02-24T07:43:05.447 INFO:teuthology.orchestra.run.trial035.stdout: "mon_clock_drift_allowed": "1.000000", 2026-02-24T07:43:05.447 INFO:teuthology.orchestra.run.trial035.stdout: "mon_clock_drift_warn_backoff": "5.000000", 2026-02-24T07:43:05.447 INFO:teuthology.orchestra.run.trial035.stdout: "mon_cluster_log_file": "default=/var/log/ceph/ceph.$channel.log cluster=/var/log/ceph/ceph.log", 2026-02-24T07:43:05.447 INFO:teuthology.orchestra.run.trial035.stdout: "mon_cluster_log_level": "debug", 2026-02-24T07:43:05.447 INFO:teuthology.orchestra.run.trial035.stdout: "mon_cluster_log_to_file": "true", 2026-02-24T07:43:05.447 INFO:teuthology.orchestra.run.trial035.stdout: "mon_cluster_log_to_graylog": "false", 2026-02-24T07:43:05.447 INFO:teuthology.orchestra.run.trial035.stdout: "mon_cluster_log_to_graylog_host": "127.0.0.1", 2026-02-24T07:43:05.447 INFO:teuthology.orchestra.run.trial035.stdout: "mon_cluster_log_to_graylog_port": "12201", 2026-02-24T07:43:05.447 INFO:teuthology.orchestra.run.trial035.stdout: "mon_cluster_log_to_journald": "true", 2026-02-24T07:43:05.447 INFO:teuthology.orchestra.run.trial035.stdout: "mon_cluster_log_to_stderr": "false", 2026-02-24T07:43:05.447 INFO:teuthology.orchestra.run.trial035.stdout: "mon_cluster_log_to_syslog": "default=false", 2026-02-24T07:43:05.447 INFO:teuthology.orchestra.run.trial035.stdout: "mon_cluster_log_to_syslog_facility": "daemon", 2026-02-24T07:43:05.447 INFO:teuthology.orchestra.run.trial035.stdout: "mon_compact_on_bootstrap": "false", 2026-02-24T07:43:05.447 INFO:teuthology.orchestra.run.trial035.stdout: "mon_compact_on_start": "false", 2026-02-24T07:43:05.447 INFO:teuthology.orchestra.run.trial035.stdout: "mon_compact_on_trim": "true", 2026-02-24T07:43:05.447 INFO:teuthology.orchestra.run.trial035.stdout: "mon_con_tracker_persist_interval": "10", 2026-02-24T07:43:05.447 INFO:teuthology.orchestra.run.trial035.stdout: "mon_con_tracker_score_halflife": "43200", 2026-02-24T07:43:05.448 INFO:teuthology.orchestra.run.trial035.stdout: "mon_config_key_max_entry_size": "65536", 2026-02-24T07:43:05.448 INFO:teuthology.orchestra.run.trial035.stdout: "mon_cpu_threads": "4", 2026-02-24T07:43:05.448 INFO:teuthology.orchestra.run.trial035.stdout: "mon_crush_min_required_version": "hammer", 2026-02-24T07:43:05.448 INFO:teuthology.orchestra.run.trial035.stdout: "mon_daemon_bytes": "419430400", 2026-02-24T07:43:05.448 INFO:teuthology.orchestra.run.trial035.stdout: "mon_data": "/var/lib/ceph/mon/ceph-a", 2026-02-24T07:43:05.448 INFO:teuthology.orchestra.run.trial035.stdout: "mon_data_avail_crit": "5", 2026-02-24T07:43:05.448 INFO:teuthology.orchestra.run.trial035.stdout: "mon_data_avail_warn": "5", 2026-02-24T07:43:05.448 INFO:teuthology.orchestra.run.trial035.stdout: "mon_data_size_warn": "16106127360", 2026-02-24T07:43:05.448 INFO:teuthology.orchestra.run.trial035.stdout: "mon_debug_block_osdmap_trim": "false", 2026-02-24T07:43:05.448 INFO:teuthology.orchestra.run.trial035.stdout: "mon_debug_deprecated_as_obsolete": "false", 2026-02-24T07:43:05.448 INFO:teuthology.orchestra.run.trial035.stdout: "mon_debug_dump_json": "false", 2026-02-24T07:43:05.448 INFO:teuthology.orchestra.run.trial035.stdout: "mon_debug_dump_location": "/var/log/ceph/ceph-mon.a.tdump", 2026-02-24T07:43:05.448 INFO:teuthology.orchestra.run.trial035.stdout: "mon_debug_dump_transactions": "false", 2026-02-24T07:43:05.448 INFO:teuthology.orchestra.run.trial035.stdout: "mon_debug_extra_checks": "false", 2026-02-24T07:43:05.448 INFO:teuthology.orchestra.run.trial035.stdout: "mon_debug_no_initial_persistent_features": "false", 2026-02-24T07:43:05.448 INFO:teuthology.orchestra.run.trial035.stdout: "mon_debug_no_require_bluestore_for_ec_overwrites": "false", 2026-02-24T07:43:05.448 INFO:teuthology.orchestra.run.trial035.stdout: "mon_debug_no_require_squid": "false", 2026-02-24T07:43:05.448 INFO:teuthology.orchestra.run.trial035.stdout: "mon_debug_no_require_tentacle": "false", 2026-02-24T07:43:05.448 INFO:teuthology.orchestra.run.trial035.stdout: "mon_debug_unsafe_allow_tier_with_nonempty_snaps": "false", 2026-02-24T07:43:05.448 INFO:teuthology.orchestra.run.trial035.stdout: "mon_delta_reset_interval": "10.000000", 2026-02-24T07:43:05.448 INFO:teuthology.orchestra.run.trial035.stdout: "mon_dns_srv_name": "ceph-mon", 2026-02-24T07:43:05.448 INFO:teuthology.orchestra.run.trial035.stdout: "mon_down_added_grace": "180", 2026-02-24T07:43:05.448 INFO:teuthology.orchestra.run.trial035.stdout: "mon_down_mkfs_grace": "60", 2026-02-24T07:43:05.448 INFO:teuthology.orchestra.run.trial035.stdout: "mon_down_uptime_grace": "60", 2026-02-24T07:43:05.448 INFO:teuthology.orchestra.run.trial035.stdout: "mon_election_default_strategy": "1", 2026-02-24T07:43:05.448 INFO:teuthology.orchestra.run.trial035.stdout: "mon_election_timeout": "5.000000", 2026-02-24T07:43:05.449 INFO:teuthology.orchestra.run.trial035.stdout: "mon_elector_ignore_propose_margin": "0.000500", 2026-02-24T07:43:05.449 INFO:teuthology.orchestra.run.trial035.stdout: "mon_elector_ping_divisor": "2", 2026-02-24T07:43:05.449 INFO:teuthology.orchestra.run.trial035.stdout: "mon_elector_ping_timeout": "2.000000", 2026-02-24T07:43:05.449 INFO:teuthology.orchestra.run.trial035.stdout: "mon_enable_op_tracker": "true", 2026-02-24T07:43:05.449 INFO:teuthology.orchestra.run.trial035.stdout: "mon_fake_pool_delete": "false", 2026-02-24T07:43:05.449 INFO:teuthology.orchestra.run.trial035.stdout: "mon_force_quorum_join": "false", 2026-02-24T07:43:05.449 INFO:teuthology.orchestra.run.trial035.stdout: "mon_fsmap_prune_threshold": "300", 2026-02-24T07:43:05.449 INFO:teuthology.orchestra.run.trial035.stdout: "mon_globalid_prealloc": "10000", 2026-02-24T07:43:05.449 INFO:teuthology.orchestra.run.trial035.stdout: "mon_health_detail_to_clog": "true", 2026-02-24T07:43:05.449 INFO:teuthology.orchestra.run.trial035.stdout: "mon_health_log_update_period": "5", 2026-02-24T07:43:05.449 INFO:teuthology.orchestra.run.trial035.stdout: "mon_health_max_detail": "50", 2026-02-24T07:43:05.449 INFO:teuthology.orchestra.run.trial035.stdout: "mon_health_to_clog": "true", 2026-02-24T07:43:05.449 INFO:teuthology.orchestra.run.trial035.stdout: "mon_health_to_clog_interval": "600", 2026-02-24T07:43:05.449 INFO:teuthology.orchestra.run.trial035.stdout: "mon_health_to_clog_tick_interval": "60.000000", 2026-02-24T07:43:05.449 INFO:teuthology.orchestra.run.trial035.stdout: "mon_host": "[v2:10.20.193.35:3300/0,v1:10.20.193.35:6789/0]", 2026-02-24T07:43:05.449 INFO:teuthology.orchestra.run.trial035.stdout: "mon_host_override": "", 2026-02-24T07:43:05.449 INFO:teuthology.orchestra.run.trial035.stdout: "mon_initial_members": "", 2026-02-24T07:43:05.449 INFO:teuthology.orchestra.run.trial035.stdout: "mon_inject_pg_merge_bounce_probability": "0.000000", 2026-02-24T07:43:05.449 INFO:teuthology.orchestra.run.trial035.stdout: "mon_inject_sync_get_chunk_delay": "0.000000", 2026-02-24T07:43:05.449 INFO:teuthology.orchestra.run.trial035.stdout: "mon_inject_transaction_delay_max": "10.000000", 2026-02-24T07:43:05.449 INFO:teuthology.orchestra.run.trial035.stdout: "mon_inject_transaction_delay_probability": "0.000000", 2026-02-24T07:43:05.449 INFO:teuthology.orchestra.run.trial035.stdout: "mon_keyvaluedb": "rocksdb", 2026-02-24T07:43:05.449 INFO:teuthology.orchestra.run.trial035.stdout: "mon_lease": "5.000000", 2026-02-24T07:43:05.449 INFO:teuthology.orchestra.run.trial035.stdout: "mon_lease_ack_timeout_factor": "2.000000", 2026-02-24T07:43:05.449 INFO:teuthology.orchestra.run.trial035.stdout: "mon_lease_renew_interval_factor": "0.600000", 2026-02-24T07:43:05.449 INFO:teuthology.orchestra.run.trial035.stdout: "mon_log_full_interval": "50", 2026-02-24T07:43:05.449 INFO:teuthology.orchestra.run.trial035.stdout: "mon_log_max": "10000", 2026-02-24T07:43:05.450 INFO:teuthology.orchestra.run.trial035.stdout: "mon_log_max_summary": "50", 2026-02-24T07:43:05.450 INFO:teuthology.orchestra.run.trial035.stdout: "mon_max_log_entries_per_event": "4096", 2026-02-24T07:43:05.450 INFO:teuthology.orchestra.run.trial035.stdout: "mon_max_log_epochs": "500", 2026-02-24T07:43:05.450 INFO:teuthology.orchestra.run.trial035.stdout: "mon_max_mdsmap_epochs": "500", 2026-02-24T07:43:05.450 INFO:teuthology.orchestra.run.trial035.stdout: "mon_max_mgrmap_epochs": "500", 2026-02-24T07:43:05.450 INFO:teuthology.orchestra.run.trial035.stdout: "mon_max_nvmeof_epochs": "500", 2026-02-24T07:43:05.450 INFO:teuthology.orchestra.run.trial035.stdout: "mon_max_osd": "10000", 2026-02-24T07:43:05.450 INFO:teuthology.orchestra.run.trial035.stdout: "mon_max_pg_per_osd": "10000", 2026-02-24T07:43:05.450 INFO:teuthology.orchestra.run.trial035.stdout: "mon_max_pool_pg_num": "65536", 2026-02-24T07:43:05.450 INFO:teuthology.orchestra.run.trial035.stdout: "mon_max_snap_prune_per_epoch": "100", 2026-02-24T07:43:05.450 INFO:teuthology.orchestra.run.trial035.stdout: "mon_mds_blocklist_interval": "86400.000000", 2026-02-24T07:43:05.450 INFO:teuthology.orchestra.run.trial035.stdout: "mon_mds_force_trim_to": "0", 2026-02-24T07:43:05.450 INFO:teuthology.orchestra.run.trial035.stdout: "mon_mds_skip_sanity": "false", 2026-02-24T07:43:05.450 INFO:teuthology.orchestra.run.trial035.stdout: "mon_memory_autotune": "true", 2026-02-24T07:43:05.450 INFO:teuthology.orchestra.run.trial035.stdout: "mon_memory_target": "2147483648", 2026-02-24T07:43:05.450 INFO:teuthology.orchestra.run.trial035.stdout: "mon_mgr_beacon_grace": "30", 2026-02-24T07:43:05.450 INFO:teuthology.orchestra.run.trial035.stdout: "mon_mgr_blocklist_interval": "86400.000000", 2026-02-24T07:43:05.450 INFO:teuthology.orchestra.run.trial035.stdout: "mon_mgr_digest_period": "5", 2026-02-24T07:43:05.450 INFO:teuthology.orchestra.run.trial035.stdout: "mon_mgr_inactive_grace": "60", 2026-02-24T07:43:05.450 INFO:teuthology.orchestra.run.trial035.stdout: "mon_mgr_mkfs_grace": "240", 2026-02-24T07:43:05.450 INFO:teuthology.orchestra.run.trial035.stdout: "mon_mgr_proxy_client_bytes_ratio": "0.300000", 2026-02-24T07:43:05.450 INFO:teuthology.orchestra.run.trial035.stdout: "mon_min_osdmap_epochs": "50", 2026-02-24T07:43:05.450 INFO:teuthology.orchestra.run.trial035.stdout: "mon_netsplit_grace_period": "9", 2026-02-24T07:43:05.450 INFO:teuthology.orchestra.run.trial035.stdout: "mon_nvmeofgw_beacon_grace": "7", 2026-02-24T07:43:05.450 INFO:teuthology.orchestra.run.trial035.stdout: "mon_nvmeofgw_beacons_till_ack": "15", 2026-02-24T07:43:05.450 INFO:teuthology.orchestra.run.trial035.stdout: "mon_nvmeofgw_delete_grace": "900", 2026-02-24T07:43:05.451 INFO:teuthology.orchestra.run.trial035.stdout: "mon_nvmeofgw_set_group_id_retry": "1000", 2026-02-24T07:43:05.451 INFO:teuthology.orchestra.run.trial035.stdout: "mon_nvmeofgw_skip_failovers_interval": "16", 2026-02-24T07:43:05.451 INFO:teuthology.orchestra.run.trial035.stdout: "mon_nvmeofgw_wrong_map_ignore_sec": "15", 2026-02-24T07:43:05.451 INFO:teuthology.orchestra.run.trial035.stdout: "mon_op_complaint_time": "30", 2026-02-24T07:43:05.451 INFO:teuthology.orchestra.run.trial035.stdout: "mon_op_history_duration": "600", 2026-02-24T07:43:05.451 INFO:teuthology.orchestra.run.trial035.stdout: "mon_op_history_size": "20", 2026-02-24T07:43:05.451 INFO:teuthology.orchestra.run.trial035.stdout: "mon_op_history_slow_op_size": "20", 2026-02-24T07:43:05.451 INFO:teuthology.orchestra.run.trial035.stdout: "mon_op_history_slow_op_threshold": "10", 2026-02-24T07:43:05.451 INFO:teuthology.orchestra.run.trial035.stdout: "mon_op_log_threshold": "5", 2026-02-24T07:43:05.451 INFO:teuthology.orchestra.run.trial035.stdout: "mon_osd_adjust_down_out_interval": "true", 2026-02-24T07:43:05.451 INFO:teuthology.orchestra.run.trial035.stdout: "mon_osd_adjust_heartbeat_grace": "true", 2026-02-24T07:43:05.451 INFO:teuthology.orchestra.run.trial035.stdout: "mon_osd_auto_mark_auto_out_in": "true", 2026-02-24T07:43:05.451 INFO:teuthology.orchestra.run.trial035.stdout: "mon_osd_auto_mark_in": "false", 2026-02-24T07:43:05.451 INFO:teuthology.orchestra.run.trial035.stdout: "mon_osd_auto_mark_new_in": "true", 2026-02-24T07:43:05.451 INFO:teuthology.orchestra.run.trial035.stdout: "mon_osd_backfillfull_ratio": "0.900000", 2026-02-24T07:43:05.451 INFO:teuthology.orchestra.run.trial035.stdout: "mon_osd_blocklist_default_expire": "3600.000000", 2026-02-24T07:43:05.451 INFO:teuthology.orchestra.run.trial035.stdout: "mon_osd_cache_size": "500", 2026-02-24T07:43:05.451 INFO:teuthology.orchestra.run.trial035.stdout: "mon_osd_cache_size_min": "134217728", 2026-02-24T07:43:05.451 INFO:teuthology.orchestra.run.trial035.stdout: "mon_osd_crush_smoke_test": "true", 2026-02-24T07:43:05.451 INFO:teuthology.orchestra.run.trial035.stdout: "mon_osd_destroyed_out_interval": "600", 2026-02-24T07:43:05.451 INFO:teuthology.orchestra.run.trial035.stdout: "mon_osd_down_out_interval": "600", 2026-02-24T07:43:05.451 INFO:teuthology.orchestra.run.trial035.stdout: "mon_osd_down_out_subtree_limit": "rack", 2026-02-24T07:43:05.451 INFO:teuthology.orchestra.run.trial035.stdout: "mon_osd_err_op_age_ratio": "128.000000", 2026-02-24T07:43:05.451 INFO:teuthology.orchestra.run.trial035.stdout: "mon_osd_force_trim_to": "0", 2026-02-24T07:43:05.451 INFO:teuthology.orchestra.run.trial035.stdout: "mon_osd_full_ratio": "0.950000", 2026-02-24T07:43:05.451 INFO:teuthology.orchestra.run.trial035.stdout: "mon_osd_initial_require_min_compat_client": "luminous", 2026-02-24T07:43:05.451 INFO:teuthology.orchestra.run.trial035.stdout: "mon_osd_laggy_halflife": "3600", 2026-02-24T07:43:05.452 INFO:teuthology.orchestra.run.trial035.stdout: "mon_osd_laggy_max_interval": "300", 2026-02-24T07:43:05.452 INFO:teuthology.orchestra.run.trial035.stdout: "mon_osd_laggy_weight": "0.300000", 2026-02-24T07:43:05.452 INFO:teuthology.orchestra.run.trial035.stdout: "mon_osd_mapping_pgs_per_chunk": "4096", 2026-02-24T07:43:05.452 INFO:teuthology.orchestra.run.trial035.stdout: "mon_osd_max_initial_pgs": "1024", 2026-02-24T07:43:05.452 INFO:teuthology.orchestra.run.trial035.stdout: "mon_osd_min_down_reporters": "2", 2026-02-24T07:43:05.452 INFO:teuthology.orchestra.run.trial035.stdout: "mon_osd_min_in_ratio": "0.750000", 2026-02-24T07:43:05.452 INFO:teuthology.orchestra.run.trial035.stdout: "mon_osd_min_up_ratio": "0.300000", 2026-02-24T07:43:05.452 INFO:teuthology.orchestra.run.trial035.stdout: "mon_osd_nearfull_ratio": "0.850000", 2026-02-24T07:43:05.452 INFO:teuthology.orchestra.run.trial035.stdout: "mon_osd_prime_pg_temp": "true", 2026-02-24T07:43:05.452 INFO:teuthology.orchestra.run.trial035.stdout: "mon_osd_prime_pg_temp_max_estimate": "0.250000", 2026-02-24T07:43:05.452 INFO:teuthology.orchestra.run.trial035.stdout: "mon_osd_prime_pg_temp_max_time": "0.500000", 2026-02-24T07:43:05.452 INFO:teuthology.orchestra.run.trial035.stdout: "mon_osd_report_timeout": "900", 2026-02-24T07:43:05.452 INFO:teuthology.orchestra.run.trial035.stdout: "mon_osd_reporter_subtree_level": "osd", 2026-02-24T07:43:05.452 INFO:teuthology.orchestra.run.trial035.stdout: "mon_osd_snap_trim_queue_warn_on": "32768", 2026-02-24T07:43:05.452 INFO:teuthology.orchestra.run.trial035.stdout: "mon_osd_warn_num_repaired": "10", 2026-02-24T07:43:05.452 INFO:teuthology.orchestra.run.trial035.stdout: "mon_osd_warn_op_age": "32.000000", 2026-02-24T07:43:05.452 INFO:teuthology.orchestra.run.trial035.stdout: "mon_osdmap_full_prune_enabled": "true", 2026-02-24T07:43:05.452 INFO:teuthology.orchestra.run.trial035.stdout: "mon_osdmap_full_prune_interval": "2", 2026-02-24T07:43:05.452 INFO:teuthology.orchestra.run.trial035.stdout: "mon_osdmap_full_prune_min": "15", 2026-02-24T07:43:05.452 INFO:teuthology.orchestra.run.trial035.stdout: "mon_osdmap_full_prune_txsize": "2", 2026-02-24T07:43:05.452 INFO:teuthology.orchestra.run.trial035.stdout: "mon_pg_check_down_all_threshold": "0.500000", 2026-02-24T07:43:05.452 INFO:teuthology.orchestra.run.trial035.stdout: "mon_pg_stuck_threshold": "60", 2026-02-24T07:43:05.452 INFO:teuthology.orchestra.run.trial035.stdout: "mon_pg_warn_max_object_skew": "0.000000", 2026-02-24T07:43:05.452 INFO:teuthology.orchestra.run.trial035.stdout: "mon_pg_warn_min_objects": "10000", 2026-02-24T07:43:05.452 INFO:teuthology.orchestra.run.trial035.stdout: "mon_pg_warn_min_per_osd": "0", 2026-02-24T07:43:05.452 INFO:teuthology.orchestra.run.trial035.stdout: "mon_pg_warn_min_pool_objects": "1000", 2026-02-24T07:43:05.452 INFO:teuthology.orchestra.run.trial035.stdout: "mon_pool_quota_crit_threshold": "0", 2026-02-24T07:43:05.453 INFO:teuthology.orchestra.run.trial035.stdout: "mon_pool_quota_warn_threshold": "0", 2026-02-24T07:43:05.453 INFO:teuthology.orchestra.run.trial035.stdout: "mon_probe_timeout": "2.000000", 2026-02-24T07:43:05.453 INFO:teuthology.orchestra.run.trial035.stdout: "mon_reweight_max_change": "0.050000", 2026-02-24T07:43:05.453 INFO:teuthology.orchestra.run.trial035.stdout: "mon_reweight_max_osds": "4", 2026-02-24T07:43:05.453 INFO:teuthology.orchestra.run.trial035.stdout: "mon_reweight_min_bytes_per_osd": "10", 2026-02-24T07:43:05.453 INFO:teuthology.orchestra.run.trial035.stdout: "mon_reweight_min_pgs_per_osd": "4", 2026-02-24T07:43:05.453 INFO:teuthology.orchestra.run.trial035.stdout: "mon_rocksdb_options": "write_buffer_size=33554432,compression=kNoCompression,level_compaction_dynamic_level_bytes=true", 2026-02-24T07:43:05.453 INFO:teuthology.orchestra.run.trial035.stdout: "mon_scrub_inject_crc_mismatch": "0.000000", 2026-02-24T07:43:05.453 INFO:teuthology.orchestra.run.trial035.stdout: "mon_scrub_inject_missing_keys": "0.000000", 2026-02-24T07:43:05.453 INFO:teuthology.orchestra.run.trial035.stdout: "mon_scrub_interval": "86400", 2026-02-24T07:43:05.453 INFO:teuthology.orchestra.run.trial035.stdout: "mon_scrub_max_keys": "100", 2026-02-24T07:43:05.453 INFO:teuthology.orchestra.run.trial035.stdout: "mon_scrub_timeout": "300", 2026-02-24T07:43:05.453 INFO:teuthology.orchestra.run.trial035.stdout: "mon_session_timeout": "300", 2026-02-24T07:43:05.453 INFO:teuthology.orchestra.run.trial035.stdout: "mon_smart_report_timeout": "5", 2026-02-24T07:43:05.453 INFO:teuthology.orchestra.run.trial035.stdout: "mon_stat_smooth_intervals": "6", 2026-02-24T07:43:05.453 INFO:teuthology.orchestra.run.trial035.stdout: "mon_stretch_cluster_recovery_ratio": "0.600000", 2026-02-24T07:43:05.453 INFO:teuthology.orchestra.run.trial035.stdout: "mon_stretch_pool_min_size": "2", 2026-02-24T07:43:05.453 INFO:teuthology.orchestra.run.trial035.stdout: "mon_stretch_pool_size": "4", 2026-02-24T07:43:05.453 INFO:teuthology.orchestra.run.trial035.stdout: "mon_stretch_recovery_min_wait": "15.000000", 2026-02-24T07:43:05.453 INFO:teuthology.orchestra.run.trial035.stdout: "mon_subscribe_interval": "86400.000000", 2026-02-24T07:43:05.453 INFO:teuthology.orchestra.run.trial035.stdout: "mon_sync_debug": "false", 2026-02-24T07:43:05.453 INFO:teuthology.orchestra.run.trial035.stdout: "mon_sync_max_payload_keys": "2000", 2026-02-24T07:43:05.453 INFO:teuthology.orchestra.run.trial035.stdout: "mon_sync_max_payload_size": "1048576", 2026-02-24T07:43:05.453 INFO:teuthology.orchestra.run.trial035.stdout: "mon_sync_provider_kill_at": "0", 2026-02-24T07:43:05.453 INFO:teuthology.orchestra.run.trial035.stdout: "mon_sync_requester_kill_at": "0", 2026-02-24T07:43:05.453 INFO:teuthology.orchestra.run.trial035.stdout: "mon_sync_timeout": "60.000000", 2026-02-24T07:43:05.453 INFO:teuthology.orchestra.run.trial035.stdout: "mon_target_pg_per_osd": "100", 2026-02-24T07:43:05.454 INFO:teuthology.orchestra.run.trial035.stdout: "mon_tick_interval": "5", 2026-02-24T07:43:05.454 INFO:teuthology.orchestra.run.trial035.stdout: "mon_timecheck_interval": "300.000000", 2026-02-24T07:43:05.454 INFO:teuthology.orchestra.run.trial035.stdout: "mon_timecheck_skew_interval": "30.000000", 2026-02-24T07:43:05.454 INFO:teuthology.orchestra.run.trial035.stdout: "mon_use_min_delay_socket": "false", 2026-02-24T07:43:05.454 INFO:teuthology.orchestra.run.trial035.stdout: "mon_warn_older_version_delay": "604800", 2026-02-24T07:43:05.454 INFO:teuthology.orchestra.run.trial035.stdout: "mon_warn_on_cache_pools_without_hit_sets": "true", 2026-02-24T07:43:05.454 INFO:teuthology.orchestra.run.trial035.stdout: "mon_warn_on_crush_straw_calc_version_zero": "false", 2026-02-24T07:43:05.454 INFO:teuthology.orchestra.run.trial035.stdout: "mon_warn_on_degraded_stretch_mode": "true", 2026-02-24T07:43:05.454 INFO:teuthology.orchestra.run.trial035.stdout: "mon_warn_on_filestore_osds": "true", 2026-02-24T07:43:05.454 INFO:teuthology.orchestra.run.trial035.stdout: "mon_warn_on_insecure_global_id_reclaim": "false", 2026-02-24T07:43:05.454 INFO:teuthology.orchestra.run.trial035.stdout: "mon_warn_on_insecure_global_id_reclaim_allowed": "false", 2026-02-24T07:43:05.454 INFO:teuthology.orchestra.run.trial035.stdout: "mon_warn_on_legacy_crush_tunables": "false", 2026-02-24T07:43:05.454 INFO:teuthology.orchestra.run.trial035.stdout: "mon_warn_on_misplaced": "false", 2026-02-24T07:43:05.454 INFO:teuthology.orchestra.run.trial035.stdout: "mon_warn_on_msgr2_not_enabled": "true", 2026-02-24T07:43:05.454 INFO:teuthology.orchestra.run.trial035.stdout: "mon_warn_on_older_version": "true", 2026-02-24T07:43:05.454 INFO:teuthology.orchestra.run.trial035.stdout: "mon_warn_on_osd_down_out_interval_zero": "false", 2026-02-24T07:43:05.454 INFO:teuthology.orchestra.run.trial035.stdout: "mon_warn_on_pool_no_app": "true", 2026-02-24T07:43:05.454 INFO:teuthology.orchestra.run.trial035.stdout: "mon_warn_on_pool_no_app_grace": "300", 2026-02-24T07:43:05.454 INFO:teuthology.orchestra.run.trial035.stdout: "mon_warn_on_pool_no_redundancy": "true", 2026-02-24T07:43:05.454 INFO:teuthology.orchestra.run.trial035.stdout: "mon_warn_on_pool_pg_num_not_power_of_two": "false", 2026-02-24T07:43:05.454 INFO:teuthology.orchestra.run.trial035.stdout: "mon_warn_on_slow_ping_ratio": "0.050000", 2026-02-24T07:43:05.454 INFO:teuthology.orchestra.run.trial035.stdout: "mon_warn_on_slow_ping_time": "0.000000", 2026-02-24T07:43:05.454 INFO:teuthology.orchestra.run.trial035.stdout: "mon_warn_on_too_few_osds": "false", 2026-02-24T07:43:05.454 INFO:teuthology.orchestra.run.trial035.stdout: "mon_warn_pg_not_deep_scrubbed_ratio": "0.750000", 2026-02-24T07:43:05.454 INFO:teuthology.orchestra.run.trial035.stdout: "mon_warn_pg_not_scrubbed_ratio": "0.500000", 2026-02-24T07:43:05.454 INFO:teuthology.orchestra.run.trial035.stdout: "monmap": "", 2026-02-24T07:43:05.454 INFO:teuthology.orchestra.run.trial035.stdout: "motr_admin_endpoint": "192.168.180.182@tcp:12345:4:1", 2026-02-24T07:43:05.454 INFO:teuthology.orchestra.run.trial035.stdout: "motr_admin_fid": "0x7200000000000001:0x0", 2026-02-24T07:43:05.455 INFO:teuthology.orchestra.run.trial035.stdout: "motr_ha_endpoint": "192.168.180.182@tcp:12345:1:1", 2026-02-24T07:43:05.455 INFO:teuthology.orchestra.run.trial035.stdout: "motr_my_endpoint": "192.168.180.182@tcp:12345:4:1", 2026-02-24T07:43:05.455 INFO:teuthology.orchestra.run.trial035.stdout: "motr_my_fid": "0x7200000000000001:0x0", 2026-02-24T07:43:05.455 INFO:teuthology.orchestra.run.trial035.stdout: "motr_profile_fid": "0x7000000000000001:0x0", 2026-02-24T07:43:05.455 INFO:teuthology.orchestra.run.trial035.stdout: "motr_tracing_enabled": "false", 2026-02-24T07:43:05.455 INFO:teuthology.orchestra.run.trial035.stdout: "ms_async_op_threads": "3", 2026-02-24T07:43:05.455 INFO:teuthology.orchestra.run.trial035.stdout: "ms_async_rdma_buffer_size": "131072", 2026-02-24T07:43:05.455 INFO:teuthology.orchestra.run.trial035.stdout: "ms_async_rdma_cm": "false", 2026-02-24T07:43:05.455 INFO:teuthology.orchestra.run.trial035.stdout: "ms_async_rdma_device_name": "", 2026-02-24T07:43:05.455 INFO:teuthology.orchestra.run.trial035.stdout: "ms_async_rdma_dscp": "96", 2026-02-24T07:43:05.455 INFO:teuthology.orchestra.run.trial035.stdout: "ms_async_rdma_enable_hugepage": "false", 2026-02-24T07:43:05.455 INFO:teuthology.orchestra.run.trial035.stdout: "ms_async_rdma_gid_idx": "0", 2026-02-24T07:43:05.455 INFO:teuthology.orchestra.run.trial035.stdout: "ms_async_rdma_local_gid": "", 2026-02-24T07:43:05.455 INFO:teuthology.orchestra.run.trial035.stdout: "ms_async_rdma_polling_us": "1000", 2026-02-24T07:43:05.455 INFO:teuthology.orchestra.run.trial035.stdout: "ms_async_rdma_port_num": "1", 2026-02-24T07:43:05.455 INFO:teuthology.orchestra.run.trial035.stdout: "ms_async_rdma_receive_buffers": "32768", 2026-02-24T07:43:05.455 INFO:teuthology.orchestra.run.trial035.stdout: "ms_async_rdma_receive_queue_len": "4096", 2026-02-24T07:43:05.455 INFO:teuthology.orchestra.run.trial035.stdout: "ms_async_rdma_roce_ver": "1", 2026-02-24T07:43:05.455 INFO:teuthology.orchestra.run.trial035.stdout: "ms_async_rdma_send_buffers": "1024", 2026-02-24T07:43:05.455 INFO:teuthology.orchestra.run.trial035.stdout: "ms_async_rdma_sl": "3", 2026-02-24T07:43:05.455 INFO:teuthology.orchestra.run.trial035.stdout: "ms_async_rdma_support_srq": "true", 2026-02-24T07:43:05.455 INFO:teuthology.orchestra.run.trial035.stdout: "ms_async_rdma_type": "ib", 2026-02-24T07:43:05.455 INFO:teuthology.orchestra.run.trial035.stdout: "ms_async_reap_threshold": "5", 2026-02-24T07:43:05.455 INFO:teuthology.orchestra.run.trial035.stdout: "ms_bind_before_connect": "false", 2026-02-24T07:43:05.455 INFO:teuthology.orchestra.run.trial035.stdout: "ms_bind_ipv4": "true", 2026-02-24T07:43:05.455 INFO:teuthology.orchestra.run.trial035.stdout: "ms_bind_ipv6": "false", 2026-02-24T07:43:05.455 INFO:teuthology.orchestra.run.trial035.stdout: "ms_bind_msgr1": "true", 2026-02-24T07:43:05.455 INFO:teuthology.orchestra.run.trial035.stdout: "ms_bind_msgr2": "true", 2026-02-24T07:43:05.455 INFO:teuthology.orchestra.run.trial035.stdout: "ms_bind_port_max": "7568", 2026-02-24T07:43:05.455 INFO:teuthology.orchestra.run.trial035.stdout: "ms_bind_port_min": "6800", 2026-02-24T07:43:05.456 INFO:teuthology.orchestra.run.trial035.stdout: "ms_bind_prefer_ipv4": "false", 2026-02-24T07:43:05.456 INFO:teuthology.orchestra.run.trial035.stdout: "ms_bind_retry_count": "3", 2026-02-24T07:43:05.456 INFO:teuthology.orchestra.run.trial035.stdout: "ms_bind_retry_delay": "5", 2026-02-24T07:43:05.456 INFO:teuthology.orchestra.run.trial035.stdout: "ms_blackhole_client": "false", 2026-02-24T07:43:05.456 INFO:teuthology.orchestra.run.trial035.stdout: "ms_blackhole_mds": "false", 2026-02-24T07:43:05.456 INFO:teuthology.orchestra.run.trial035.stdout: "ms_blackhole_mgr": "false", 2026-02-24T07:43:05.456 INFO:teuthology.orchestra.run.trial035.stdout: "ms_blackhole_mon": "false", 2026-02-24T07:43:05.456 INFO:teuthology.orchestra.run.trial035.stdout: "ms_blackhole_osd": "false", 2026-02-24T07:43:05.456 INFO:teuthology.orchestra.run.trial035.stdout: "ms_client_mode": "crc", 2026-02-24T07:43:05.456 INFO:teuthology.orchestra.run.trial035.stdout: "ms_client_throttle_retry_time_interval": "5000", 2026-02-24T07:43:05.456 INFO:teuthology.orchestra.run.trial035.stdout: "ms_cluster_mode": "crc", 2026-02-24T07:43:05.456 INFO:teuthology.orchestra.run.trial035.stdout: "ms_cluster_type": "", 2026-02-24T07:43:05.456 INFO:teuthology.orchestra.run.trial035.stdout: "ms_compress_secure": "false", 2026-02-24T07:43:05.456 INFO:teuthology.orchestra.run.trial035.stdout: "ms_connection_idle_timeout": "900", 2026-02-24T07:43:05.456 INFO:teuthology.orchestra.run.trial035.stdout: "ms_connection_ready_timeout": "10", 2026-02-24T07:43:05.456 INFO:teuthology.orchestra.run.trial035.stdout: "ms_crc_data": "true", 2026-02-24T07:43:05.456 INFO:teuthology.orchestra.run.trial035.stdout: "ms_crc_header": "true", 2026-02-24T07:43:05.456 INFO:teuthology.orchestra.run.trial035.stdout: "ms_die_on_bad_msg": "false", 2026-02-24T07:43:05.456 INFO:teuthology.orchestra.run.trial035.stdout: "ms_die_on_bug": "true", 2026-02-24T07:43:05.456 INFO:teuthology.orchestra.run.trial035.stdout: "ms_die_on_old_message": "true", 2026-02-24T07:43:05.456 INFO:teuthology.orchestra.run.trial035.stdout: "ms_die_on_skipped_message": "false", 2026-02-24T07:43:05.456 INFO:teuthology.orchestra.run.trial035.stdout: "ms_die_on_unhandled_msg": "false", 2026-02-24T07:43:05.456 INFO:teuthology.orchestra.run.trial035.stdout: "ms_dispatch_throttle_bytes": "104857600", 2026-02-24T07:43:05.456 INFO:teuthology.orchestra.run.trial035.stdout: "ms_dpdk_coremask": "0xF", 2026-02-24T07:43:05.456 INFO:teuthology.orchestra.run.trial035.stdout: "ms_dpdk_debug_allow_loopback": "false", 2026-02-24T07:43:05.456 INFO:teuthology.orchestra.run.trial035.stdout: "ms_dpdk_devs_allowlist": "", 2026-02-24T07:43:05.456 INFO:teuthology.orchestra.run.trial035.stdout: "ms_dpdk_enable_tso": "true", 2026-02-24T07:43:05.456 INFO:teuthology.orchestra.run.trial035.stdout: "ms_dpdk_gateway_ipv4_addr": "", 2026-02-24T07:43:05.456 INFO:teuthology.orchestra.run.trial035.stdout: "ms_dpdk_host_ipv4_addr": "", 2026-02-24T07:43:05.456 INFO:teuthology.orchestra.run.trial035.stdout: "ms_dpdk_hugepages": "", 2026-02-24T07:43:05.456 INFO:teuthology.orchestra.run.trial035.stdout: "ms_dpdk_hw_flow_control": "true", 2026-02-24T07:43:05.457 INFO:teuthology.orchestra.run.trial035.stdout: "ms_dpdk_hw_queue_weight": "1.000000", 2026-02-24T07:43:05.457 INFO:teuthology.orchestra.run.trial035.stdout: "ms_dpdk_lro": "true", 2026-02-24T07:43:05.457 INFO:teuthology.orchestra.run.trial035.stdout: "ms_dpdk_memory_channel": "4", 2026-02-24T07:43:05.457 INFO:teuthology.orchestra.run.trial035.stdout: "ms_dpdk_netmask_ipv4_addr": "", 2026-02-24T07:43:05.457 INFO:teuthology.orchestra.run.trial035.stdout: "ms_dpdk_pmd": "", 2026-02-24T07:43:05.457 INFO:teuthology.orchestra.run.trial035.stdout: "ms_dpdk_port_id": "0", 2026-02-24T07:43:05.457 INFO:teuthology.orchestra.run.trial035.stdout: "ms_dpdk_rx_buffer_count_per_core": "8192", 2026-02-24T07:43:05.457 INFO:teuthology.orchestra.run.trial035.stdout: "ms_dump_corrupt_message_level": "1", 2026-02-24T07:43:05.457 INFO:teuthology.orchestra.run.trial035.stdout: "ms_dump_on_send": "false", 2026-02-24T07:43:05.457 INFO:teuthology.orchestra.run.trial035.stdout: "ms_initial_backoff": "0.200000", 2026-02-24T07:43:05.457 INFO:teuthology.orchestra.run.trial035.stdout: "ms_inject_delay_max": "1.000000", 2026-02-24T07:43:05.457 INFO:teuthology.orchestra.run.trial035.stdout: "ms_inject_delay_probability": "0.000000", 2026-02-24T07:43:05.457 INFO:teuthology.orchestra.run.trial035.stdout: "ms_inject_delay_type": "", 2026-02-24T07:43:05.457 INFO:teuthology.orchestra.run.trial035.stdout: "ms_inject_internal_delays": "0.000000", 2026-02-24T07:43:05.457 INFO:teuthology.orchestra.run.trial035.stdout: "ms_inject_network_congestion": "0", 2026-02-24T07:43:05.457 INFO:teuthology.orchestra.run.trial035.stdout: "ms_inject_socket_failures": "0", 2026-02-24T07:43:05.457 INFO:teuthology.orchestra.run.trial035.stdout: "ms_learn_addr_from_peer": "true", 2026-02-24T07:43:05.457 INFO:teuthology.orchestra.run.trial035.stdout: "ms_max_accept_failures": "4", 2026-02-24T07:43:05.457 INFO:teuthology.orchestra.run.trial035.stdout: "ms_max_backoff": "15.000000", 2026-02-24T07:43:05.457 INFO:teuthology.orchestra.run.trial035.stdout: "ms_mon_client_mode": "crc", 2026-02-24T07:43:05.457 INFO:teuthology.orchestra.run.trial035.stdout: "ms_mon_cluster_mode": "crc", 2026-02-24T07:43:05.457 INFO:teuthology.orchestra.run.trial035.stdout: "ms_mon_service_mode": "crc", 2026-02-24T07:43:05.457 INFO:teuthology.orchestra.run.trial035.stdout: "ms_osd_compress_min_size": "1024", 2026-02-24T07:43:05.457 INFO:teuthology.orchestra.run.trial035.stdout: "ms_osd_compress_mode": "none", 2026-02-24T07:43:05.457 INFO:teuthology.orchestra.run.trial035.stdout: "ms_osd_compression_algorithm": "snappy", 2026-02-24T07:43:05.457 INFO:teuthology.orchestra.run.trial035.stdout: "ms_pq_max_tokens_per_priority": "16777216", 2026-02-24T07:43:05.457 INFO:teuthology.orchestra.run.trial035.stdout: "ms_pq_min_cost": "65536", 2026-02-24T07:43:05.457 INFO:teuthology.orchestra.run.trial035.stdout: "ms_public_type": "", 2026-02-24T07:43:05.457 INFO:teuthology.orchestra.run.trial035.stdout: "ms_service_mode": "crc", 2026-02-24T07:43:05.457 INFO:teuthology.orchestra.run.trial035.stdout: "ms_tcp_listen_backlog": "512", 2026-02-24T07:43:05.457 INFO:teuthology.orchestra.run.trial035.stdout: "ms_tcp_nodelay": "true", 2026-02-24T07:43:05.458 INFO:teuthology.orchestra.run.trial035.stdout: "ms_tcp_prefetch_max_size": "65536", 2026-02-24T07:43:05.458 INFO:teuthology.orchestra.run.trial035.stdout: "ms_tcp_rcvbuf": "0", 2026-02-24T07:43:05.458 INFO:teuthology.orchestra.run.trial035.stdout: "ms_time_events_min_wait_interval": "1000", 2026-02-24T07:43:05.458 INFO:teuthology.orchestra.run.trial035.stdout: "ms_type": "async+posix", 2026-02-24T07:43:05.458 INFO:teuthology.orchestra.run.trial035.stdout: "no_config_file": "false", 2026-02-24T07:43:05.458 INFO:teuthology.orchestra.run.trial035.stdout: "nvmeof_mon_client_connect_panic": "30", 2026-02-24T07:43:05.458 INFO:teuthology.orchestra.run.trial035.stdout: "nvmeof_mon_client_disconnect_panic": "100", 2026-02-24T07:43:05.458 INFO:teuthology.orchestra.run.trial035.stdout: "nvmeof_mon_client_tick_period": "1", 2026-02-24T07:43:05.458 INFO:teuthology.orchestra.run.trial035.stdout: "objecter_completion_locks_per_session": "32", 2026-02-24T07:43:05.458 INFO:teuthology.orchestra.run.trial035.stdout: "objecter_debug_inject_relock_delay": "false", 2026-02-24T07:43:05.458 INFO:teuthology.orchestra.run.trial035.stdout: "objecter_inflight_op_bytes": "104857600", 2026-02-24T07:43:05.458 INFO:teuthology.orchestra.run.trial035.stdout: "objecter_inflight_ops": "1024", 2026-02-24T07:43:05.458 INFO:teuthology.orchestra.run.trial035.stdout: "objecter_inject_no_watch_ping": "false", 2026-02-24T07:43:05.458 INFO:teuthology.orchestra.run.trial035.stdout: "objecter_retry_writes_after_first_reply": "false", 2026-02-24T07:43:05.458 INFO:teuthology.orchestra.run.trial035.stdout: "objecter_tick_interval": "5.000000", 2026-02-24T07:43:05.458 INFO:teuthology.orchestra.run.trial035.stdout: "objecter_timeout": "10.000000", 2026-02-24T07:43:05.458 INFO:teuthology.orchestra.run.trial035.stdout: "objectstore_blackhole": "false", 2026-02-24T07:43:05.458 INFO:teuthology.orchestra.run.trial035.stdout: "objectstore_debug_throw_on_failed_txc": "false", 2026-02-24T07:43:05.458 INFO:teuthology.orchestra.run.trial035.stdout: "openssl_conf": "", 2026-02-24T07:43:05.458 INFO:teuthology.orchestra.run.trial035.stdout: "osd_agent_delay_time": "5.000000", 2026-02-24T07:43:05.458 INFO:teuthology.orchestra.run.trial035.stdout: "osd_agent_hist_halflife": "1000", 2026-02-24T07:43:05.458 INFO:teuthology.orchestra.run.trial035.stdout: "osd_agent_max_low_ops": "2", 2026-02-24T07:43:05.458 INFO:teuthology.orchestra.run.trial035.stdout: "osd_agent_max_ops": "4", 2026-02-24T07:43:05.458 INFO:teuthology.orchestra.run.trial035.stdout: "osd_agent_min_evict_effort": "0.100000", 2026-02-24T07:43:05.458 INFO:teuthology.orchestra.run.trial035.stdout: "osd_agent_quantize_effort": "0.100000", 2026-02-24T07:43:05.458 INFO:teuthology.orchestra.run.trial035.stdout: "osd_agent_slop": "0.020000", 2026-02-24T07:43:05.458 INFO:teuthology.orchestra.run.trial035.stdout: "osd_aggregated_slow_ops_logging": "true", 2026-02-24T07:43:05.458 INFO:teuthology.orchestra.run.trial035.stdout: "osd_allow_recovery_below_min_size": "true", 2026-02-24T07:43:05.458 INFO:teuthology.orchestra.run.trial035.stdout: "osd_asio_thread_count": "2", 2026-02-24T07:43:05.458 INFO:teuthology.orchestra.run.trial035.stdout: "osd_async_recovery_min_cost": "100", 2026-02-24T07:43:05.458 INFO:teuthology.orchestra.run.trial035.stdout: "osd_auto_mark_unfound_lost": "false", 2026-02-24T07:43:05.459 INFO:teuthology.orchestra.run.trial035.stdout: "osd_backfill_retry_interval": "30.000000", 2026-02-24T07:43:05.459 INFO:teuthology.orchestra.run.trial035.stdout: "osd_backfill_scan_max": "512", 2026-02-24T07:43:05.459 INFO:teuthology.orchestra.run.trial035.stdout: "osd_backfill_scan_min": "64", 2026-02-24T07:43:05.459 INFO:teuthology.orchestra.run.trial035.stdout: "osd_backoff_on_degraded": "false", 2026-02-24T07:43:05.459 INFO:teuthology.orchestra.run.trial035.stdout: "osd_backoff_on_peering": "false", 2026-02-24T07:43:05.459 INFO:teuthology.orchestra.run.trial035.stdout: "osd_backoff_on_unfound": "true", 2026-02-24T07:43:05.459 INFO:teuthology.orchestra.run.trial035.stdout: "osd_beacon_report_interval": "300", 2026-02-24T07:43:05.459 INFO:teuthology.orchestra.run.trial035.stdout: "osd_bench_duration": "30", 2026-02-24T07:43:05.459 INFO:teuthology.orchestra.run.trial035.stdout: "osd_bench_large_size_max_throughput": "104857600", 2026-02-24T07:43:05.459 INFO:teuthology.orchestra.run.trial035.stdout: "osd_bench_max_block_size": "67108864", 2026-02-24T07:43:05.459 INFO:teuthology.orchestra.run.trial035.stdout: "osd_bench_small_size_max_iops": "100", 2026-02-24T07:43:05.459 INFO:teuthology.orchestra.run.trial035.stdout: "osd_blkin_trace_all": "false", 2026-02-24T07:43:05.459 INFO:teuthology.orchestra.run.trial035.stdout: "osd_blocked_scrub_grace_period": "120", 2026-02-24T07:43:05.459 INFO:teuthology.orchestra.run.trial035.stdout: "osd_calc_pg_upmaps_aggressively": "true", 2026-02-24T07:43:05.459 INFO:teuthology.orchestra.run.trial035.stdout: "osd_calc_pg_upmaps_aggressively_fast": "true", 2026-02-24T07:43:05.459 INFO:teuthology.orchestra.run.trial035.stdout: "osd_calc_pg_upmaps_local_fallback_retries": "100", 2026-02-24T07:43:05.459 INFO:teuthology.orchestra.run.trial035.stdout: "osd_check_for_log_corruption": "false", 2026-02-24T07:43:05.459 INFO:teuthology.orchestra.run.trial035.stdout: "osd_check_max_object_name_len_on_startup": "true", 2026-02-24T07:43:05.459 INFO:teuthology.orchestra.run.trial035.stdout: "osd_class_default_list": "cephfs hello journal lock log numops otp rbd refcount rgw rgw_gc timeindex user version cas cmpomap queue 2pc_queue fifo sem_set", 2026-02-24T07:43:05.459 INFO:teuthology.orchestra.run.trial035.stdout: "osd_class_dir": "/usr/lib64/rados-classes", 2026-02-24T07:43:05.459 INFO:teuthology.orchestra.run.trial035.stdout: "osd_class_load_list": "cephfs hello journal lock log numops otp rbd refcount rgw rgw_gc timeindex user version cas cmpomap queue 2pc_queue fifo sem_set", 2026-02-24T07:43:05.459 INFO:teuthology.orchestra.run.trial035.stdout: "osd_class_update_on_start": "true", 2026-02-24T07:43:05.459 INFO:teuthology.orchestra.run.trial035.stdout: "osd_client_message_cap": "256", 2026-02-24T07:43:05.459 INFO:teuthology.orchestra.run.trial035.stdout: "osd_client_message_size_cap": "524288000", 2026-02-24T07:43:05.459 INFO:teuthology.orchestra.run.trial035.stdout: "osd_client_op_priority": "63", 2026-02-24T07:43:05.459 INFO:teuthology.orchestra.run.trial035.stdout: "osd_client_watch_timeout": "30", 2026-02-24T07:43:05.459 INFO:teuthology.orchestra.run.trial035.stdout: "osd_command_max_records": "256", 2026-02-24T07:43:05.459 INFO:teuthology.orchestra.run.trial035.stdout: "osd_compact_on_start": "false", 2026-02-24T07:43:05.459 INFO:teuthology.orchestra.run.trial035.stdout: "osd_copyfrom_max_chunk": "8388608", 2026-02-24T07:43:05.459 INFO:teuthology.orchestra.run.trial035.stdout: "osd_crush_chooseleaf_type": "1", 2026-02-24T07:43:05.459 INFO:teuthology.orchestra.run.trial035.stdout: "osd_crush_initial_weight": "-1.000000", 2026-02-24T07:43:05.459 INFO:teuthology.orchestra.run.trial035.stdout: "osd_crush_update_on_start": "true", 2026-02-24T07:43:05.460 INFO:teuthology.orchestra.run.trial035.stdout: "osd_crush_update_weight_set": "true", 2026-02-24T07:43:05.460 INFO:teuthology.orchestra.run.trial035.stdout: "osd_data": "/var/lib/ceph/osd/ceph-a", 2026-02-24T07:43:05.460 INFO:teuthology.orchestra.run.trial035.stdout: "osd_debug_crash_on_ignored_backoff": "false", 2026-02-24T07:43:05.460 INFO:teuthology.orchestra.run.trial035.stdout: "osd_debug_deep_scrub_sleep": "0.000000", 2026-02-24T07:43:05.460 INFO:teuthology.orchestra.run.trial035.stdout: "osd_debug_drop_ping_duration": "0", 2026-02-24T07:43:05.460 INFO:teuthology.orchestra.run.trial035.stdout: "osd_debug_drop_ping_probability": "0.000000", 2026-02-24T07:43:05.460 INFO:teuthology.orchestra.run.trial035.stdout: "osd_debug_feed_pullee": "-1", 2026-02-24T07:43:05.460 INFO:teuthology.orchestra.run.trial035.stdout: "osd_debug_inject_copyfrom_error": "false", 2026-02-24T07:43:05.460 INFO:teuthology.orchestra.run.trial035.stdout: "osd_debug_inject_dispatch_delay_duration": "0.100000", 2026-02-24T07:43:05.460 INFO:teuthology.orchestra.run.trial035.stdout: "osd_debug_inject_dispatch_delay_probability": "0.000000", 2026-02-24T07:43:05.460 INFO:teuthology.orchestra.run.trial035.stdout: "osd_debug_misdirected_ops": "false", 2026-02-24T07:43:05.460 INFO:teuthology.orchestra.run.trial035.stdout: "osd_debug_no_acting_change": "false", 2026-02-24T07:43:05.460 INFO:teuthology.orchestra.run.trial035.stdout: "osd_debug_no_purge_strays": "false", 2026-02-24T07:43:05.460 INFO:teuthology.orchestra.run.trial035.stdout: "osd_debug_op_order": "false", 2026-02-24T07:43:05.460 INFO:teuthology.orchestra.run.trial035.stdout: "osd_debug_pg_log_writeout": "false", 2026-02-24T07:43:05.460 INFO:teuthology.orchestra.run.trial035.stdout: "osd_debug_pretend_recovery_active": "false", 2026-02-24T07:43:05.460 INFO:teuthology.orchestra.run.trial035.stdout: "osd_debug_random_push_read_error": "0.000000", 2026-02-24T07:43:05.460 INFO:teuthology.orchestra.run.trial035.stdout: "osd_debug_reject_backfill_probability": "0.000000", 2026-02-24T07:43:05.460 INFO:teuthology.orchestra.run.trial035.stdout: "osd_debug_shutdown": "false", 2026-02-24T07:43:05.460 INFO:teuthology.orchestra.run.trial035.stdout: "osd_debug_skip_full_check_in_backfill_reservation": "false", 2026-02-24T07:43:05.460 INFO:teuthology.orchestra.run.trial035.stdout: "osd_debug_skip_full_check_in_recovery": "false", 2026-02-24T07:43:05.460 INFO:teuthology.orchestra.run.trial035.stdout: "osd_debug_trim_objects": "false", 2026-02-24T07:43:05.460 INFO:teuthology.orchestra.run.trial035.stdout: "osd_debug_verify_cached_snaps": "false", 2026-02-24T07:43:05.460 INFO:teuthology.orchestra.run.trial035.stdout: "osd_debug_verify_missing_on_start": "false", 2026-02-24T07:43:05.460 INFO:teuthology.orchestra.run.trial035.stdout: "osd_debug_verify_snaps": "false", 2026-02-24T07:43:05.460 INFO:teuthology.orchestra.run.trial035.stdout: "osd_debug_verify_stray_on_activate": "false", 2026-02-24T07:43:05.460 INFO:teuthology.orchestra.run.trial035.stdout: "osd_deep_scrub_interval": "604800.000000", 2026-02-24T07:43:05.460 INFO:teuthology.orchestra.run.trial035.stdout: "osd_deep_scrub_interval_cv": "0.200000", 2026-02-24T07:43:05.460 INFO:teuthology.orchestra.run.trial035.stdout: "osd_deep_scrub_keys": "1024", 2026-02-24T07:43:05.460 INFO:teuthology.orchestra.run.trial035.stdout: "osd_deep_scrub_large_omap_object_key_threshold": "200000", 2026-02-24T07:43:05.460 INFO:teuthology.orchestra.run.trial035.stdout: "osd_deep_scrub_large_omap_object_value_sum_threshold": "1073741824", 2026-02-24T07:43:05.461 INFO:teuthology.orchestra.run.trial035.stdout: "osd_deep_scrub_randomize_ratio": "0.150000", 2026-02-24T07:43:05.461 INFO:teuthology.orchestra.run.trial035.stdout: "osd_deep_scrub_stride": "4194304", 2026-02-24T07:43:05.461 INFO:teuthology.orchestra.run.trial035.stdout: "osd_deep_scrub_update_digest_min_age": "7200", 2026-02-24T07:43:05.461 INFO:teuthology.orchestra.run.trial035.stdout: "osd_default_data_pool_replay_window": "45", 2026-02-24T07:43:05.461 INFO:teuthology.orchestra.run.trial035.stdout: "osd_default_notify_timeout": "30", 2026-02-24T07:43:05.461 INFO:teuthology.orchestra.run.trial035.stdout: "osd_delete_sleep": "0.000000", 2026-02-24T07:43:05.461 INFO:teuthology.orchestra.run.trial035.stdout: "osd_delete_sleep_hdd": "5.000000", 2026-02-24T07:43:05.461 INFO:teuthology.orchestra.run.trial035.stdout: "osd_delete_sleep_hybrid": "1.000000", 2026-02-24T07:43:05.461 INFO:teuthology.orchestra.run.trial035.stdout: "osd_delete_sleep_ssd": "1.000000", 2026-02-24T07:43:05.461 INFO:teuthology.orchestra.run.trial035.stdout: "osd_discard_disconnected_ops": "true", 2026-02-24T07:43:05.461 INFO:teuthology.orchestra.run.trial035.stdout: "osd_ec_partial_reads": "true", 2026-02-24T07:43:05.461 INFO:teuthology.orchestra.run.trial035.stdout: "osd_enable_op_tracker": "true", 2026-02-24T07:43:05.461 INFO:teuthology.orchestra.run.trial035.stdout: "osd_erasure_code_plugins": "jerasure lrc", 2026-02-24T07:43:05.461 INFO:teuthology.orchestra.run.trial035.stdout: "osd_extblkdev_plugins": "vdo", 2026-02-24T07:43:05.461 INFO:teuthology.orchestra.run.trial035.stdout: "osd_failsafe_full_ratio": "0.970000", 2026-02-24T07:43:05.461 INFO:teuthology.orchestra.run.trial035.stdout: "osd_fast_fail_on_connection_refused": "true", 2026-02-24T07:43:05.461 INFO:teuthology.orchestra.run.trial035.stdout: "osd_fast_info": "true", 2026-02-24T07:43:05.461 INFO:teuthology.orchestra.run.trial035.stdout: "osd_fast_shutdown": "true", 2026-02-24T07:43:05.461 INFO:teuthology.orchestra.run.trial035.stdout: "osd_fast_shutdown_notify_mon": "true", 2026-02-24T07:43:05.461 INFO:teuthology.orchestra.run.trial035.stdout: "osd_fast_shutdown_timeout": "15", 2026-02-24T07:43:05.461 INFO:teuthology.orchestra.run.trial035.stdout: "osd_find_best_info_ignore_history_les": "false", 2026-02-24T07:43:05.461 INFO:teuthology.orchestra.run.trial035.stdout: "osd_force_auth_primary_missing_objects": "100", 2026-02-24T07:43:05.461 INFO:teuthology.orchestra.run.trial035.stdout: "osd_force_recovery_pg_log_entries_factor": "1.300000", 2026-02-24T07:43:05.461 INFO:teuthology.orchestra.run.trial035.stdout: "osd_function_tracing": "false", 2026-02-24T07:43:05.461 INFO:teuthology.orchestra.run.trial035.stdout: "osd_heartbeat_grace": "20", 2026-02-24T07:43:05.461 INFO:teuthology.orchestra.run.trial035.stdout: "osd_heartbeat_interval": "6", 2026-02-24T07:43:05.461 INFO:teuthology.orchestra.run.trial035.stdout: "osd_heartbeat_min_healthy_ratio": "0.330000", 2026-02-24T07:43:05.461 INFO:teuthology.orchestra.run.trial035.stdout: "osd_heartbeat_min_peers": "10", 2026-02-24T07:43:05.461 INFO:teuthology.orchestra.run.trial035.stdout: "osd_heartbeat_min_size": "2000", 2026-02-24T07:43:05.461 INFO:teuthology.orchestra.run.trial035.stdout: "osd_heartbeat_stale": "600", 2026-02-24T07:43:05.461 INFO:teuthology.orchestra.run.trial035.stdout: "osd_heartbeat_use_min_delay_socket": "false", 2026-02-24T07:43:05.461 INFO:teuthology.orchestra.run.trial035.stdout: "osd_hit_set_max_size": "100000", 2026-02-24T07:43:05.461 INFO:teuthology.orchestra.run.trial035.stdout: "osd_hit_set_min_size": "1000", 2026-02-24T07:43:05.461 INFO:teuthology.orchestra.run.trial035.stdout: "osd_hit_set_namespace": ".ceph-internal", 2026-02-24T07:43:05.461 INFO:teuthology.orchestra.run.trial035.stdout: "osd_ignore_stale_divergent_priors": "false", 2026-02-24T07:43:05.462 INFO:teuthology.orchestra.run.trial035.stdout: "osd_inject_bad_map_crc_probability": "0.000000", 2026-02-24T07:43:05.462 INFO:teuthology.orchestra.run.trial035.stdout: "osd_inject_failure_on_pg_removal": "false", 2026-02-24T07:43:05.462 INFO:teuthology.orchestra.run.trial035.stdout: "osd_journal": "/var/lib/ceph/osd/ceph-a/journal", 2026-02-24T07:43:05.462 INFO:teuthology.orchestra.run.trial035.stdout: "osd_journal_flush_on_shutdown": "true", 2026-02-24T07:43:05.462 INFO:teuthology.orchestra.run.trial035.stdout: "osd_journal_size": "5120", 2026-02-24T07:43:05.462 INFO:teuthology.orchestra.run.trial035.stdout: "osd_kill_backfill_at": "0", 2026-02-24T07:43:05.462 INFO:teuthology.orchestra.run.trial035.stdout: "osd_loop_before_reset_tphandle": "64", 2026-02-24T07:43:05.462 INFO:teuthology.orchestra.run.trial035.stdout: "osd_map_cache_size": "50", 2026-02-24T07:43:05.462 INFO:teuthology.orchestra.run.trial035.stdout: "osd_map_dedup": "true", 2026-02-24T07:43:05.462 INFO:teuthology.orchestra.run.trial035.stdout: "osd_map_message_max": "40", 2026-02-24T07:43:05.462 INFO:teuthology.orchestra.run.trial035.stdout: "osd_map_message_max_bytes": "10485760", 2026-02-24T07:43:05.462 INFO:teuthology.orchestra.run.trial035.stdout: "osd_map_share_max_epochs": "40", 2026-02-24T07:43:05.462 INFO:teuthology.orchestra.run.trial035.stdout: "osd_max_attr_name_len": "100", 2026-02-24T07:43:05.462 INFO:teuthology.orchestra.run.trial035.stdout: "osd_max_attr_size": "0", 2026-02-24T07:43:05.462 INFO:teuthology.orchestra.run.trial035.stdout: "osd_max_backfills": "1", 2026-02-24T07:43:05.462 INFO:teuthology.orchestra.run.trial035.stdout: "osd_max_markdown_count": "5", 2026-02-24T07:43:05.462 INFO:teuthology.orchestra.run.trial035.stdout: "osd_max_markdown_period": "600", 2026-02-24T07:43:05.462 INFO:teuthology.orchestra.run.trial035.stdout: "osd_max_object_name_len": "2048", 2026-02-24T07:43:05.462 INFO:teuthology.orchestra.run.trial035.stdout: "osd_max_object_namespace_len": "256", 2026-02-24T07:43:05.462 INFO:teuthology.orchestra.run.trial035.stdout: "osd_max_object_size": "134217728", 2026-02-24T07:43:05.462 INFO:teuthology.orchestra.run.trial035.stdout: "osd_max_omap_bytes_per_request": "1073741824", 2026-02-24T07:43:05.462 INFO:teuthology.orchestra.run.trial035.stdout: "osd_max_omap_entries_per_request": "1024", 2026-02-24T07:43:05.462 INFO:teuthology.orchestra.run.trial035.stdout: "osd_max_pg_blocked_by": "16", 2026-02-24T07:43:05.462 INFO:teuthology.orchestra.run.trial035.stdout: "osd_max_pg_log_entries": "2", 2026-02-24T07:43:05.462 INFO:teuthology.orchestra.run.trial035.stdout: "osd_max_pg_per_osd_hard_ratio": "3.000000", 2026-02-24T07:43:05.462 INFO:teuthology.orchestra.run.trial035.stdout: "osd_max_pgls": "1024", 2026-02-24T07:43:05.462 INFO:teuthology.orchestra.run.trial035.stdout: "osd_max_push_cost": "8388608", 2026-02-24T07:43:05.462 INFO:teuthology.orchestra.run.trial035.stdout: "osd_max_push_objects": "10", 2026-02-24T07:43:05.462 INFO:teuthology.orchestra.run.trial035.stdout: "osd_max_scrubs": "3", 2026-02-24T07:43:05.462 INFO:teuthology.orchestra.run.trial035.stdout: "osd_max_snap_prune_intervals_per_epoch": "512", 2026-02-24T07:43:05.462 INFO:teuthology.orchestra.run.trial035.stdout: "osd_max_trimming_pgs": "2", 2026-02-24T07:43:05.462 INFO:teuthology.orchestra.run.trial035.stdout: "osd_max_write_op_reply_len": "64", 2026-02-24T07:43:05.462 INFO:teuthology.orchestra.run.trial035.stdout: "osd_max_write_size": "90", 2026-02-24T07:43:05.462 INFO:teuthology.orchestra.run.trial035.stdout: "osd_mclock_force_run_benchmark_on_init": "false", 2026-02-24T07:43:05.463 INFO:teuthology.orchestra.run.trial035.stdout: "osd_mclock_iops_capacity_low_threshold_hdd": "50.000000", 2026-02-24T07:43:05.463 INFO:teuthology.orchestra.run.trial035.stdout: "osd_mclock_iops_capacity_low_threshold_ssd": "1000.000000", 2026-02-24T07:43:05.463 INFO:teuthology.orchestra.run.trial035.stdout: "osd_mclock_iops_capacity_threshold_hdd": "500.000000", 2026-02-24T07:43:05.463 INFO:teuthology.orchestra.run.trial035.stdout: "osd_mclock_iops_capacity_threshold_ssd": "80000.000000", 2026-02-24T07:43:05.463 INFO:teuthology.orchestra.run.trial035.stdout: "osd_mclock_max_capacity_iops_hdd": "315.000000", 2026-02-24T07:43:05.463 INFO:teuthology.orchestra.run.trial035.stdout: "osd_mclock_max_capacity_iops_ssd": "21500.000000", 2026-02-24T07:43:05.463 INFO:teuthology.orchestra.run.trial035.stdout: "osd_mclock_max_sequential_bandwidth_hdd": "157286400", 2026-02-24T07:43:05.463 INFO:teuthology.orchestra.run.trial035.stdout: "osd_mclock_max_sequential_bandwidth_ssd": "1258291200", 2026-02-24T07:43:05.463 INFO:teuthology.orchestra.run.trial035.stdout: "osd_mclock_override_recovery_settings": "false", 2026-02-24T07:43:05.463 INFO:teuthology.orchestra.run.trial035.stdout: "osd_mclock_profile": "balanced", 2026-02-24T07:43:05.463 INFO:teuthology.orchestra.run.trial035.stdout: "osd_mclock_scheduler_anticipation_timeout": "0.000000", 2026-02-24T07:43:05.463 INFO:teuthology.orchestra.run.trial035.stdout: "osd_mclock_scheduler_background_best_effort_lim": "0.000000", 2026-02-24T07:43:05.463 INFO:teuthology.orchestra.run.trial035.stdout: "osd_mclock_scheduler_background_best_effort_res": "0.000000", 2026-02-24T07:43:05.463 INFO:teuthology.orchestra.run.trial035.stdout: "osd_mclock_scheduler_background_best_effort_wgt": "1", 2026-02-24T07:43:05.463 INFO:teuthology.orchestra.run.trial035.stdout: "osd_mclock_scheduler_background_recovery_lim": "0.000000", 2026-02-24T07:43:05.463 INFO:teuthology.orchestra.run.trial035.stdout: "osd_mclock_scheduler_background_recovery_res": "0.000000", 2026-02-24T07:43:05.463 INFO:teuthology.orchestra.run.trial035.stdout: "osd_mclock_scheduler_background_recovery_wgt": "1", 2026-02-24T07:43:05.463 INFO:teuthology.orchestra.run.trial035.stdout: "osd_mclock_scheduler_client_lim": "0.000000", 2026-02-24T07:43:05.463 INFO:teuthology.orchestra.run.trial035.stdout: "osd_mclock_scheduler_client_res": "0.000000", 2026-02-24T07:43:05.463 INFO:teuthology.orchestra.run.trial035.stdout: "osd_mclock_scheduler_client_wgt": "1", 2026-02-24T07:43:05.464 INFO:teuthology.orchestra.run.trial035.stdout: "osd_mclock_skip_benchmark": "false", 2026-02-24T07:43:05.464 INFO:teuthology.orchestra.run.trial035.stdout: "osd_memory_base": "805306368", 2026-02-24T07:43:05.464 INFO:teuthology.orchestra.run.trial035.stdout: "osd_memory_cache_min": "134217728", 2026-02-24T07:43:05.464 INFO:teuthology.orchestra.run.trial035.stdout: "osd_memory_cache_resize_interval": "1.000000", 2026-02-24T07:43:05.464 INFO:teuthology.orchestra.run.trial035.stdout: "osd_memory_expected_fragmentation": "0.150000", 2026-02-24T07:43:05.464 INFO:teuthology.orchestra.run.trial035.stdout: "osd_memory_target": "4294967296", 2026-02-24T07:43:05.464 INFO:teuthology.orchestra.run.trial035.stdout: "osd_memory_target_autotune": "false", 2026-02-24T07:43:05.464 INFO:teuthology.orchestra.run.trial035.stdout: "osd_memory_target_cgroup_limit_ratio": "0.800000", 2026-02-24T07:43:05.464 INFO:teuthology.orchestra.run.trial035.stdout: "osd_min_pg_log_entries": "1", 2026-02-24T07:43:05.464 INFO:teuthology.orchestra.run.trial035.stdout: "osd_min_recovery_priority": "0", 2026-02-24T07:43:05.464 INFO:teuthology.orchestra.run.trial035.stdout: "osd_mon_heartbeat_interval": "30", 2026-02-24T07:43:05.464 INFO:teuthology.orchestra.run.trial035.stdout: "osd_mon_heartbeat_stat_stale": "3600", 2026-02-24T07:43:05.464 INFO:teuthology.orchestra.run.trial035.stdout: "osd_mon_report_interval": "5", 2026-02-24T07:43:05.464 INFO:teuthology.orchestra.run.trial035.stdout: "osd_mon_report_max_in_flight": "2", 2026-02-24T07:43:05.464 INFO:teuthology.orchestra.run.trial035.stdout: "osd_mon_shutdown_timeout": "5.000000", 2026-02-24T07:43:05.464 INFO:teuthology.orchestra.run.trial035.stdout: "osd_num_cache_shards": "32", 2026-02-24T07:43:05.465 INFO:teuthology.orchestra.run.trial035.stdout: "osd_num_op_tracker_shard": "32", 2026-02-24T07:43:05.465 INFO:teuthology.orchestra.run.trial035.stdout: "osd_numa_auto_affinity": "true", 2026-02-24T07:43:05.465 INFO:teuthology.orchestra.run.trial035.stdout: "osd_numa_node": "-1", 2026-02-24T07:43:05.465 INFO:teuthology.orchestra.run.trial035.stdout: "osd_numa_prefer_iface": "true", 2026-02-24T07:43:05.465 INFO:teuthology.orchestra.run.trial035.stdout: "osd_object_clean_region_max_num_intervals": "10", 2026-02-24T07:43:05.465 INFO:teuthology.orchestra.run.trial035.stdout: "osd_objecter_finishers": "1", 2026-02-24T07:43:05.465 INFO:teuthology.orchestra.run.trial035.stdout: "osd_objectstore": "bluestore", 2026-02-24T07:43:05.465 INFO:teuthology.orchestra.run.trial035.stdout: "osd_objectstore_fuse": "false", 2026-02-24T07:43:05.465 INFO:teuthology.orchestra.run.trial035.stdout: "osd_objectstore_ideal_list_max": "64", 2026-02-24T07:43:05.465 INFO:teuthology.orchestra.run.trial035.stdout: "osd_objectstore_tracing": "false", 2026-02-24T07:43:05.465 INFO:teuthology.orchestra.run.trial035.stdout: "osd_op_complaint_time": "30.000000", 2026-02-24T07:43:05.465 INFO:teuthology.orchestra.run.trial035.stdout: "osd_op_history_duration": "600", 2026-02-24T07:43:05.465 INFO:teuthology.orchestra.run.trial035.stdout: "osd_op_history_size": "20", 2026-02-24T07:43:05.465 INFO:teuthology.orchestra.run.trial035.stdout: "osd_op_history_slow_op_size": "20", 2026-02-24T07:43:05.465 INFO:teuthology.orchestra.run.trial035.stdout: "osd_op_history_slow_op_threshold": "10.000000", 2026-02-24T07:43:05.465 INFO:teuthology.orchestra.run.trial035.stdout: "osd_op_log_threshold": "5", 2026-02-24T07:43:05.465 INFO:teuthology.orchestra.run.trial035.stdout: "osd_op_num_shards": "0", 2026-02-24T07:43:05.465 INFO:teuthology.orchestra.run.trial035.stdout: "osd_op_num_shards_hdd": "1", 2026-02-24T07:43:05.465 INFO:teuthology.orchestra.run.trial035.stdout: "osd_op_num_shards_ssd": "8", 2026-02-24T07:43:05.465 INFO:teuthology.orchestra.run.trial035.stdout: "osd_op_num_threads_per_shard": "0", 2026-02-24T07:43:05.465 INFO:teuthology.orchestra.run.trial035.stdout: "osd_op_num_threads_per_shard_hdd": "5", 2026-02-24T07:43:05.465 INFO:teuthology.orchestra.run.trial035.stdout: "osd_op_num_threads_per_shard_ssd": "2", 2026-02-24T07:43:05.465 INFO:teuthology.orchestra.run.trial035.stdout: "osd_op_pq_max_tokens_per_priority": "4194304", 2026-02-24T07:43:05.465 INFO:teuthology.orchestra.run.trial035.stdout: "osd_op_pq_min_cost": "65536", 2026-02-24T07:43:05.465 INFO:teuthology.orchestra.run.trial035.stdout: "osd_op_queue": "mclock_scheduler", 2026-02-24T07:43:05.465 INFO:teuthology.orchestra.run.trial035.stdout: "osd_op_queue_cut_off": "high", 2026-02-24T07:43:05.465 INFO:teuthology.orchestra.run.trial035.stdout: "osd_op_thread_suicide_timeout": "150", 2026-02-24T07:43:05.465 INFO:teuthology.orchestra.run.trial035.stdout: "osd_op_thread_timeout": "15", 2026-02-24T07:43:05.465 INFO:teuthology.orchestra.run.trial035.stdout: "osd_open_classes_on_start": "true", 2026-02-24T07:43:05.465 INFO:teuthology.orchestra.run.trial035.stdout: "osd_os_flags": "0", 2026-02-24T07:43:05.465 INFO:teuthology.orchestra.run.trial035.stdout: "osd_peering_op_priority": "255", 2026-02-24T07:43:05.465 INFO:teuthology.orchestra.run.trial035.stdout: "osd_pg_delete_cost": "1048576", 2026-02-24T07:43:05.465 INFO:teuthology.orchestra.run.trial035.stdout: "osd_pg_delete_priority": "5", 2026-02-24T07:43:05.465 INFO:teuthology.orchestra.run.trial035.stdout: "osd_pg_epoch_max_lag_factor": "2.000000", 2026-02-24T07:43:05.466 INFO:teuthology.orchestra.run.trial035.stdout: "osd_pg_epoch_persisted_max_stale": "40", 2026-02-24T07:43:05.466 INFO:teuthology.orchestra.run.trial035.stdout: "osd_pg_log_dups_tracked": "3000", 2026-02-24T07:43:05.466 INFO:teuthology.orchestra.run.trial035.stdout: "osd_pg_log_trim_max": "10000", 2026-02-24T07:43:05.466 INFO:teuthology.orchestra.run.trial035.stdout: "osd_pg_log_trim_min": "0", 2026-02-24T07:43:05.466 INFO:teuthology.orchestra.run.trial035.stdout: "osd_pg_max_concurrent_snap_trims": "2", 2026-02-24T07:43:05.466 INFO:teuthology.orchestra.run.trial035.stdout: "osd_pg_object_context_cache_count": "64", 2026-02-24T07:43:05.466 INFO:teuthology.orchestra.run.trial035.stdout: "osd_pg_stat_report_interval_max_epochs": "500", 2026-02-24T07:43:05.466 INFO:teuthology.orchestra.run.trial035.stdout: "osd_pg_stat_report_interval_max_seconds": "5", 2026-02-24T07:43:05.466 INFO:teuthology.orchestra.run.trial035.stdout: "osd_pool_default_cache_max_evict_check_size": "10", 2026-02-24T07:43:05.466 INFO:teuthology.orchestra.run.trial035.stdout: "osd_pool_default_cache_min_evict_age": "0", 2026-02-24T07:43:05.466 INFO:teuthology.orchestra.run.trial035.stdout: "osd_pool_default_cache_min_flush_age": "0", 2026-02-24T07:43:05.466 INFO:teuthology.orchestra.run.trial035.stdout: "osd_pool_default_cache_target_dirty_high_ratio": "0.600000", 2026-02-24T07:43:05.466 INFO:teuthology.orchestra.run.trial035.stdout: "osd_pool_default_cache_target_dirty_ratio": "0.400000", 2026-02-24T07:43:05.466 INFO:teuthology.orchestra.run.trial035.stdout: "osd_pool_default_cache_target_full_ratio": "0.800000", 2026-02-24T07:43:05.466 INFO:teuthology.orchestra.run.trial035.stdout: "osd_pool_default_crimson": "true", 2026-02-24T07:43:05.466 INFO:teuthology.orchestra.run.trial035.stdout: "osd_pool_default_crush_rule": "-1", 2026-02-24T07:43:05.466 INFO:teuthology.orchestra.run.trial035.stdout: "osd_pool_default_ec_fast_read": "false", 2026-02-24T07:43:05.466 INFO:teuthology.orchestra.run.trial035.stdout: "osd_pool_default_erasure_code_profile": "plugin=isa technique=reed_sol_van k=2 m=1 crush-failure-domain=osd", 2026-02-24T07:43:05.466 INFO:teuthology.orchestra.run.trial035.stdout: "osd_pool_default_flag_bulk": "false", 2026-02-24T07:43:05.466 INFO:teuthology.orchestra.run.trial035.stdout: "osd_pool_default_flag_ec_optimizations": "false", 2026-02-24T07:43:05.466 INFO:teuthology.orchestra.run.trial035.stdout: "osd_pool_default_flag_hashpspool": "true", 2026-02-24T07:43:05.466 INFO:teuthology.orchestra.run.trial035.stdout: "osd_pool_default_flag_nodelete": "false", 2026-02-24T07:43:05.466 INFO:teuthology.orchestra.run.trial035.stdout: "osd_pool_default_flag_nopgchange": "false", 2026-02-24T07:43:05.466 INFO:teuthology.orchestra.run.trial035.stdout: "osd_pool_default_flag_nosizechange": "false", 2026-02-24T07:43:05.466 INFO:teuthology.orchestra.run.trial035.stdout: "osd_pool_default_flags": "0", 2026-02-24T07:43:05.466 INFO:teuthology.orchestra.run.trial035.stdout: "osd_pool_default_hit_set_bloom_fpp": "0.050000", 2026-02-24T07:43:05.466 INFO:teuthology.orchestra.run.trial035.stdout: "osd_pool_default_min_size": "2", 2026-02-24T07:43:05.466 INFO:teuthology.orchestra.run.trial035.stdout: "osd_pool_default_pg_autoscale_mode": "off", 2026-02-24T07:43:05.466 INFO:teuthology.orchestra.run.trial035.stdout: "osd_pool_default_pg_num": "32", 2026-02-24T07:43:05.466 INFO:teuthology.orchestra.run.trial035.stdout: "osd_pool_default_pgp_num": "0", 2026-02-24T07:43:05.466 INFO:teuthology.orchestra.run.trial035.stdout: "osd_pool_default_read_lease_ratio": "0.800000", 2026-02-24T07:43:05.466 INFO:teuthology.orchestra.run.trial035.stdout: "osd_pool_default_read_ratio": "70", 2026-02-24T07:43:05.466 INFO:teuthology.orchestra.run.trial035.stdout: "osd_pool_default_size": "3", 2026-02-24T07:43:05.466 INFO:teuthology.orchestra.run.trial035.stdout: "osd_pool_default_type": "replicated", 2026-02-24T07:43:05.467 INFO:teuthology.orchestra.run.trial035.stdout: "osd_pool_erasure_code_stripe_unit": "0", 2026-02-24T07:43:05.467 INFO:teuthology.orchestra.run.trial035.stdout: "osd_pool_use_gmt_hitset": "true", 2026-02-24T07:43:05.467 INFO:teuthology.orchestra.run.trial035.stdout: "osd_push_per_object_cost": "1000", 2026-02-24T07:43:05.467 INFO:teuthology.orchestra.run.trial035.stdout: "osd_read_ec_check_for_errors": "false", 2026-02-24T07:43:05.467 INFO:teuthology.orchestra.run.trial035.stdout: "osd_recover_clone_overlap": "true", 2026-02-24T07:43:05.467 INFO:teuthology.orchestra.run.trial035.stdout: "osd_recover_clone_overlap_limit": "10", 2026-02-24T07:43:05.467 INFO:teuthology.orchestra.run.trial035.stdout: "osd_recovery_cost": "20971520", 2026-02-24T07:43:05.467 INFO:teuthology.orchestra.run.trial035.stdout: "osd_recovery_delay_start": "0.000000", 2026-02-24T07:43:05.467 INFO:teuthology.orchestra.run.trial035.stdout: "osd_recovery_max_active": "0", 2026-02-24T07:43:05.467 INFO:teuthology.orchestra.run.trial035.stdout: "osd_recovery_max_active_hdd": "3", 2026-02-24T07:43:05.467 INFO:teuthology.orchestra.run.trial035.stdout: "osd_recovery_max_active_ssd": "10", 2026-02-24T07:43:05.467 INFO:teuthology.orchestra.run.trial035.stdout: "osd_recovery_max_chunk": "8388608", 2026-02-24T07:43:05.467 INFO:teuthology.orchestra.run.trial035.stdout: "osd_recovery_max_omap_entries_per_chunk": "8096", 2026-02-24T07:43:05.467 INFO:teuthology.orchestra.run.trial035.stdout: "osd_recovery_max_single_start": "1", 2026-02-24T07:43:05.467 INFO:teuthology.orchestra.run.trial035.stdout: "osd_recovery_op_priority": "3", 2026-02-24T07:43:05.467 INFO:teuthology.orchestra.run.trial035.stdout: "osd_recovery_op_warn_multiple": "16", 2026-02-24T07:43:05.467 INFO:teuthology.orchestra.run.trial035.stdout: "osd_recovery_priority": "5", 2026-02-24T07:43:05.467 INFO:teuthology.orchestra.run.trial035.stdout: "osd_recovery_retry_interval": "30.000000", 2026-02-24T07:43:05.467 INFO:teuthology.orchestra.run.trial035.stdout: "osd_recovery_sleep": "0.000000", 2026-02-24T07:43:05.467 INFO:teuthology.orchestra.run.trial035.stdout: "osd_recovery_sleep_degraded": "0.000000", 2026-02-24T07:43:05.467 INFO:teuthology.orchestra.run.trial035.stdout: "osd_recovery_sleep_degraded_hdd": "0.100000", 2026-02-24T07:43:05.467 INFO:teuthology.orchestra.run.trial035.stdout: "osd_recovery_sleep_degraded_hybrid": "0.025000", 2026-02-24T07:43:05.467 INFO:teuthology.orchestra.run.trial035.stdout: "osd_recovery_sleep_degraded_ssd": "0.000000", 2026-02-24T07:43:05.467 INFO:teuthology.orchestra.run.trial035.stdout: "osd_recovery_sleep_hdd": "0.100000", 2026-02-24T07:43:05.467 INFO:teuthology.orchestra.run.trial035.stdout: "osd_recovery_sleep_hybrid": "0.025000", 2026-02-24T07:43:05.467 INFO:teuthology.orchestra.run.trial035.stdout: "osd_recovery_sleep_ssd": "0.000000", 2026-02-24T07:43:05.467 INFO:teuthology.orchestra.run.trial035.stdout: "osd_requested_scrub_priority": "5", 2026-02-24T07:43:05.467 INFO:teuthology.orchestra.run.trial035.stdout: "osd_rocksdb_iterator_bounds_enabled": "true", 2026-02-24T07:43:05.467 INFO:teuthology.orchestra.run.trial035.stdout: "osd_rollback_to_cluster_snap": "", 2026-02-24T07:43:05.467 INFO:teuthology.orchestra.run.trial035.stdout: "osd_scrub_auto_repair": "false", 2026-02-24T07:43:05.467 INFO:teuthology.orchestra.run.trial035.stdout: "osd_scrub_auto_repair_num_errors": "5", 2026-02-24T07:43:05.467 INFO:teuthology.orchestra.run.trial035.stdout: "osd_scrub_backoff_ratio": "0.660000", 2026-02-24T07:43:05.467 INFO:teuthology.orchestra.run.trial035.stdout: "osd_scrub_begin_hour": "0", 2026-02-24T07:43:05.467 INFO:teuthology.orchestra.run.trial035.stdout: "osd_scrub_begin_week_day": "0", 2026-02-24T07:43:05.467 INFO:teuthology.orchestra.run.trial035.stdout: "osd_scrub_chunk_max": "15", 2026-02-24T07:43:05.467 INFO:teuthology.orchestra.run.trial035.stdout: "osd_scrub_chunk_min": "5", 2026-02-24T07:43:05.468 INFO:teuthology.orchestra.run.trial035.stdout: "osd_scrub_cost": "52428800", 2026-02-24T07:43:05.468 INFO:teuthology.orchestra.run.trial035.stdout: "osd_scrub_disable_reservation_queuing": "false", 2026-02-24T07:43:05.468 INFO:teuthology.orchestra.run.trial035.stdout: "osd_scrub_during_recovery": "false", 2026-02-24T07:43:05.468 INFO:teuthology.orchestra.run.trial035.stdout: "osd_scrub_end_hour": "0", 2026-02-24T07:43:05.468 INFO:teuthology.orchestra.run.trial035.stdout: "osd_scrub_end_week_day": "0", 2026-02-24T07:43:05.468 INFO:teuthology.orchestra.run.trial035.stdout: "osd_scrub_event_cost": "4096", 2026-02-24T07:43:05.468 INFO:teuthology.orchestra.run.trial035.stdout: "osd_scrub_extended_sleep": "0.000000", 2026-02-24T07:43:05.468 INFO:teuthology.orchestra.run.trial035.stdout: "osd_scrub_interval_randomize_ratio": "0.500000", 2026-02-24T07:43:05.468 INFO:teuthology.orchestra.run.trial035.stdout: "osd_scrub_invalid_stats": "true", 2026-02-24T07:43:05.468 INFO:teuthology.orchestra.run.trial035.stdout: "osd_scrub_load_threshold": "10.000000", 2026-02-24T07:43:05.468 INFO:teuthology.orchestra.run.trial035.stdout: "osd_scrub_max_interval": "604800.000000", 2026-02-24T07:43:05.468 INFO:teuthology.orchestra.run.trial035.stdout: "osd_scrub_max_preemptions": "5", 2026-02-24T07:43:05.468 INFO:teuthology.orchestra.run.trial035.stdout: "osd_scrub_min_interval": "86400.000000", 2026-02-24T07:43:05.468 INFO:teuthology.orchestra.run.trial035.stdout: "osd_scrub_priority": "5", 2026-02-24T07:43:05.468 INFO:teuthology.orchestra.run.trial035.stdout: "osd_scrub_retry_after_noscrub": "60", 2026-02-24T07:43:05.468 INFO:teuthology.orchestra.run.trial035.stdout: "osd_scrub_retry_delay": "30", 2026-02-24T07:43:05.468 INFO:teuthology.orchestra.run.trial035.stdout: "osd_scrub_retry_new_interval": "10", 2026-02-24T07:43:05.468 INFO:teuthology.orchestra.run.trial035.stdout: "osd_scrub_retry_pg_state": "60", 2026-02-24T07:43:05.468 INFO:teuthology.orchestra.run.trial035.stdout: "osd_scrub_retry_trimming": "10", 2026-02-24T07:43:05.468 INFO:teuthology.orchestra.run.trial035.stdout: "osd_scrub_sleep": "0.000000", 2026-02-24T07:43:05.468 INFO:teuthology.orchestra.run.trial035.stdout: "osd_shallow_scrub_chunk_max": "100", 2026-02-24T07:43:05.468 INFO:teuthology.orchestra.run.trial035.stdout: "osd_shallow_scrub_chunk_min": "50", 2026-02-24T07:43:05.468 INFO:teuthology.orchestra.run.trial035.stdout: "osd_shutdown_pgref_assert": "false", 2026-02-24T07:43:05.468 INFO:teuthology.orchestra.run.trial035.stdout: "osd_skip_check_past_interval_bounds": "false", 2026-02-24T07:43:05.468 INFO:teuthology.orchestra.run.trial035.stdout: "osd_skip_data_digest": "false", 2026-02-24T07:43:05.468 INFO:teuthology.orchestra.run.trial035.stdout: "osd_smart_report_timeout": "5", 2026-02-24T07:43:05.468 INFO:teuthology.orchestra.run.trial035.stdout: "osd_snap_trim_cost": "1048576", 2026-02-24T07:43:05.468 INFO:teuthology.orchestra.run.trial035.stdout: "osd_snap_trim_priority": "5", 2026-02-24T07:43:05.468 INFO:teuthology.orchestra.run.trial035.stdout: "osd_snap_trim_sleep": "0.000000", 2026-02-24T07:43:05.468 INFO:teuthology.orchestra.run.trial035.stdout: "osd_snap_trim_sleep_hdd": "5.000000", 2026-02-24T07:43:05.468 INFO:teuthology.orchestra.run.trial035.stdout: "osd_snap_trim_sleep_hybrid": "2.000000", 2026-02-24T07:43:05.468 INFO:teuthology.orchestra.run.trial035.stdout: "osd_snap_trim_sleep_ssd": "0.000000", 2026-02-24T07:43:05.468 INFO:teuthology.orchestra.run.trial035.stdout: "osd_stats_update_period_not_scrubbing": "120", 2026-02-24T07:43:05.468 INFO:teuthology.orchestra.run.trial035.stdout: "osd_stats_update_period_scrubbing": "15", 2026-02-24T07:43:05.468 INFO:teuthology.orchestra.run.trial035.stdout: "osd_target_pg_log_entries_per_osd": "300000", 2026-02-24T07:43:05.468 INFO:teuthology.orchestra.run.trial035.stdout: "osd_target_transaction_size": "30", 2026-02-24T07:43:05.468 INFO:teuthology.orchestra.run.trial035.stdout: "osd_tier_default_cache_hit_set_count": "4", 2026-02-24T07:43:05.468 INFO:teuthology.orchestra.run.trial035.stdout: "osd_tier_default_cache_hit_set_grade_decay_rate": "20", 2026-02-24T07:43:05.468 INFO:teuthology.orchestra.run.trial035.stdout: "osd_tier_default_cache_hit_set_period": "1200", 2026-02-24T07:43:05.469 INFO:teuthology.orchestra.run.trial035.stdout: "osd_tier_default_cache_hit_set_search_last_n": "1", 2026-02-24T07:43:05.469 INFO:teuthology.orchestra.run.trial035.stdout: "osd_tier_default_cache_hit_set_type": "bloom", 2026-02-24T07:43:05.469 INFO:teuthology.orchestra.run.trial035.stdout: "osd_tier_default_cache_min_read_recency_for_promote": "1", 2026-02-24T07:43:05.469 INFO:teuthology.orchestra.run.trial035.stdout: "osd_tier_default_cache_min_write_recency_for_promote": "1", 2026-02-24T07:43:05.469 INFO:teuthology.orchestra.run.trial035.stdout: "osd_tier_default_cache_mode": "writeback", 2026-02-24T07:43:05.469 INFO:teuthology.orchestra.run.trial035.stdout: "osd_tier_promote_max_bytes_sec": "5242880", 2026-02-24T07:43:05.469 INFO:teuthology.orchestra.run.trial035.stdout: "osd_tier_promote_max_objects_sec": "25", 2026-02-24T07:43:05.469 INFO:teuthology.orchestra.run.trial035.stdout: "osd_tracing": "false", 2026-02-24T07:43:05.469 INFO:teuthology.orchestra.run.trial035.stdout: "osd_use_stale_snap": "false", 2026-02-24T07:43:05.469 INFO:teuthology.orchestra.run.trial035.stdout: "osd_uuid": "00000000-0000-0000-0000-000000000000", 2026-02-24T07:43:05.469 INFO:teuthology.orchestra.run.trial035.stdout: "osdc_blkin_trace_all": "false", 2026-02-24T07:43:05.469 INFO:teuthology.orchestra.run.trial035.stdout: "paxos_kill_at": "0", 2026-02-24T07:43:05.469 INFO:teuthology.orchestra.run.trial035.stdout: "paxos_max_join_drift": "10", 2026-02-24T07:43:05.469 INFO:teuthology.orchestra.run.trial035.stdout: "paxos_min": "500", 2026-02-24T07:43:05.469 INFO:teuthology.orchestra.run.trial035.stdout: "paxos_min_wait": "0.050000", 2026-02-24T07:43:05.469 INFO:teuthology.orchestra.run.trial035.stdout: "paxos_propose_interval": "1.000000", 2026-02-24T07:43:05.469 INFO:teuthology.orchestra.run.trial035.stdout: "paxos_service_trim_max": "500", 2026-02-24T07:43:05.469 INFO:teuthology.orchestra.run.trial035.stdout: "paxos_service_trim_max_multiplier": "20", 2026-02-24T07:43:05.469 INFO:teuthology.orchestra.run.trial035.stdout: "paxos_service_trim_min": "10", 2026-02-24T07:43:05.469 INFO:teuthology.orchestra.run.trial035.stdout: "paxos_stash_full_interval": "25", 2026-02-24T07:43:05.469 INFO:teuthology.orchestra.run.trial035.stdout: "paxos_trim_max": "500", 2026-02-24T07:43:05.469 INFO:teuthology.orchestra.run.trial035.stdout: "paxos_trim_min": "250", 2026-02-24T07:43:05.469 INFO:teuthology.orchestra.run.trial035.stdout: "perf": "true", 2026-02-24T07:43:05.469 INFO:teuthology.orchestra.run.trial035.stdout: "pid_file": "", 2026-02-24T07:43:05.469 INFO:teuthology.orchestra.run.trial035.stdout: "plugin_crypto_accelerator": "crypto_isal", 2026-02-24T07:43:05.469 INFO:teuthology.orchestra.run.trial035.stdout: "plugin_dir": "/usr/lib64/ceph", 2026-02-24T07:43:05.469 INFO:teuthology.orchestra.run.trial035.stdout: "pool_availability_update_interval": "1.000000", 2026-02-24T07:43:05.469 INFO:teuthology.orchestra.run.trial035.stdout: "public_addr": "-", 2026-02-24T07:43:05.469 INFO:teuthology.orchestra.run.trial035.stdout: "public_addrv": "", 2026-02-24T07:43:05.469 INFO:teuthology.orchestra.run.trial035.stdout: "public_bind_addr": "-", 2026-02-24T07:43:05.469 INFO:teuthology.orchestra.run.trial035.stdout: "public_network": "", 2026-02-24T07:43:05.469 INFO:teuthology.orchestra.run.trial035.stdout: "public_network_interface": "", 2026-02-24T07:43:05.469 INFO:teuthology.orchestra.run.trial035.stdout: "qat_compressor_busy_polling": "false", 2026-02-24T07:43:05.469 INFO:teuthology.orchestra.run.trial035.stdout: "qat_compressor_enabled": "false", 2026-02-24T07:43:05.469 INFO:teuthology.orchestra.run.trial035.stdout: "qat_compressor_session_max_number": "256", 2026-02-24T07:43:05.469 INFO:teuthology.orchestra.run.trial035.stdout: "rados_mon_op_timeout": "0", 2026-02-24T07:43:05.469 INFO:teuthology.orchestra.run.trial035.stdout: "rados_osd_op_timeout": "0", 2026-02-24T07:43:05.469 INFO:teuthology.orchestra.run.trial035.stdout: "rados_replica_read_policy": "default", 2026-02-24T07:43:05.470 INFO:teuthology.orchestra.run.trial035.stdout: "rados_replica_read_policy_on_objclass": "false", 2026-02-24T07:43:05.470 INFO:teuthology.orchestra.run.trial035.stdout: "rados_tracing": "false", 2026-02-24T07:43:05.470 INFO:teuthology.orchestra.run.trial035.stdout: "rbd_atime_update_interval": "60", 2026-02-24T07:43:05.470 INFO:teuthology.orchestra.run.trial035.stdout: "rbd_auto_exclusive_lock_until_manual_request": "true", 2026-02-24T07:43:05.470 INFO:teuthology.orchestra.run.trial035.stdout: "rbd_balance_parent_reads": "false", 2026-02-24T07:43:05.470 INFO:teuthology.orchestra.run.trial035.stdout: "rbd_balance_snap_reads": "false", 2026-02-24T07:43:05.470 INFO:teuthology.orchestra.run.trial035.stdout: "rbd_blkin_trace_all": "false", 2026-02-24T07:43:05.470 INFO:teuthology.orchestra.run.trial035.stdout: "rbd_blocklist_expire_seconds": "0", 2026-02-24T07:43:05.470 INFO:teuthology.orchestra.run.trial035.stdout: "rbd_blocklist_on_break_lock": "true", 2026-02-24T07:43:05.470 INFO:teuthology.orchestra.run.trial035.stdout: "rbd_cache": "true", 2026-02-24T07:43:05.470 INFO:teuthology.orchestra.run.trial035.stdout: "rbd_cache_block_writes_upfront": "false", 2026-02-24T07:43:05.470 INFO:teuthology.orchestra.run.trial035.stdout: "rbd_cache_max_dirty": "25165824", 2026-02-24T07:43:05.470 INFO:teuthology.orchestra.run.trial035.stdout: "rbd_cache_max_dirty_age": "1.000000", 2026-02-24T07:43:05.470 INFO:teuthology.orchestra.run.trial035.stdout: "rbd_cache_max_dirty_object": "0", 2026-02-24T07:43:05.470 INFO:teuthology.orchestra.run.trial035.stdout: "rbd_cache_policy": "writearound", 2026-02-24T07:43:05.470 INFO:teuthology.orchestra.run.trial035.stdout: "rbd_cache_size": "33554432", 2026-02-24T07:43:05.470 INFO:teuthology.orchestra.run.trial035.stdout: "rbd_cache_target_dirty": "16777216", 2026-02-24T07:43:05.470 INFO:teuthology.orchestra.run.trial035.stdout: "rbd_cache_writethrough_until_flush": "true", 2026-02-24T07:43:05.470 INFO:teuthology.orchestra.run.trial035.stdout: "rbd_clone_copy_on_read": "false", 2026-02-24T07:43:05.470 INFO:teuthology.orchestra.run.trial035.stdout: "rbd_compression_hint": "none", 2026-02-24T07:43:05.470 INFO:teuthology.orchestra.run.trial035.stdout: "rbd_concurrent_management_ops": "10", 2026-02-24T07:43:05.470 INFO:teuthology.orchestra.run.trial035.stdout: "rbd_config_pool_override_update_timestamp": "0", 2026-02-24T07:43:05.470 INFO:teuthology.orchestra.run.trial035.stdout: "rbd_default_clone_format": "auto", 2026-02-24T07:43:05.470 INFO:teuthology.orchestra.run.trial035.stdout: "rbd_default_data_pool": "", 2026-02-24T07:43:05.470 INFO:teuthology.orchestra.run.trial035.stdout: "rbd_default_features": "61", 2026-02-24T07:43:05.470 INFO:teuthology.orchestra.run.trial035.stdout: "rbd_default_format": "2", 2026-02-24T07:43:05.470 INFO:teuthology.orchestra.run.trial035.stdout: "rbd_default_map_options": "", 2026-02-24T07:43:05.470 INFO:teuthology.orchestra.run.trial035.stdout: "rbd_default_order": "22", 2026-02-24T07:43:05.470 INFO:teuthology.orchestra.run.trial035.stdout: "rbd_default_pool": "rbd", 2026-02-24T07:43:05.470 INFO:teuthology.orchestra.run.trial035.stdout: "rbd_default_snapshot_quiesce_mode": "required", 2026-02-24T07:43:05.470 INFO:teuthology.orchestra.run.trial035.stdout: "rbd_default_stripe_count": "0", 2026-02-24T07:43:05.470 INFO:teuthology.orchestra.run.trial035.stdout: "rbd_default_stripe_unit": "0", 2026-02-24T07:43:05.470 INFO:teuthology.orchestra.run.trial035.stdout: "rbd_disable_zero_copy_writes": "true", 2026-02-24T07:43:05.470 INFO:teuthology.orchestra.run.trial035.stdout: "rbd_discard_granularity_bytes": "65536", 2026-02-24T07:43:05.470 INFO:teuthology.orchestra.run.trial035.stdout: "rbd_discard_on_zeroed_write_same": "true", 2026-02-24T07:43:05.470 INFO:teuthology.orchestra.run.trial035.stdout: "rbd_enable_alloc_hint": "true", 2026-02-24T07:43:05.470 INFO:teuthology.orchestra.run.trial035.stdout: "rbd_invalidate_object_map_on_timeout": "true", 2026-02-24T07:43:05.470 INFO:teuthology.orchestra.run.trial035.stdout: "rbd_io_scheduler": "simple", 2026-02-24T07:43:05.471 INFO:teuthology.orchestra.run.trial035.stdout: "rbd_io_scheduler_simple_max_delay": "0", 2026-02-24T07:43:05.471 INFO:teuthology.orchestra.run.trial035.stdout: "rbd_journal_commit_age": "5.000000", 2026-02-24T07:43:05.471 INFO:teuthology.orchestra.run.trial035.stdout: "rbd_journal_max_concurrent_object_sets": "0", 2026-02-24T07:43:05.471 INFO:teuthology.orchestra.run.trial035.stdout: "rbd_journal_max_payload_bytes": "16384", 2026-02-24T07:43:05.471 INFO:teuthology.orchestra.run.trial035.stdout: "rbd_journal_object_flush_age": "0.000000", 2026-02-24T07:43:05.471 INFO:teuthology.orchestra.run.trial035.stdout: "rbd_journal_object_flush_bytes": "1048576", 2026-02-24T07:43:05.471 INFO:teuthology.orchestra.run.trial035.stdout: "rbd_journal_object_flush_interval": "0", 2026-02-24T07:43:05.471 INFO:teuthology.orchestra.run.trial035.stdout: "rbd_journal_object_max_in_flight_appends": "0", 2026-02-24T07:43:05.471 INFO:teuthology.orchestra.run.trial035.stdout: "rbd_journal_object_writethrough_until_flush": "true", 2026-02-24T07:43:05.471 INFO:teuthology.orchestra.run.trial035.stdout: "rbd_journal_order": "24", 2026-02-24T07:43:05.471 INFO:teuthology.orchestra.run.trial035.stdout: "rbd_journal_pool": "", 2026-02-24T07:43:05.471 INFO:teuthology.orchestra.run.trial035.stdout: "rbd_journal_splay_width": "4", 2026-02-24T07:43:05.471 INFO:teuthology.orchestra.run.trial035.stdout: "rbd_localize_parent_reads": "false", 2026-02-24T07:43:05.471 INFO:teuthology.orchestra.run.trial035.stdout: "rbd_localize_snap_reads": "false", 2026-02-24T07:43:05.471 INFO:teuthology.orchestra.run.trial035.stdout: "rbd_mirror_concurrent_image_deletions": "1", 2026-02-24T07:43:05.471 INFO:teuthology.orchestra.run.trial035.stdout: "rbd_mirror_concurrent_image_syncs": "5", 2026-02-24T07:43:05.471 INFO:teuthology.orchestra.run.trial035.stdout: "rbd_mirror_delete_retry_interval": "30.000000", 2026-02-24T07:43:05.471 INFO:teuthology.orchestra.run.trial035.stdout: "rbd_mirror_image_perf_stats_prio": "5", 2026-02-24T07:43:05.471 INFO:teuthology.orchestra.run.trial035.stdout: "rbd_mirror_image_policy_migration_throttle": "300", 2026-02-24T07:43:05.471 INFO:teuthology.orchestra.run.trial035.stdout: "rbd_mirror_image_policy_rebalance_timeout": "0.000000", 2026-02-24T07:43:05.471 INFO:teuthology.orchestra.run.trial035.stdout: "rbd_mirror_image_policy_type": "simple", 2026-02-24T07:43:05.471 INFO:teuthology.orchestra.run.trial035.stdout: "rbd_mirror_image_policy_update_throttle_interval": "1.000000", 2026-02-24T07:43:05.471 INFO:teuthology.orchestra.run.trial035.stdout: "rbd_mirror_image_state_check_interval": "30", 2026-02-24T07:43:05.471 INFO:teuthology.orchestra.run.trial035.stdout: "rbd_mirror_journal_commit_age": "5.000000", 2026-02-24T07:43:05.471 INFO:teuthology.orchestra.run.trial035.stdout: "rbd_mirror_journal_poll_age": "5.000000", 2026-02-24T07:43:05.471 INFO:teuthology.orchestra.run.trial035.stdout: "rbd_mirror_leader_heartbeat_interval": "5", 2026-02-24T07:43:05.471 INFO:teuthology.orchestra.run.trial035.stdout: "rbd_mirror_leader_max_acquire_attempts_before_break": "3", 2026-02-24T07:43:05.471 INFO:teuthology.orchestra.run.trial035.stdout: "rbd_mirror_leader_max_missed_heartbeats": "2", 2026-02-24T07:43:05.471 INFO:teuthology.orchestra.run.trial035.stdout: "rbd_mirror_memory_autotune": "true", 2026-02-24T07:43:05.471 INFO:teuthology.orchestra.run.trial035.stdout: "rbd_mirror_memory_base": "805306368", 2026-02-24T07:43:05.471 INFO:teuthology.orchestra.run.trial035.stdout: "rbd_mirror_memory_cache_autotune_interval": "30.000000", 2026-02-24T07:43:05.471 INFO:teuthology.orchestra.run.trial035.stdout: "rbd_mirror_memory_cache_min": "134217728", 2026-02-24T07:43:05.471 INFO:teuthology.orchestra.run.trial035.stdout: "rbd_mirror_memory_cache_resize_interval": "5.000000", 2026-02-24T07:43:05.471 INFO:teuthology.orchestra.run.trial035.stdout: "rbd_mirror_memory_expected_fragmentation": "0.150000", 2026-02-24T07:43:05.471 INFO:teuthology.orchestra.run.trial035.stdout: "rbd_mirror_memory_target": "4294967296", 2026-02-24T07:43:05.471 INFO:teuthology.orchestra.run.trial035.stdout: "rbd_mirror_perf_stats_prio": "5", 2026-02-24T07:43:05.471 INFO:teuthology.orchestra.run.trial035.stdout: "rbd_mirror_pool_replayers_refresh_interval": "30", 2026-02-24T07:43:05.471 INFO:teuthology.orchestra.run.trial035.stdout: "rbd_mirror_sync_point_update_age": "30.000000", 2026-02-24T07:43:05.471 INFO:teuthology.orchestra.run.trial035.stdout: "rbd_mirroring_delete_delay": "0", 2026-02-24T07:43:05.471 INFO:teuthology.orchestra.run.trial035.stdout: "rbd_mirroring_max_mirroring_snapshots": "5", 2026-02-24T07:43:05.472 INFO:teuthology.orchestra.run.trial035.stdout: "rbd_mirroring_replay_delay": "0", 2026-02-24T07:43:05.472 INFO:teuthology.orchestra.run.trial035.stdout: "rbd_mirroring_resync_after_disconnect": "false", 2026-02-24T07:43:05.472 INFO:teuthology.orchestra.run.trial035.stdout: "rbd_move_parent_to_trash_on_remove": "false", 2026-02-24T07:43:05.472 INFO:teuthology.orchestra.run.trial035.stdout: "rbd_move_to_trash_on_remove": "false", 2026-02-24T07:43:05.472 INFO:teuthology.orchestra.run.trial035.stdout: "rbd_move_to_trash_on_remove_expire_seconds": "0", 2026-02-24T07:43:05.472 INFO:teuthology.orchestra.run.trial035.stdout: "rbd_mtime_update_interval": "60", 2026-02-24T07:43:05.472 INFO:teuthology.orchestra.run.trial035.stdout: "rbd_non_blocking_aio": "true", 2026-02-24T07:43:05.472 INFO:teuthology.orchestra.run.trial035.stdout: "rbd_op_thread_timeout": "60", 2026-02-24T07:43:05.472 INFO:teuthology.orchestra.run.trial035.stdout: "rbd_op_threads": "1", 2026-02-24T07:43:05.472 INFO:teuthology.orchestra.run.trial035.stdout: "rbd_parent_cache_enabled": "false", 2026-02-24T07:43:05.472 INFO:teuthology.orchestra.run.trial035.stdout: "rbd_persistent_cache_mode": "disabled", 2026-02-24T07:43:05.472 INFO:teuthology.orchestra.run.trial035.stdout: "rbd_persistent_cache_path": "/tmp", 2026-02-24T07:43:05.472 INFO:teuthology.orchestra.run.trial035.stdout: "rbd_persistent_cache_size": "1073741824", 2026-02-24T07:43:05.472 INFO:teuthology.orchestra.run.trial035.stdout: "rbd_plugins": "", 2026-02-24T07:43:05.472 INFO:teuthology.orchestra.run.trial035.stdout: "rbd_qos_bps_burst": "0", 2026-02-24T07:43:05.472 INFO:teuthology.orchestra.run.trial035.stdout: "rbd_qos_bps_burst_seconds": "1", 2026-02-24T07:43:05.472 INFO:teuthology.orchestra.run.trial035.stdout: "rbd_qos_bps_limit": "0", 2026-02-24T07:43:05.472 INFO:teuthology.orchestra.run.trial035.stdout: "rbd_qos_exclude_ops": "0", 2026-02-24T07:43:05.472 INFO:teuthology.orchestra.run.trial035.stdout: "rbd_qos_iops_burst": "0", 2026-02-24T07:43:05.472 INFO:teuthology.orchestra.run.trial035.stdout: "rbd_qos_iops_burst_seconds": "1", 2026-02-24T07:43:05.472 INFO:teuthology.orchestra.run.trial035.stdout: "rbd_qos_iops_limit": "0", 2026-02-24T07:43:05.472 INFO:teuthology.orchestra.run.trial035.stdout: "rbd_qos_read_bps_burst": "0", 2026-02-24T07:43:05.472 INFO:teuthology.orchestra.run.trial035.stdout: "rbd_qos_read_bps_burst_seconds": "1", 2026-02-24T07:43:05.472 INFO:teuthology.orchestra.run.trial035.stdout: "rbd_qos_read_bps_limit": "0", 2026-02-24T07:43:05.472 INFO:teuthology.orchestra.run.trial035.stdout: "rbd_qos_read_iops_burst": "0", 2026-02-24T07:43:05.472 INFO:teuthology.orchestra.run.trial035.stdout: "rbd_qos_read_iops_burst_seconds": "1", 2026-02-24T07:43:05.472 INFO:teuthology.orchestra.run.trial035.stdout: "rbd_qos_read_iops_limit": "0", 2026-02-24T07:43:05.472 INFO:teuthology.orchestra.run.trial035.stdout: "rbd_qos_schedule_tick_min": "50", 2026-02-24T07:43:05.472 INFO:teuthology.orchestra.run.trial035.stdout: "rbd_qos_write_bps_burst": "0", 2026-02-24T07:43:05.472 INFO:teuthology.orchestra.run.trial035.stdout: "rbd_qos_write_bps_burst_seconds": "1", 2026-02-24T07:43:05.472 INFO:teuthology.orchestra.run.trial035.stdout: "rbd_qos_write_bps_limit": "0", 2026-02-24T07:43:05.472 INFO:teuthology.orchestra.run.trial035.stdout: "rbd_qos_write_iops_burst": "0", 2026-02-24T07:43:05.472 INFO:teuthology.orchestra.run.trial035.stdout: "rbd_qos_write_iops_burst_seconds": "1", 2026-02-24T07:43:05.472 INFO:teuthology.orchestra.run.trial035.stdout: "rbd_qos_write_iops_limit": "0", 2026-02-24T07:43:05.472 INFO:teuthology.orchestra.run.trial035.stdout: "rbd_quiesce_notification_attempts": "10", 2026-02-24T07:43:05.472 INFO:teuthology.orchestra.run.trial035.stdout: "rbd_read_from_replica_policy": "default", 2026-02-24T07:43:05.472 INFO:teuthology.orchestra.run.trial035.stdout: "rbd_readahead_disable_after_bytes": "52428800", 2026-02-24T07:43:05.472 INFO:teuthology.orchestra.run.trial035.stdout: "rbd_readahead_max_bytes": "524288", 2026-02-24T07:43:05.472 INFO:teuthology.orchestra.run.trial035.stdout: "rbd_readahead_trigger_requests": "10", 2026-02-24T07:43:05.472 INFO:teuthology.orchestra.run.trial035.stdout: "rbd_request_timed_out_seconds": "30", 2026-02-24T07:43:05.472 INFO:teuthology.orchestra.run.trial035.stdout: "rbd_skip_partial_discard": "true", 2026-02-24T07:43:05.473 INFO:teuthology.orchestra.run.trial035.stdout: "rbd_sparse_read_threshold_bytes": "65536", 2026-02-24T07:43:05.473 INFO:teuthology.orchestra.run.trial035.stdout: "rbd_tracing": "false", 2026-02-24T07:43:05.473 INFO:teuthology.orchestra.run.trial035.stdout: "rbd_validate_names": "true", 2026-02-24T07:43:05.473 INFO:teuthology.orchestra.run.trial035.stdout: "rbd_validate_pool": "true", 2026-02-24T07:43:05.473 INFO:teuthology.orchestra.run.trial035.stdout: "restapi_base_url": "", 2026-02-24T07:43:05.473 INFO:teuthology.orchestra.run.trial035.stdout: "restapi_log_level": "", 2026-02-24T07:43:05.473 INFO:teuthology.orchestra.run.trial035.stdout: "rgw_account_default_quota_max_objects": "-1", 2026-02-24T07:43:05.473 INFO:teuthology.orchestra.run.trial035.stdout: "rgw_account_default_quota_max_size": "-1", 2026-02-24T07:43:05.473 INFO:teuthology.orchestra.run.trial035.stdout: "rgw_acl_grants_max_num": "100", 2026-02-24T07:43:05.473 INFO:teuthology.orchestra.run.trial035.stdout: "rgw_admin_entry": "admin", 2026-02-24T07:43:05.473 INFO:teuthology.orchestra.run.trial035.stdout: "rgw_allow_notification_secrets_in_cleartext": "false", 2026-02-24T07:43:05.473 INFO:teuthology.orchestra.run.trial035.stdout: "rgw_asio_assert_yielding": "false", 2026-02-24T07:43:05.473 INFO:teuthology.orchestra.run.trial035.stdout: "rgw_backend_store": "rados", 2026-02-24T07:43:05.473 INFO:teuthology.orchestra.run.trial035.stdout: "rgw_barbican_url": "", 2026-02-24T07:43:05.473 INFO:teuthology.orchestra.run.trial035.stdout: "rgw_beast_enable_async": "true", 2026-02-24T07:43:05.473 INFO:teuthology.orchestra.run.trial035.stdout: "rgw_bucket_counters_cache": "false", 2026-02-24T07:43:05.473 INFO:teuthology.orchestra.run.trial035.stdout: "rgw_bucket_counters_cache_size": "10000", 2026-02-24T07:43:05.473 INFO:teuthology.orchestra.run.trial035.stdout: "rgw_bucket_default_quota_max_objects": "-1", 2026-02-24T07:43:05.473 INFO:teuthology.orchestra.run.trial035.stdout: "rgw_bucket_default_quota_max_size": "-1", 2026-02-24T07:43:05.473 INFO:teuthology.orchestra.run.trial035.stdout: "rgw_bucket_eexist_override": "false", 2026-02-24T07:43:05.473 INFO:teuthology.orchestra.run.trial035.stdout: "rgw_bucket_index_max_aio": "128", 2026-02-24T07:43:05.473 INFO:teuthology.orchestra.run.trial035.stdout: "rgw_bucket_index_transaction_instrumentation": "false", 2026-02-24T07:43:05.473 INFO:teuthology.orchestra.run.trial035.stdout: "rgw_bucket_logging_obj_roll_time": "300", 2026-02-24T07:43:05.473 INFO:teuthology.orchestra.run.trial035.stdout: "rgw_bucket_persistent_notif_num_shards": "11", 2026-02-24T07:43:05.473 INFO:teuthology.orchestra.run.trial035.stdout: "rgw_bucket_quota_cache_size": "10000", 2026-02-24T07:43:05.473 INFO:teuthology.orchestra.run.trial035.stdout: "rgw_bucket_quota_ttl": "600", 2026-02-24T07:43:05.473 INFO:teuthology.orchestra.run.trial035.stdout: "rgw_bucket_sync_spawn_window": "20", 2026-02-24T07:43:05.473 INFO:teuthology.orchestra.run.trial035.stdout: "rgw_cache_enabled": "true", 2026-02-24T07:43:05.473 INFO:teuthology.orchestra.run.trial035.stdout: "rgw_cache_expiry_interval": "900", 2026-02-24T07:43:05.473 INFO:teuthology.orchestra.run.trial035.stdout: "rgw_cache_lru_size": "25000", 2026-02-24T07:43:05.473 INFO:teuthology.orchestra.run.trial035.stdout: "rgw_config_store": "rados", 2026-02-24T07:43:05.473 INFO:teuthology.orchestra.run.trial035.stdout: "rgw_content_length_compat": "false", 2026-02-24T07:43:05.473 INFO:teuthology.orchestra.run.trial035.stdout: "rgw_copy_obj_progress": "true", 2026-02-24T07:43:05.473 INFO:teuthology.orchestra.run.trial035.stdout: "rgw_copy_obj_progress_every_bytes": "1048576", 2026-02-24T07:43:05.474 INFO:teuthology.orchestra.run.trial035.stdout: "rgw_cors_rules_max_num": "100", 2026-02-24T07:43:05.474 INFO:teuthology.orchestra.run.trial035.stdout: "rgw_cross_domain_policy": "", 2026-02-24T07:43:05.474 INFO:teuthology.orchestra.run.trial035.stdout: "rgw_crypt_default_encryption_key": "", 2026-02-24T07:43:05.474 INFO:teuthology.orchestra.run.trial035.stdout: "rgw_crypt_kmip_addr": "", 2026-02-24T07:43:05.474 INFO:teuthology.orchestra.run.trial035.stdout: "rgw_crypt_kmip_ca_path": "", 2026-02-24T07:43:05.474 INFO:teuthology.orchestra.run.trial035.stdout: "rgw_crypt_kmip_client_cert": "", 2026-02-24T07:43:05.474 INFO:teuthology.orchestra.run.trial035.stdout: "rgw_crypt_kmip_client_key": "", 2026-02-24T07:43:05.474 INFO:teuthology.orchestra.run.trial035.stdout: "rgw_crypt_kmip_kms_key_template": "", 2026-02-24T07:43:05.474 INFO:teuthology.orchestra.run.trial035.stdout: "rgw_crypt_kmip_password": "", 2026-02-24T07:43:05.474 INFO:teuthology.orchestra.run.trial035.stdout: "rgw_crypt_kmip_s3_key_template": "$keyid", 2026-02-24T07:43:05.474 INFO:teuthology.orchestra.run.trial035.stdout: "rgw_crypt_kmip_username": "", 2026-02-24T07:43:05.474 INFO:teuthology.orchestra.run.trial035.stdout: "rgw_crypt_require_ssl": "true", 2026-02-24T07:43:05.474 INFO:teuthology.orchestra.run.trial035.stdout: "rgw_crypt_s3_kms_backend": "barbican", 2026-02-24T07:43:05.474 INFO:teuthology.orchestra.run.trial035.stdout: "rgw_crypt_s3_kms_encryption_keys": "", 2026-02-24T07:43:05.474 INFO:teuthology.orchestra.run.trial035.stdout: "rgw_crypt_sse_s3_backend": "vault", 2026-02-24T07:43:05.474 INFO:teuthology.orchestra.run.trial035.stdout: "rgw_crypt_sse_s3_key_template": "%bucket_id", 2026-02-24T07:43:05.474 INFO:teuthology.orchestra.run.trial035.stdout: "rgw_crypt_sse_s3_vault_addr": "", 2026-02-24T07:43:05.474 INFO:teuthology.orchestra.run.trial035.stdout: "rgw_crypt_sse_s3_vault_auth": "token", 2026-02-24T07:43:05.474 INFO:teuthology.orchestra.run.trial035.stdout: "rgw_crypt_sse_s3_vault_namespace": "", 2026-02-24T07:43:05.474 INFO:teuthology.orchestra.run.trial035.stdout: "rgw_crypt_sse_s3_vault_prefix": "", 2026-02-24T07:43:05.474 INFO:teuthology.orchestra.run.trial035.stdout: "rgw_crypt_sse_s3_vault_secret_engine": "transit", 2026-02-24T07:43:05.474 INFO:teuthology.orchestra.run.trial035.stdout: "rgw_crypt_sse_s3_vault_ssl_cacert": "", 2026-02-24T07:43:05.474 INFO:teuthology.orchestra.run.trial035.stdout: "rgw_crypt_sse_s3_vault_ssl_clientcert": "", 2026-02-24T07:43:05.474 INFO:teuthology.orchestra.run.trial035.stdout: "rgw_crypt_sse_s3_vault_ssl_clientkey": "", 2026-02-24T07:43:05.474 INFO:teuthology.orchestra.run.trial035.stdout: "rgw_crypt_sse_s3_vault_token_file": "", 2026-02-24T07:43:05.474 INFO:teuthology.orchestra.run.trial035.stdout: "rgw_crypt_sse_s3_vault_verify_ssl": "true", 2026-02-24T07:43:05.474 INFO:teuthology.orchestra.run.trial035.stdout: "rgw_crypt_suppress_logs": "true", 2026-02-24T07:43:05.474 INFO:teuthology.orchestra.run.trial035.stdout: "rgw_crypt_vault_addr": "", 2026-02-24T07:43:05.474 INFO:teuthology.orchestra.run.trial035.stdout: "rgw_crypt_vault_auth": "token", 2026-02-24T07:43:05.474 INFO:teuthology.orchestra.run.trial035.stdout: "rgw_crypt_vault_namespace": "", 2026-02-24T07:43:05.474 INFO:teuthology.orchestra.run.trial035.stdout: "rgw_crypt_vault_prefix": "", 2026-02-24T07:43:05.474 INFO:teuthology.orchestra.run.trial035.stdout: "rgw_crypt_vault_secret_engine": "transit", 2026-02-24T07:43:05.474 INFO:teuthology.orchestra.run.trial035.stdout: "rgw_crypt_vault_ssl_cacert": "", 2026-02-24T07:43:05.474 INFO:teuthology.orchestra.run.trial035.stdout: "rgw_crypt_vault_ssl_clientcert": "", 2026-02-24T07:43:05.474 INFO:teuthology.orchestra.run.trial035.stdout: "rgw_crypt_vault_ssl_clientkey": "", 2026-02-24T07:43:05.474 INFO:teuthology.orchestra.run.trial035.stdout: "rgw_crypt_vault_token_file": "", 2026-02-24T07:43:05.474 INFO:teuthology.orchestra.run.trial035.stdout: "rgw_crypt_vault_verify_ssl": "true", 2026-02-24T07:43:05.474 INFO:teuthology.orchestra.run.trial035.stdout: "rgw_curl_buffersize": "524288", 2026-02-24T07:43:05.474 INFO:teuthology.orchestra.run.trial035.stdout: "rgw_curl_low_speed_limit": "1024", 2026-02-24T07:43:05.474 INFO:teuthology.orchestra.run.trial035.stdout: "rgw_curl_low_speed_time": "300", 2026-02-24T07:43:05.475 INFO:teuthology.orchestra.run.trial035.stdout: "rgw_curl_tcp_keepalive": "0", 2026-02-24T07:43:05.475 INFO:teuthology.orchestra.run.trial035.stdout: "rgw_curl_wait_timeout_ms": "1000", 2026-02-24T07:43:05.475 INFO:teuthology.orchestra.run.trial035.stdout: "rgw_d3n_l1_datacache_persistent_path": "/tmp/rgw_datacache/", 2026-02-24T07:43:05.475 INFO:teuthology.orchestra.run.trial035.stdout: "rgw_d3n_l1_datacache_size": "1073741824", 2026-02-24T07:43:05.475 INFO:teuthology.orchestra.run.trial035.stdout: "rgw_d3n_l1_evict_cache_on_start": "true", 2026-02-24T07:43:05.475 INFO:teuthology.orchestra.run.trial035.stdout: "rgw_d3n_l1_eviction_policy": "lru", 2026-02-24T07:43:05.475 INFO:teuthology.orchestra.run.trial035.stdout: "rgw_d3n_l1_fadvise": "4", 2026-02-24T07:43:05.475 INFO:teuthology.orchestra.run.trial035.stdout: "rgw_d3n_l1_local_datacache_enabled": "false", 2026-02-24T07:43:05.475 INFO:teuthology.orchestra.run.trial035.stdout: "rgw_d3n_libaio_aio_num": "64", 2026-02-24T07:43:05.475 INFO:teuthology.orchestra.run.trial035.stdout: "rgw_d3n_libaio_aio_threads": "20", 2026-02-24T07:43:05.475 INFO:teuthology.orchestra.run.trial035.stdout: "rgw_d4n_address": "127.0.0.1:6379", 2026-02-24T07:43:05.475 INFO:teuthology.orchestra.run.trial035.stdout: "rgw_d4n_backend_address": "127.0.0.1:6379", 2026-02-24T07:43:05.475 INFO:teuthology.orchestra.run.trial035.stdout: "rgw_d4n_cache_cleaning_interval": "1000", 2026-02-24T07:43:05.475 INFO:teuthology.orchestra.run.trial035.stdout: "rgw_d4n_l1_datacache_address": "127.0.0.1:6379", 2026-02-24T07:43:05.475 INFO:teuthology.orchestra.run.trial035.stdout: "rgw_d4n_l1_datacache_disk_reserve": "1073741824", 2026-02-24T07:43:05.475 INFO:teuthology.orchestra.run.trial035.stdout: "rgw_d4n_l1_datacache_persistent_path": "/tmp/rgw_d4n_datacache/", 2026-02-24T07:43:05.475 INFO:teuthology.orchestra.run.trial035.stdout: "rgw_d4n_l1_evict_cache_on_start": "true", 2026-02-24T07:43:05.475 INFO:teuthology.orchestra.run.trial035.stdout: "rgw_d4n_l1_fadvise": "4", 2026-02-24T07:43:05.475 INFO:teuthology.orchestra.run.trial035.stdout: "rgw_d4n_l1_write_open_flags": "4096", 2026-02-24T07:43:05.475 INFO:teuthology.orchestra.run.trial035.stdout: "rgw_d4n_libaio_aio_num": "64", 2026-02-24T07:43:05.475 INFO:teuthology.orchestra.run.trial035.stdout: "rgw_d4n_libaio_aio_threads": "20", 2026-02-24T07:43:05.475 INFO:teuthology.orchestra.run.trial035.stdout: "rgw_d4n_local_rgw_address": "127.0.0.1:8000", 2026-02-24T07:43:05.475 INFO:teuthology.orchestra.run.trial035.stdout: "rgw_data": "/var/lib/ceph/radosgw/ceph-a", 2026-02-24T07:43:05.475 INFO:teuthology.orchestra.run.trial035.stdout: "rgw_data_log_changes_size": "1000", 2026-02-24T07:43:05.475 INFO:teuthology.orchestra.run.trial035.stdout: "rgw_data_log_num_shards": "128", 2026-02-24T07:43:05.475 INFO:teuthology.orchestra.run.trial035.stdout: "rgw_data_log_window": "30", 2026-02-24T07:43:05.475 INFO:teuthology.orchestra.run.trial035.stdout: "rgw_data_notify_interval_msec": "0", 2026-02-24T07:43:05.475 INFO:teuthology.orchestra.run.trial035.stdout: "rgw_data_sync_poll_interval": "20", 2026-02-24T07:43:05.475 INFO:teuthology.orchestra.run.trial035.stdout: "rgw_data_sync_spawn_window": "20", 2026-02-24T07:43:05.475 INFO:teuthology.orchestra.run.trial035.stdout: "rgw_debug_inject_latency_bi_unlink": "0", 2026-02-24T07:43:05.475 INFO:teuthology.orchestra.run.trial035.stdout: "rgw_debug_inject_olh_cancel_modification_err": "false", 2026-02-24T07:43:05.475 INFO:teuthology.orchestra.run.trial035.stdout: "rgw_debug_inject_set_olh_err": "0", 2026-02-24T07:43:05.475 INFO:teuthology.orchestra.run.trial035.stdout: "rgw_default_data_log_backing": "fifo", 2026-02-24T07:43:05.475 INFO:teuthology.orchestra.run.trial035.stdout: "rgw_default_realm_info_oid": "default.realm", 2026-02-24T07:43:05.475 INFO:teuthology.orchestra.run.trial035.stdout: "rgw_default_region_info_oid": "default.region", 2026-02-24T07:43:05.475 INFO:teuthology.orchestra.run.trial035.stdout: "rgw_default_zone_info_oid": "default.zone", 2026-02-24T07:43:05.475 INFO:teuthology.orchestra.run.trial035.stdout: "rgw_default_zonegroup_info_oid": "default.zonegroup", 2026-02-24T07:43:05.475 INFO:teuthology.orchestra.run.trial035.stdout: "rgw_defer_to_bucket_acls": "", 2026-02-24T07:43:05.475 INFO:teuthology.orchestra.run.trial035.stdout: "rgw_delete_multi_obj_max_num": "1000", 2026-02-24T07:43:05.475 INFO:teuthology.orchestra.run.trial035.stdout: "rgw_disable_s3select": "false", 2026-02-24T07:43:05.475 INFO:teuthology.orchestra.run.trial035.stdout: "rgw_dmclock_admin_lim": "0.000000", 2026-02-24T07:43:05.476 INFO:teuthology.orchestra.run.trial035.stdout: "rgw_dmclock_admin_res": "100.000000", 2026-02-24T07:43:05.476 INFO:teuthology.orchestra.run.trial035.stdout: "rgw_dmclock_admin_wgt": "100.000000", 2026-02-24T07:43:05.476 INFO:teuthology.orchestra.run.trial035.stdout: "rgw_dmclock_auth_lim": "0.000000", 2026-02-24T07:43:05.476 INFO:teuthology.orchestra.run.trial035.stdout: "rgw_dmclock_auth_res": "200.000000", 2026-02-24T07:43:05.476 INFO:teuthology.orchestra.run.trial035.stdout: "rgw_dmclock_auth_wgt": "100.000000", 2026-02-24T07:43:05.476 INFO:teuthology.orchestra.run.trial035.stdout: "rgw_dmclock_data_lim": "0.000000", 2026-02-24T07:43:05.476 INFO:teuthology.orchestra.run.trial035.stdout: "rgw_dmclock_data_res": "500.000000", 2026-02-24T07:43:05.476 INFO:teuthology.orchestra.run.trial035.stdout: "rgw_dmclock_data_wgt": "500.000000", 2026-02-24T07:43:05.476 INFO:teuthology.orchestra.run.trial035.stdout: "rgw_dmclock_metadata_lim": "0.000000", 2026-02-24T07:43:05.476 INFO:teuthology.orchestra.run.trial035.stdout: "rgw_dmclock_metadata_res": "500.000000", 2026-02-24T07:43:05.476 INFO:teuthology.orchestra.run.trial035.stdout: "rgw_dmclock_metadata_wgt": "500.000000", 2026-02-24T07:43:05.476 INFO:teuthology.orchestra.run.trial035.stdout: "rgw_dns_name": "", 2026-02-24T07:43:05.476 INFO:teuthology.orchestra.run.trial035.stdout: "rgw_dns_s3website_name": "", 2026-02-24T07:43:05.476 INFO:teuthology.orchestra.run.trial035.stdout: "rgw_dynamic_resharding": "true", 2026-02-24T07:43:05.476 INFO:teuthology.orchestra.run.trial035.stdout: "rgw_dynamic_resharding_may_reduce": "true", 2026-02-24T07:43:05.476 INFO:teuthology.orchestra.run.trial035.stdout: "rgw_dynamic_resharding_reduction_wait": "120", 2026-02-24T07:43:05.476 INFO:teuthology.orchestra.run.trial035.stdout: "rgw_enable_apis": "s3, s3website, swift, swift_auth, admin, sts, iam, notifications", 2026-02-24T07:43:05.476 INFO:teuthology.orchestra.run.trial035.stdout: "rgw_enable_gc_threads": "true", 2026-02-24T07:43:05.476 INFO:teuthology.orchestra.run.trial035.stdout: "rgw_enable_jwks_url_verification": "false", 2026-02-24T07:43:05.476 INFO:teuthology.orchestra.run.trial035.stdout: "rgw_enable_lc_threads": "true", 2026-02-24T07:43:05.476 INFO:teuthology.orchestra.run.trial035.stdout: "rgw_enable_mdsearch": "true", 2026-02-24T07:43:05.476 INFO:teuthology.orchestra.run.trial035.stdout: "rgw_enable_ops_log": "false", 2026-02-24T07:43:05.476 INFO:teuthology.orchestra.run.trial035.stdout: "rgw_enable_quota_threads": "true", 2026-02-24T07:43:05.476 INFO:teuthology.orchestra.run.trial035.stdout: "rgw_enable_restore_threads": "true", 2026-02-24T07:43:05.476 INFO:teuthology.orchestra.run.trial035.stdout: "rgw_enable_static_website": "false", 2026-02-24T07:43:05.476 INFO:teuthology.orchestra.run.trial035.stdout: "rgw_enable_usage_log": "false", 2026-02-24T07:43:05.476 INFO:teuthology.orchestra.run.trial035.stdout: "rgw_enforce_swift_acls": "true", 2026-02-24T07:43:05.476 INFO:teuthology.orchestra.run.trial035.stdout: "rgw_exit_timeout_secs": "120", 2026-02-24T07:43:05.476 INFO:teuthology.orchestra.run.trial035.stdout: "rgw_expose_bucket": "false", 2026-02-24T07:43:05.476 INFO:teuthology.orchestra.run.trial035.stdout: "rgw_extended_http_attrs": "", 2026-02-24T07:43:05.476 INFO:teuthology.orchestra.run.trial035.stdout: "rgw_filter": "none", 2026-02-24T07:43:05.476 INFO:teuthology.orchestra.run.trial035.stdout: "rgw_frontend_defaults": "beast ssl_certificate=config://rgw/cert/$realm/$zone.crt ssl_private_key=config://rgw/cert/$realm/$zone.key", 2026-02-24T07:43:05.476 INFO:teuthology.orchestra.run.trial035.stdout: "rgw_frontends": "beast port=7480", 2026-02-24T07:43:05.476 INFO:teuthology.orchestra.run.trial035.stdout: "rgw_gc_max_concurrent_io": "10", 2026-02-24T07:43:05.476 INFO:teuthology.orchestra.run.trial035.stdout: "rgw_gc_max_deferred": "50", 2026-02-24T07:43:05.476 INFO:teuthology.orchestra.run.trial035.stdout: "rgw_gc_max_deferred_entries_size": "3072", 2026-02-24T07:43:05.476 INFO:teuthology.orchestra.run.trial035.stdout: "rgw_gc_max_objs": "32", 2026-02-24T07:43:05.476 INFO:teuthology.orchestra.run.trial035.stdout: "rgw_gc_max_queue_size": "134213632", 2026-02-24T07:43:05.476 INFO:teuthology.orchestra.run.trial035.stdout: "rgw_gc_max_trim_chunk": "16", 2026-02-24T07:43:05.476 INFO:teuthology.orchestra.run.trial035.stdout: "rgw_gc_obj_min_wait": "7200", 2026-02-24T07:43:05.476 INFO:teuthology.orchestra.run.trial035.stdout: "rgw_gc_processor_max_time": "3600", 2026-02-24T07:43:05.476 INFO:teuthology.orchestra.run.trial035.stdout: "rgw_gc_processor_period": "3600", 2026-02-24T07:43:05.476 INFO:teuthology.orchestra.run.trial035.stdout: "rgw_get_obj_max_req_size": "4194304", 2026-02-24T07:43:05.476 INFO:teuthology.orchestra.run.trial035.stdout: "rgw_get_obj_window_size": "16777216", 2026-02-24T07:43:05.477 INFO:teuthology.orchestra.run.trial035.stdout: "rgw_graceful_stop": "false", 2026-02-24T07:43:05.477 INFO:teuthology.orchestra.run.trial035.stdout: "rgw_healthcheck_disabling_path": "", 2026-02-24T07:43:05.477 INFO:teuthology.orchestra.run.trial035.stdout: "rgw_http_notif_connection_timeout": "5", 2026-02-24T07:43:05.477 INFO:teuthology.orchestra.run.trial035.stdout: "rgw_http_notif_max_inflight": "8192", 2026-02-24T07:43:05.477 INFO:teuthology.orchestra.run.trial035.stdout: "rgw_http_notif_message_timeout": "10", 2026-02-24T07:43:05.477 INFO:teuthology.orchestra.run.trial035.stdout: "rgw_ignore_get_invalid_range": "false", 2026-02-24T07:43:05.477 INFO:teuthology.orchestra.run.trial035.stdout: "rgw_init_timeout": "300", 2026-02-24T07:43:05.477 INFO:teuthology.orchestra.run.trial035.stdout: "rgw_inject_notify_timeout_probability": "0.000000", 2026-02-24T07:43:05.477 INFO:teuthology.orchestra.run.trial035.stdout: "rgw_json_config": "/var/lib/ceph/radosgw/config.json", 2026-02-24T07:43:05.477 INFO:teuthology.orchestra.run.trial035.stdout: "rgw_kafka_connection_idle": "300", 2026-02-24T07:43:05.477 INFO:teuthology.orchestra.run.trial035.stdout: "rgw_kafka_message_timeout": "5000", 2026-02-24T07:43:05.477 INFO:teuthology.orchestra.run.trial035.stdout: "rgw_kafka_sleep_timeout": "10", 2026-02-24T07:43:05.477 INFO:teuthology.orchestra.run.trial035.stdout: "rgw_keystone_accepted_admin_roles": "", 2026-02-24T07:43:05.477 INFO:teuthology.orchestra.run.trial035.stdout: "rgw_keystone_accepted_reader_roles": "", 2026-02-24T07:43:05.477 INFO:teuthology.orchestra.run.trial035.stdout: "rgw_keystone_accepted_roles": "Member, admin", 2026-02-24T07:43:05.477 INFO:teuthology.orchestra.run.trial035.stdout: "rgw_keystone_admin_domain": "", 2026-02-24T07:43:05.477 INFO:teuthology.orchestra.run.trial035.stdout: "rgw_keystone_admin_password": "", 2026-02-24T07:43:05.477 INFO:teuthology.orchestra.run.trial035.stdout: "rgw_keystone_admin_password_path": "", 2026-02-24T07:43:05.477 INFO:teuthology.orchestra.run.trial035.stdout: "rgw_keystone_admin_project": "", 2026-02-24T07:43:05.477 INFO:teuthology.orchestra.run.trial035.stdout: "rgw_keystone_admin_tenant": "", 2026-02-24T07:43:05.477 INFO:teuthology.orchestra.run.trial035.stdout: "rgw_keystone_admin_user": "", 2026-02-24T07:43:05.477 INFO:teuthology.orchestra.run.trial035.stdout: "rgw_keystone_barbican_domain": "", 2026-02-24T07:43:05.477 INFO:teuthology.orchestra.run.trial035.stdout: "rgw_keystone_barbican_password": "", 2026-02-24T07:43:05.477 INFO:teuthology.orchestra.run.trial035.stdout: "rgw_keystone_barbican_project": "", 2026-02-24T07:43:05.477 INFO:teuthology.orchestra.run.trial035.stdout: "rgw_keystone_barbican_tenant": "", 2026-02-24T07:43:05.477 INFO:teuthology.orchestra.run.trial035.stdout: "rgw_keystone_barbican_user": "", 2026-02-24T07:43:05.477 INFO:teuthology.orchestra.run.trial035.stdout: "rgw_keystone_expired_token_cache_expiration": "3600", 2026-02-24T07:43:05.477 INFO:teuthology.orchestra.run.trial035.stdout: "rgw_keystone_implicit_tenants": "false", 2026-02-24T07:43:05.477 INFO:teuthology.orchestra.run.trial035.stdout: "rgw_keystone_service_token_accepted_roles": "admin", 2026-02-24T07:43:05.477 INFO:teuthology.orchestra.run.trial035.stdout: "rgw_keystone_service_token_enabled": "false", 2026-02-24T07:43:05.477 INFO:teuthology.orchestra.run.trial035.stdout: "rgw_keystone_token_cache_size": "10000", 2026-02-24T07:43:05.477 INFO:teuthology.orchestra.run.trial035.stdout: "rgw_keystone_token_cache_ttl": "300", 2026-02-24T07:43:05.477 INFO:teuthology.orchestra.run.trial035.stdout: "rgw_keystone_url": "", 2026-02-24T07:43:05.477 INFO:teuthology.orchestra.run.trial035.stdout: "rgw_keystone_verify_ssl": "true", 2026-02-24T07:43:05.477 INFO:teuthology.orchestra.run.trial035.stdout: "rgw_lc_debug_interval": "-1", 2026-02-24T07:43:05.477 INFO:teuthology.orchestra.run.trial035.stdout: "rgw_lc_list_cnt": "1000", 2026-02-24T07:43:05.477 INFO:teuthology.orchestra.run.trial035.stdout: "rgw_lc_lock_max_time": "90", 2026-02-24T07:43:05.477 INFO:teuthology.orchestra.run.trial035.stdout: "rgw_lc_max_objs": "32", 2026-02-24T07:43:05.477 INFO:teuthology.orchestra.run.trial035.stdout: "rgw_lc_max_rules": "1000", 2026-02-24T07:43:05.477 INFO:teuthology.orchestra.run.trial035.stdout: "rgw_lc_max_worker": "3", 2026-02-24T07:43:05.477 INFO:teuthology.orchestra.run.trial035.stdout: "rgw_lc_max_wp_worker": "128", 2026-02-24T07:43:05.477 INFO:teuthology.orchestra.run.trial035.stdout: "rgw_lc_ordered_list_threshold": "500", 2026-02-24T07:43:05.477 INFO:teuthology.orchestra.run.trial035.stdout: "rgw_lc_thread_delay": "0", 2026-02-24T07:43:05.478 INFO:teuthology.orchestra.run.trial035.stdout: "rgw_ldap_binddn": "uid=admin,cn=users,dc=example,dc=com", 2026-02-24T07:43:05.478 INFO:teuthology.orchestra.run.trial035.stdout: "rgw_ldap_dnattr": "uid", 2026-02-24T07:43:05.478 INFO:teuthology.orchestra.run.trial035.stdout: "rgw_ldap_searchdn": "cn=users,cn=accounts,dc=example,dc=com", 2026-02-24T07:43:05.478 INFO:teuthology.orchestra.run.trial035.stdout: "rgw_ldap_searchfilter": "", 2026-02-24T07:43:05.478 INFO:teuthology.orchestra.run.trial035.stdout: "rgw_ldap_secret": "/etc/openldap/secret", 2026-02-24T07:43:05.478 INFO:teuthology.orchestra.run.trial035.stdout: "rgw_ldap_uri": "", 2026-02-24T07:43:05.478 INFO:teuthology.orchestra.run.trial035.stdout: "rgw_lfuda_sync_frequency": "60", 2026-02-24T07:43:05.478 INFO:teuthology.orchestra.run.trial035.stdout: "rgw_lifecycle_work_time": "00:00-06:00", 2026-02-24T07:43:05.478 INFO:teuthology.orchestra.run.trial035.stdout: "rgw_list_bucket_min_readahead": "1000", 2026-02-24T07:43:05.478 INFO:teuthology.orchestra.run.trial035.stdout: "rgw_list_buckets_max_chunk": "1000", 2026-02-24T07:43:05.478 INFO:teuthology.orchestra.run.trial035.stdout: "rgw_log_http_headers": "", 2026-02-24T07:43:05.478 INFO:teuthology.orchestra.run.trial035.stdout: "rgw_log_nonexistent_bucket": "false", 2026-02-24T07:43:05.478 INFO:teuthology.orchestra.run.trial035.stdout: "rgw_log_object_name": "%Y-%m-%d-%H-%i-%n", 2026-02-24T07:43:05.478 INFO:teuthology.orchestra.run.trial035.stdout: "rgw_log_object_name_utc": "false", 2026-02-24T07:43:05.478 INFO:teuthology.orchestra.run.trial035.stdout: "rgw_lua_enable": "true", 2026-02-24T07:43:05.478 INFO:teuthology.orchestra.run.trial035.stdout: "rgw_lua_max_memory_per_state": "128000", 2026-02-24T07:43:05.478 INFO:teuthology.orchestra.run.trial035.stdout: "rgw_lua_max_runtime_per_state": "1000", 2026-02-24T07:43:05.478 INFO:teuthology.orchestra.run.trial035.stdout: "rgw_luarocks_location": "/tmp/rgw_luarocks/mon.a", 2026-02-24T07:43:05.478 INFO:teuthology.orchestra.run.trial035.stdout: "rgw_max_attr_name_len": "0", 2026-02-24T07:43:05.478 INFO:teuthology.orchestra.run.trial035.stdout: "rgw_max_attr_size": "0", 2026-02-24T07:43:05.478 INFO:teuthology.orchestra.run.trial035.stdout: "rgw_max_attrs_num_in_req": "0", 2026-02-24T07:43:05.478 INFO:teuthology.orchestra.run.trial035.stdout: "rgw_max_chunk_size": "4194304", 2026-02-24T07:43:05.478 INFO:teuthology.orchestra.run.trial035.stdout: "rgw_max_concurrent_requests": "1024", 2026-02-24T07:43:05.478 INFO:teuthology.orchestra.run.trial035.stdout: "rgw_max_control_aio": "8", 2026-02-24T07:43:05.478 INFO:teuthology.orchestra.run.trial035.stdout: "rgw_max_copy_obj_concurrent_io": "10", 2026-02-24T07:43:05.478 INFO:teuthology.orchestra.run.trial035.stdout: "rgw_max_dynamic_shards": "1999", 2026-02-24T07:43:05.478 INFO:teuthology.orchestra.run.trial035.stdout: "rgw_max_listing_results": "5000", 2026-02-24T07:43:05.478 INFO:teuthology.orchestra.run.trial035.stdout: "rgw_max_notify_retries": "10", 2026-02-24T07:43:05.478 INFO:teuthology.orchestra.run.trial035.stdout: "rgw_max_objs_per_shard": "100000", 2026-02-24T07:43:05.478 INFO:teuthology.orchestra.run.trial035.stdout: "rgw_max_put_param_size": "1048576", 2026-02-24T07:43:05.478 INFO:teuthology.orchestra.run.trial035.stdout: "rgw_max_put_size": "5368709120", 2026-02-24T07:43:05.478 INFO:teuthology.orchestra.run.trial035.stdout: "rgw_max_slo_entries": "1000", 2026-02-24T07:43:05.478 INFO:teuthology.orchestra.run.trial035.stdout: "rgw_md_log_max_shards": "64", 2026-02-24T07:43:05.478 INFO:teuthology.orchestra.run.trial035.stdout: "rgw_md_notify_interval_msec": "200", 2026-02-24T07:43:05.478 INFO:teuthology.orchestra.run.trial035.stdout: "rgw_meta_sync_poll_interval": "20", 2026-02-24T07:43:05.478 INFO:teuthology.orchestra.run.trial035.stdout: "rgw_meta_sync_spawn_window": "20", 2026-02-24T07:43:05.478 INFO:teuthology.orchestra.run.trial035.stdout: "rgw_mime_types_file": "/etc/mime.types", 2026-02-24T07:43:05.478 INFO:teuthology.orchestra.run.trial035.stdout: "rgw_mp_lock_max_time": "600", 2026-02-24T07:43:05.478 INFO:teuthology.orchestra.run.trial035.stdout: "rgw_multi_obj_del_max_aio": "16", 2026-02-24T07:43:05.478 INFO:teuthology.orchestra.run.trial035.stdout: "rgw_multipart_min_part_size": "5242880", 2026-02-24T07:43:05.478 INFO:teuthology.orchestra.run.trial035.stdout: "rgw_multipart_part_upload_limit": "10000", 2026-02-24T07:43:05.478 INFO:teuthology.orchestra.run.trial035.stdout: "rgw_nfs_fhcache_partitions": "3", 2026-02-24T07:43:05.478 INFO:teuthology.orchestra.run.trial035.stdout: "rgw_nfs_fhcache_size": "2017", 2026-02-24T07:43:05.478 INFO:teuthology.orchestra.run.trial035.stdout: "rgw_nfs_frontends": "rgw-nfs", 2026-02-24T07:43:05.479 INFO:teuthology.orchestra.run.trial035.stdout: "rgw_nfs_lru_lane_hiwat": "911", 2026-02-24T07:43:05.479 INFO:teuthology.orchestra.run.trial035.stdout: "rgw_nfs_lru_lanes": "5", 2026-02-24T07:43:05.479 INFO:teuthology.orchestra.run.trial035.stdout: "rgw_nfs_max_gc": "300", 2026-02-24T07:43:05.479 INFO:teuthology.orchestra.run.trial035.stdout: "rgw_nfs_namespace_expire_secs": "300", 2026-02-24T07:43:05.479 INFO:teuthology.orchestra.run.trial035.stdout: "rgw_nfs_run_gc_threads": "false", 2026-02-24T07:43:05.479 INFO:teuthology.orchestra.run.trial035.stdout: "rgw_nfs_run_lc_threads": "false", 2026-02-24T07:43:05.479 INFO:teuthology.orchestra.run.trial035.stdout: "rgw_nfs_run_quota_threads": "false", 2026-02-24T07:43:05.479 INFO:teuthology.orchestra.run.trial035.stdout: "rgw_nfs_run_restore_threads": "false", 2026-02-24T07:43:05.479 INFO:teuthology.orchestra.run.trial035.stdout: "rgw_nfs_run_sync_thread": "false", 2026-02-24T07:43:05.479 INFO:teuthology.orchestra.run.trial035.stdout: "rgw_nfs_s3_fast_attrs": "false", 2026-02-24T07:43:05.479 INFO:teuthology.orchestra.run.trial035.stdout: "rgw_nfs_write_completion_interval_s": "10", 2026-02-24T07:43:05.479 INFO:teuthology.orchestra.run.trial035.stdout: "rgw_num_async_rados_threads": "32", 2026-02-24T07:43:05.479 INFO:teuthology.orchestra.run.trial035.stdout: "rgw_num_control_oids": "8", 2026-02-24T07:43:05.479 INFO:teuthology.orchestra.run.trial035.stdout: "rgw_numa_node": "-1", 2026-02-24T07:43:05.479 INFO:teuthology.orchestra.run.trial035.stdout: "rgw_obj_stripe_size": "4194304", 2026-02-24T07:43:05.479 INFO:teuthology.orchestra.run.trial035.stdout: "rgw_obj_tombstone_cache_size": "1000", 2026-02-24T07:43:05.479 INFO:teuthology.orchestra.run.trial035.stdout: "rgw_objexp_chunk_size": "100", 2026-02-24T07:43:05.479 INFO:teuthology.orchestra.run.trial035.stdout: "rgw_objexp_gc_interval": "600", 2026-02-24T07:43:05.479 INFO:teuthology.orchestra.run.trial035.stdout: "rgw_objexp_hints_num_shards": "127", 2026-02-24T07:43:05.479 INFO:teuthology.orchestra.run.trial035.stdout: "rgw_olh_pending_timeout_sec": "3600", 2026-02-24T07:43:05.479 INFO:teuthology.orchestra.run.trial035.stdout: "rgw_op_thread_suicide_timeout": "0", 2026-02-24T07:43:05.479 INFO:teuthology.orchestra.run.trial035.stdout: "rgw_op_thread_timeout": "600", 2026-02-24T07:43:05.479 INFO:teuthology.orchestra.run.trial035.stdout: "rgw_op_tracing": "false", 2026-02-24T07:43:05.479 INFO:teuthology.orchestra.run.trial035.stdout: "rgw_opa_token": "", 2026-02-24T07:43:05.479 INFO:teuthology.orchestra.run.trial035.stdout: "rgw_opa_url": "", 2026-02-24T07:43:05.479 INFO:teuthology.orchestra.run.trial035.stdout: "rgw_opa_verify_ssl": "true", 2026-02-24T07:43:05.479 INFO:teuthology.orchestra.run.trial035.stdout: "rgw_ops_log_data_backlog": "5242880", 2026-02-24T07:43:05.479 INFO:teuthology.orchestra.run.trial035.stdout: "rgw_ops_log_file_path": "/var/log/ceph/ops-log-ceph-mon.a.log", 2026-02-24T07:43:05.479 INFO:teuthology.orchestra.run.trial035.stdout: "rgw_ops_log_rados": "false", 2026-02-24T07:43:05.479 INFO:teuthology.orchestra.run.trial035.stdout: "rgw_ops_log_socket_path": "", 2026-02-24T07:43:05.479 INFO:teuthology.orchestra.run.trial035.stdout: "rgw_override_bucket_index_max_shards": "0", 2026-02-24T07:43:05.479 INFO:teuthology.orchestra.run.trial035.stdout: "rgw_parquet_buffer_size": "16777216", 2026-02-24T07:43:05.479 INFO:teuthology.orchestra.run.trial035.stdout: "rgw_pending_bucket_index_op_expiration": "120", 2026-02-24T07:43:05.479 INFO:teuthology.orchestra.run.trial035.stdout: "rgw_period_latest_epoch_info_oid": ".latest_epoch", 2026-02-24T07:43:05.479 INFO:teuthology.orchestra.run.trial035.stdout: "rgw_period_push_interval": "2.000000", 2026-02-24T07:43:05.479 INFO:teuthology.orchestra.run.trial035.stdout: "rgw_period_push_interval_max": "30.000000", 2026-02-24T07:43:05.479 INFO:teuthology.orchestra.run.trial035.stdout: "rgw_period_root_pool": ".rgw.root", 2026-02-24T07:43:05.479 INFO:teuthology.orchestra.run.trial035.stdout: "rgw_policy_reject_invalid_principals": "true", 2026-02-24T07:43:05.479 INFO:teuthology.orchestra.run.trial035.stdout: "rgw_posix_base_path": "/tmp/rgw_posix_driver", 2026-02-24T07:43:05.479 INFO:teuthology.orchestra.run.trial035.stdout: "rgw_posix_cache_lanes": "3", 2026-02-24T07:43:05.479 INFO:teuthology.orchestra.run.trial035.stdout: "rgw_posix_cache_lmdb_count": "3", 2026-02-24T07:43:05.480 INFO:teuthology.orchestra.run.trial035.stdout: "rgw_posix_cache_max_buckets": "100", 2026-02-24T07:43:05.480 INFO:teuthology.orchestra.run.trial035.stdout: "rgw_posix_cache_partitions": "3", 2026-02-24T07:43:05.480 INFO:teuthology.orchestra.run.trial035.stdout: "rgw_posix_database_root": "/var/lib/ceph/radosgw", 2026-02-24T07:43:05.480 INFO:teuthology.orchestra.run.trial035.stdout: "rgw_posix_userdb_dir": "/var/lib/ceph/radosgw", 2026-02-24T07:43:05.480 INFO:teuthology.orchestra.run.trial035.stdout: "rgw_print_continue": "true", 2026-02-24T07:43:05.480 INFO:teuthology.orchestra.run.trial035.stdout: "rgw_print_prohibited_content_length": "false", 2026-02-24T07:43:05.480 INFO:teuthology.orchestra.run.trial035.stdout: "rgw_put_obj_max_window_size": "67108864", 2026-02-24T07:43:05.480 INFO:teuthology.orchestra.run.trial035.stdout: "rgw_put_obj_min_window_size": "16777216", 2026-02-24T07:43:05.480 INFO:teuthology.orchestra.run.trial035.stdout: "rgw_rados_pool_autoscale_bias": "4.000000", 2026-02-24T07:43:05.480 INFO:teuthology.orchestra.run.trial035.stdout: "rgw_rados_pool_recovery_priority": "5", 2026-02-24T07:43:05.480 INFO:teuthology.orchestra.run.trial035.stdout: "rgw_rados_tracing": "false", 2026-02-24T07:43:05.480 INFO:teuthology.orchestra.run.trial035.stdout: "rgw_ratelimit_interval": "60", 2026-02-24T07:43:05.480 INFO:teuthology.orchestra.run.trial035.stdout: "rgw_realm": "", 2026-02-24T07:43:05.480 INFO:teuthology.orchestra.run.trial035.stdout: "rgw_realm_id": "", 2026-02-24T07:43:05.480 INFO:teuthology.orchestra.run.trial035.stdout: "rgw_realm_root_pool": ".rgw.root", 2026-02-24T07:43:05.480 INFO:teuthology.orchestra.run.trial035.stdout: "rgw_redis_connection_pool_size": "512", 2026-02-24T07:43:05.480 INFO:teuthology.orchestra.run.trial035.stdout: "rgw_region": "", 2026-02-24T07:43:05.480 INFO:teuthology.orchestra.run.trial035.stdout: "rgw_region_root_pool": ".rgw.root", 2026-02-24T07:43:05.480 INFO:teuthology.orchestra.run.trial035.stdout: "rgw_relaxed_region_enforcement": "false", 2026-02-24T07:43:05.480 INFO:teuthology.orchestra.run.trial035.stdout: "rgw_relaxed_s3_bucket_names": "false", 2026-02-24T07:43:05.480 INFO:teuthology.orchestra.run.trial035.stdout: "rgw_remote_addr_param": "REMOTE_ADDR", 2026-02-24T07:43:05.480 INFO:teuthology.orchestra.run.trial035.stdout: "rgw_request_uri": "", 2026-02-24T07:43:05.480 INFO:teuthology.orchestra.run.trial035.stdout: "rgw_reshard_batch_size": "64", 2026-02-24T07:43:05.480 INFO:teuthology.orchestra.run.trial035.stdout: "rgw_reshard_bucket_lock_duration": "360", 2026-02-24T07:43:05.480 INFO:teuthology.orchestra.run.trial035.stdout: "rgw_reshard_debug_interval": "-1", 2026-02-24T07:43:05.480 INFO:teuthology.orchestra.run.trial035.stdout: "rgw_reshard_max_aio": "128", 2026-02-24T07:43:05.480 INFO:teuthology.orchestra.run.trial035.stdout: "rgw_reshard_num_logs": "16", 2026-02-24T07:43:05.480 INFO:teuthology.orchestra.run.trial035.stdout: "rgw_reshard_progress_judge_interval": "120", 2026-02-24T07:43:05.480 INFO:teuthology.orchestra.run.trial035.stdout: "rgw_reshard_progress_judge_ratio": "0.500000", 2026-02-24T07:43:05.480 INFO:teuthology.orchestra.run.trial035.stdout: "rgw_reshard_thread_interval": "600", 2026-02-24T07:43:05.480 INFO:teuthology.orchestra.run.trial035.stdout: "rgw_reshardlog_threshold": "30000", 2026-02-24T07:43:05.480 INFO:teuthology.orchestra.run.trial035.stdout: "rgw_resolve_cname": "false", 2026-02-24T07:43:05.480 INFO:teuthology.orchestra.run.trial035.stdout: "rgw_rest_getusage_op_compat": "false", 2026-02-24T07:43:05.480 INFO:teuthology.orchestra.run.trial035.stdout: "rgw_restore_debug_interval": "-1", 2026-02-24T07:43:05.480 INFO:teuthology.orchestra.run.trial035.stdout: "rgw_restore_lock_max_time": "90", 2026-02-24T07:43:05.480 INFO:teuthology.orchestra.run.trial035.stdout: "rgw_restore_max_objs": "32", 2026-02-24T07:43:05.480 INFO:teuthology.orchestra.run.trial035.stdout: "rgw_restore_processor_period": "900", 2026-02-24T07:43:05.480 INFO:teuthology.orchestra.run.trial035.stdout: "rgw_run_sync_thread": "true", 2026-02-24T07:43:05.480 INFO:teuthology.orchestra.run.trial035.stdout: "rgw_s3_auth_disable_signature_url": "false", 2026-02-24T07:43:05.480 INFO:teuthology.orchestra.run.trial035.stdout: "rgw_s3_auth_order": "sts, external, local", 2026-02-24T07:43:05.480 INFO:teuthology.orchestra.run.trial035.stdout: "rgw_s3_auth_use_keystone": "false", 2026-02-24T07:43:05.480 INFO:teuthology.orchestra.run.trial035.stdout: "rgw_s3_auth_use_ldap": "false", 2026-02-24T07:43:05.480 INFO:teuthology.orchestra.run.trial035.stdout: "rgw_s3_auth_use_rados": "true", 2026-02-24T07:43:05.481 INFO:teuthology.orchestra.run.trial035.stdout: "rgw_s3_auth_use_sts": "false", 2026-02-24T07:43:05.481 INFO:teuthology.orchestra.run.trial035.stdout: "rgw_s3_client_max_sig_ver": "-1", 2026-02-24T07:43:05.481 INFO:teuthology.orchestra.run.trial035.stdout: "rgw_s3_success_create_obj_status": "0", 2026-02-24T07:43:05.481 INFO:teuthology.orchestra.run.trial035.stdout: "rgw_safe_max_objects_per_shard": "102400", 2026-02-24T07:43:05.481 INFO:teuthology.orchestra.run.trial035.stdout: "rgw_scheduler_type": "throttler", 2026-02-24T07:43:05.481 INFO:teuthology.orchestra.run.trial035.stdout: "rgw_script_uri": "", 2026-02-24T07:43:05.481 INFO:teuthology.orchestra.run.trial035.stdout: "rgw_service_provider_name": "", 2026-02-24T07:43:05.481 INFO:teuthology.orchestra.run.trial035.stdout: "rgw_shard_warning_threshold": "90.000000", 2026-02-24T07:43:05.481 INFO:teuthology.orchestra.run.trial035.stdout: "rgw_sts_client_id": "", 2026-02-24T07:43:05.481 INFO:teuthology.orchestra.run.trial035.stdout: "rgw_sts_client_secret": "", 2026-02-24T07:43:05.481 INFO:teuthology.orchestra.run.trial035.stdout: "rgw_sts_entry": "sts", 2026-02-24T07:43:05.481 INFO:teuthology.orchestra.run.trial035.stdout: "rgw_sts_key": "", 2026-02-24T07:43:05.481 INFO:teuthology.orchestra.run.trial035.stdout: "rgw_sts_max_session_duration": "43200", 2026-02-24T07:43:05.481 INFO:teuthology.orchestra.run.trial035.stdout: "rgw_sts_min_session_duration": "900", 2026-02-24T07:43:05.481 INFO:teuthology.orchestra.run.trial035.stdout: "rgw_sts_token_introspection_url": "", 2026-02-24T07:43:05.481 INFO:teuthology.orchestra.run.trial035.stdout: "rgw_swift_account_in_url": "false", 2026-02-24T07:43:05.481 INFO:teuthology.orchestra.run.trial035.stdout: "rgw_swift_auth_entry": "auth", 2026-02-24T07:43:05.481 INFO:teuthology.orchestra.run.trial035.stdout: "rgw_swift_auth_url": "", 2026-02-24T07:43:05.481 INFO:teuthology.orchestra.run.trial035.stdout: "rgw_swift_custom_header": "", 2026-02-24T07:43:05.481 INFO:teuthology.orchestra.run.trial035.stdout: "rgw_swift_enforce_content_length": "false", 2026-02-24T07:43:05.481 INFO:teuthology.orchestra.run.trial035.stdout: "rgw_swift_need_stats": "true", 2026-02-24T07:43:05.481 INFO:teuthology.orchestra.run.trial035.stdout: "rgw_swift_tenant_name": "", 2026-02-24T07:43:05.481 INFO:teuthology.orchestra.run.trial035.stdout: "rgw_swift_token_expiration": "86400", 2026-02-24T07:43:05.481 INFO:teuthology.orchestra.run.trial035.stdout: "rgw_swift_url": "", 2026-02-24T07:43:05.481 INFO:teuthology.orchestra.run.trial035.stdout: "rgw_swift_url_prefix": "swift", 2026-02-24T07:43:05.481 INFO:teuthology.orchestra.run.trial035.stdout: "rgw_swift_versioning_enabled": "false", 2026-02-24T07:43:05.481 INFO:teuthology.orchestra.run.trial035.stdout: "rgw_sync_data_full_inject_err_probability": "0.000000", 2026-02-24T07:43:05.481 INFO:teuthology.orchestra.run.trial035.stdout: "rgw_sync_data_inject_err_probability": "0.000000", 2026-02-24T07:43:05.481 INFO:teuthology.orchestra.run.trial035.stdout: "rgw_sync_lease_period": "120", 2026-02-24T07:43:05.481 INFO:teuthology.orchestra.run.trial035.stdout: "rgw_sync_log_trim_concurrent_buckets": "4", 2026-02-24T07:43:05.481 INFO:teuthology.orchestra.run.trial035.stdout: "rgw_sync_log_trim_interval": "1200", 2026-02-24T07:43:05.481 INFO:teuthology.orchestra.run.trial035.stdout: "rgw_sync_log_trim_max_buckets": "16", 2026-02-24T07:43:05.481 INFO:teuthology.orchestra.run.trial035.stdout: "rgw_sync_log_trim_min_cold_buckets": "4", 2026-02-24T07:43:05.481 INFO:teuthology.orchestra.run.trial035.stdout: "rgw_sync_meta_inject_err_probability": "0.000000", 2026-02-24T07:43:05.481 INFO:teuthology.orchestra.run.trial035.stdout: "rgw_sync_obj_etag_verify": "false", 2026-02-24T07:43:05.481 INFO:teuthology.orchestra.run.trial035.stdout: "rgw_sync_trace_history_size": "4096", 2026-02-24T07:43:05.481 INFO:teuthology.orchestra.run.trial035.stdout: "rgw_sync_trace_per_node_log_size": "32", 2026-02-24T07:43:05.481 INFO:teuthology.orchestra.run.trial035.stdout: "rgw_sync_trace_servicemap_update_interval": "10", 2026-02-24T07:43:05.481 INFO:teuthology.orchestra.run.trial035.stdout: "rgw_thread_pool_size": "512", 2026-02-24T07:43:05.481 INFO:teuthology.orchestra.run.trial035.stdout: "rgw_topic_persistency_max_retries": "0", 2026-02-24T07:43:05.481 INFO:teuthology.orchestra.run.trial035.stdout: "rgw_topic_persistency_sleep_duration": "0", 2026-02-24T07:43:05.481 INFO:teuthology.orchestra.run.trial035.stdout: "rgw_topic_persistency_time_to_live": "0", 2026-02-24T07:43:05.481 INFO:teuthology.orchestra.run.trial035.stdout: "rgw_topic_require_publish_policy": "false", 2026-02-24T07:43:05.481 INFO:teuthology.orchestra.run.trial035.stdout: "rgw_torrent_comment": "", 2026-02-24T07:43:05.481 INFO:teuthology.orchestra.run.trial035.stdout: "rgw_torrent_createby": "", 2026-02-24T07:43:05.481 INFO:teuthology.orchestra.run.trial035.stdout: "rgw_torrent_encoding": "", 2026-02-24T07:43:05.482 INFO:teuthology.orchestra.run.trial035.stdout: "rgw_torrent_flag": "false", 2026-02-24T07:43:05.482 INFO:teuthology.orchestra.run.trial035.stdout: "rgw_torrent_max_size": "5368709120", 2026-02-24T07:43:05.482 INFO:teuthology.orchestra.run.trial035.stdout: "rgw_torrent_origin": "", 2026-02-24T07:43:05.482 INFO:teuthology.orchestra.run.trial035.stdout: "rgw_torrent_sha_unit": "524288", 2026-02-24T07:43:05.482 INFO:teuthology.orchestra.run.trial035.stdout: "rgw_torrent_tracker": "", 2026-02-24T07:43:05.482 INFO:teuthology.orchestra.run.trial035.stdout: "rgw_trust_forwarded_https": "false", 2026-02-24T07:43:05.482 INFO:teuthology.orchestra.run.trial035.stdout: "rgw_usage_log_flush_threshold": "1024", 2026-02-24T07:43:05.482 INFO:teuthology.orchestra.run.trial035.stdout: "rgw_usage_log_key_transition": "true", 2026-02-24T07:43:05.482 INFO:teuthology.orchestra.run.trial035.stdout: "rgw_usage_log_tick_interval": "30", 2026-02-24T07:43:05.482 INFO:teuthology.orchestra.run.trial035.stdout: "rgw_usage_max_shards": "32", 2026-02-24T07:43:05.482 INFO:teuthology.orchestra.run.trial035.stdout: "rgw_usage_max_user_shards": "1", 2026-02-24T07:43:05.482 INFO:teuthology.orchestra.run.trial035.stdout: "rgw_use_opa_authz": "false", 2026-02-24T07:43:05.482 INFO:teuthology.orchestra.run.trial035.stdout: "rgw_user_counters_cache": "false", 2026-02-24T07:43:05.482 INFO:teuthology.orchestra.run.trial035.stdout: "rgw_user_counters_cache_size": "10000", 2026-02-24T07:43:05.482 INFO:teuthology.orchestra.run.trial035.stdout: "rgw_user_default_quota_max_objects": "-1", 2026-02-24T07:43:05.482 INFO:teuthology.orchestra.run.trial035.stdout: "rgw_user_default_quota_max_size": "-1", 2026-02-24T07:43:05.482 INFO:teuthology.orchestra.run.trial035.stdout: "rgw_user_max_buckets": "1000", 2026-02-24T07:43:05.482 INFO:teuthology.orchestra.run.trial035.stdout: "rgw_user_policies_max_num": "100", 2026-02-24T07:43:05.482 INFO:teuthology.orchestra.run.trial035.stdout: "rgw_user_quota_bucket_sync_interval": "180", 2026-02-24T07:43:05.482 INFO:teuthology.orchestra.run.trial035.stdout: "rgw_user_quota_sync_idle_users": "false", 2026-02-24T07:43:05.482 INFO:teuthology.orchestra.run.trial035.stdout: "rgw_user_quota_sync_interval": "86400", 2026-02-24T07:43:05.482 INFO:teuthology.orchestra.run.trial035.stdout: "rgw_user_quota_sync_wait_time": "86400", 2026-02-24T07:43:05.482 INFO:teuthology.orchestra.run.trial035.stdout: "rgw_user_unique_email": "true", 2026-02-24T07:43:05.482 INFO:teuthology.orchestra.run.trial035.stdout: "rgw_verify_ssl": "true", 2026-02-24T07:43:05.482 INFO:teuthology.orchestra.run.trial035.stdout: "rgw_website_routing_rules_max_num": "50", 2026-02-24T07:43:05.482 INFO:teuthology.orchestra.run.trial035.stdout: "rgw_zone": "", 2026-02-24T07:43:05.482 INFO:teuthology.orchestra.run.trial035.stdout: "rgw_zone_id": "", 2026-02-24T07:43:05.482 INFO:teuthology.orchestra.run.trial035.stdout: "rgw_zone_root_pool": ".rgw.root", 2026-02-24T07:43:05.482 INFO:teuthology.orchestra.run.trial035.stdout: "rgw_zonegroup": "", 2026-02-24T07:43:05.482 INFO:teuthology.orchestra.run.trial035.stdout: "rgw_zonegroup_id": "", 2026-02-24T07:43:05.482 INFO:teuthology.orchestra.run.trial035.stdout: "rgw_zonegroup_root_pool": ".rgw.root", 2026-02-24T07:43:05.482 INFO:teuthology.orchestra.run.trial035.stdout: "rgwlc_auto_session_clear": "true", 2026-02-24T07:43:05.482 INFO:teuthology.orchestra.run.trial035.stdout: "rgwlc_skip_bucket_step": "false", 2026-02-24T07:43:05.482 INFO:teuthology.orchestra.run.trial035.stdout: "rocksdb_block_size": "4096", 2026-02-24T07:43:05.482 INFO:teuthology.orchestra.run.trial035.stdout: "rocksdb_bloom_bits_per_key": "20", 2026-02-24T07:43:05.482 INFO:teuthology.orchestra.run.trial035.stdout: "rocksdb_cache_index_and_filter_blocks": "true", 2026-02-24T07:43:05.482 INFO:teuthology.orchestra.run.trial035.stdout: "rocksdb_cache_index_and_filter_blocks_with_high_priority": "false", 2026-02-24T07:43:05.482 INFO:teuthology.orchestra.run.trial035.stdout: "rocksdb_cache_row_ratio": "0.000000", 2026-02-24T07:43:05.482 INFO:teuthology.orchestra.run.trial035.stdout: "rocksdb_cache_shard_bits": "4", 2026-02-24T07:43:05.482 INFO:teuthology.orchestra.run.trial035.stdout: "rocksdb_cache_size": "536870912", 2026-02-24T07:43:05.482 INFO:teuthology.orchestra.run.trial035.stdout: "rocksdb_cache_type": "binned_lru", 2026-02-24T07:43:05.482 INFO:teuthology.orchestra.run.trial035.stdout: "rocksdb_cf_compact_on_deletion": "true", 2026-02-24T07:43:05.482 INFO:teuthology.orchestra.run.trial035.stdout: "rocksdb_cf_compact_on_deletion_sliding_window": "32768", 2026-02-24T07:43:05.482 INFO:teuthology.orchestra.run.trial035.stdout: "rocksdb_cf_compact_on_deletion_trigger": "16384", 2026-02-24T07:43:05.482 INFO:teuthology.orchestra.run.trial035.stdout: "rocksdb_collect_compaction_stats": "false", 2026-02-24T07:43:05.482 INFO:teuthology.orchestra.run.trial035.stdout: "rocksdb_collect_extended_stats": "false", 2026-02-24T07:43:05.482 INFO:teuthology.orchestra.run.trial035.stdout: "rocksdb_collect_memory_stats": "false", 2026-02-24T07:43:05.482 INFO:teuthology.orchestra.run.trial035.stdout: "rocksdb_delete_range_threshold": "1048576", 2026-02-24T07:43:05.483 INFO:teuthology.orchestra.run.trial035.stdout: "rocksdb_index_type": "binary_search", 2026-02-24T07:43:05.483 INFO:teuthology.orchestra.run.trial035.stdout: "rocksdb_log_to_ceph_log": "true", 2026-02-24T07:43:05.483 INFO:teuthology.orchestra.run.trial035.stdout: "rocksdb_metadata_block_size": "4096", 2026-02-24T07:43:05.483 INFO:teuthology.orchestra.run.trial035.stdout: "rocksdb_partition_filters": "false", 2026-02-24T07:43:05.483 INFO:teuthology.orchestra.run.trial035.stdout: "rocksdb_perf": "false", 2026-02-24T07:43:05.483 INFO:teuthology.orchestra.run.trial035.stdout: "rocksdb_pin_l0_filter_and_index_blocks_in_cache": "false", 2026-02-24T07:43:05.483 INFO:teuthology.orchestra.run.trial035.stdout: "rotating_keys_bootstrap_timeout": "30", 2026-02-24T07:43:05.483 INFO:teuthology.orchestra.run.trial035.stdout: "rotating_keys_renewal_timeout": "10", 2026-02-24T07:43:05.483 INFO:teuthology.orchestra.run.trial035.stdout: "run_dir": "/var/run/ceph", 2026-02-24T07:43:05.483 INFO:teuthology.orchestra.run.trial035.stdout: "seastore_block_create": "true", 2026-02-24T07:43:05.483 INFO:teuthology.orchestra.run.trial035.stdout: "seastore_cachepin_2q_in_ratio": "0.500000", 2026-02-24T07:43:05.483 INFO:teuthology.orchestra.run.trial035.stdout: "seastore_cachepin_2q_out_ratio": "0.500000", 2026-02-24T07:43:05.483 INFO:teuthology.orchestra.run.trial035.stdout: "seastore_cachepin_size_pershard": "2147483648", 2026-02-24T07:43:05.483 INFO:teuthology.orchestra.run.trial035.stdout: "seastore_cachepin_type": "LRU", 2026-02-24T07:43:05.483 INFO:teuthology.orchestra.run.trial035.stdout: "seastore_cbjournal_size": "5368709120", 2026-02-24T07:43:05.483 INFO:teuthology.orchestra.run.trial035.stdout: "seastore_cold_tier_generations": "3", 2026-02-24T07:43:05.483 INFO:teuthology.orchestra.run.trial035.stdout: "seastore_data_delta_based_overwrite": "0", 2026-02-24T07:43:05.483 INFO:teuthology.orchestra.run.trial035.stdout: "seastore_default_max_object_size": "16777216", 2026-02-24T07:43:05.483 INFO:teuthology.orchestra.run.trial035.stdout: "seastore_default_object_metadata_reservation": "16777216", 2026-02-24T07:43:05.483 INFO:teuthology.orchestra.run.trial035.stdout: "seastore_device_size": "53687091200", 2026-02-24T07:43:05.483 INFO:teuthology.orchestra.run.trial035.stdout: "seastore_disable_end_to_end_data_protection": "true", 2026-02-24T07:43:05.483 INFO:teuthology.orchestra.run.trial035.stdout: "seastore_full_integrity_check": "false", 2026-02-24T07:43:05.483 INFO:teuthology.orchestra.run.trial035.stdout: "seastore_hot_tier_generations": "5", 2026-02-24T07:43:05.483 INFO:teuthology.orchestra.run.trial035.stdout: "seastore_journal_batch_capacity": "16", 2026-02-24T07:43:05.483 INFO:teuthology.orchestra.run.trial035.stdout: "seastore_journal_batch_flush_size": "16777216", 2026-02-24T07:43:05.483 INFO:teuthology.orchestra.run.trial035.stdout: "seastore_journal_batch_preferred_fullness": "0.950000", 2026-02-24T07:43:05.483 INFO:teuthology.orchestra.run.trial035.stdout: "seastore_journal_iodepth_limit": "5", 2026-02-24T07:43:05.483 INFO:teuthology.orchestra.run.trial035.stdout: "seastore_main_device_type": "SSD", 2026-02-24T07:43:05.483 INFO:teuthology.orchestra.run.trial035.stdout: "seastore_max_concurrent_transactions": "128", 2026-02-24T07:43:05.483 INFO:teuthology.orchestra.run.trial035.stdout: "seastore_max_data_allocation_size": "0", 2026-02-24T07:43:05.483 INFO:teuthology.orchestra.run.trial035.stdout: "seastore_multiple_tiers_default_evict_ratio": "0.600000", 2026-02-24T07:43:05.483 INFO:teuthology.orchestra.run.trial035.stdout: "seastore_multiple_tiers_fast_evict_ratio": "0.700000", 2026-02-24T07:43:05.483 INFO:teuthology.orchestra.run.trial035.stdout: "seastore_multiple_tiers_stop_evict_ratio": "0.500000", 2026-02-24T07:43:05.483 INFO:teuthology.orchestra.run.trial035.stdout: "seastore_segment_cleaner_gc_formula": "cost_benefit", 2026-02-24T07:43:05.483 INFO:teuthology.orchestra.run.trial035.stdout: "seastore_segment_size": "67108864", 2026-02-24T07:43:05.483 INFO:teuthology.orchestra.run.trial035.stdout: "service_unique_id": "", 2026-02-24T07:43:05.483 INFO:teuthology.orchestra.run.trial035.stdout: "set_keepcaps": "false", 2026-02-24T07:43:05.483 INFO:teuthology.orchestra.run.trial035.stdout: "setgroup": "ceph", 2026-02-24T07:43:05.483 INFO:teuthology.orchestra.run.trial035.stdout: "setuser": "ceph", 2026-02-24T07:43:05.483 INFO:teuthology.orchestra.run.trial035.stdout: "setuser_match_path": "", 2026-02-24T07:43:05.483 INFO:teuthology.orchestra.run.trial035.stdout: "subv_metrics_window_interval": "30", 2026-02-24T07:43:05.483 INFO:teuthology.orchestra.run.trial035.stdout: "target_max_misplaced_ratio": "0.050000", 2026-02-24T07:43:05.483 INFO:teuthology.orchestra.run.trial035.stdout: "thp": "false", 2026-02-24T07:43:05.483 INFO:teuthology.orchestra.run.trial035.stdout: "threadpool_default_timeout": "60", 2026-02-24T07:43:05.484 INFO:teuthology.orchestra.run.trial035.stdout: "threadpool_empty_queue_max_wait": "2", 2026-02-24T07:43:05.484 INFO:teuthology.orchestra.run.trial035.stdout: "throttler_perf_counter": "true", 2026-02-24T07:43:05.484 INFO:teuthology.orchestra.run.trial035.stdout: "tmp_dir": "/tmp", 2026-02-24T07:43:05.484 INFO:teuthology.orchestra.run.trial035.stdout: "tmp_file_template": "/tmp/ceph-mon.a.XXXXXX", 2026-02-24T07:43:05.484 INFO:teuthology.orchestra.run.trial035.stdout: "uadk_compressor_enabled": "false", 2026-02-24T07:43:05.484 INFO:teuthology.orchestra.run.trial035.stdout: "uadk_wd_sync_ctx_num": "2" 2026-02-24T07:43:05.484 INFO:teuthology.orchestra.run.trial035.stdout:} 2026-02-24T07:43:05.484 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph -- tell 'mon.*' injectargs --mon_osd_down_out_interval=0 2026-02-24T07:43:05.598 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:43:05.697 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:43:05.696+0000 7faf9c8c2640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:43:05.697 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:43:05.696+0000 7faf9c8c2640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:43:05.704 INFO:teuthology.orchestra.run.trial035.stdout:{} 2026-02-24T07:43:05.704 INFO:teuthology.orchestra.run.trial035.stderr:mon_osd_down_out_interval = '' (not observed, change may require restart) 2026-02-24T07:43:05.736 INFO:teuthology.run_tasks:Running task full_sequential... 2026-02-24T07:43:05.750 INFO:teuthology.task.full_sequential:In full_sequential, running task radosbench... 2026-02-24T07:43:05.753 INFO:tasks.radosbench:Beginning radosbench... 2026-02-24T07:43:05.753 INFO:tasks.cephadm.ceph_manager.ceph:creating pool_name unique_pool_0 2026-02-24T07:43:05.754 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph osd pool create unique_pool_0 16 2026-02-24T07:43:05.754 INFO:tasks.thrashosds.thrasher:starting do_thrash 2026-02-24T07:43:05.754 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2026-02-24T07:43:05.754 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2026-02-24T07:43:05.755 INFO:tasks.thrashosds.thrasher:starting do_optrack_toggle with a delay of 2.0 2026-02-24T07:43:05.755 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2026-02-24T07:43:05.870 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:43:05.871 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:43:05.894 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:43:05 trial035 ceph-mon[39284]: Reconfiguring prometheus.a deps ['8765', 'alertmanager_configured:False', 'ceph-exporter_configured:False', 'ingress_configured:False', 'mgmt-gateway_configured:False', 'mgr.x', 'nfs_configured:False', 'node-exporter_configured:False', 'nvmeof_configured:False', 'oauth2-proxy_configured:False', 'secure_monitoring_stack:False', 'smb_configured:False'] -> ['8765', 'alertmanager_configured:False', 'ceph-exporter_configured:False', 'ingress_configured:False', 'mgmt-gateway_configured:False', 'mgr.x', 'nfs_configured:False', 'node-exporter_configured:True', 'nvmeof_configured:False', 'oauth2-proxy_configured:False', 'secure_monitoring_stack:False', 'smb_configured:False'] (diff {'node-exporter_configured:False', 'node-exporter_configured:True'}) 2026-02-24T07:43:05.894 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:43:05 trial035 ceph-mon[39284]: Reconfiguring daemon prometheus.a on trial091 2026-02-24T07:43:05.894 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:43:05 trial035 ceph-mon[39284]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "orch get-security-config"}]: dispatch 2026-02-24T07:43:05.894 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:43:05 trial035 ceph-mon[39284]: from='client.14320 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2026-02-24T07:43:05.894 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:43:05 trial035 ceph-mon[39284]: from='client.14322 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2026-02-24T07:43:05.894 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:43:05 trial035 ceph-mon[39284]: from='mgr.14284 10.20.193.35:0/3681373222' entity='mgr.x' 2026-02-24T07:43:05.894 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:43:05 trial035 ceph-mon[39284]: from='mgr.14284 10.20.193.35:0/3681373222' entity='mgr.x' 2026-02-24T07:43:05.894 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:43:05 trial035 ceph-mon[39284]: from='mgr.14284 10.20.193.35:0/3681373222' entity='mgr.x' 2026-02-24T07:43:05.894 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:43:05 trial035 ceph-mon[39284]: from='mgr.14284 10.20.193.35:0/3681373222' entity='mgr.x' 2026-02-24T07:43:05.895 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:43:05 trial035 ceph-mon[39284]: from='mgr.14284 10.20.193.35:0/3681373222' entity='mgr.x' cmd={"prefix": "dashboard get-prometheus-api-host"} : dispatch 2026-02-24T07:43:05.895 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:43:05 trial035 ceph-mon[39284]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2026-02-24T07:43:05.895 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:43:05 trial035 ceph-mon[39284]: from='mgr.14284 10.20.193.35:0/3681373222' entity='mgr.x' cmd={"prefix": "dashboard set-prometheus-api-host", "value": "http://trial091:9095"} : dispatch 2026-02-24T07:43:05.895 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:43:05 trial035 ceph-mon[39284]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://trial091:9095"}]: dispatch 2026-02-24T07:43:05.895 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:43:05 trial035 ceph-mon[39284]: from='mgr.14284 10.20.193.35:0/3681373222' entity='mgr.x' 2026-02-24T07:43:05.895 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:43:05 trial035 ceph-mon[39284]: from='mgr.14284 10.20.193.35:0/3681373222' entity='mgr.x' 2026-02-24T07:43:05.895 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:43:05 trial035 ceph-mon[39284]: from='mgr.14284 10.20.193.35:0/3681373222' entity='mgr.x' cmd={"prefix": "orch get-security-config"} : dispatch 2026-02-24T07:43:05.895 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:43:05 trial035 ceph-mon[39284]: from='mgr.14284 10.20.193.35:0/3681373222' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2026-02-24T07:43:05.895 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:43:05 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/689268168' entity='client.admin' cmd={"prefix": "osd dump"} : dispatch 2026-02-24T07:43:05.895 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:43:05 trial035 ceph-mon[39284]: from='mgr.14284 10.20.193.35:0/3681373222' entity='mgr.x' 2026-02-24T07:43:05.895 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:43:05 trial035 ceph-mon[39284]: from='mgr.14284 10.20.193.35:0/3681373222' entity='mgr.x' 2026-02-24T07:43:05.895 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:43:05 trial035 ceph-mon[39284]: from='mgr.14284 10.20.193.35:0/3681373222' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-24T07:43:05.896 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:43:05 trial035 ceph-mon[39284]: from='mgr.14284 10.20.193.35:0/3681373222' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2026-02-24T07:43:05.896 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:43:05 trial035 ceph-mon[39284]: from='mgr.14284 10.20.193.35:0/3681373222' entity='mgr.x' 2026-02-24T07:43:05.896 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:43:05 trial035 ceph-mon[39284]: from='mgr.14284 10.20.193.35:0/3681373222' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2026-02-24T07:43:05.896 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:43:05 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/893000590' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2026-02-24T07:43:05.976 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:43:05.974+0000 7f115fe9f640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:43:05.976 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:43:05.975+0000 7f115fe9f640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:43:05.978 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:43:05.977+0000 7ffb01ff7640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:43:05.978 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:43:05.977+0000 7ffb01ff7640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:43:05.988 INFO:teuthology.orchestra.run.trial035.stdout:osd.0: {} 2026-02-24T07:43:05.988 INFO:teuthology.orchestra.run.trial035.stderr:osd.0: osd_enable_op_tracker = '' 2026-02-24T07:43:05.998 INFO:teuthology.orchestra.run.trial035.stdout:osd.1: {} 2026-02-24T07:43:05.999 INFO:teuthology.orchestra.run.trial035.stderr:osd.1: osd_enable_op_tracker = '' 2026-02-24T07:43:06.010 INFO:teuthology.orchestra.run.trial035.stdout:osd.2: {} 2026-02-24T07:43:06.010 INFO:teuthology.orchestra.run.trial035.stderr:osd.2: osd_enable_op_tracker = '' 2026-02-24T07:43:06.021 INFO:teuthology.orchestra.run.trial035.stdout:osd.3: {} 2026-02-24T07:43:06.021 INFO:teuthology.orchestra.run.trial035.stderr:osd.3: osd_enable_op_tracker = '' 2026-02-24T07:43:06.835 INFO:teuthology.orchestra.run.trial035.stderr:pool 'unique_pool_0' created 2026-02-24T07:43:06.874 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph osd pool application enable unique_pool_0 rados --yes-i-really-mean-it || true 2026-02-24T07:43:06.991 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:43:07.092 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:43:07.091+0000 7f078e3b1640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:43:07.093 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:43:06 trial035 ceph-mon[39284]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "orch get-security-config"}]: dispatch 2026-02-24T07:43:07.093 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:43:06 trial035 ceph-mon[39284]: Certificate for "cephadm-signed_agent_cert (trial035)" is still valid for 1824 days. 2026-02-24T07:43:07.093 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:43:06 trial035 ceph-mon[39284]: pgmap v6: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 2.7 TiB / 2.7 TiB avail 2026-02-24T07:43:07.093 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:43:06 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/2981558394' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2026-02-24T07:43:07.093 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:43:06 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/1568989745' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "unique_pool_0", "pg_num": 16} : dispatch 2026-02-24T07:43:07.093 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:43:07.092+0000 7f078e3b1640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:43:07.836 INFO:teuthology.orchestra.run.trial035.stderr:enabled application 'rados' on pool 'unique_pool_0' 2026-02-24T07:43:08.058 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2026-02-24T07:43:08.175 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:43:08.196 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:43:07 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/1568989745' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "unique_pool_0", "pg_num": 16}]': finished 2026-02-24T07:43:08.196 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:43:07 trial035 ceph-mon[39284]: osdmap e30: 4 total, 4 up, 4 in 2026-02-24T07:43:08.196 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:43:07 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/2520054589' entity='client.admin' cmd={"prefix": "osd pool application enable", "pool": "unique_pool_0", "app": "rados", "yes_i_really_mean_it": true} : dispatch 2026-02-24T07:43:08.282 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:43:08.280+0000 7fa221be3640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:43:08.282 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:43:08.281+0000 7fa221be3640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:43:08.294 INFO:teuthology.orchestra.run.trial035.stdout:osd.0: {} 2026-02-24T07:43:08.294 INFO:teuthology.orchestra.run.trial035.stderr:osd.0: osd_enable_op_tracker = '' 2026-02-24T07:43:08.303 INFO:teuthology.orchestra.run.trial035.stdout:osd.1: {} 2026-02-24T07:43:08.304 INFO:teuthology.orchestra.run.trial035.stderr:osd.1: osd_enable_op_tracker = '' 2026-02-24T07:43:08.315 INFO:teuthology.orchestra.run.trial035.stdout:osd.2: {} 2026-02-24T07:43:08.315 INFO:teuthology.orchestra.run.trial035.stderr:osd.2: osd_enable_op_tracker = '' 2026-02-24T07:43:08.326 INFO:teuthology.orchestra.run.trial035.stdout:osd.3: {} 2026-02-24T07:43:08.327 INFO:teuthology.orchestra.run.trial035.stderr:osd.3: osd_enable_op_tracker = '' 2026-02-24T07:43:08.878 DEBUG:teuthology.orchestra.run.trial035:> /bin/sh -c 'adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --no-log-to-stderr --name client.0 -t 16 -b 65536 --object-size 65536 -p unique_pool_0 bench 90 write' 2026-02-24T07:43:08.879 INFO:tasks.thrashosds.thrasher:Growing pool .mgr 2026-02-24T07:43:08.879 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph pg dump --format=json 2026-02-24T07:43:08.882 INFO:tasks.radosbench:joining radosbench (timing out after 3000s) 2026-02-24T07:43:08.882 INFO:teuthology.orchestra.run:waiting for 3000 2026-02-24T07:43:08.929 INFO:tasks.radosbench.radosbench.0.trial035.stdout:hints = 1 2026-02-24T07:43:08.929 INFO:tasks.radosbench.radosbench.0.trial035.stdout:Maintaining 16 concurrent writes of 65536 bytes to objects of size 65536 for up to 90 seconds or 0 objects 2026-02-24T07:43:08.930 INFO:tasks.radosbench.radosbench.0.trial035.stdout:Object prefix: benchmark_data_trial035_77811 2026-02-24T07:43:08.930 INFO:tasks.radosbench.radosbench.0.trial035.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2026-02-24T07:43:08.930 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 0 0 0 0 0 0 - 0 2026-02-24T07:43:09.014 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:43:09.129 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:43:08 trial035 ceph-mon[39284]: pgmap v8: 17 pgs: 16 unknown, 1 active+clean; 577 KiB data, 108 MiB used, 2.7 TiB / 2.7 TiB avail; 25 KiB/s rd, 0 B/s wr, 10 op/s 2026-02-24T07:43:09.130 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:43:08 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/2520054589' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "unique_pool_0", "app": "rados", "yes_i_really_mean_it": true}]': finished 2026-02-24T07:43:09.130 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:43:08 trial035 ceph-mon[39284]: osdmap e31: 4 total, 4 up, 4 in 2026-02-24T07:43:09.130 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:43:08 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/3474496959' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2026-02-24T07:43:09.131 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:43:09.129+0000 7f64d8d74640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:43:09.131 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:43:09.130+0000 7f64d8d74640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:43:09.261 INFO:teuthology.orchestra.run.trial035.stdout: 2026-02-24T07:43:09.262 INFO:teuthology.orchestra.run.trial035.stderr:dumped all 2026-02-24T07:43:09.302 INFO:teuthology.orchestra.run.trial035.stdout:{"pg_ready":true,"pg_map":{"version":10,"stamp":"2026-02-24T07:43:08.839963+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":240,"num_write":135,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":3,"acting":3,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":3,"num_osds":4,"num_per_pool_osds":4,"num_per_pool_omap_osds":3,"kb":2936012800,"kb_used":110556,"kb_used_data":2700,"kb_used_omap":45,"kb_used_meta":107730,"kb_avail":2935902244,"statfs":{"total":3006477107200,"available":3006363897856,"internally_reserved":0,"allocated":2764800,"data_stored":2108276,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":46187,"internal_metadata":110316437},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":60,"num_read_kb":176,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.180539"},"pg_stats":[{"pgid":"2.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2026-02-24T07:43:06.832892+0000","last_change":"2026-02-24T07:43:06.832892+0000","last_active":"2026-02-24T07:43:06.832892+0000","last_peered":"2026-02-24T07:43:06.832892+0000","last_clean":"2026-02-24T07:43:06.832892+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2026-02-24T07:43:06.832892+0000","last_undegraded":"2026-02-24T07:43:06.832892+0000","last_fullsized":"2026-02-24T07:43:06.832892+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_clean_scrub_stamp":"2026-02-24T07:43:06.832892+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2026-02-24T07:43:06.832892+0000","last_change":"2026-02-24T07:43:06.832892+0000","last_active":"2026-02-24T07:43:06.832892+0000","last_peered":"2026-02-24T07:43:06.832892+0000","last_clean":"2026-02-24T07:43:06.832892+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2026-02-24T07:43:06.832892+0000","last_undegraded":"2026-02-24T07:43:06.832892+0000","last_fullsized":"2026-02-24T07:43:06.832892+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_clean_scrub_stamp":"2026-02-24T07:43:06.832892+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2026-02-24T07:43:06.832892+0000","last_change":"2026-02-24T07:43:06.832892+0000","last_active":"2026-02-24T07:43:06.832892+0000","last_peered":"2026-02-24T07:43:06.832892+0000","last_clean":"2026-02-24T07:43:06.832892+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2026-02-24T07:43:06.832892+0000","last_undegraded":"2026-02-24T07:43:06.832892+0000","last_fullsized":"2026-02-24T07:43:06.832892+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_clean_scrub_stamp":"2026-02-24T07:43:06.832892+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2026-02-24T07:43:06.832892+0000","last_change":"2026-02-24T07:43:06.832892+0000","last_active":"2026-02-24T07:43:06.832892+0000","last_peered":"2026-02-24T07:43:06.832892+0000","last_clean":"2026-02-24T07:43:06.832892+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2026-02-24T07:43:06.832892+0000","last_undegraded":"2026-02-24T07:43:06.832892+0000","last_fullsized":"2026-02-24T07:43:06.832892+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_clean_scrub_stamp":"2026-02-24T07:43:06.832892+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"29'99","reported_seq":179,"reported_epoch":29,"state":"active+clean","last_fresh":"2026-02-24T07:43:01.869387+0000","last_change":"2026-02-24T07:42:54.186862+0000","last_active":"2026-02-24T07:43:01.869387+0000","last_peered":"2026-02-24T07:43:01.869387+0000","last_clean":"2026-02-24T07:43:01.869387+0000","last_became_active":"2026-02-24T07:42:54.186719+0000","last_became_peered":"2026-02-24T07:42:54.186719+0000","last_unstale":"2026-02-24T07:43:01.869387+0000","last_undegraded":"2026-02-24T07:43:01.869387+0000","last_fullsized":"2026-02-24T07:43:01.869387+0000","mapping_epoch":27,"log_start":"29'96","ondisk_log_start":"29'96","created":20,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:42:43.640776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:42:43.640776+0000","last_clean_scrub_stamp":"2026-02-24T07:42:43.640776+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":96,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:43:57.783028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":240,"num_write":135,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2026-02-24T07:43:06.832892+0000","last_change":"2026-02-24T07:43:06.832892+0000","last_active":"2026-02-24T07:43:06.832892+0000","last_peered":"2026-02-24T07:43:06.832892+0000","last_clean":"2026-02-24T07:43:06.832892+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2026-02-24T07:43:06.832892+0000","last_undegraded":"2026-02-24T07:43:06.832892+0000","last_fullsized":"2026-02-24T07:43:06.832892+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_clean_scrub_stamp":"2026-02-24T07:43:06.832892+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2026-02-24T07:43:06.832892+0000","last_change":"2026-02-24T07:43:06.832892+0000","last_active":"2026-02-24T07:43:06.832892+0000","last_peered":"2026-02-24T07:43:06.832892+0000","last_clean":"2026-02-24T07:43:06.832892+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2026-02-24T07:43:06.832892+0000","last_undegraded":"2026-02-24T07:43:06.832892+0000","last_fullsized":"2026-02-24T07:43:06.832892+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_clean_scrub_stamp":"2026-02-24T07:43:06.832892+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2026-02-24T07:43:06.832892+0000","last_change":"2026-02-24T07:43:06.832892+0000","last_active":"2026-02-24T07:43:06.832892+0000","last_peered":"2026-02-24T07:43:06.832892+0000","last_clean":"2026-02-24T07:43:06.832892+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2026-02-24T07:43:06.832892+0000","last_undegraded":"2026-02-24T07:43:06.832892+0000","last_fullsized":"2026-02-24T07:43:06.832892+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_clean_scrub_stamp":"2026-02-24T07:43:06.832892+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2026-02-24T07:43:06.832892+0000","last_change":"2026-02-24T07:43:06.832892+0000","last_active":"2026-02-24T07:43:06.832892+0000","last_peered":"2026-02-24T07:43:06.832892+0000","last_clean":"2026-02-24T07:43:06.832892+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2026-02-24T07:43:06.832892+0000","last_undegraded":"2026-02-24T07:43:06.832892+0000","last_fullsized":"2026-02-24T07:43:06.832892+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_clean_scrub_stamp":"2026-02-24T07:43:06.832892+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2026-02-24T07:43:06.832892+0000","last_change":"2026-02-24T07:43:06.832892+0000","last_active":"2026-02-24T07:43:06.832892+0000","last_peered":"2026-02-24T07:43:06.832892+0000","last_clean":"2026-02-24T07:43:06.832892+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2026-02-24T07:43:06.832892+0000","last_undegraded":"2026-02-24T07:43:06.832892+0000","last_fullsized":"2026-02-24T07:43:06.832892+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_clean_scrub_stamp":"2026-02-24T07:43:06.832892+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2026-02-24T07:43:06.832892+0000","last_change":"2026-02-24T07:43:06.832892+0000","last_active":"2026-02-24T07:43:06.832892+0000","last_peered":"2026-02-24T07:43:06.832892+0000","last_clean":"2026-02-24T07:43:06.832892+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2026-02-24T07:43:06.832892+0000","last_undegraded":"2026-02-24T07:43:06.832892+0000","last_fullsized":"2026-02-24T07:43:06.832892+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_clean_scrub_stamp":"2026-02-24T07:43:06.832892+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2026-02-24T07:43:06.832892+0000","last_change":"2026-02-24T07:43:06.832892+0000","last_active":"2026-02-24T07:43:06.832892+0000","last_peered":"2026-02-24T07:43:06.832892+0000","last_clean":"2026-02-24T07:43:06.832892+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2026-02-24T07:43:06.832892+0000","last_undegraded":"2026-02-24T07:43:06.832892+0000","last_fullsized":"2026-02-24T07:43:06.832892+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_clean_scrub_stamp":"2026-02-24T07:43:06.832892+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2026-02-24T07:43:06.832892+0000","last_change":"2026-02-24T07:43:06.832892+0000","last_active":"2026-02-24T07:43:06.832892+0000","last_peered":"2026-02-24T07:43:06.832892+0000","last_clean":"2026-02-24T07:43:06.832892+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2026-02-24T07:43:06.832892+0000","last_undegraded":"2026-02-24T07:43:06.832892+0000","last_fullsized":"2026-02-24T07:43:06.832892+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_clean_scrub_stamp":"2026-02-24T07:43:06.832892+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2026-02-24T07:43:06.832892+0000","last_change":"2026-02-24T07:43:06.832892+0000","last_active":"2026-02-24T07:43:06.832892+0000","last_peered":"2026-02-24T07:43:06.832892+0000","last_clean":"2026-02-24T07:43:06.832892+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2026-02-24T07:43:06.832892+0000","last_undegraded":"2026-02-24T07:43:06.832892+0000","last_fullsized":"2026-02-24T07:43:06.832892+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_clean_scrub_stamp":"2026-02-24T07:43:06.832892+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2026-02-24T07:43:06.832892+0000","last_change":"2026-02-24T07:43:06.832892+0000","last_active":"2026-02-24T07:43:06.832892+0000","last_peered":"2026-02-24T07:43:06.832892+0000","last_clean":"2026-02-24T07:43:06.832892+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2026-02-24T07:43:06.832892+0000","last_undegraded":"2026-02-24T07:43:06.832892+0000","last_fullsized":"2026-02-24T07:43:06.832892+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_clean_scrub_stamp":"2026-02-24T07:43:06.832892+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2026-02-24T07:43:06.832892+0000","last_change":"2026-02-24T07:43:06.832892+0000","last_active":"2026-02-24T07:43:06.832892+0000","last_peered":"2026-02-24T07:43:06.832892+0000","last_clean":"2026-02-24T07:43:06.832892+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2026-02-24T07:43:06.832892+0000","last_undegraded":"2026-02-24T07:43:06.832892+0000","last_fullsized":"2026-02-24T07:43:06.832892+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_clean_scrub_stamp":"2026-02-24T07:43:06.832892+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2026-02-24T07:43:06.832892+0000","last_change":"2026-02-24T07:43:06.832892+0000","last_active":"2026-02-24T07:43:06.832892+0000","last_peered":"2026-02-24T07:43:06.832892+0000","last_clean":"2026-02-24T07:43:06.832892+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2026-02-24T07:43:06.832892+0000","last_undegraded":"2026-02-24T07:43:06.832892+0000","last_fullsized":"2026-02-24T07:43:06.832892+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_clean_scrub_stamp":"2026-02-24T07:43:06.832892+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":240,"num_write":135,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":3,"up_from":25,"seq":107374182405,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":27784,"kb_used_data":820,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":733975416,"statfs":{"total":751619276800,"available":751590825984,"internally_reserved":0,"allocated":839680,"data_stored":674661,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6823,"internal_metadata":27583833},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":19,"seq":81604378631,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":27784,"kb_used_data":820,"kb_used_omap":10,"kb_used_meta":26933,"kb_avail":733975416,"statfs":{"total":751619276800,"available":751590825984,"internally_reserved":0,"allocated":839680,"data_stored":674661,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":10725,"internal_metadata":27579931},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542152,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":734003200,"kb_used":27204,"kb_used_data":240,"kb_used_omap":11,"kb_used_meta":26932,"kb_avail":733975996,"statfs":{"total":751619276800,"available":751591419904,"internally_reserved":0,"allocated":245760,"data_stored":84293,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12041,"internal_metadata":27578615},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705675,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":27784,"kb_used_data":820,"kb_used_omap":16,"kb_used_meta":26927,"kb_avail":733975416,"statfs":{"total":751619276800,"available":751590825984,"internally_reserved":0,"allocated":839680,"data_stored":674661,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16598,"internal_metadata":27574058},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-24T07:43:09.303 INFO:tasks.cephadm.ceph_manager.ceph:increase pool size by 10 2026-02-24T07:43:09.303 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph osd pool set .mgr pg_num 11 2026-02-24T07:43:09.430 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:43:09.549 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:43:09.547+0000 7fdb28b93640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:43:09.549 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:43:09.547+0000 7fdb28b93640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:43:09.694 INFO:teuthology.orchestra.run.trial035.stderr:Error EPERM: crimson_allow_pg_split is false; pg_num increase denied for crimson pool 2026-02-24T07:43:09.731 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-24T07:43:09.930 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 1 16 7102 7086 442.834 442.875 0.00168918 0.0022533 2026-02-24T07:43:10.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:43:09 trial035 ceph-mon[39284]: osdmap e32: 4 total, 4 up, 4 in 2026-02-24T07:43:10.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:43:09 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/800941745' entity='client.admin' cmd={"prefix": "osd pool set", "pool": ".mgr", "var": "pg_num", "val": "11"} : dispatch 2026-02-24T07:43:10.362 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2026-02-24T07:43:10.493 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:43:10.605 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:43:10.602+0000 7f15fdd49640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:43:10.605 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:43:10.603+0000 7f15fdd49640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:43:10.620 INFO:teuthology.orchestra.run.trial035.stdout:osd.0: {} 2026-02-24T07:43:10.620 INFO:teuthology.orchestra.run.trial035.stderr:osd.0: osd_enable_op_tracker = '' 2026-02-24T07:43:10.633 INFO:teuthology.orchestra.run.trial035.stdout:osd.1: {} 2026-02-24T07:43:10.633 INFO:teuthology.orchestra.run.trial035.stderr:osd.1: osd_enable_op_tracker = '' 2026-02-24T07:43:10.644 INFO:teuthology.orchestra.run.trial035.stdout:osd.2: {} 2026-02-24T07:43:10.645 INFO:teuthology.orchestra.run.trial035.stderr:osd.2: osd_enable_op_tracker = '' 2026-02-24T07:43:10.656 INFO:teuthology.orchestra.run.trial035.stdout:osd.3: {} 2026-02-24T07:43:10.656 INFO:teuthology.orchestra.run.trial035.stderr:osd.3: osd_enable_op_tracker = '' 2026-02-24T07:43:10.930 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 2 16 13862 13846 432.652 422.5 0.00269996 0.00230857 2026-02-24T07:43:10.954 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:43:10 trial035 ceph-mon[39284]: from='client.14341 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2026-02-24T07:43:10.954 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:43:10 trial035 ceph-mon[39284]: pgmap v11: 17 pgs: 16 unknown, 1 active+clean; 577 KiB data, 108 MiB used, 2.7 TiB / 2.7 TiB avail; 29 KiB/s rd, 0 B/s wr, 11 op/s 2026-02-24T07:43:10.954 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:43:10 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/4007349192' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2026-02-24T07:43:11.930 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 3 15 20828 20813 433.569 435.438 0.00301819 0.00230451 2026-02-24T07:43:12.696 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2026-02-24T07:43:12.833 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:43:12.930 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 4 15 27798 27783 434.075 435.625 0.0017889 0.0023019 2026-02-24T07:43:12.949 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:43:12.946+0000 7fe009d73640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:43:12.949 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:43:12.947+0000 7fe009d73640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:43:12.963 INFO:teuthology.orchestra.run.trial035.stdout:osd.0: {} 2026-02-24T07:43:12.963 INFO:teuthology.orchestra.run.trial035.stderr:osd.0: osd_enable_op_tracker = '' 2026-02-24T07:43:12.975 INFO:teuthology.orchestra.run.trial035.stdout:osd.1: {} 2026-02-24T07:43:12.975 INFO:teuthology.orchestra.run.trial035.stderr:osd.1: osd_enable_op_tracker = '' 2026-02-24T07:43:12.976 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:43:12 trial035 ceph-mon[39284]: pgmap v12: 17 pgs: 17 active+clean; 752 MiB data, 1.6 GiB used, 2.7 TiB / 2.7 TiB avail; 29 KiB/s rd, 125 MiB/s wr, 2.01k op/s 2026-02-24T07:43:12.987 INFO:teuthology.orchestra.run.trial035.stdout:osd.2: {} 2026-02-24T07:43:12.988 INFO:teuthology.orchestra.run.trial035.stderr:osd.2: osd_enable_op_tracker = '' 2026-02-24T07:43:13.000 INFO:teuthology.orchestra.run.trial035.stdout:osd.3: {} 2026-02-24T07:43:13.001 INFO:teuthology.orchestra.run.trial035.stderr:osd.3: osd_enable_op_tracker = '' 2026-02-24T07:43:13.930 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 5 16 34937 34921 436.478 446.125 0.00362017 0.00228926 2026-02-24T07:43:14.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:43:13 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/47155957' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2026-02-24T07:43:14.733 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2026-02-24T07:43:14.733 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph osd reweight 0 1 2026-02-24T07:43:14.876 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:43:14.930 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 6 16 42272 42256 440.102 458.438 0.00216897 0.0022708 2026-02-24T07:43:14.993 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:43:14.990+0000 7f137cdc5640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:43:14.993 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:43:14.991+0000 7f137cdc5640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:43:15.043 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2026-02-24T07:43:15.069 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:43:14 trial035 ceph-mon[39284]: pgmap v13: 17 pgs: 17 active+clean; 752 MiB data, 1.6 GiB used, 2.7 TiB / 2.7 TiB avail; 110 MiB/s wr, 1.76k op/s 2026-02-24T07:43:15.186 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:43:15.306 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:43:15.304+0000 7fef2e0d5640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:43:15.306 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:43:15.304+0000 7fef2e0d5640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:43:15.321 INFO:teuthology.orchestra.run.trial035.stdout:osd.0: {} 2026-02-24T07:43:15.322 INFO:teuthology.orchestra.run.trial035.stderr:osd.0: osd_enable_op_tracker = '' 2026-02-24T07:43:15.334 INFO:teuthology.orchestra.run.trial035.stdout:osd.1: {} 2026-02-24T07:43:15.335 INFO:teuthology.orchestra.run.trial035.stderr:osd.1: osd_enable_op_tracker = '' 2026-02-24T07:43:15.347 INFO:teuthology.orchestra.run.trial035.stdout:osd.2: {} 2026-02-24T07:43:15.347 INFO:teuthology.orchestra.run.trial035.stderr:osd.2: osd_enable_op_tracker = '' 2026-02-24T07:43:15.361 INFO:teuthology.orchestra.run.trial035.stdout:osd.3: {} 2026-02-24T07:43:15.361 INFO:teuthology.orchestra.run.trial035.stderr:osd.3: osd_enable_op_tracker = '' 2026-02-24T07:43:15.849 INFO:teuthology.orchestra.run.trial035.stderr:reweighted osd.0 to 1 (10000) 2026-02-24T07:43:15.931 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 7 16 49364 49348 440.547 443.25 0.00256964 0.00226859 2026-02-24T07:43:16.048 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph osd reweight 1 1 2026-02-24T07:43:16.183 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:43:16.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:43:15 trial035 ceph-mon[39284]: from='mgr.14284 10.20.193.35:0/3681373222' entity='mgr.x' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2026-02-24T07:43:16.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:43:15 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/3465554337' entity='client.admin' cmd={"prefix": "osd reweight", "id": 0, "weight": 1.0} : dispatch 2026-02-24T07:43:16.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:43:15 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/408098104' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2026-02-24T07:43:16.301 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:43:16.298+0000 7f880d9dd640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:43:16.301 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:43:16.299+0000 7f880d9dd640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:43:16.850 INFO:teuthology.orchestra.run.trial035.stderr:reweighted osd.1 to 1 (10000) 2026-02-24T07:43:16.894 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph osd reweight 2 1 2026-02-24T07:43:16.930 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 8 16 56070 56054 437.866 419.125 0.000878201 0.0022821 2026-02-24T07:43:17.029 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:43:17.102 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:43:16 trial035 ceph-mon[39284]: pgmap v14: 17 pgs: 17 active+clean; 1.8 GiB data, 4.9 GiB used, 2.7 TiB / 2.7 TiB avail; 232 MiB/s wr, 3.71k op/s 2026-02-24T07:43:17.102 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:43:16 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/3465554337' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 0, "weight": 1.0}]': finished 2026-02-24T07:43:17.102 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:43:16 trial035 ceph-mon[39284]: osdmap e33: 4 total, 4 up, 4 in 2026-02-24T07:43:17.102 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:43:16 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/3479524932' entity='client.admin' cmd={"prefix": "osd reweight", "id": 1, "weight": 1.0} : dispatch 2026-02-24T07:43:17.103 INFO:journalctl@ceph.mgr.x.trial035.stdout:Feb 24 07:43:16 trial035 ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-mgr-x[39621]: ::ffff:10.20.193.91 - - [24/Feb/2026:07:43:16] "GET /metrics HTTP/1.1" 200 4213 "" "Prometheus/3.6.0" 2026-02-24T07:43:17.155 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:43:17.152+0000 7fe3ad6f0640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:43:17.155 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:43:17.153+0000 7fe3ad6f0640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:43:17.356 INFO:journalctl@ceph.prometheus.a.trial091.stdout:Feb 24 07:43:16 trial091 ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-prometheus-a[66007]: time=2026-02-24T07:43:16.974Z level=ERROR source=scrape.go:1630 msg="Failed to determine correct type of scrape target." component="scrape manager" scrape_pool=ceph target=http://trial035:9283/metrics content_type="text/html;charset=utf-8" fallback_media_type="" err="received unsupported Content-Type \"text/html;charset=utf-8\" and no fallback_scrape_protocol specified for target" 2026-02-24T07:43:17.404 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2026-02-24T07:43:17.540 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:43:17.854 INFO:teuthology.orchestra.run.trial035.stderr:reweighted osd.2 to 1 (10000) 2026-02-24T07:43:17.931 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 9 16 62368 62352 432.947 393.625 0.00119372 0.00230855 2026-02-24T07:43:17.954 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:43:17 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/3479524932' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 1, "weight": 1.0}]': finished 2026-02-24T07:43:17.954 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:43:17 trial035 ceph-mon[39284]: osdmap e34: 4 total, 4 up, 4 in 2026-02-24T07:43:17.954 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:43:17 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/906882929' entity='client.admin' cmd={"prefix": "osd reweight", "id": 2, "weight": 1.0} : dispatch 2026-02-24T07:43:17.975 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:43:17.952+0000 7fdc510ff640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:43:17.975 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:43:17.952+0000 7fdc510ff640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:43:17.975 INFO:teuthology.orchestra.run.trial035.stderr:osd.0: osd_enable_op_tracker = '' 2026-02-24T07:43:17.975 INFO:teuthology.orchestra.run.trial035.stdout:osd.0: {} 2026-02-24T07:43:17.988 INFO:teuthology.orchestra.run.trial035.stdout:osd.1: {} 2026-02-24T07:43:17.988 INFO:teuthology.orchestra.run.trial035.stderr:osd.1: osd_enable_op_tracker = '' 2026-02-24T07:43:18.001 INFO:teuthology.orchestra.run.trial035.stdout:osd.2: {} 2026-02-24T07:43:18.001 INFO:teuthology.orchestra.run.trial035.stderr:osd.2: osd_enable_op_tracker = '' 2026-02-24T07:43:18.014 INFO:teuthology.orchestra.run.trial035.stdout:osd.3: {} 2026-02-24T07:43:18.015 INFO:teuthology.orchestra.run.trial035.stderr:osd.3: osd_enable_op_tracker = '' 2026-02-24T07:43:18.607 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph osd reweight 3 1 2026-02-24T07:43:18.744 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:43:18.931 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 10 16 66928 66912 418.151 285 0.00135694 0.00238992 2026-02-24T07:43:19.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:43:18 trial035 ceph-mon[39284]: pgmap v17: 17 pgs: 17 active+clean; 2.9 GiB data, 8.4 GiB used, 2.7 TiB / 2.7 TiB avail; 372 MiB/s wr, 5.95k op/s 2026-02-24T07:43:19.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:43:18 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/906882929' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 2, "weight": 1.0}]': finished 2026-02-24T07:43:19.205 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:43:18 trial035 ceph-mon[39284]: osdmap e35: 4 total, 4 up, 4 in 2026-02-24T07:43:19.205 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:43:18 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/2340198824' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2026-02-24T07:43:19.512 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:43:19.509+0000 7ffbda26c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:43:19.512 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:43:19.510+0000 7ffbda26c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:43:19.857 INFO:teuthology.orchestra.run.trial035.stderr:reweighted osd.3 to 1 (10000) 2026-02-24T07:43:19.902 INFO:tasks.cephadm.ceph_manager.ceph:waiting for recovery to complete 2026-02-24T07:43:19.902 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph pg dump --format=json 2026-02-24T07:43:19.931 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 11 15 73209 73194 415.828 392.625 0.00220792 0.00240356 2026-02-24T07:43:20.042 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:43:20.107 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:43:19 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/1211062642' entity='client.admin' cmd={"prefix": "osd reweight", "id": 3, "weight": 1.0} : dispatch 2026-02-24T07:43:20.107 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:43:19 trial035 ceph-mon[39284]: pgmap v19: 17 pgs: 17 active+clean; 2.9 GiB data, 8.4 GiB used, 2.7 TiB / 2.7 TiB avail; 371 MiB/s wr, 5.93k op/s 2026-02-24T07:43:20.174 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:43:20.171+0000 7ff1d1d10640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:43:20.174 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:43:20.172+0000 7ff1d1d10640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:43:20.316 INFO:teuthology.orchestra.run.trial035.stdout: 2026-02-24T07:43:20.316 INFO:teuthology.orchestra.run.trial035.stderr:dumped all 2026-02-24T07:43:20.357 INFO:teuthology.orchestra.run.trial035.stdout:{"pg_ready":true,"pg_map":{"version":20,"stamp":"2026-02-24T07:43:19.856869+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":3120235040,"num_objects":47604,"num_object_clones":0,"num_object_copies":142812,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":47604,"num_whiteouts":0,"num_read":136,"num_read_kb":240,"num_write":47737,"num_write_kb":3048678,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":51,"ondisk_log_size":51,"up":51,"acting":51,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":51,"num_osds":4,"num_per_pool_osds":4,"num_per_pool_omap_osds":4,"kb":2936012800,"kb_used":8840940,"kb_used_data":8372124,"kb_used_omap":52,"kb_used_meta":468683,"kb_avail":2927171860,"statfs":{"total":3006477107200,"available":2997423984640,"internally_reserved":0,"allocated":8573054976,"data_stored":8572308096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":53361,"internal_metadata":479932303},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1174929408,"num_objects":17928,"num_object_clones":0,"num_object_copies":53784,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17928,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17928,"num_write_kb":1147392,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.195739"},"pg_stats":[{"pgid":"2.f","version":"32'2892","reported_seq":2906,"reported_epoch":32,"state":"active+clean","last_fresh":"2026-02-24T07:43:15.659713+0000","last_change":"2026-02-24T07:43:07.853034+0000","last_active":"2026-02-24T07:43:15.659713+0000","last_peered":"2026-02-24T07:43:15.659713+0000","last_clean":"2026-02-24T07:43:15.659713+0000","last_became_active":"2026-02-24T07:43:07.852899+0000","last_became_peered":"2026-02-24T07:43:07.852899+0000","last_unstale":"2026-02-24T07:43:15.659713+0000","last_undegraded":"2026-02-24T07:43:15.659713+0000","last_fullsized":"2026-02-24T07:43:15.659713+0000","mapping_epoch":30,"log_start":"32'2889","ondisk_log_start":"32'2889","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_clean_scrub_stamp":"2026-02-24T07:43:06.832892+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":2889,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:44:18.843618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":189530112,"num_objects":2892,"num_object_clones":0,"num_object_copies":8676,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2892,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2892,"num_write_kb":185088,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.e","version":"32'2920","reported_seq":2934,"reported_epoch":32,"state":"active+clean","last_fresh":"2026-02-24T07:43:15.658012+0000","last_change":"2026-02-24T07:43:07.853009+0000","last_active":"2026-02-24T07:43:15.658012+0000","last_peered":"2026-02-24T07:43:15.658012+0000","last_clean":"2026-02-24T07:43:15.658012+0000","last_became_active":"2026-02-24T07:43:07.852815+0000","last_became_peered":"2026-02-24T07:43:07.852815+0000","last_unstale":"2026-02-24T07:43:15.658012+0000","last_undegraded":"2026-02-24T07:43:15.658012+0000","last_fullsized":"2026-02-24T07:43:15.658012+0000","mapping_epoch":30,"log_start":"32'2917","ondisk_log_start":"32'2917","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_clean_scrub_stamp":"2026-02-24T07:43:06.832892+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":2917,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:44:11.750665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":191365120,"num_objects":2920,"num_object_clones":0,"num_object_copies":8760,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2920,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2920,"num_write_kb":186880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.d","version":"32'3004","reported_seq":3018,"reported_epoch":32,"state":"active+clean","last_fresh":"2026-02-24T07:43:15.654975+0000","last_change":"2026-02-24T07:43:07.854403+0000","last_active":"2026-02-24T07:43:15.654975+0000","last_peered":"2026-02-24T07:43:15.654975+0000","last_clean":"2026-02-24T07:43:15.654975+0000","last_became_active":"2026-02-24T07:43:07.853975+0000","last_became_peered":"2026-02-24T07:43:07.853975+0000","last_unstale":"2026-02-24T07:43:15.654975+0000","last_undegraded":"2026-02-24T07:43:15.654975+0000","last_fullsized":"2026-02-24T07:43:15.654975+0000","mapping_epoch":30,"log_start":"32'3001","ondisk_log_start":"32'3001","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_clean_scrub_stamp":"2026-02-24T07:43:06.832892+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:44:14.520704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":196870144,"num_objects":3004,"num_object_clones":0,"num_object_copies":9012,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3004,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3004,"num_write_kb":192256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"32'2886","reported_seq":2900,"reported_epoch":32,"state":"active+clean","last_fresh":"2026-02-24T07:43:15.658620+0000","last_change":"2026-02-24T07:43:07.854780+0000","last_active":"2026-02-24T07:43:15.658620+0000","last_peered":"2026-02-24T07:43:15.658620+0000","last_clean":"2026-02-24T07:43:15.658620+0000","last_became_active":"2026-02-24T07:43:07.854428+0000","last_became_peered":"2026-02-24T07:43:07.854428+0000","last_unstale":"2026-02-24T07:43:15.658620+0000","last_undegraded":"2026-02-24T07:43:15.658620+0000","last_fullsized":"2026-02-24T07:43:15.658620+0000","mapping_epoch":30,"log_start":"32'2883","ondisk_log_start":"32'2883","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_clean_scrub_stamp":"2026-02-24T07:43:06.832892+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":2883,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:44:16.718883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":189136896,"num_objects":2886,"num_object_clones":0,"num_object_copies":8658,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2886,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2886,"num_write_kb":184704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"29'99","reported_seq":185,"reported_epoch":32,"state":"active+clean","last_fresh":"2026-02-24T07:43:08.932602+0000","last_change":"2026-02-24T07:42:54.186862+0000","last_active":"2026-02-24T07:43:08.932602+0000","last_peered":"2026-02-24T07:43:08.932602+0000","last_clean":"2026-02-24T07:43:08.932602+0000","last_became_active":"2026-02-24T07:42:54.186719+0000","last_became_peered":"2026-02-24T07:42:54.186719+0000","last_unstale":"2026-02-24T07:43:08.932602+0000","last_undegraded":"2026-02-24T07:43:08.932602+0000","last_fullsized":"2026-02-24T07:43:08.932602+0000","mapping_epoch":27,"log_start":"29'96","ondisk_log_start":"29'96","created":20,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:42:43.640776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:42:43.640776+0000","last_clean_scrub_stamp":"2026-02-24T07:42:43.640776+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":96,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:43:57.783028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":240,"num_write":135,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"32'2971","reported_seq":2985,"reported_epoch":32,"state":"active+clean","last_fresh":"2026-02-24T07:43:15.661760+0000","last_change":"2026-02-24T07:43:07.842967+0000","last_active":"2026-02-24T07:43:15.661760+0000","last_peered":"2026-02-24T07:43:15.661760+0000","last_clean":"2026-02-24T07:43:15.661760+0000","last_became_active":"2026-02-24T07:43:07.842530+0000","last_became_peered":"2026-02-24T07:43:07.842530+0000","last_unstale":"2026-02-24T07:43:15.661760+0000","last_undegraded":"2026-02-24T07:43:15.661760+0000","last_fullsized":"2026-02-24T07:43:15.661760+0000","mapping_epoch":30,"log_start":"32'2968","ondisk_log_start":"32'2968","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_clean_scrub_stamp":"2026-02-24T07:43:06.832892+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":2968,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:44:14.900069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":194707456,"num_objects":2971,"num_object_clones":0,"num_object_copies":8913,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2971,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2971,"num_write_kb":190144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"32'3047","reported_seq":3061,"reported_epoch":32,"state":"active+clean","last_fresh":"2026-02-24T07:43:15.656625+0000","last_change":"2026-02-24T07:43:07.854082+0000","last_active":"2026-02-24T07:43:15.656625+0000","last_peered":"2026-02-24T07:43:15.656625+0000","last_clean":"2026-02-24T07:43:15.656625+0000","last_became_active":"2026-02-24T07:43:07.853885+0000","last_became_peered":"2026-02-24T07:43:07.853885+0000","last_unstale":"2026-02-24T07:43:15.656625+0000","last_undegraded":"2026-02-24T07:43:15.656625+0000","last_fullsized":"2026-02-24T07:43:15.656625+0000","mapping_epoch":30,"log_start":"32'3044","ondisk_log_start":"32'3044","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_clean_scrub_stamp":"2026-02-24T07:43:06.832892+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:44:20.512438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":199688192,"num_objects":3047,"num_object_clones":0,"num_object_copies":9141,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3047,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3047,"num_write_kb":195008,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"32'2998","reported_seq":3012,"reported_epoch":32,"state":"active+clean","last_fresh":"2026-02-24T07:43:15.659672+0000","last_change":"2026-02-24T07:43:07.854497+0000","last_active":"2026-02-24T07:43:15.659672+0000","last_peered":"2026-02-24T07:43:15.659672+0000","last_clean":"2026-02-24T07:43:15.659672+0000","last_became_active":"2026-02-24T07:43:07.854212+0000","last_became_peered":"2026-02-24T07:43:07.854212+0000","last_unstale":"2026-02-24T07:43:15.659672+0000","last_undegraded":"2026-02-24T07:43:15.659672+0000","last_fullsized":"2026-02-24T07:43:15.659672+0000","mapping_epoch":30,"log_start":"32'2995","ondisk_log_start":"32'2995","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_clean_scrub_stamp":"2026-02-24T07:43:06.832892+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":2995,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:44:21.531086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":196476928,"num_objects":2998,"num_object_clones":0,"num_object_copies":8994,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2998,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2998,"num_write_kb":191872,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[1,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"32'3125","reported_seq":3139,"reported_epoch":32,"state":"active+clean","last_fresh":"2026-02-24T07:43:15.656470+0000","last_change":"2026-02-24T07:43:07.843123+0000","last_active":"2026-02-24T07:43:15.656470+0000","last_peered":"2026-02-24T07:43:15.656470+0000","last_clean":"2026-02-24T07:43:15.656470+0000","last_became_active":"2026-02-24T07:43:07.842864+0000","last_became_peered":"2026-02-24T07:43:07.842864+0000","last_unstale":"2026-02-24T07:43:15.656470+0000","last_undegraded":"2026-02-24T07:43:15.656470+0000","last_fullsized":"2026-02-24T07:43:15.656470+0000","mapping_epoch":30,"log_start":"32'3122","ondisk_log_start":"32'3122","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_clean_scrub_stamp":"2026-02-24T07:43:06.832892+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:44:34.876596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":204800000,"num_objects":3125,"num_object_clones":0,"num_object_copies":9375,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3125,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3125,"num_write_kb":200000,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.4","version":"32'2927","reported_seq":2941,"reported_epoch":32,"state":"active+clean","last_fresh":"2026-02-24T07:43:15.659653+0000","last_change":"2026-02-24T07:43:07.854462+0000","last_active":"2026-02-24T07:43:15.659653+0000","last_peered":"2026-02-24T07:43:15.659653+0000","last_clean":"2026-02-24T07:43:15.659653+0000","last_became_active":"2026-02-24T07:43:07.853966+0000","last_became_peered":"2026-02-24T07:43:07.853966+0000","last_unstale":"2026-02-24T07:43:15.659653+0000","last_undegraded":"2026-02-24T07:43:15.659653+0000","last_fullsized":"2026-02-24T07:43:15.659653+0000","mapping_epoch":30,"log_start":"32'2924","ondisk_log_start":"32'2924","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_clean_scrub_stamp":"2026-02-24T07:43:06.832892+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":2924,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:44:13.031431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":191823872,"num_objects":2927,"num_object_clones":0,"num_object_copies":8781,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2927,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2927,"num_write_kb":187328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"32'2990","reported_seq":3004,"reported_epoch":32,"state":"active+clean","last_fresh":"2026-02-24T07:43:15.662256+0000","last_change":"2026-02-24T07:43:07.854184+0000","last_active":"2026-02-24T07:43:15.662256+0000","last_peered":"2026-02-24T07:43:15.662256+0000","last_clean":"2026-02-24T07:43:15.662256+0000","last_became_active":"2026-02-24T07:43:07.853878+0000","last_became_peered":"2026-02-24T07:43:07.853878+0000","last_unstale":"2026-02-24T07:43:15.662256+0000","last_undegraded":"2026-02-24T07:43:15.662256+0000","last_fullsized":"2026-02-24T07:43:15.662256+0000","mapping_epoch":30,"log_start":"32'2987","ondisk_log_start":"32'2987","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_clean_scrub_stamp":"2026-02-24T07:43:06.832892+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":2987,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:44:31.507398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":195952640,"num_objects":2990,"num_object_clones":0,"num_object_copies":8970,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2990,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2990,"num_write_kb":191360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"32'3045","reported_seq":3059,"reported_epoch":32,"state":"active+clean","last_fresh":"2026-02-24T07:43:15.659483+0000","last_change":"2026-02-24T07:43:07.842180+0000","last_active":"2026-02-24T07:43:15.659483+0000","last_peered":"2026-02-24T07:43:15.659483+0000","last_clean":"2026-02-24T07:43:15.659483+0000","last_became_active":"2026-02-24T07:43:07.841993+0000","last_became_peered":"2026-02-24T07:43:07.841993+0000","last_unstale":"2026-02-24T07:43:15.659483+0000","last_undegraded":"2026-02-24T07:43:15.659483+0000","last_fullsized":"2026-02-24T07:43:15.659483+0000","mapping_epoch":30,"log_start":"32'3042","ondisk_log_start":"32'3042","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_clean_scrub_stamp":"2026-02-24T07:43:06.832892+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:44:19.519502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":199557120,"num_objects":3045,"num_object_clones":0,"num_object_copies":9135,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3045,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3045,"num_write_kb":194880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"32'2939","reported_seq":2953,"reported_epoch":32,"state":"active+clean","last_fresh":"2026-02-24T07:43:15.661636+0000","last_change":"2026-02-24T07:43:07.842375+0000","last_active":"2026-02-24T07:43:15.661636+0000","last_peered":"2026-02-24T07:43:15.661636+0000","last_clean":"2026-02-24T07:43:15.661636+0000","last_became_active":"2026-02-24T07:43:07.841920+0000","last_became_peered":"2026-02-24T07:43:07.841920+0000","last_unstale":"2026-02-24T07:43:15.661636+0000","last_undegraded":"2026-02-24T07:43:15.661636+0000","last_fullsized":"2026-02-24T07:43:15.661636+0000","mapping_epoch":30,"log_start":"32'2936","ondisk_log_start":"32'2936","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_clean_scrub_stamp":"2026-02-24T07:43:06.832892+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":2936,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:44:14.681944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":192610304,"num_objects":2939,"num_object_clones":0,"num_object_copies":8817,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2939,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2939,"num_write_kb":188096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.8","version":"32'2901","reported_seq":2915,"reported_epoch":32,"state":"active+clean","last_fresh":"2026-02-24T07:43:15.653814+0000","last_change":"2026-02-24T07:43:07.853992+0000","last_active":"2026-02-24T07:43:15.653814+0000","last_peered":"2026-02-24T07:43:15.653814+0000","last_clean":"2026-02-24T07:43:15.653814+0000","last_became_active":"2026-02-24T07:43:07.853758+0000","last_became_peered":"2026-02-24T07:43:07.853758+0000","last_unstale":"2026-02-24T07:43:15.653814+0000","last_undegraded":"2026-02-24T07:43:15.653814+0000","last_fullsized":"2026-02-24T07:43:15.653814+0000","mapping_epoch":30,"log_start":"32'2898","ondisk_log_start":"32'2898","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_clean_scrub_stamp":"2026-02-24T07:43:06.832892+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":2898,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:44:27.436119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":190119936,"num_objects":2901,"num_object_clones":0,"num_object_copies":8703,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2901,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2901,"num_write_kb":185664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"32'2945","reported_seq":2959,"reported_epoch":32,"state":"active+clean","last_fresh":"2026-02-24T07:43:15.656482+0000","last_change":"2026-02-24T07:43:07.854687+0000","last_active":"2026-02-24T07:43:15.656482+0000","last_peered":"2026-02-24T07:43:15.656482+0000","last_clean":"2026-02-24T07:43:15.656482+0000","last_became_active":"2026-02-24T07:43:07.854377+0000","last_became_peered":"2026-02-24T07:43:07.854377+0000","last_unstale":"2026-02-24T07:43:15.656482+0000","last_undegraded":"2026-02-24T07:43:15.656482+0000","last_fullsized":"2026-02-24T07:43:15.656482+0000","mapping_epoch":30,"log_start":"32'2942","ondisk_log_start":"32'2942","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_clean_scrub_stamp":"2026-02-24T07:43:06.832892+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":2942,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:44:34.215480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":193003520,"num_objects":2945,"num_object_clones":0,"num_object_copies":8835,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2945,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2945,"num_write_kb":188480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.a","version":"32'2981","reported_seq":2995,"reported_epoch":32,"state":"active+clean","last_fresh":"2026-02-24T07:43:15.657324+0000","last_change":"2026-02-24T07:43:07.841933+0000","last_active":"2026-02-24T07:43:15.657324+0000","last_peered":"2026-02-24T07:43:15.657324+0000","last_clean":"2026-02-24T07:43:15.657324+0000","last_became_active":"2026-02-24T07:43:07.841721+0000","last_became_peered":"2026-02-24T07:43:07.841721+0000","last_unstale":"2026-02-24T07:43:15.657324+0000","last_undegraded":"2026-02-24T07:43:15.657324+0000","last_fullsized":"2026-02-24T07:43:15.657324+0000","mapping_epoch":30,"log_start":"32'2978","ondisk_log_start":"32'2978","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_clean_scrub_stamp":"2026-02-24T07:43:06.832892+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":2978,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:44:14.991691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":195362816,"num_objects":2981,"num_object_clones":0,"num_object_copies":8943,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2981,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2981,"num_write_kb":190784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.b","version":"32'3031","reported_seq":3045,"reported_epoch":32,"state":"active+clean","last_fresh":"2026-02-24T07:43:15.655967+0000","last_change":"2026-02-24T07:43:07.853283+0000","last_active":"2026-02-24T07:43:15.655967+0000","last_peered":"2026-02-24T07:43:15.655967+0000","last_clean":"2026-02-24T07:43:15.655967+0000","last_became_active":"2026-02-24T07:43:07.853069+0000","last_became_peered":"2026-02-24T07:43:07.853069+0000","last_unstale":"2026-02-24T07:43:15.655967+0000","last_undegraded":"2026-02-24T07:43:15.655967+0000","last_fullsized":"2026-02-24T07:43:15.655967+0000","mapping_epoch":30,"log_start":"32'3028","ondisk_log_start":"32'3028","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_clean_scrub_stamp":"2026-02-24T07:43:06.832892+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:44:32.581403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":198639616,"num_objects":3031,"num_object_clones":0,"num_object_copies":9093,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3031,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3031,"num_write_kb":193984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":3119644672,"num_objects":47602,"num_object_clones":0,"num_object_copies":142806,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":47602,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":47602,"num_write_kb":3046528,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":9359523840,"data_stored":9359523840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":48,"ondisk_log_size":48,"up":48,"acting":48,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":240,"num_write":135,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":3,"up_from":25,"seq":107374182407,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":2750828,"kb_used_data":2615576,"kb_used_omap":7,"kb_used_meta":135224,"kb_avail":731252372,"statfs":{"total":751619276800,"available":748802428928,"internally_reserved":0,"allocated":2678349824,"data_stored":2678162216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":7473,"internal_metadata":138470095},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":19,"seq":81604378633,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":1914540,"kb_used_data":1815384,"kb_used_omap":12,"kb_used_meta":99123,"kb_avail":732088660,"statfs":{"total":751619276800,"available":749658787840,"internally_reserved":0,"allocated":1858953216,"data_stored":1858765608,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12692,"internal_metadata":101502572},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542154,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":2189672,"kb_used_data":2054420,"kb_used_omap":14,"kb_used_meta":135217,"kb_avail":731813528,"statfs":{"total":751619276800,"available":749377052672,"internally_reserved":0,"allocated":2103726080,"data_stored":2103542024,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":14645,"internal_metadata":138462923},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705677,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":1985900,"kb_used_data":1886744,"kb_used_omap":18,"kb_used_meta":99117,"kb_avail":732017300,"statfs":{"total":751619276800,"available":749585715200,"internally_reserved":0,"allocated":1932025856,"data_stored":1931838248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":18551,"internal_metadata":101496713},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":2132541440,"data_stored":2132541440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":2345205760,"data_stored":2345205760,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":2144665600,"data_stored":2144665600,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":2737111040,"data_stored":2737111040,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-24T07:43:20.358 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph pg dump --format=json 2026-02-24T07:43:20.541 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:43:20.667 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:43:20.664+0000 7f3ef4b49640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:43:20.667 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:43:20.665+0000 7f3ef4b49640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:43:20.806 INFO:teuthology.orchestra.run.trial035.stdout: 2026-02-24T07:43:20.807 INFO:teuthology.orchestra.run.trial035.stderr:dumped all 2026-02-24T07:43:20.849 INFO:teuthology.orchestra.run.trial035.stdout:{"pg_ready":true,"pg_map":{"version":20,"stamp":"2026-02-24T07:43:19.856869+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":3120235040,"num_objects":47604,"num_object_clones":0,"num_object_copies":142812,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":47604,"num_whiteouts":0,"num_read":136,"num_read_kb":240,"num_write":47737,"num_write_kb":3048678,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":51,"ondisk_log_size":51,"up":51,"acting":51,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":51,"num_osds":4,"num_per_pool_osds":4,"num_per_pool_omap_osds":4,"kb":2936012800,"kb_used":8840940,"kb_used_data":8372124,"kb_used_omap":52,"kb_used_meta":468683,"kb_avail":2927171860,"statfs":{"total":3006477107200,"available":2997423984640,"internally_reserved":0,"allocated":8573054976,"data_stored":8572308096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":53361,"internal_metadata":479932303},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1174929408,"num_objects":17928,"num_object_clones":0,"num_object_copies":53784,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17928,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17928,"num_write_kb":1147392,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.195739"},"pg_stats":[{"pgid":"2.f","version":"32'2892","reported_seq":2906,"reported_epoch":32,"state":"active+clean","last_fresh":"2026-02-24T07:43:15.659713+0000","last_change":"2026-02-24T07:43:07.853034+0000","last_active":"2026-02-24T07:43:15.659713+0000","last_peered":"2026-02-24T07:43:15.659713+0000","last_clean":"2026-02-24T07:43:15.659713+0000","last_became_active":"2026-02-24T07:43:07.852899+0000","last_became_peered":"2026-02-24T07:43:07.852899+0000","last_unstale":"2026-02-24T07:43:15.659713+0000","last_undegraded":"2026-02-24T07:43:15.659713+0000","last_fullsized":"2026-02-24T07:43:15.659713+0000","mapping_epoch":30,"log_start":"32'2889","ondisk_log_start":"32'2889","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_clean_scrub_stamp":"2026-02-24T07:43:06.832892+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":2889,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:44:18.843618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":189530112,"num_objects":2892,"num_object_clones":0,"num_object_copies":8676,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2892,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2892,"num_write_kb":185088,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.e","version":"32'2920","reported_seq":2934,"reported_epoch":32,"state":"active+clean","last_fresh":"2026-02-24T07:43:15.658012+0000","last_change":"2026-02-24T07:43:07.853009+0000","last_active":"2026-02-24T07:43:15.658012+0000","last_peered":"2026-02-24T07:43:15.658012+0000","last_clean":"2026-02-24T07:43:15.658012+0000","last_became_active":"2026-02-24T07:43:07.852815+0000","last_became_peered":"2026-02-24T07:43:07.852815+0000","last_unstale":"2026-02-24T07:43:15.658012+0000","last_undegraded":"2026-02-24T07:43:15.658012+0000","last_fullsized":"2026-02-24T07:43:15.658012+0000","mapping_epoch":30,"log_start":"32'2917","ondisk_log_start":"32'2917","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_clean_scrub_stamp":"2026-02-24T07:43:06.832892+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":2917,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:44:11.750665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":191365120,"num_objects":2920,"num_object_clones":0,"num_object_copies":8760,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2920,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2920,"num_write_kb":186880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.d","version":"32'3004","reported_seq":3018,"reported_epoch":32,"state":"active+clean","last_fresh":"2026-02-24T07:43:15.654975+0000","last_change":"2026-02-24T07:43:07.854403+0000","last_active":"2026-02-24T07:43:15.654975+0000","last_peered":"2026-02-24T07:43:15.654975+0000","last_clean":"2026-02-24T07:43:15.654975+0000","last_became_active":"2026-02-24T07:43:07.853975+0000","last_became_peered":"2026-02-24T07:43:07.853975+0000","last_unstale":"2026-02-24T07:43:15.654975+0000","last_undegraded":"2026-02-24T07:43:15.654975+0000","last_fullsized":"2026-02-24T07:43:15.654975+0000","mapping_epoch":30,"log_start":"32'3001","ondisk_log_start":"32'3001","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_clean_scrub_stamp":"2026-02-24T07:43:06.832892+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:44:14.520704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":196870144,"num_objects":3004,"num_object_clones":0,"num_object_copies":9012,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3004,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3004,"num_write_kb":192256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"32'2886","reported_seq":2900,"reported_epoch":32,"state":"active+clean","last_fresh":"2026-02-24T07:43:15.658620+0000","last_change":"2026-02-24T07:43:07.854780+0000","last_active":"2026-02-24T07:43:15.658620+0000","last_peered":"2026-02-24T07:43:15.658620+0000","last_clean":"2026-02-24T07:43:15.658620+0000","last_became_active":"2026-02-24T07:43:07.854428+0000","last_became_peered":"2026-02-24T07:43:07.854428+0000","last_unstale":"2026-02-24T07:43:15.658620+0000","last_undegraded":"2026-02-24T07:43:15.658620+0000","last_fullsized":"2026-02-24T07:43:15.658620+0000","mapping_epoch":30,"log_start":"32'2883","ondisk_log_start":"32'2883","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_clean_scrub_stamp":"2026-02-24T07:43:06.832892+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":2883,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:44:16.718883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":189136896,"num_objects":2886,"num_object_clones":0,"num_object_copies":8658,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2886,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2886,"num_write_kb":184704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"29'99","reported_seq":185,"reported_epoch":32,"state":"active+clean","last_fresh":"2026-02-24T07:43:08.932602+0000","last_change":"2026-02-24T07:42:54.186862+0000","last_active":"2026-02-24T07:43:08.932602+0000","last_peered":"2026-02-24T07:43:08.932602+0000","last_clean":"2026-02-24T07:43:08.932602+0000","last_became_active":"2026-02-24T07:42:54.186719+0000","last_became_peered":"2026-02-24T07:42:54.186719+0000","last_unstale":"2026-02-24T07:43:08.932602+0000","last_undegraded":"2026-02-24T07:43:08.932602+0000","last_fullsized":"2026-02-24T07:43:08.932602+0000","mapping_epoch":27,"log_start":"29'96","ondisk_log_start":"29'96","created":20,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:42:43.640776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:42:43.640776+0000","last_clean_scrub_stamp":"2026-02-24T07:42:43.640776+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":96,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:43:57.783028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":240,"num_write":135,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"32'2971","reported_seq":2985,"reported_epoch":32,"state":"active+clean","last_fresh":"2026-02-24T07:43:15.661760+0000","last_change":"2026-02-24T07:43:07.842967+0000","last_active":"2026-02-24T07:43:15.661760+0000","last_peered":"2026-02-24T07:43:15.661760+0000","last_clean":"2026-02-24T07:43:15.661760+0000","last_became_active":"2026-02-24T07:43:07.842530+0000","last_became_peered":"2026-02-24T07:43:07.842530+0000","last_unstale":"2026-02-24T07:43:15.661760+0000","last_undegraded":"2026-02-24T07:43:15.661760+0000","last_fullsized":"2026-02-24T07:43:15.661760+0000","mapping_epoch":30,"log_start":"32'2968","ondisk_log_start":"32'2968","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_clean_scrub_stamp":"2026-02-24T07:43:06.832892+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":2968,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:44:14.900069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":194707456,"num_objects":2971,"num_object_clones":0,"num_object_copies":8913,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2971,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2971,"num_write_kb":190144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"32'3047","reported_seq":3061,"reported_epoch":32,"state":"active+clean","last_fresh":"2026-02-24T07:43:15.656625+0000","last_change":"2026-02-24T07:43:07.854082+0000","last_active":"2026-02-24T07:43:15.656625+0000","last_peered":"2026-02-24T07:43:15.656625+0000","last_clean":"2026-02-24T07:43:15.656625+0000","last_became_active":"2026-02-24T07:43:07.853885+0000","last_became_peered":"2026-02-24T07:43:07.853885+0000","last_unstale":"2026-02-24T07:43:15.656625+0000","last_undegraded":"2026-02-24T07:43:15.656625+0000","last_fullsized":"2026-02-24T07:43:15.656625+0000","mapping_epoch":30,"log_start":"32'3044","ondisk_log_start":"32'3044","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_clean_scrub_stamp":"2026-02-24T07:43:06.832892+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:44:20.512438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":199688192,"num_objects":3047,"num_object_clones":0,"num_object_copies":9141,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3047,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3047,"num_write_kb":195008,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"32'2998","reported_seq":3012,"reported_epoch":32,"state":"active+clean","last_fresh":"2026-02-24T07:43:15.659672+0000","last_change":"2026-02-24T07:43:07.854497+0000","last_active":"2026-02-24T07:43:15.659672+0000","last_peered":"2026-02-24T07:43:15.659672+0000","last_clean":"2026-02-24T07:43:15.659672+0000","last_became_active":"2026-02-24T07:43:07.854212+0000","last_became_peered":"2026-02-24T07:43:07.854212+0000","last_unstale":"2026-02-24T07:43:15.659672+0000","last_undegraded":"2026-02-24T07:43:15.659672+0000","last_fullsized":"2026-02-24T07:43:15.659672+0000","mapping_epoch":30,"log_start":"32'2995","ondisk_log_start":"32'2995","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_clean_scrub_stamp":"2026-02-24T07:43:06.832892+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":2995,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:44:21.531086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":196476928,"num_objects":2998,"num_object_clones":0,"num_object_copies":8994,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2998,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2998,"num_write_kb":191872,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[1,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"32'3125","reported_seq":3139,"reported_epoch":32,"state":"active+clean","last_fresh":"2026-02-24T07:43:15.656470+0000","last_change":"2026-02-24T07:43:07.843123+0000","last_active":"2026-02-24T07:43:15.656470+0000","last_peered":"2026-02-24T07:43:15.656470+0000","last_clean":"2026-02-24T07:43:15.656470+0000","last_became_active":"2026-02-24T07:43:07.842864+0000","last_became_peered":"2026-02-24T07:43:07.842864+0000","last_unstale":"2026-02-24T07:43:15.656470+0000","last_undegraded":"2026-02-24T07:43:15.656470+0000","last_fullsized":"2026-02-24T07:43:15.656470+0000","mapping_epoch":30,"log_start":"32'3122","ondisk_log_start":"32'3122","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_clean_scrub_stamp":"2026-02-24T07:43:06.832892+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:44:34.876596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":204800000,"num_objects":3125,"num_object_clones":0,"num_object_copies":9375,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3125,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3125,"num_write_kb":200000,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.4","version":"32'2927","reported_seq":2941,"reported_epoch":32,"state":"active+clean","last_fresh":"2026-02-24T07:43:15.659653+0000","last_change":"2026-02-24T07:43:07.854462+0000","last_active":"2026-02-24T07:43:15.659653+0000","last_peered":"2026-02-24T07:43:15.659653+0000","last_clean":"2026-02-24T07:43:15.659653+0000","last_became_active":"2026-02-24T07:43:07.853966+0000","last_became_peered":"2026-02-24T07:43:07.853966+0000","last_unstale":"2026-02-24T07:43:15.659653+0000","last_undegraded":"2026-02-24T07:43:15.659653+0000","last_fullsized":"2026-02-24T07:43:15.659653+0000","mapping_epoch":30,"log_start":"32'2924","ondisk_log_start":"32'2924","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_clean_scrub_stamp":"2026-02-24T07:43:06.832892+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":2924,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:44:13.031431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":191823872,"num_objects":2927,"num_object_clones":0,"num_object_copies":8781,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2927,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2927,"num_write_kb":187328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"32'2990","reported_seq":3004,"reported_epoch":32,"state":"active+clean","last_fresh":"2026-02-24T07:43:15.662256+0000","last_change":"2026-02-24T07:43:07.854184+0000","last_active":"2026-02-24T07:43:15.662256+0000","last_peered":"2026-02-24T07:43:15.662256+0000","last_clean":"2026-02-24T07:43:15.662256+0000","last_became_active":"2026-02-24T07:43:07.853878+0000","last_became_peered":"2026-02-24T07:43:07.853878+0000","last_unstale":"2026-02-24T07:43:15.662256+0000","last_undegraded":"2026-02-24T07:43:15.662256+0000","last_fullsized":"2026-02-24T07:43:15.662256+0000","mapping_epoch":30,"log_start":"32'2987","ondisk_log_start":"32'2987","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_clean_scrub_stamp":"2026-02-24T07:43:06.832892+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":2987,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:44:31.507398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":195952640,"num_objects":2990,"num_object_clones":0,"num_object_copies":8970,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2990,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2990,"num_write_kb":191360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"32'3045","reported_seq":3059,"reported_epoch":32,"state":"active+clean","last_fresh":"2026-02-24T07:43:15.659483+0000","last_change":"2026-02-24T07:43:07.842180+0000","last_active":"2026-02-24T07:43:15.659483+0000","last_peered":"2026-02-24T07:43:15.659483+0000","last_clean":"2026-02-24T07:43:15.659483+0000","last_became_active":"2026-02-24T07:43:07.841993+0000","last_became_peered":"2026-02-24T07:43:07.841993+0000","last_unstale":"2026-02-24T07:43:15.659483+0000","last_undegraded":"2026-02-24T07:43:15.659483+0000","last_fullsized":"2026-02-24T07:43:15.659483+0000","mapping_epoch":30,"log_start":"32'3042","ondisk_log_start":"32'3042","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_clean_scrub_stamp":"2026-02-24T07:43:06.832892+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:44:19.519502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":199557120,"num_objects":3045,"num_object_clones":0,"num_object_copies":9135,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3045,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3045,"num_write_kb":194880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"32'2939","reported_seq":2953,"reported_epoch":32,"state":"active+clean","last_fresh":"2026-02-24T07:43:15.661636+0000","last_change":"2026-02-24T07:43:07.842375+0000","last_active":"2026-02-24T07:43:15.661636+0000","last_peered":"2026-02-24T07:43:15.661636+0000","last_clean":"2026-02-24T07:43:15.661636+0000","last_became_active":"2026-02-24T07:43:07.841920+0000","last_became_peered":"2026-02-24T07:43:07.841920+0000","last_unstale":"2026-02-24T07:43:15.661636+0000","last_undegraded":"2026-02-24T07:43:15.661636+0000","last_fullsized":"2026-02-24T07:43:15.661636+0000","mapping_epoch":30,"log_start":"32'2936","ondisk_log_start":"32'2936","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_clean_scrub_stamp":"2026-02-24T07:43:06.832892+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":2936,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:44:14.681944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":192610304,"num_objects":2939,"num_object_clones":0,"num_object_copies":8817,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2939,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2939,"num_write_kb":188096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.8","version":"32'2901","reported_seq":2915,"reported_epoch":32,"state":"active+clean","last_fresh":"2026-02-24T07:43:15.653814+0000","last_change":"2026-02-24T07:43:07.853992+0000","last_active":"2026-02-24T07:43:15.653814+0000","last_peered":"2026-02-24T07:43:15.653814+0000","last_clean":"2026-02-24T07:43:15.653814+0000","last_became_active":"2026-02-24T07:43:07.853758+0000","last_became_peered":"2026-02-24T07:43:07.853758+0000","last_unstale":"2026-02-24T07:43:15.653814+0000","last_undegraded":"2026-02-24T07:43:15.653814+0000","last_fullsized":"2026-02-24T07:43:15.653814+0000","mapping_epoch":30,"log_start":"32'2898","ondisk_log_start":"32'2898","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_clean_scrub_stamp":"2026-02-24T07:43:06.832892+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":2898,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:44:27.436119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":190119936,"num_objects":2901,"num_object_clones":0,"num_object_copies":8703,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2901,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2901,"num_write_kb":185664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"32'2945","reported_seq":2959,"reported_epoch":32,"state":"active+clean","last_fresh":"2026-02-24T07:43:15.656482+0000","last_change":"2026-02-24T07:43:07.854687+0000","last_active":"2026-02-24T07:43:15.656482+0000","last_peered":"2026-02-24T07:43:15.656482+0000","last_clean":"2026-02-24T07:43:15.656482+0000","last_became_active":"2026-02-24T07:43:07.854377+0000","last_became_peered":"2026-02-24T07:43:07.854377+0000","last_unstale":"2026-02-24T07:43:15.656482+0000","last_undegraded":"2026-02-24T07:43:15.656482+0000","last_fullsized":"2026-02-24T07:43:15.656482+0000","mapping_epoch":30,"log_start":"32'2942","ondisk_log_start":"32'2942","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_clean_scrub_stamp":"2026-02-24T07:43:06.832892+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":2942,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:44:34.215480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":193003520,"num_objects":2945,"num_object_clones":0,"num_object_copies":8835,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2945,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2945,"num_write_kb":188480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.a","version":"32'2981","reported_seq":2995,"reported_epoch":32,"state":"active+clean","last_fresh":"2026-02-24T07:43:15.657324+0000","last_change":"2026-02-24T07:43:07.841933+0000","last_active":"2026-02-24T07:43:15.657324+0000","last_peered":"2026-02-24T07:43:15.657324+0000","last_clean":"2026-02-24T07:43:15.657324+0000","last_became_active":"2026-02-24T07:43:07.841721+0000","last_became_peered":"2026-02-24T07:43:07.841721+0000","last_unstale":"2026-02-24T07:43:15.657324+0000","last_undegraded":"2026-02-24T07:43:15.657324+0000","last_fullsized":"2026-02-24T07:43:15.657324+0000","mapping_epoch":30,"log_start":"32'2978","ondisk_log_start":"32'2978","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_clean_scrub_stamp":"2026-02-24T07:43:06.832892+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":2978,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:44:14.991691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":195362816,"num_objects":2981,"num_object_clones":0,"num_object_copies":8943,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2981,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2981,"num_write_kb":190784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.b","version":"32'3031","reported_seq":3045,"reported_epoch":32,"state":"active+clean","last_fresh":"2026-02-24T07:43:15.655967+0000","last_change":"2026-02-24T07:43:07.853283+0000","last_active":"2026-02-24T07:43:15.655967+0000","last_peered":"2026-02-24T07:43:15.655967+0000","last_clean":"2026-02-24T07:43:15.655967+0000","last_became_active":"2026-02-24T07:43:07.853069+0000","last_became_peered":"2026-02-24T07:43:07.853069+0000","last_unstale":"2026-02-24T07:43:15.655967+0000","last_undegraded":"2026-02-24T07:43:15.655967+0000","last_fullsized":"2026-02-24T07:43:15.655967+0000","mapping_epoch":30,"log_start":"32'3028","ondisk_log_start":"32'3028","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_clean_scrub_stamp":"2026-02-24T07:43:06.832892+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:44:32.581403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":198639616,"num_objects":3031,"num_object_clones":0,"num_object_copies":9093,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3031,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3031,"num_write_kb":193984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":3119644672,"num_objects":47602,"num_object_clones":0,"num_object_copies":142806,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":47602,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":47602,"num_write_kb":3046528,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":9359523840,"data_stored":9359523840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":48,"ondisk_log_size":48,"up":48,"acting":48,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":240,"num_write":135,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":3,"up_from":25,"seq":107374182407,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":2750828,"kb_used_data":2615576,"kb_used_omap":7,"kb_used_meta":135224,"kb_avail":731252372,"statfs":{"total":751619276800,"available":748802428928,"internally_reserved":0,"allocated":2678349824,"data_stored":2678162216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":7473,"internal_metadata":138470095},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":19,"seq":81604378633,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":1914540,"kb_used_data":1815384,"kb_used_omap":12,"kb_used_meta":99123,"kb_avail":732088660,"statfs":{"total":751619276800,"available":749658787840,"internally_reserved":0,"allocated":1858953216,"data_stored":1858765608,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12692,"internal_metadata":101502572},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542154,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":2189672,"kb_used_data":2054420,"kb_used_omap":14,"kb_used_meta":135217,"kb_avail":731813528,"statfs":{"total":751619276800,"available":749377052672,"internally_reserved":0,"allocated":2103726080,"data_stored":2103542024,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":14645,"internal_metadata":138462923},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705677,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":1985900,"kb_used_data":1886744,"kb_used_omap":18,"kb_used_meta":99117,"kb_avail":732017300,"statfs":{"total":751619276800,"available":749585715200,"internally_reserved":0,"allocated":1932025856,"data_stored":1931838248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":18551,"internal_metadata":101496713},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":2132541440,"data_stored":2132541440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":2345205760,"data_stored":2345205760,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":2144665600,"data_stored":2144665600,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":2737111040,"data_stored":2737111040,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-24T07:43:20.851 INFO:tasks.cephadm.ceph_manager.ceph:recovered! 2026-02-24T07:43:20.852 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2026-02-24T07:43:20.852 INFO:tasks.thrashosds.thrasher:Growing pool .mgr 2026-02-24T07:43:20.852 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph pg dump --format=json 2026-02-24T07:43:20.931 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 12 16 80337 80321 418.292 445.438 0.00299548 0.00238938 2026-02-24T07:43:20.988 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:43:21.058 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:43:20 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/1211062642' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 3, "weight": 1.0}]': finished 2026-02-24T07:43:21.058 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:43:20 trial035 ceph-mon[39284]: osdmap e36: 4 total, 4 up, 4 in 2026-02-24T07:43:21.058 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:43:20 trial035 ceph-mon[39284]: from='client.14361 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2026-02-24T07:43:21.058 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:43:20 trial035 ceph-mon[39284]: from='client.14363 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2026-02-24T07:43:21.089 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2026-02-24T07:43:21.113 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:43:21.111+0000 7f98b4738640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:43:21.113 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:43:21.111+0000 7f98b4738640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:43:21.226 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:43:21.262 INFO:teuthology.orchestra.run.trial035.stdout: 2026-02-24T07:43:21.263 INFO:teuthology.orchestra.run.trial035.stderr:dumped all 2026-02-24T07:43:21.307 INFO:teuthology.orchestra.run.trial035.stdout:{"pg_ready":true,"pg_map":{"version":20,"stamp":"2026-02-24T07:43:19.856869+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":3120235040,"num_objects":47604,"num_object_clones":0,"num_object_copies":142812,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":47604,"num_whiteouts":0,"num_read":136,"num_read_kb":240,"num_write":47737,"num_write_kb":3048678,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":51,"ondisk_log_size":51,"up":51,"acting":51,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":51,"num_osds":4,"num_per_pool_osds":4,"num_per_pool_omap_osds":4,"kb":2936012800,"kb_used":8840940,"kb_used_data":8372124,"kb_used_omap":52,"kb_used_meta":468683,"kb_avail":2927171860,"statfs":{"total":3006477107200,"available":2997423984640,"internally_reserved":0,"allocated":8573054976,"data_stored":8572308096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":53361,"internal_metadata":479932303},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1174929408,"num_objects":17928,"num_object_clones":0,"num_object_copies":53784,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17928,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17928,"num_write_kb":1147392,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.195739"},"pg_stats":[{"pgid":"2.f","version":"32'2892","reported_seq":2906,"reported_epoch":32,"state":"active+clean","last_fresh":"2026-02-24T07:43:15.659713+0000","last_change":"2026-02-24T07:43:07.853034+0000","last_active":"2026-02-24T07:43:15.659713+0000","last_peered":"2026-02-24T07:43:15.659713+0000","last_clean":"2026-02-24T07:43:15.659713+0000","last_became_active":"2026-02-24T07:43:07.852899+0000","last_became_peered":"2026-02-24T07:43:07.852899+0000","last_unstale":"2026-02-24T07:43:15.659713+0000","last_undegraded":"2026-02-24T07:43:15.659713+0000","last_fullsized":"2026-02-24T07:43:15.659713+0000","mapping_epoch":30,"log_start":"32'2889","ondisk_log_start":"32'2889","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_clean_scrub_stamp":"2026-02-24T07:43:06.832892+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":2889,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:44:18.843618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":189530112,"num_objects":2892,"num_object_clones":0,"num_object_copies":8676,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2892,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2892,"num_write_kb":185088,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.e","version":"32'2920","reported_seq":2934,"reported_epoch":32,"state":"active+clean","last_fresh":"2026-02-24T07:43:15.658012+0000","last_change":"2026-02-24T07:43:07.853009+0000","last_active":"2026-02-24T07:43:15.658012+0000","last_peered":"2026-02-24T07:43:15.658012+0000","last_clean":"2026-02-24T07:43:15.658012+0000","last_became_active":"2026-02-24T07:43:07.852815+0000","last_became_peered":"2026-02-24T07:43:07.852815+0000","last_unstale":"2026-02-24T07:43:15.658012+0000","last_undegraded":"2026-02-24T07:43:15.658012+0000","last_fullsized":"2026-02-24T07:43:15.658012+0000","mapping_epoch":30,"log_start":"32'2917","ondisk_log_start":"32'2917","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_clean_scrub_stamp":"2026-02-24T07:43:06.832892+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":2917,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:44:11.750665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":191365120,"num_objects":2920,"num_object_clones":0,"num_object_copies":8760,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2920,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2920,"num_write_kb":186880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.d","version":"32'3004","reported_seq":3018,"reported_epoch":32,"state":"active+clean","last_fresh":"2026-02-24T07:43:15.654975+0000","last_change":"2026-02-24T07:43:07.854403+0000","last_active":"2026-02-24T07:43:15.654975+0000","last_peered":"2026-02-24T07:43:15.654975+0000","last_clean":"2026-02-24T07:43:15.654975+0000","last_became_active":"2026-02-24T07:43:07.853975+0000","last_became_peered":"2026-02-24T07:43:07.853975+0000","last_unstale":"2026-02-24T07:43:15.654975+0000","last_undegraded":"2026-02-24T07:43:15.654975+0000","last_fullsized":"2026-02-24T07:43:15.654975+0000","mapping_epoch":30,"log_start":"32'3001","ondisk_log_start":"32'3001","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_clean_scrub_stamp":"2026-02-24T07:43:06.832892+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:44:14.520704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":196870144,"num_objects":3004,"num_object_clones":0,"num_object_copies":9012,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3004,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3004,"num_write_kb":192256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"32'2886","reported_seq":2900,"reported_epoch":32,"state":"active+clean","last_fresh":"2026-02-24T07:43:15.658620+0000","last_change":"2026-02-24T07:43:07.854780+0000","last_active":"2026-02-24T07:43:15.658620+0000","last_peered":"2026-02-24T07:43:15.658620+0000","last_clean":"2026-02-24T07:43:15.658620+0000","last_became_active":"2026-02-24T07:43:07.854428+0000","last_became_peered":"2026-02-24T07:43:07.854428+0000","last_unstale":"2026-02-24T07:43:15.658620+0000","last_undegraded":"2026-02-24T07:43:15.658620+0000","last_fullsized":"2026-02-24T07:43:15.658620+0000","mapping_epoch":30,"log_start":"32'2883","ondisk_log_start":"32'2883","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_clean_scrub_stamp":"2026-02-24T07:43:06.832892+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":2883,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:44:16.718883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":189136896,"num_objects":2886,"num_object_clones":0,"num_object_copies":8658,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2886,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2886,"num_write_kb":184704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"29'99","reported_seq":185,"reported_epoch":32,"state":"active+clean","last_fresh":"2026-02-24T07:43:08.932602+0000","last_change":"2026-02-24T07:42:54.186862+0000","last_active":"2026-02-24T07:43:08.932602+0000","last_peered":"2026-02-24T07:43:08.932602+0000","last_clean":"2026-02-24T07:43:08.932602+0000","last_became_active":"2026-02-24T07:42:54.186719+0000","last_became_peered":"2026-02-24T07:42:54.186719+0000","last_unstale":"2026-02-24T07:43:08.932602+0000","last_undegraded":"2026-02-24T07:43:08.932602+0000","last_fullsized":"2026-02-24T07:43:08.932602+0000","mapping_epoch":27,"log_start":"29'96","ondisk_log_start":"29'96","created":20,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:42:43.640776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:42:43.640776+0000","last_clean_scrub_stamp":"2026-02-24T07:42:43.640776+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":96,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:43:57.783028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":240,"num_write":135,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"32'2971","reported_seq":2985,"reported_epoch":32,"state":"active+clean","last_fresh":"2026-02-24T07:43:15.661760+0000","last_change":"2026-02-24T07:43:07.842967+0000","last_active":"2026-02-24T07:43:15.661760+0000","last_peered":"2026-02-24T07:43:15.661760+0000","last_clean":"2026-02-24T07:43:15.661760+0000","last_became_active":"2026-02-24T07:43:07.842530+0000","last_became_peered":"2026-02-24T07:43:07.842530+0000","last_unstale":"2026-02-24T07:43:15.661760+0000","last_undegraded":"2026-02-24T07:43:15.661760+0000","last_fullsized":"2026-02-24T07:43:15.661760+0000","mapping_epoch":30,"log_start":"32'2968","ondisk_log_start":"32'2968","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_clean_scrub_stamp":"2026-02-24T07:43:06.832892+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":2968,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:44:14.900069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":194707456,"num_objects":2971,"num_object_clones":0,"num_object_copies":8913,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2971,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2971,"num_write_kb":190144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"32'3047","reported_seq":3061,"reported_epoch":32,"state":"active+clean","last_fresh":"2026-02-24T07:43:15.656625+0000","last_change":"2026-02-24T07:43:07.854082+0000","last_active":"2026-02-24T07:43:15.656625+0000","last_peered":"2026-02-24T07:43:15.656625+0000","last_clean":"2026-02-24T07:43:15.656625+0000","last_became_active":"2026-02-24T07:43:07.853885+0000","last_became_peered":"2026-02-24T07:43:07.853885+0000","last_unstale":"2026-02-24T07:43:15.656625+0000","last_undegraded":"2026-02-24T07:43:15.656625+0000","last_fullsized":"2026-02-24T07:43:15.656625+0000","mapping_epoch":30,"log_start":"32'3044","ondisk_log_start":"32'3044","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_clean_scrub_stamp":"2026-02-24T07:43:06.832892+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:44:20.512438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":199688192,"num_objects":3047,"num_object_clones":0,"num_object_copies":9141,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3047,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3047,"num_write_kb":195008,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"32'2998","reported_seq":3012,"reported_epoch":32,"state":"active+clean","last_fresh":"2026-02-24T07:43:15.659672+0000","last_change":"2026-02-24T07:43:07.854497+0000","last_active":"2026-02-24T07:43:15.659672+0000","last_peered":"2026-02-24T07:43:15.659672+0000","last_clean":"2026-02-24T07:43:15.659672+0000","last_became_active":"2026-02-24T07:43:07.854212+0000","last_became_peered":"2026-02-24T07:43:07.854212+0000","last_unstale":"2026-02-24T07:43:15.659672+0000","last_undegraded":"2026-02-24T07:43:15.659672+0000","last_fullsized":"2026-02-24T07:43:15.659672+0000","mapping_epoch":30,"log_start":"32'2995","ondisk_log_start":"32'2995","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_clean_scrub_stamp":"2026-02-24T07:43:06.832892+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":2995,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:44:21.531086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":196476928,"num_objects":2998,"num_object_clones":0,"num_object_copies":8994,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2998,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2998,"num_write_kb":191872,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[1,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"32'3125","reported_seq":3139,"reported_epoch":32,"state":"active+clean","last_fresh":"2026-02-24T07:43:15.656470+0000","last_change":"2026-02-24T07:43:07.843123+0000","last_active":"2026-02-24T07:43:15.656470+0000","last_peered":"2026-02-24T07:43:15.656470+0000","last_clean":"2026-02-24T07:43:15.656470+0000","last_became_active":"2026-02-24T07:43:07.842864+0000","last_became_peered":"2026-02-24T07:43:07.842864+0000","last_unstale":"2026-02-24T07:43:15.656470+0000","last_undegraded":"2026-02-24T07:43:15.656470+0000","last_fullsized":"2026-02-24T07:43:15.656470+0000","mapping_epoch":30,"log_start":"32'3122","ondisk_log_start":"32'3122","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_clean_scrub_stamp":"2026-02-24T07:43:06.832892+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:44:34.876596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":204800000,"num_objects":3125,"num_object_clones":0,"num_object_copies":9375,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3125,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3125,"num_write_kb":200000,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.4","version":"32'2927","reported_seq":2941,"reported_epoch":32,"state":"active+clean","last_fresh":"2026-02-24T07:43:15.659653+0000","last_change":"2026-02-24T07:43:07.854462+0000","last_active":"2026-02-24T07:43:15.659653+0000","last_peered":"2026-02-24T07:43:15.659653+0000","last_clean":"2026-02-24T07:43:15.659653+0000","last_became_active":"2026-02-24T07:43:07.853966+0000","last_became_peered":"2026-02-24T07:43:07.853966+0000","last_unstale":"2026-02-24T07:43:15.659653+0000","last_undegraded":"2026-02-24T07:43:15.659653+0000","last_fullsized":"2026-02-24T07:43:15.659653+0000","mapping_epoch":30,"log_start":"32'2924","ondisk_log_start":"32'2924","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_clean_scrub_stamp":"2026-02-24T07:43:06.832892+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":2924,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:44:13.031431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":191823872,"num_objects":2927,"num_object_clones":0,"num_object_copies":8781,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2927,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2927,"num_write_kb":187328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"32'2990","reported_seq":3004,"reported_epoch":32,"state":"active+clean","last_fresh":"2026-02-24T07:43:15.662256+0000","last_change":"2026-02-24T07:43:07.854184+0000","last_active":"2026-02-24T07:43:15.662256+0000","last_peered":"2026-02-24T07:43:15.662256+0000","last_clean":"2026-02-24T07:43:15.662256+0000","last_became_active":"2026-02-24T07:43:07.853878+0000","last_became_peered":"2026-02-24T07:43:07.853878+0000","last_unstale":"2026-02-24T07:43:15.662256+0000","last_undegraded":"2026-02-24T07:43:15.662256+0000","last_fullsized":"2026-02-24T07:43:15.662256+0000","mapping_epoch":30,"log_start":"32'2987","ondisk_log_start":"32'2987","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_clean_scrub_stamp":"2026-02-24T07:43:06.832892+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":2987,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:44:31.507398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":195952640,"num_objects":2990,"num_object_clones":0,"num_object_copies":8970,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2990,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2990,"num_write_kb":191360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"32'3045","reported_seq":3059,"reported_epoch":32,"state":"active+clean","last_fresh":"2026-02-24T07:43:15.659483+0000","last_change":"2026-02-24T07:43:07.842180+0000","last_active":"2026-02-24T07:43:15.659483+0000","last_peered":"2026-02-24T07:43:15.659483+0000","last_clean":"2026-02-24T07:43:15.659483+0000","last_became_active":"2026-02-24T07:43:07.841993+0000","last_became_peered":"2026-02-24T07:43:07.841993+0000","last_unstale":"2026-02-24T07:43:15.659483+0000","last_undegraded":"2026-02-24T07:43:15.659483+0000","last_fullsized":"2026-02-24T07:43:15.659483+0000","mapping_epoch":30,"log_start":"32'3042","ondisk_log_start":"32'3042","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_clean_scrub_stamp":"2026-02-24T07:43:06.832892+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:44:19.519502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":199557120,"num_objects":3045,"num_object_clones":0,"num_object_copies":9135,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3045,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3045,"num_write_kb":194880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"32'2939","reported_seq":2953,"reported_epoch":32,"state":"active+clean","last_fresh":"2026-02-24T07:43:15.661636+0000","last_change":"2026-02-24T07:43:07.842375+0000","last_active":"2026-02-24T07:43:15.661636+0000","last_peered":"2026-02-24T07:43:15.661636+0000","last_clean":"2026-02-24T07:43:15.661636+0000","last_became_active":"2026-02-24T07:43:07.841920+0000","last_became_peered":"2026-02-24T07:43:07.841920+0000","last_unstale":"2026-02-24T07:43:15.661636+0000","last_undegraded":"2026-02-24T07:43:15.661636+0000","last_fullsized":"2026-02-24T07:43:15.661636+0000","mapping_epoch":30,"log_start":"32'2936","ondisk_log_start":"32'2936","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_clean_scrub_stamp":"2026-02-24T07:43:06.832892+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":2936,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:44:14.681944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":192610304,"num_objects":2939,"num_object_clones":0,"num_object_copies":8817,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2939,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2939,"num_write_kb":188096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.8","version":"32'2901","reported_seq":2915,"reported_epoch":32,"state":"active+clean","last_fresh":"2026-02-24T07:43:15.653814+0000","last_change":"2026-02-24T07:43:07.853992+0000","last_active":"2026-02-24T07:43:15.653814+0000","last_peered":"2026-02-24T07:43:15.653814+0000","last_clean":"2026-02-24T07:43:15.653814+0000","last_became_active":"2026-02-24T07:43:07.853758+0000","last_became_peered":"2026-02-24T07:43:07.853758+0000","last_unstale":"2026-02-24T07:43:15.653814+0000","last_undegraded":"2026-02-24T07:43:15.653814+0000","last_fullsized":"2026-02-24T07:43:15.653814+0000","mapping_epoch":30,"log_start":"32'2898","ondisk_log_start":"32'2898","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_clean_scrub_stamp":"2026-02-24T07:43:06.832892+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":2898,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:44:27.436119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":190119936,"num_objects":2901,"num_object_clones":0,"num_object_copies":8703,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2901,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2901,"num_write_kb":185664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"32'2945","reported_seq":2959,"reported_epoch":32,"state":"active+clean","last_fresh":"2026-02-24T07:43:15.656482+0000","last_change":"2026-02-24T07:43:07.854687+0000","last_active":"2026-02-24T07:43:15.656482+0000","last_peered":"2026-02-24T07:43:15.656482+0000","last_clean":"2026-02-24T07:43:15.656482+0000","last_became_active":"2026-02-24T07:43:07.854377+0000","last_became_peered":"2026-02-24T07:43:07.854377+0000","last_unstale":"2026-02-24T07:43:15.656482+0000","last_undegraded":"2026-02-24T07:43:15.656482+0000","last_fullsized":"2026-02-24T07:43:15.656482+0000","mapping_epoch":30,"log_start":"32'2942","ondisk_log_start":"32'2942","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_clean_scrub_stamp":"2026-02-24T07:43:06.832892+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":2942,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:44:34.215480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":193003520,"num_objects":2945,"num_object_clones":0,"num_object_copies":8835,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2945,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2945,"num_write_kb":188480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.a","version":"32'2981","reported_seq":2995,"reported_epoch":32,"state":"active+clean","last_fresh":"2026-02-24T07:43:15.657324+0000","last_change":"2026-02-24T07:43:07.841933+0000","last_active":"2026-02-24T07:43:15.657324+0000","last_peered":"2026-02-24T07:43:15.657324+0000","last_clean":"2026-02-24T07:43:15.657324+0000","last_became_active":"2026-02-24T07:43:07.841721+0000","last_became_peered":"2026-02-24T07:43:07.841721+0000","last_unstale":"2026-02-24T07:43:15.657324+0000","last_undegraded":"2026-02-24T07:43:15.657324+0000","last_fullsized":"2026-02-24T07:43:15.657324+0000","mapping_epoch":30,"log_start":"32'2978","ondisk_log_start":"32'2978","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_clean_scrub_stamp":"2026-02-24T07:43:06.832892+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":2978,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:44:14.991691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":195362816,"num_objects":2981,"num_object_clones":0,"num_object_copies":8943,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2981,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2981,"num_write_kb":190784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.b","version":"32'3031","reported_seq":3045,"reported_epoch":32,"state":"active+clean","last_fresh":"2026-02-24T07:43:15.655967+0000","last_change":"2026-02-24T07:43:07.853283+0000","last_active":"2026-02-24T07:43:15.655967+0000","last_peered":"2026-02-24T07:43:15.655967+0000","last_clean":"2026-02-24T07:43:15.655967+0000","last_became_active":"2026-02-24T07:43:07.853069+0000","last_became_peered":"2026-02-24T07:43:07.853069+0000","last_unstale":"2026-02-24T07:43:15.655967+0000","last_undegraded":"2026-02-24T07:43:15.655967+0000","last_fullsized":"2026-02-24T07:43:15.655967+0000","mapping_epoch":30,"log_start":"32'3028","ondisk_log_start":"32'3028","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_clean_scrub_stamp":"2026-02-24T07:43:06.832892+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:44:32.581403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":198639616,"num_objects":3031,"num_object_clones":0,"num_object_copies":9093,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3031,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3031,"num_write_kb":193984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":3119644672,"num_objects":47602,"num_object_clones":0,"num_object_copies":142806,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":47602,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":47602,"num_write_kb":3046528,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":9359523840,"data_stored":9359523840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":48,"ondisk_log_size":48,"up":48,"acting":48,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":240,"num_write":135,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":3,"up_from":25,"seq":107374182407,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":2750828,"kb_used_data":2615576,"kb_used_omap":7,"kb_used_meta":135224,"kb_avail":731252372,"statfs":{"total":751619276800,"available":748802428928,"internally_reserved":0,"allocated":2678349824,"data_stored":2678162216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":7473,"internal_metadata":138470095},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":19,"seq":81604378633,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":1914540,"kb_used_data":1815384,"kb_used_omap":12,"kb_used_meta":99123,"kb_avail":732088660,"statfs":{"total":751619276800,"available":749658787840,"internally_reserved":0,"allocated":1858953216,"data_stored":1858765608,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12692,"internal_metadata":101502572},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542154,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":2189672,"kb_used_data":2054420,"kb_used_omap":14,"kb_used_meta":135217,"kb_avail":731813528,"statfs":{"total":751619276800,"available":749377052672,"internally_reserved":0,"allocated":2103726080,"data_stored":2103542024,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":14645,"internal_metadata":138462923},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705677,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":1985900,"kb_used_data":1886744,"kb_used_omap":18,"kb_used_meta":99117,"kb_avail":732017300,"statfs":{"total":751619276800,"available":749585715200,"internally_reserved":0,"allocated":1932025856,"data_stored":1931838248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":18551,"internal_metadata":101496713},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":2132541440,"data_stored":2132541440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":2345205760,"data_stored":2345205760,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":2144665600,"data_stored":2144665600,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":2737111040,"data_stored":2737111040,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-24T07:43:21.310 INFO:tasks.cephadm.ceph_manager.ceph:increase pool size by 10 2026-02-24T07:43:21.310 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph osd pool set .mgr pg_num 21 2026-02-24T07:43:21.335 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:43:21.333+0000 7f11ff971640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:43:21.336 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:43:21.333+0000 7f11ff971640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:43:21.352 INFO:teuthology.orchestra.run.trial035.stdout:osd.0: {} 2026-02-24T07:43:21.352 INFO:teuthology.orchestra.run.trial035.stderr:osd.0: osd_enable_op_tracker = '' 2026-02-24T07:43:21.370 INFO:teuthology.orchestra.run.trial035.stdout:osd.1: {} 2026-02-24T07:43:21.370 INFO:teuthology.orchestra.run.trial035.stderr:osd.1: osd_enable_op_tracker = '' 2026-02-24T07:43:21.382 INFO:teuthology.orchestra.run.trial035.stdout:osd.2: {} 2026-02-24T07:43:21.382 INFO:teuthology.orchestra.run.trial035.stderr:osd.2: osd_enable_op_tracker = '' 2026-02-24T07:43:21.395 INFO:teuthology.orchestra.run.trial035.stdout:osd.3: {} 2026-02-24T07:43:21.395 INFO:teuthology.orchestra.run.trial035.stderr:osd.3: osd_enable_op_tracker = '' 2026-02-24T07:43:21.473 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:43:21.595 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:43:21.592+0000 7ff2dcfad640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:43:21.595 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:43:21.593+0000 7ff2dcfad640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:43:21.734 INFO:teuthology.orchestra.run.trial035.stderr:Error EPERM: crimson_allow_pg_split is false; pg_num increase denied for crimson pool 2026-02-24T07:43:21.772 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-24T07:43:21.931 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 13 16 87835 87819 422.161 468.625 0.00241398 0.00236756 2026-02-24T07:43:22.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:43:21 trial035 ceph-mon[39284]: from='client.14365 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2026-02-24T07:43:22.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:43:21 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/3225187751' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2026-02-24T07:43:22.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:43:21 trial035 ceph-mon[39284]: pgmap v21: 17 pgs: 17 active+clean; 4.8 GiB data, 14 GiB used, 2.7 TiB / 2.7 TiB avail; 332 MiB/s wr, 5.31k op/s 2026-02-24T07:43:22.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:43:21 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/715905529' entity='client.admin' cmd={"prefix": "osd pool set", "pool": ".mgr", "var": "pg_num", "val": "21"} : dispatch 2026-02-24T07:43:22.931 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 14 16 95138 95122 424.607 456.438 0.00213033 0.00235386 2026-02-24T07:43:23.437 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2026-02-24T07:43:23.574 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:43:23.694 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:43:23.692+0000 7f1b38b4e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:43:23.694 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:43:23.692+0000 7f1b38b4e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:43:23.709 INFO:teuthology.orchestra.run.trial035.stdout:osd.0: {} 2026-02-24T07:43:23.710 INFO:teuthology.orchestra.run.trial035.stderr:osd.0: osd_enable_op_tracker = '' 2026-02-24T07:43:23.723 INFO:teuthology.orchestra.run.trial035.stdout:osd.1: {} 2026-02-24T07:43:23.723 INFO:teuthology.orchestra.run.trial035.stderr:osd.1: osd_enable_op_tracker = '' 2026-02-24T07:43:23.736 INFO:teuthology.orchestra.run.trial035.stdout:osd.2: {} 2026-02-24T07:43:23.736 INFO:teuthology.orchestra.run.trial035.stderr:osd.2: osd_enable_op_tracker = '' 2026-02-24T07:43:23.749 INFO:teuthology.orchestra.run.trial035.stdout:osd.3: {} 2026-02-24T07:43:23.749 INFO:teuthology.orchestra.run.trial035.stderr:osd.3: osd_enable_op_tracker = '' 2026-02-24T07:43:23.931 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 15 16 101909 101893 424.51 423.188 0.00306587 0.00235438 2026-02-24T07:43:24.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:43:23 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/2011196027' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2026-02-24T07:43:24.931 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 16 16 108675 108659 424.406 422.875 0.0026663 0.002355 2026-02-24T07:43:25.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:43:24 trial035 ceph-mon[39284]: pgmap v22: 17 pgs: 17 active+clean; 4.8 GiB data, 14 GiB used, 2.7 TiB / 2.7 TiB avail; 283 MiB/s wr, 4.53k op/s 2026-02-24T07:43:25.792 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2026-02-24T07:43:25.923 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:43:25.931 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 17 16 115078 115062 422.98 400.188 0.00253899 0.00236298 2026-02-24T07:43:26.048 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:43:26.045+0000 7f2da19c0640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:43:26.048 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:43:26.046+0000 7f2da19c0640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:43:26.063 INFO:teuthology.orchestra.run.trial035.stdout:osd.0: {} 2026-02-24T07:43:26.064 INFO:teuthology.orchestra.run.trial035.stderr:osd.0: osd_enable_op_tracker = '' 2026-02-24T07:43:26.080 INFO:teuthology.orchestra.run.trial035.stdout:osd.1: {} 2026-02-24T07:43:26.080 INFO:teuthology.orchestra.run.trial035.stderr:osd.1: osd_enable_op_tracker = '' 2026-02-24T07:43:26.091 INFO:teuthology.orchestra.run.trial035.stdout:osd.2: {} 2026-02-24T07:43:26.091 INFO:teuthology.orchestra.run.trial035.stderr:osd.2: osd_enable_op_tracker = '' 2026-02-24T07:43:26.106 INFO:teuthology.orchestra.run.trial035.stdout:osd.3: {} 2026-02-24T07:43:26.107 INFO:teuthology.orchestra.run.trial035.stderr:osd.3: osd_enable_op_tracker = '' 2026-02-24T07:43:26.773 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2026-02-24T07:43:26.774 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2026-02-24T07:43:26.774 INFO:tasks.thrashosds.thrasher:fixing pg num pool .mgr 2026-02-24T07:43:26.774 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph pg dump --format=json 2026-02-24T07:43:26.915 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:43:26.931 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 18 15 121940 121925 423.309 428.938 0.00132635 0.0023612 2026-02-24T07:43:27.031 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:43:26 trial035 ceph-mon[39284]: pgmap v23: 17 pgs: 17 active+clean; 4.8 GiB data, 16 GiB used, 2.7 TiB / 2.7 TiB avail; 241 MiB/s wr, 3.86k op/s 2026-02-24T07:43:27.031 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:43:26 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/111423304' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2026-02-24T07:43:27.032 INFO:journalctl@ceph.mgr.x.trial035.stdout:Feb 24 07:43:26 trial035 ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-mgr-x[39621]: ::ffff:10.20.193.91 - - [24/Feb/2026:07:43:26] "GET /metrics HTTP/1.1" 200 4213 "" "Prometheus/3.6.0" 2026-02-24T07:43:27.037 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:43:27.034+0000 7f2bcf75a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:43:27.037 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:43:27.035+0000 7f2bcf75a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:43:27.175 INFO:teuthology.orchestra.run.trial035.stdout: 2026-02-24T07:43:27.175 INFO:teuthology.orchestra.run.trial035.stderr:dumped all 2026-02-24T07:43:27.220 INFO:teuthology.orchestra.run.trial035.stdout:{"pg_ready":true,"pg_map":{"version":23,"stamp":"2026-02-24T07:43:25.662355+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":5143134752,"num_objects":78471,"num_object_clones":0,"num_object_copies":235413,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":78471,"num_whiteouts":0,"num_read":136,"num_read_kb":240,"num_write":78604,"num_write_kb":5024166,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":51,"ondisk_log_size":51,"up":51,"acting":51,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":51,"num_osds":4,"num_per_pool_osds":4,"num_per_pool_omap_osds":4,"kb":2936012800,"kb_used":16733024,"kb_used_data":15831056,"kb_used_omap":62,"kb_used_meta":901825,"kb_avail":2919279776,"statfs":{"total":3006477107200,"available":2989342490624,"internally_reserved":0,"allocated":16211001344,"data_stored":16210136279,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":64422,"internal_metadata":923468890},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[9,0,3],"upper_bound":8},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2022899712,"num_objects":30867,"num_object_clones":0,"num_object_copies":92601,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30867,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":30867,"num_write_kb":1975488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.000971"},"pg_stats":[{"pgid":"2.f","version":"36'4708","reported_seq":4730,"reported_epoch":36,"state":"active+clean","last_fresh":"2026-02-24T07:43:20.656431+0000","last_change":"2026-02-24T07:43:07.853034+0000","last_active":"2026-02-24T07:43:20.656431+0000","last_peered":"2026-02-24T07:43:20.656431+0000","last_clean":"2026-02-24T07:43:20.656431+0000","last_became_active":"2026-02-24T07:43:07.852899+0000","last_became_peered":"2026-02-24T07:43:07.852899+0000","last_unstale":"2026-02-24T07:43:20.656431+0000","last_undegraded":"2026-02-24T07:43:20.656431+0000","last_fullsized":"2026-02-24T07:43:20.656431+0000","mapping_epoch":30,"log_start":"36'4705","ondisk_log_start":"36'4705","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_clean_scrub_stamp":"2026-02-24T07:43:06.832892+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:44:18.843618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":308543488,"num_objects":4708,"num_object_clones":0,"num_object_copies":14124,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4708,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4708,"num_write_kb":301312,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.e","version":"36'4889","reported_seq":4911,"reported_epoch":36,"state":"active+clean","last_fresh":"2026-02-24T07:43:20.656242+0000","last_change":"2026-02-24T07:43:07.853009+0000","last_active":"2026-02-24T07:43:20.656242+0000","last_peered":"2026-02-24T07:43:20.656242+0000","last_clean":"2026-02-24T07:43:20.656242+0000","last_became_active":"2026-02-24T07:43:07.852815+0000","last_became_peered":"2026-02-24T07:43:07.852815+0000","last_unstale":"2026-02-24T07:43:20.656242+0000","last_undegraded":"2026-02-24T07:43:20.656242+0000","last_fullsized":"2026-02-24T07:43:20.656242+0000","mapping_epoch":30,"log_start":"36'4886","ondisk_log_start":"36'4886","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_clean_scrub_stamp":"2026-02-24T07:43:06.832892+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:44:11.750665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":320405504,"num_objects":4889,"num_object_clones":0,"num_object_copies":14667,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4889,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4889,"num_write_kb":312896,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.d","version":"36'4975","reported_seq":4997,"reported_epoch":36,"state":"active+clean","last_fresh":"2026-02-24T07:43:20.653847+0000","last_change":"2026-02-24T07:43:07.854403+0000","last_active":"2026-02-24T07:43:20.653847+0000","last_peered":"2026-02-24T07:43:20.653847+0000","last_clean":"2026-02-24T07:43:20.653847+0000","last_became_active":"2026-02-24T07:43:07.853975+0000","last_became_peered":"2026-02-24T07:43:07.853975+0000","last_unstale":"2026-02-24T07:43:20.653847+0000","last_undegraded":"2026-02-24T07:43:20.653847+0000","last_fullsized":"2026-02-24T07:43:20.653847+0000","mapping_epoch":30,"log_start":"36'4972","ondisk_log_start":"36'4972","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_clean_scrub_stamp":"2026-02-24T07:43:06.832892+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:44:14.520704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":326041600,"num_objects":4975,"num_object_clones":0,"num_object_copies":14925,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4975,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4975,"num_write_kb":318400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"36'4807","reported_seq":4827,"reported_epoch":36,"state":"active+clean","last_fresh":"2026-02-24T07:43:20.657827+0000","last_change":"2026-02-24T07:43:07.854780+0000","last_active":"2026-02-24T07:43:20.657827+0000","last_peered":"2026-02-24T07:43:20.657827+0000","last_clean":"2026-02-24T07:43:20.657827+0000","last_became_active":"2026-02-24T07:43:07.854428+0000","last_became_peered":"2026-02-24T07:43:07.854428+0000","last_unstale":"2026-02-24T07:43:20.657827+0000","last_undegraded":"2026-02-24T07:43:20.657827+0000","last_fullsized":"2026-02-24T07:43:20.657827+0000","mapping_epoch":30,"log_start":"36'4804","ondisk_log_start":"36'4804","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_clean_scrub_stamp":"2026-02-24T07:43:06.832892+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:44:16.718883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":315031552,"num_objects":4807,"num_object_clones":0,"num_object_copies":14421,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4807,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4807,"num_write_kb":307648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"29'99","reported_seq":193,"reported_epoch":36,"state":"active+clean","last_fresh":"2026-02-24T07:43:19.857485+0000","last_change":"2026-02-24T07:42:54.186862+0000","last_active":"2026-02-24T07:43:19.857485+0000","last_peered":"2026-02-24T07:43:19.857485+0000","last_clean":"2026-02-24T07:43:19.857485+0000","last_became_active":"2026-02-24T07:42:54.186719+0000","last_became_peered":"2026-02-24T07:42:54.186719+0000","last_unstale":"2026-02-24T07:43:19.857485+0000","last_undegraded":"2026-02-24T07:43:19.857485+0000","last_fullsized":"2026-02-24T07:43:19.857485+0000","mapping_epoch":27,"log_start":"29'96","ondisk_log_start":"29'96","created":20,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:42:43.640776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:42:43.640776+0000","last_clean_scrub_stamp":"2026-02-24T07:42:43.640776+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":96,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:43:57.783028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":240,"num_write":135,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"36'4924","reported_seq":4946,"reported_epoch":36,"state":"active+clean","last_fresh":"2026-02-24T07:43:20.659080+0000","last_change":"2026-02-24T07:43:07.842967+0000","last_active":"2026-02-24T07:43:20.659080+0000","last_peered":"2026-02-24T07:43:20.659080+0000","last_clean":"2026-02-24T07:43:20.659080+0000","last_became_active":"2026-02-24T07:43:07.842530+0000","last_became_peered":"2026-02-24T07:43:07.842530+0000","last_unstale":"2026-02-24T07:43:20.659080+0000","last_undegraded":"2026-02-24T07:43:20.659080+0000","last_fullsized":"2026-02-24T07:43:20.659080+0000","mapping_epoch":30,"log_start":"36'4921","ondisk_log_start":"36'4921","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_clean_scrub_stamp":"2026-02-24T07:43:06.832892+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:44:14.900069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":322699264,"num_objects":4924,"num_object_clones":0,"num_object_copies":14772,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4924,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4924,"num_write_kb":315136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"36'4961","reported_seq":4983,"reported_epoch":36,"state":"active+clean","last_fresh":"2026-02-24T07:43:20.659434+0000","last_change":"2026-02-24T07:43:07.854082+0000","last_active":"2026-02-24T07:43:20.659434+0000","last_peered":"2026-02-24T07:43:20.659434+0000","last_clean":"2026-02-24T07:43:20.659434+0000","last_became_active":"2026-02-24T07:43:07.853885+0000","last_became_peered":"2026-02-24T07:43:07.853885+0000","last_unstale":"2026-02-24T07:43:20.659434+0000","last_undegraded":"2026-02-24T07:43:20.659434+0000","last_fullsized":"2026-02-24T07:43:20.659434+0000","mapping_epoch":30,"log_start":"36'4958","ondisk_log_start":"36'4958","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_clean_scrub_stamp":"2026-02-24T07:43:06.832892+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:44:20.512438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":325124096,"num_objects":4961,"num_object_clones":0,"num_object_copies":14883,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4961,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4961,"num_write_kb":317504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"36'4982","reported_seq":5004,"reported_epoch":36,"state":"active+clean","last_fresh":"2026-02-24T07:43:20.658582+0000","last_change":"2026-02-24T07:43:07.854497+0000","last_active":"2026-02-24T07:43:20.658582+0000","last_peered":"2026-02-24T07:43:20.658582+0000","last_clean":"2026-02-24T07:43:20.658582+0000","last_became_active":"2026-02-24T07:43:07.854212+0000","last_became_peered":"2026-02-24T07:43:07.854212+0000","last_unstale":"2026-02-24T07:43:20.658582+0000","last_undegraded":"2026-02-24T07:43:20.658582+0000","last_fullsized":"2026-02-24T07:43:20.658582+0000","mapping_epoch":30,"log_start":"36'4979","ondisk_log_start":"36'4979","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_clean_scrub_stamp":"2026-02-24T07:43:06.832892+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:44:21.531086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":326500352,"num_objects":4982,"num_object_clones":0,"num_object_copies":14946,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4982,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4982,"num_write_kb":318848,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[1,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"36'5055","reported_seq":5077,"reported_epoch":36,"state":"active+clean","last_fresh":"2026-02-24T07:43:20.659186+0000","last_change":"2026-02-24T07:43:07.843123+0000","last_active":"2026-02-24T07:43:20.659186+0000","last_peered":"2026-02-24T07:43:20.659186+0000","last_clean":"2026-02-24T07:43:20.659186+0000","last_became_active":"2026-02-24T07:43:07.842864+0000","last_became_peered":"2026-02-24T07:43:07.842864+0000","last_unstale":"2026-02-24T07:43:20.659186+0000","last_undegraded":"2026-02-24T07:43:20.659186+0000","last_fullsized":"2026-02-24T07:43:20.659186+0000","mapping_epoch":30,"log_start":"36'5052","ondisk_log_start":"36'5052","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_clean_scrub_stamp":"2026-02-24T07:43:06.832892+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:44:34.876596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":331284480,"num_objects":5055,"num_object_clones":0,"num_object_copies":15165,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5055,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5055,"num_write_kb":323520,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.4","version":"36'4775","reported_seq":4797,"reported_epoch":36,"state":"active+clean","last_fresh":"2026-02-24T07:43:20.653793+0000","last_change":"2026-02-24T07:43:07.854462+0000","last_active":"2026-02-24T07:43:20.653793+0000","last_peered":"2026-02-24T07:43:20.653793+0000","last_clean":"2026-02-24T07:43:20.653793+0000","last_became_active":"2026-02-24T07:43:07.853966+0000","last_became_peered":"2026-02-24T07:43:07.853966+0000","last_unstale":"2026-02-24T07:43:20.653793+0000","last_undegraded":"2026-02-24T07:43:20.653793+0000","last_fullsized":"2026-02-24T07:43:20.653793+0000","mapping_epoch":30,"log_start":"36'4772","ondisk_log_start":"36'4772","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_clean_scrub_stamp":"2026-02-24T07:43:06.832892+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:44:13.031431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":312934400,"num_objects":4775,"num_object_clones":0,"num_object_copies":14325,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4775,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4775,"num_write_kb":305600,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"36'4946","reported_seq":4968,"reported_epoch":36,"state":"active+clean","last_fresh":"2026-02-24T07:43:20.655473+0000","last_change":"2026-02-24T07:43:07.854184+0000","last_active":"2026-02-24T07:43:20.655473+0000","last_peered":"2026-02-24T07:43:20.655473+0000","last_clean":"2026-02-24T07:43:20.655473+0000","last_became_active":"2026-02-24T07:43:07.853878+0000","last_became_peered":"2026-02-24T07:43:07.853878+0000","last_unstale":"2026-02-24T07:43:20.655473+0000","last_undegraded":"2026-02-24T07:43:20.655473+0000","last_fullsized":"2026-02-24T07:43:20.655473+0000","mapping_epoch":30,"log_start":"36'4943","ondisk_log_start":"36'4943","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_clean_scrub_stamp":"2026-02-24T07:43:06.832892+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:44:31.507398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":324141056,"num_objects":4946,"num_object_clones":0,"num_object_copies":14838,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4946,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4946,"num_write_kb":316544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"36'4978","reported_seq":5000,"reported_epoch":36,"state":"active+clean","last_fresh":"2026-02-24T07:43:20.656108+0000","last_change":"2026-02-24T07:43:07.842180+0000","last_active":"2026-02-24T07:43:20.656108+0000","last_peered":"2026-02-24T07:43:20.656108+0000","last_clean":"2026-02-24T07:43:20.656108+0000","last_became_active":"2026-02-24T07:43:07.841993+0000","last_became_peered":"2026-02-24T07:43:07.841993+0000","last_unstale":"2026-02-24T07:43:20.656108+0000","last_undegraded":"2026-02-24T07:43:20.656108+0000","last_fullsized":"2026-02-24T07:43:20.656108+0000","mapping_epoch":30,"log_start":"36'4975","ondisk_log_start":"36'4975","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_clean_scrub_stamp":"2026-02-24T07:43:06.832892+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:44:19.519502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":326238208,"num_objects":4978,"num_object_clones":0,"num_object_copies":14934,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4978,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4978,"num_write_kb":318592,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"36'4787","reported_seq":4809,"reported_epoch":36,"state":"active+clean","last_fresh":"2026-02-24T07:43:20.662213+0000","last_change":"2026-02-24T07:43:07.842375+0000","last_active":"2026-02-24T07:43:20.662213+0000","last_peered":"2026-02-24T07:43:20.662213+0000","last_clean":"2026-02-24T07:43:20.662213+0000","last_became_active":"2026-02-24T07:43:07.841920+0000","last_became_peered":"2026-02-24T07:43:07.841920+0000","last_unstale":"2026-02-24T07:43:20.662213+0000","last_undegraded":"2026-02-24T07:43:20.662213+0000","last_fullsized":"2026-02-24T07:43:20.662213+0000","mapping_epoch":30,"log_start":"36'4784","ondisk_log_start":"36'4784","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_clean_scrub_stamp":"2026-02-24T07:43:06.832892+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:44:14.681944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":313720832,"num_objects":4787,"num_object_clones":0,"num_object_copies":14361,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4787,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4787,"num_write_kb":306368,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.8","version":"36'4862","reported_seq":4884,"reported_epoch":36,"state":"active+clean","last_fresh":"2026-02-24T07:43:20.661445+0000","last_change":"2026-02-24T07:43:07.853992+0000","last_active":"2026-02-24T07:43:20.661445+0000","last_peered":"2026-02-24T07:43:20.661445+0000","last_clean":"2026-02-24T07:43:20.661445+0000","last_became_active":"2026-02-24T07:43:07.853758+0000","last_became_peered":"2026-02-24T07:43:07.853758+0000","last_unstale":"2026-02-24T07:43:20.661445+0000","last_undegraded":"2026-02-24T07:43:20.661445+0000","last_fullsized":"2026-02-24T07:43:20.661445+0000","mapping_epoch":30,"log_start":"36'4859","ondisk_log_start":"36'4859","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_clean_scrub_stamp":"2026-02-24T07:43:06.832892+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:44:27.436119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":318636032,"num_objects":4862,"num_object_clones":0,"num_object_copies":14586,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4862,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4862,"num_write_kb":311168,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"36'4895","reported_seq":4915,"reported_epoch":36,"state":"active+clean","last_fresh":"2026-02-24T07:43:20.656261+0000","last_change":"2026-02-24T07:43:07.854687+0000","last_active":"2026-02-24T07:43:20.656261+0000","last_peered":"2026-02-24T07:43:20.656261+0000","last_clean":"2026-02-24T07:43:20.656261+0000","last_became_active":"2026-02-24T07:43:07.854377+0000","last_became_peered":"2026-02-24T07:43:07.854377+0000","last_unstale":"2026-02-24T07:43:20.656261+0000","last_undegraded":"2026-02-24T07:43:20.656261+0000","last_fullsized":"2026-02-24T07:43:20.656261+0000","mapping_epoch":30,"log_start":"36'4892","ondisk_log_start":"36'4892","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_clean_scrub_stamp":"2026-02-24T07:43:06.832892+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:44:34.215480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":320798720,"num_objects":4895,"num_object_clones":0,"num_object_copies":14685,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4895,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4895,"num_write_kb":313280,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.a","version":"36'4917","reported_seq":4939,"reported_epoch":36,"state":"active+clean","last_fresh":"2026-02-24T07:43:20.651284+0000","last_change":"2026-02-24T07:43:07.841933+0000","last_active":"2026-02-24T07:43:20.651284+0000","last_peered":"2026-02-24T07:43:20.651284+0000","last_clean":"2026-02-24T07:43:20.651284+0000","last_became_active":"2026-02-24T07:43:07.841721+0000","last_became_peered":"2026-02-24T07:43:07.841721+0000","last_unstale":"2026-02-24T07:43:20.651284+0000","last_undegraded":"2026-02-24T07:43:20.651284+0000","last_fullsized":"2026-02-24T07:43:20.651284+0000","mapping_epoch":30,"log_start":"36'4914","ondisk_log_start":"36'4914","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_clean_scrub_stamp":"2026-02-24T07:43:06.832892+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:44:14.991691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":322240512,"num_objects":4917,"num_object_clones":0,"num_object_copies":14751,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4917,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4917,"num_write_kb":314688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.b","version":"36'5008","reported_seq":5030,"reported_epoch":36,"state":"active+clean","last_fresh":"2026-02-24T07:43:20.656564+0000","last_change":"2026-02-24T07:43:07.853283+0000","last_active":"2026-02-24T07:43:20.656564+0000","last_peered":"2026-02-24T07:43:20.656564+0000","last_clean":"2026-02-24T07:43:20.656564+0000","last_became_active":"2026-02-24T07:43:07.853069+0000","last_became_peered":"2026-02-24T07:43:07.853069+0000","last_unstale":"2026-02-24T07:43:20.656564+0000","last_undegraded":"2026-02-24T07:43:20.656564+0000","last_fullsized":"2026-02-24T07:43:20.656564+0000","mapping_epoch":30,"log_start":"36'5005","ondisk_log_start":"36'5005","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_clean_scrub_stamp":"2026-02-24T07:43:06.832892+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:44:32.581403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":328204288,"num_objects":5008,"num_object_clones":0,"num_object_copies":15024,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5008,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5008,"num_write_kb":320512,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":5142544384,"num_objects":78469,"num_object_clones":0,"num_object_copies":235407,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":78469,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":78469,"num_write_kb":5022016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":16993026048,"data_stored":16993026048,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":48,"ondisk_log_size":48,"up":48,"acting":48,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":240,"num_write":135,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":3,"up_from":25,"seq":107374182408,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":4578396,"kb_used_data":4334856,"kb_used_omap":11,"kb_used_meta":243508,"kb_avail":729424804,"statfs":{"total":751619276800,"available":746930999296,"internally_reserved":0,"allocated":4438892544,"data_stored":4438673420,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":11373,"internal_metadata":249353107},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":19,"seq":81604378634,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":3331344,"kb_used_data":3159996,"kb_used_omap":14,"kb_used_meta":171313,"kb_avail":730671856,"statfs":{"total":751619276800,"available":748207980544,"internally_reserved":0,"allocated":3235835904,"data_stored":3235624659,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":14645,"internal_metadata":175425227},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542155,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":3757208,"kb_used_data":3549764,"kb_used_omap":16,"kb_used_meta":207407,"kb_avail":730245992,"statfs":{"total":751619276800,"available":747771895808,"internally_reserved":0,"allocated":3634958336,"data_stored":3634742764,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":17249,"internal_metadata":212384927},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[9,0,3],"upper_bound":8},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705679,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":5066076,"kb_used_data":4786440,"kb_used_omap":20,"kb_used_meta":279595,"kb_avail":728937124,"statfs":{"total":751619276800,"available":746431614976,"internally_reserved":0,"allocated":4901314560,"data_stored":4901095436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":21155,"internal_metadata":286305629},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":5091164160,"data_stored":5091164160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":3862888448,"data_stored":3862888448,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":3529244672,"data_stored":3529244672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4509728768,"data_stored":4509728768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-24T07:43:27.222 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph osd pool set .mgr pgp_num 21 2026-02-24T07:43:27.352 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:43:27.356 INFO:journalctl@ceph.prometheus.a.trial091.stdout:Feb 24 07:43:26 trial091 ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-prometheus-a[66007]: time=2026-02-24T07:43:26.973Z level=ERROR source=scrape.go:1630 msg="Failed to determine correct type of scrape target." component="scrape manager" scrape_pool=ceph target=http://trial035:9283/metrics content_type="text/html;charset=utf-8" fallback_media_type="" err="received unsupported Content-Type \"text/html;charset=utf-8\" and no fallback_scrape_protocol specified for target" 2026-02-24T07:43:27.481 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:43:27.478+0000 7f46325fa640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:43:27.481 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:43:27.479+0000 7f46325fa640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:43:27.610 INFO:teuthology.orchestra.run.trial035.stderr:Error EPERM: crimson_allow_pg_split is false; pgp_num increase denied 2026-02-24T07:43:27.646 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-24T07:43:27.932 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 19 15 128829 128814 423.689 430.562 0.00395483 0.00235896 2026-02-24T07:43:27.954 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:43:27 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/1362063515' entity='client.admin' cmd={"prefix": "osd pool set", "pool": ".mgr", "var": "pgp_num", "val": "21"} : dispatch 2026-02-24T07:43:28.144 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2026-02-24T07:43:28.277 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:43:28.406 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:43:28.404+0000 7fc7f282a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:43:28.406 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:43:28.405+0000 7fc7f282a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:43:28.420 INFO:teuthology.orchestra.run.trial035.stdout:osd.0: {} 2026-02-24T07:43:28.420 INFO:teuthology.orchestra.run.trial035.stderr:osd.0: osd_enable_op_tracker = '' 2026-02-24T07:43:28.433 INFO:teuthology.orchestra.run.trial035.stdout:osd.1: {} 2026-02-24T07:43:28.433 INFO:teuthology.orchestra.run.trial035.stderr:osd.1: osd_enable_op_tracker = '' 2026-02-24T07:43:28.445 INFO:teuthology.orchestra.run.trial035.stdout:osd.2: {} 2026-02-24T07:43:28.446 INFO:teuthology.orchestra.run.trial035.stderr:osd.2: osd_enable_op_tracker = '' 2026-02-24T07:43:28.459 INFO:teuthology.orchestra.run.trial035.stdout:osd.3: {} 2026-02-24T07:43:28.459 INFO:teuthology.orchestra.run.trial035.stderr:osd.3: osd_enable_op_tracker = '' 2026-02-24T07:43:28.932 INFO:tasks.radosbench.radosbench.0.trial035.stdout:2026-02-24T07:43:28.931288+0000 min lat: 0.000700241 max lat: 0.0413934 avg lat: 0.00235256 2026-02-24T07:43:28.932 INFO:tasks.radosbench.radosbench.0.trial035.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2026-02-24T07:43:28.932 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 20 16 135982 135966 424.852 447 0.00127168 0.00235256 2026-02-24T07:43:29.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:43:28 trial035 ceph-mon[39284]: from='client.14375 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2026-02-24T07:43:29.205 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:43:28 trial035 ceph-mon[39284]: pgmap v24: 17 pgs: 17 active+clean; 6.9 GiB data, 21 GiB used, 2.7 TiB / 2.7 TiB avail; 419 MiB/s wr, 6.70k op/s 2026-02-24T07:43:29.205 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:43:28 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/2018056792' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2026-02-24T07:43:29.931 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 21 16 143829 143813 427.974 490.438 0.00146189 0.00233542 2026-02-24T07:43:30.498 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2026-02-24T07:43:30.639 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:43:30.750 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:43:30.749+0000 7faa0fb9e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:43:30.750 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:43:30.749+0000 7faa0fb9e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:43:30.767 INFO:teuthology.orchestra.run.trial035.stdout:osd.0: {} 2026-02-24T07:43:30.767 INFO:teuthology.orchestra.run.trial035.stderr:osd.0: osd_enable_op_tracker = '' 2026-02-24T07:43:30.780 INFO:teuthology.orchestra.run.trial035.stdout:osd.1: {} 2026-02-24T07:43:30.780 INFO:teuthology.orchestra.run.trial035.stderr:osd.1: osd_enable_op_tracker = '' 2026-02-24T07:43:30.780 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:43:30 trial035 ceph-mon[39284]: pgmap v25: 17 pgs: 17 active+clean; 6.9 GiB data, 21 GiB used, 2.7 TiB / 2.7 TiB avail; 411 MiB/s wr, 6.57k op/s 2026-02-24T07:43:30.780 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:43:30 trial035 ceph-mon[39284]: from='mgr.14284 10.20.193.35:0/3681373222' entity='mgr.x' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2026-02-24T07:43:30.792 INFO:teuthology.orchestra.run.trial035.stdout:osd.2: {} 2026-02-24T07:43:30.792 INFO:teuthology.orchestra.run.trial035.stderr:osd.2: osd_enable_op_tracker = '' 2026-02-24T07:43:30.804 INFO:teuthology.orchestra.run.trial035.stdout:osd.3: {} 2026-02-24T07:43:30.804 INFO:teuthology.orchestra.run.trial035.stderr:osd.3: osd_enable_op_tracker = '' 2026-02-24T07:43:30.931 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 22 16 150884 150868 428.561 440.938 0.00112091 0.00233212 2026-02-24T07:43:31.932 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 23 15 157683 157668 428.405 425 0.00262276 0.00233304 2026-02-24T07:43:32.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:43:31 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/2193085795' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2026-02-24T07:43:32.648 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2026-02-24T07:43:32.648 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph osd reweight 0 1 2026-02-24T07:43:32.782 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:43:32.846 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2026-02-24T07:43:32.910 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:43:32.909+0000 7f9704ed4640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:43:32.910 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:43:32.909+0000 7f9704ed4640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:43:32.931 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 24 16 164597 164581 428.556 432.062 0.00289122 0.00233228 2026-02-24T07:43:32.954 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:43:32 trial035 ceph-mon[39284]: pgmap v26: 17 pgs: 17 active+clean; 9.1 GiB data, 28 GiB used, 2.7 TiB / 2.7 TiB avail; 537 MiB/s wr, 8.59k op/s 2026-02-24T07:43:32.981 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:43:33.096 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:43:33.094+0000 7f6dbd532640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:43:33.096 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:43:33.095+0000 7f6dbd532640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:43:33.109 INFO:teuthology.orchestra.run.trial035.stdout:osd.0: {} 2026-02-24T07:43:33.110 INFO:teuthology.orchestra.run.trial035.stderr:osd.0: osd_enable_op_tracker = '' 2026-02-24T07:43:33.123 INFO:teuthology.orchestra.run.trial035.stdout:osd.1: {} 2026-02-24T07:43:33.123 INFO:teuthology.orchestra.run.trial035.stderr:osd.1: osd_enable_op_tracker = '' 2026-02-24T07:43:33.135 INFO:teuthology.orchestra.run.trial035.stdout:osd.2: {} 2026-02-24T07:43:33.135 INFO:teuthology.orchestra.run.trial035.stderr:osd.2: osd_enable_op_tracker = '' 2026-02-24T07:43:33.147 INFO:teuthology.orchestra.run.trial035.stdout:osd.3: {} 2026-02-24T07:43:33.148 INFO:teuthology.orchestra.run.trial035.stderr:osd.3: osd_enable_op_tracker = '' 2026-02-24T07:43:33.757 INFO:teuthology.orchestra.run.trial035.stderr:reweighted osd.0 to 1 (10000) 2026-02-24T07:43:33.800 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph osd reweight 1 1 2026-02-24T07:43:33.932 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 25 16 171364 171348 428.33 422.938 0.00211568 0.00233351 2026-02-24T07:43:33.939 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:43:34.007 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:43:33 trial035 ceph-mon[39284]: purged_snaps scrub starts 2026-02-24T07:43:34.007 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:43:33 trial035 ceph-mon[39284]: purged_snaps scrub ok 2026-02-24T07:43:34.007 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:43:33 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/3853460047' entity='client.admin' cmd={"prefix": "osd reweight", "id": 0, "weight": 1.0} : dispatch 2026-02-24T07:43:34.007 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:43:33 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/1401506171' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2026-02-24T07:43:34.059 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:43:34.057+0000 7fafd26db640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:43:34.059 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:43:34.058+0000 7fafd26db640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:43:34.757 INFO:teuthology.orchestra.run.trial035.stderr:reweighted osd.1 to 1 (10000) 2026-02-24T07:43:34.799 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph osd reweight 2 1 2026-02-24T07:43:34.932 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 26 16 178357 178341 428.665 437.062 0.00190285 0.00233164 2026-02-24T07:43:34.938 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:43:35.054 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:43:34 trial035 ceph-mon[39284]: pgmap v27: 17 pgs: 17 active+clean; 9.1 GiB data, 28 GiB used, 2.7 TiB / 2.7 TiB avail; 368 MiB/s wr, 5.88k op/s 2026-02-24T07:43:35.055 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:43:34 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/3853460047' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 0, "weight": 1.0}]': finished 2026-02-24T07:43:35.055 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:43:34 trial035 ceph-mon[39284]: osdmap e37: 4 total, 4 up, 4 in 2026-02-24T07:43:35.055 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:43:34 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/3471184151' entity='client.admin' cmd={"prefix": "osd reweight", "id": 1, "weight": 1.0} : dispatch 2026-02-24T07:43:35.191 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2026-02-24T07:43:35.358 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:43:35.345+0000 7f8be2cd8640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:43:35.358 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:43:35.345+0000 7f8be2cd8640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:43:35.374 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:43:35.762 INFO:teuthology.orchestra.run.trial035.stderr:reweighted osd.2 to 1 (10000) 2026-02-24T07:43:35.855 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:43:35.808+0000 7f3022f07640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:43:35.855 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:43:35.808+0000 7f3022f07640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:43:35.856 INFO:teuthology.orchestra.run.trial035.stderr:osd.0: osd_enable_op_tracker = '' 2026-02-24T07:43:35.856 INFO:teuthology.orchestra.run.trial035.stdout:osd.0: {} 2026-02-24T07:43:35.856 INFO:teuthology.orchestra.run.trial035.stdout:osd.1: {} 2026-02-24T07:43:35.856 INFO:teuthology.orchestra.run.trial035.stderr:osd.1: osd_enable_op_tracker = '' 2026-02-24T07:43:35.860 INFO:teuthology.orchestra.run.trial035.stdout:osd.2: {} 2026-02-24T07:43:35.860 INFO:teuthology.orchestra.run.trial035.stderr:osd.2: osd_enable_op_tracker = '' 2026-02-24T07:43:35.872 INFO:teuthology.orchestra.run.trial035.stdout:osd.3: {} 2026-02-24T07:43:35.872 INFO:teuthology.orchestra.run.trial035.stderr:osd.3: osd_enable_op_tracker = '' 2026-02-24T07:43:35.932 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 27 16 182930 182914 423.373 285.812 0.00236647 0.00235843 2026-02-24T07:43:35.954 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:43:35 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/3471184151' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 1, "weight": 1.0}]': finished 2026-02-24T07:43:35.954 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:43:35 trial035 ceph-mon[39284]: osdmap e38: 4 total, 4 up, 4 in 2026-02-24T07:43:35.954 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:43:35 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/2614151183' entity='client.admin' cmd={"prefix": "osd reweight", "id": 2, "weight": 1.0} : dispatch 2026-02-24T07:43:36.528 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph osd reweight 3 1 2026-02-24T07:43:36.932 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 28 16 187545 187529 418.553 288.438 0.00150012 0.00238801 2026-02-24T07:43:37.002 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:43:37.138 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:43:36 trial035 ceph-mon[39284]: pgmap v30: 17 pgs: 17 active+clean; 9.2 GiB data, 31 GiB used, 2.7 TiB / 2.7 TiB avail; 295 MiB/s wr, 4.72k op/s 2026-02-24T07:43:37.138 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:43:36 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/2614151183' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 2, "weight": 1.0}]': finished 2026-02-24T07:43:37.138 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:43:36 trial035 ceph-mon[39284]: osdmap e39: 4 total, 4 up, 4 in 2026-02-24T07:43:37.138 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:43:36 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/1376839219' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2026-02-24T07:43:37.139 INFO:journalctl@ceph.mgr.x.trial035.stdout:Feb 24 07:43:36 trial035 ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-mgr-x[39621]: ::ffff:10.20.193.91 - - [24/Feb/2026:07:43:36] "GET /metrics HTTP/1.1" 200 4211 "" "Prometheus/3.6.0" 2026-02-24T07:43:37.356 INFO:journalctl@ceph.prometheus.a.trial091.stdout:Feb 24 07:43:36 trial091 ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-prometheus-a[66007]: time=2026-02-24T07:43:36.973Z level=ERROR source=scrape.go:1630 msg="Failed to determine correct type of scrape target." component="scrape manager" scrape_pool=ceph target=http://trial035:9283/metrics content_type="text/html;charset=utf-8" fallback_media_type="" err="received unsupported Content-Type \"text/html;charset=utf-8\" and no fallback_scrape_protocol specified for target" 2026-02-24T07:43:37.670 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:43:37.599+0000 7f5fd2dda640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:43:37.671 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:43:37.599+0000 7f5fd2dda640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:43:37.765 INFO:teuthology.orchestra.run.trial035.stderr:reweighted osd.3 to 1 (10000) 2026-02-24T07:43:37.932 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 29 16 193241 193225 416.396 356 0.00190824 0.00240039 2026-02-24T07:43:37.954 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:43:37 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/3538172453' entity='client.admin' cmd={"prefix": "osd reweight", "id": 3, "weight": 1.0} : dispatch 2026-02-24T07:43:38.576 INFO:tasks.cephadm.ceph_manager.ceph:waiting for recovery to complete 2026-02-24T07:43:38.577 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph pg dump --format=json 2026-02-24T07:43:38.723 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:43:38.840 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:43:38 trial035 ceph-mon[39284]: pgmap v32: 17 pgs: 17 active+clean; 11 GiB data, 35 GiB used, 2.7 TiB / 2.7 TiB avail; 340 MiB/s wr, 5.45k op/s 2026-02-24T07:43:38.840 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:43:38 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/3538172453' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 3, "weight": 1.0}]': finished 2026-02-24T07:43:38.840 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:43:38 trial035 ceph-mon[39284]: osdmap e40: 4 total, 4 up, 4 in 2026-02-24T07:43:38.932 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 30 16 199148 199132 414.821 369.188 0.001155 0.00240956 2026-02-24T07:43:39.159 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2026-02-24T07:43:39.326 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:43:39.250+0000 7f94c329c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:43:39.326 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:43:39.250+0000 7f94c329c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:43:39.349 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:43:39.389 INFO:teuthology.orchestra.run.trial035.stdout: 2026-02-24T07:43:39.390 INFO:teuthology.orchestra.run.trial035.stderr:dumped all 2026-02-24T07:43:39.932 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 31 16 204562 204546 412.354 338.375 0.000971143 0.00242394 2026-02-24T07:43:40.097 INFO:teuthology.orchestra.run.trial035.stdout:{"pg_ready":true,"pg_map":{"version":33,"stamp":"2026-02-24T07:43:37.764730+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":11911823904,"num_objects":181753,"num_object_clones":0,"num_object_copies":545259,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":181753,"num_whiteouts":0,"num_read":136,"num_read_kb":240,"num_write":181886,"num_write_kb":11634214,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":51,"ondisk_log_size":51,"up":51,"acting":51,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":51,"num_osds":4,"num_per_pool_osds":4,"num_per_pool_omap_osds":4,"kb":2936012800,"kb_used":36448000,"kb_used_data":34390896,"kb_used_omap":68,"kb_used_meta":2056891,"kb_avail":2899564800,"statfs":{"total":3006477107200,"available":2969154355200,"internally_reserved":0,"allocated":35216277504,"data_stored":35215349407,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":70296,"internal_metadata":2106256744},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[10,0,4],"upper_bound":8},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2142437376,"num_objects":32691,"num_object_clones":0,"num_object_copies":98073,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32691,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":32691,"num_write_kb":2092224,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.101351"},"pg_stats":[{"pgid":"2.f","version":"38'11131","reported_seq":11157,"reported_epoch":38,"state":"active+clean","last_fresh":"2026-02-24T07:43:35.655469+0000","last_change":"2026-02-24T07:43:07.853034+0000","last_active":"2026-02-24T07:43:35.655469+0000","last_peered":"2026-02-24T07:43:35.655469+0000","last_clean":"2026-02-24T07:43:35.655469+0000","last_became_active":"2026-02-24T07:43:07.852899+0000","last_became_peered":"2026-02-24T07:43:07.852899+0000","last_unstale":"2026-02-24T07:43:35.655469+0000","last_undegraded":"2026-02-24T07:43:35.655469+0000","last_fullsized":"2026-02-24T07:43:35.655469+0000","mapping_epoch":30,"log_start":"38'11128","ondisk_log_start":"38'11128","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_clean_scrub_stamp":"2026-02-24T07:43:06.832892+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:44:18.843618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":729481216,"num_objects":11131,"num_object_clones":0,"num_object_copies":33393,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11131,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11131,"num_write_kb":712384,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.e","version":"38'11437","reported_seq":11463,"reported_epoch":38,"state":"active+clean","last_fresh":"2026-02-24T07:43:35.661075+0000","last_change":"2026-02-24T07:43:07.853009+0000","last_active":"2026-02-24T07:43:35.661075+0000","last_peered":"2026-02-24T07:43:35.661075+0000","last_clean":"2026-02-24T07:43:35.661075+0000","last_became_active":"2026-02-24T07:43:07.852815+0000","last_became_peered":"2026-02-24T07:43:07.852815+0000","last_unstale":"2026-02-24T07:43:35.661075+0000","last_undegraded":"2026-02-24T07:43:35.661075+0000","last_fullsized":"2026-02-24T07:43:35.661075+0000","mapping_epoch":30,"log_start":"38'11434","ondisk_log_start":"38'11434","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_clean_scrub_stamp":"2026-02-24T07:43:06.832892+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:44:11.750665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":749535232,"num_objects":11437,"num_object_clones":0,"num_object_copies":34311,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11437,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11437,"num_write_kb":731968,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.d","version":"38'11494","reported_seq":11520,"reported_epoch":38,"state":"active+clean","last_fresh":"2026-02-24T07:43:35.658803+0000","last_change":"2026-02-24T07:43:07.854403+0000","last_active":"2026-02-24T07:43:35.658803+0000","last_peered":"2026-02-24T07:43:35.658803+0000","last_clean":"2026-02-24T07:43:35.658803+0000","last_became_active":"2026-02-24T07:43:07.853975+0000","last_became_peered":"2026-02-24T07:43:07.853975+0000","last_unstale":"2026-02-24T07:43:35.658803+0000","last_undegraded":"2026-02-24T07:43:35.658803+0000","last_fullsized":"2026-02-24T07:43:35.658803+0000","mapping_epoch":30,"log_start":"38'11491","ondisk_log_start":"38'11491","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_clean_scrub_stamp":"2026-02-24T07:43:06.832892+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:44:14.520704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":753270784,"num_objects":11494,"num_object_clones":0,"num_object_copies":34482,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11494,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11494,"num_write_kb":735616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"38'11334","reported_seq":11359,"reported_epoch":38,"state":"active+clean","last_fresh":"2026-02-24T07:43:35.663918+0000","last_change":"2026-02-24T07:43:07.854780+0000","last_active":"2026-02-24T07:43:35.663918+0000","last_peered":"2026-02-24T07:43:35.663918+0000","last_clean":"2026-02-24T07:43:35.663918+0000","last_became_active":"2026-02-24T07:43:07.854428+0000","last_became_peered":"2026-02-24T07:43:07.854428+0000","last_unstale":"2026-02-24T07:43:35.663918+0000","last_undegraded":"2026-02-24T07:43:35.663918+0000","last_fullsized":"2026-02-24T07:43:35.663918+0000","mapping_epoch":30,"log_start":"38'11331","ondisk_log_start":"38'11331","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_clean_scrub_stamp":"2026-02-24T07:43:06.832892+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:44:16.718883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":742785024,"num_objects":11334,"num_object_clones":0,"num_object_copies":34002,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11334,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11334,"num_write_kb":725376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"29'99","reported_seq":197,"reported_epoch":38,"state":"active+clean","last_fresh":"2026-02-24T07:43:34.757645+0000","last_change":"2026-02-24T07:42:54.186862+0000","last_active":"2026-02-24T07:43:34.757645+0000","last_peered":"2026-02-24T07:43:34.757645+0000","last_clean":"2026-02-24T07:43:34.757645+0000","last_became_active":"2026-02-24T07:42:54.186719+0000","last_became_peered":"2026-02-24T07:42:54.186719+0000","last_unstale":"2026-02-24T07:43:34.757645+0000","last_undegraded":"2026-02-24T07:43:34.757645+0000","last_fullsized":"2026-02-24T07:43:34.757645+0000","mapping_epoch":27,"log_start":"29'96","ondisk_log_start":"29'96","created":20,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:42:43.640776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:42:43.640776+0000","last_clean_scrub_stamp":"2026-02-24T07:42:43.640776+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":96,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:43:57.783028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":240,"num_write":135,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"38'11407","reported_seq":11433,"reported_epoch":38,"state":"active+clean","last_fresh":"2026-02-24T07:43:35.663121+0000","last_change":"2026-02-24T07:43:07.842967+0000","last_active":"2026-02-24T07:43:35.663121+0000","last_peered":"2026-02-24T07:43:35.663121+0000","last_clean":"2026-02-24T07:43:35.663121+0000","last_became_active":"2026-02-24T07:43:07.842530+0000","last_became_peered":"2026-02-24T07:43:07.842530+0000","last_unstale":"2026-02-24T07:43:35.663121+0000","last_undegraded":"2026-02-24T07:43:35.663121+0000","last_fullsized":"2026-02-24T07:43:35.663121+0000","mapping_epoch":30,"log_start":"38'11404","ondisk_log_start":"38'11404","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_clean_scrub_stamp":"2026-02-24T07:43:06.832892+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:44:14.900069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":747569152,"num_objects":11407,"num_object_clones":0,"num_object_copies":34221,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11407,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11407,"num_write_kb":730048,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"38'11434","reported_seq":11460,"reported_epoch":38,"state":"active+clean","last_fresh":"2026-02-24T07:43:35.661359+0000","last_change":"2026-02-24T07:43:07.854082+0000","last_active":"2026-02-24T07:43:35.661359+0000","last_peered":"2026-02-24T07:43:35.661359+0000","last_clean":"2026-02-24T07:43:35.661359+0000","last_became_active":"2026-02-24T07:43:07.853885+0000","last_became_peered":"2026-02-24T07:43:07.853885+0000","last_unstale":"2026-02-24T07:43:35.661359+0000","last_undegraded":"2026-02-24T07:43:35.661359+0000","last_fullsized":"2026-02-24T07:43:35.661359+0000","mapping_epoch":30,"log_start":"38'11431","ondisk_log_start":"38'11431","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_clean_scrub_stamp":"2026-02-24T07:43:06.832892+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:44:20.512438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":749338624,"num_objects":11434,"num_object_clones":0,"num_object_copies":34302,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11434,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11434,"num_write_kb":731776,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"38'11365","reported_seq":11391,"reported_epoch":38,"state":"active+clean","last_fresh":"2026-02-24T07:43:35.660812+0000","last_change":"2026-02-24T07:43:07.854497+0000","last_active":"2026-02-24T07:43:35.660812+0000","last_peered":"2026-02-24T07:43:35.660812+0000","last_clean":"2026-02-24T07:43:35.660812+0000","last_became_active":"2026-02-24T07:43:07.854212+0000","last_became_peered":"2026-02-24T07:43:07.854212+0000","last_unstale":"2026-02-24T07:43:35.660812+0000","last_undegraded":"2026-02-24T07:43:35.660812+0000","last_fullsized":"2026-02-24T07:43:35.660812+0000","mapping_epoch":30,"log_start":"38'11362","ondisk_log_start":"38'11362","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_clean_scrub_stamp":"2026-02-24T07:43:06.832892+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:44:21.531086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":744816640,"num_objects":11365,"num_object_clones":0,"num_object_copies":34095,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11365,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11365,"num_write_kb":727360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[1,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"38'11563","reported_seq":11589,"reported_epoch":38,"state":"active+clean","last_fresh":"2026-02-24T07:43:35.652634+0000","last_change":"2026-02-24T07:43:07.843123+0000","last_active":"2026-02-24T07:43:35.652634+0000","last_peered":"2026-02-24T07:43:35.652634+0000","last_clean":"2026-02-24T07:43:35.652634+0000","last_became_active":"2026-02-24T07:43:07.842864+0000","last_became_peered":"2026-02-24T07:43:07.842864+0000","last_unstale":"2026-02-24T07:43:35.652634+0000","last_undegraded":"2026-02-24T07:43:35.652634+0000","last_fullsized":"2026-02-24T07:43:35.652634+0000","mapping_epoch":30,"log_start":"38'11560","ondisk_log_start":"38'11560","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_clean_scrub_stamp":"2026-02-24T07:43:06.832892+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:44:34.876596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":757792768,"num_objects":11563,"num_object_clones":0,"num_object_copies":34689,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11563,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11563,"num_write_kb":740032,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.4","version":"38'11245","reported_seq":11271,"reported_epoch":38,"state":"active+clean","last_fresh":"2026-02-24T07:43:35.658277+0000","last_change":"2026-02-24T07:43:07.854462+0000","last_active":"2026-02-24T07:43:35.658277+0000","last_peered":"2026-02-24T07:43:35.658277+0000","last_clean":"2026-02-24T07:43:35.658277+0000","last_became_active":"2026-02-24T07:43:07.853966+0000","last_became_peered":"2026-02-24T07:43:07.853966+0000","last_unstale":"2026-02-24T07:43:35.658277+0000","last_undegraded":"2026-02-24T07:43:35.658277+0000","last_fullsized":"2026-02-24T07:43:35.658277+0000","mapping_epoch":30,"log_start":"38'11242","ondisk_log_start":"38'11242","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_clean_scrub_stamp":"2026-02-24T07:43:06.832892+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:44:13.031431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":736952320,"num_objects":11245,"num_object_clones":0,"num_object_copies":33735,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11245,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11245,"num_write_kb":719680,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"38'11430","reported_seq":11456,"reported_epoch":38,"state":"active+clean","last_fresh":"2026-02-24T07:43:35.652962+0000","last_change":"2026-02-24T07:43:07.854184+0000","last_active":"2026-02-24T07:43:35.652962+0000","last_peered":"2026-02-24T07:43:35.652962+0000","last_clean":"2026-02-24T07:43:35.652962+0000","last_became_active":"2026-02-24T07:43:07.853878+0000","last_became_peered":"2026-02-24T07:43:07.853878+0000","last_unstale":"2026-02-24T07:43:35.652962+0000","last_undegraded":"2026-02-24T07:43:35.652962+0000","last_fullsized":"2026-02-24T07:43:35.652962+0000","mapping_epoch":30,"log_start":"38'11427","ondisk_log_start":"38'11427","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_clean_scrub_stamp":"2026-02-24T07:43:06.832892+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:44:31.507398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":749076480,"num_objects":11430,"num_object_clones":0,"num_object_copies":34290,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11430,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11430,"num_write_kb":731520,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"38'11444","reported_seq":11470,"reported_epoch":38,"state":"active+clean","last_fresh":"2026-02-24T07:43:35.660446+0000","last_change":"2026-02-24T07:43:07.842180+0000","last_active":"2026-02-24T07:43:35.660446+0000","last_peered":"2026-02-24T07:43:35.660446+0000","last_clean":"2026-02-24T07:43:35.660446+0000","last_became_active":"2026-02-24T07:43:07.841993+0000","last_became_peered":"2026-02-24T07:43:07.841993+0000","last_unstale":"2026-02-24T07:43:35.660446+0000","last_undegraded":"2026-02-24T07:43:35.660446+0000","last_fullsized":"2026-02-24T07:43:35.660446+0000","mapping_epoch":30,"log_start":"38'11441","ondisk_log_start":"38'11441","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_clean_scrub_stamp":"2026-02-24T07:43:06.832892+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:44:19.519502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":749993984,"num_objects":11444,"num_object_clones":0,"num_object_copies":34332,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11444,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11444,"num_write_kb":732416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"38'11085","reported_seq":11111,"reported_epoch":38,"state":"active+clean","last_fresh":"2026-02-24T07:43:35.659003+0000","last_change":"2026-02-24T07:43:07.842375+0000","last_active":"2026-02-24T07:43:35.659003+0000","last_peered":"2026-02-24T07:43:35.659003+0000","last_clean":"2026-02-24T07:43:35.659003+0000","last_became_active":"2026-02-24T07:43:07.841920+0000","last_became_peered":"2026-02-24T07:43:07.841920+0000","last_unstale":"2026-02-24T07:43:35.659003+0000","last_undegraded":"2026-02-24T07:43:35.659003+0000","last_fullsized":"2026-02-24T07:43:35.659003+0000","mapping_epoch":30,"log_start":"38'11082","ondisk_log_start":"38'11082","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_clean_scrub_stamp":"2026-02-24T07:43:06.832892+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:44:14.681944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":726466560,"num_objects":11085,"num_object_clones":0,"num_object_copies":33255,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11085,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11085,"num_write_kb":709440,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.8","version":"38'11258","reported_seq":11284,"reported_epoch":38,"state":"active+clean","last_fresh":"2026-02-24T07:43:35.658362+0000","last_change":"2026-02-24T07:43:07.853992+0000","last_active":"2026-02-24T07:43:35.658362+0000","last_peered":"2026-02-24T07:43:35.658362+0000","last_clean":"2026-02-24T07:43:35.658362+0000","last_became_active":"2026-02-24T07:43:07.853758+0000","last_became_peered":"2026-02-24T07:43:07.853758+0000","last_unstale":"2026-02-24T07:43:35.658362+0000","last_undegraded":"2026-02-24T07:43:35.658362+0000","last_fullsized":"2026-02-24T07:43:35.658362+0000","mapping_epoch":30,"log_start":"38'11255","ondisk_log_start":"38'11255","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_clean_scrub_stamp":"2026-02-24T07:43:06.832892+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:44:27.436119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":737804288,"num_objects":11258,"num_object_clones":0,"num_object_copies":33774,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11258,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11258,"num_write_kb":720512,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"38'11250","reported_seq":11276,"reported_epoch":38,"state":"active+clean","last_fresh":"2026-02-24T07:43:35.650870+0000","last_change":"2026-02-24T07:43:07.854687+0000","last_active":"2026-02-24T07:43:35.650870+0000","last_peered":"2026-02-24T07:43:35.650870+0000","last_clean":"2026-02-24T07:43:35.650870+0000","last_became_active":"2026-02-24T07:43:07.854377+0000","last_became_peered":"2026-02-24T07:43:07.854377+0000","last_unstale":"2026-02-24T07:43:35.650870+0000","last_undegraded":"2026-02-24T07:43:35.650870+0000","last_fullsized":"2026-02-24T07:43:35.650870+0000","mapping_epoch":30,"log_start":"38'11247","ondisk_log_start":"38'11247","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_clean_scrub_stamp":"2026-02-24T07:43:06.832892+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:44:34.215480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":737280000,"num_objects":11250,"num_object_clones":0,"num_object_copies":33750,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11250,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11250,"num_write_kb":720000,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.a","version":"38'11455","reported_seq":11481,"reported_epoch":38,"state":"active+clean","last_fresh":"2026-02-24T07:43:35.657171+0000","last_change":"2026-02-24T07:43:07.841933+0000","last_active":"2026-02-24T07:43:35.657171+0000","last_peered":"2026-02-24T07:43:35.657171+0000","last_clean":"2026-02-24T07:43:35.657171+0000","last_became_active":"2026-02-24T07:43:07.841721+0000","last_became_peered":"2026-02-24T07:43:07.841721+0000","last_unstale":"2026-02-24T07:43:35.657171+0000","last_undegraded":"2026-02-24T07:43:35.657171+0000","last_fullsized":"2026-02-24T07:43:35.657171+0000","mapping_epoch":30,"log_start":"38'11452","ondisk_log_start":"38'11452","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_clean_scrub_stamp":"2026-02-24T07:43:06.832892+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:44:14.991691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":750714880,"num_objects":11455,"num_object_clones":0,"num_object_copies":34365,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11455,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11455,"num_write_kb":733120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.b","version":"38'11419","reported_seq":11445,"reported_epoch":38,"state":"active+clean","last_fresh":"2026-02-24T07:43:35.662574+0000","last_change":"2026-02-24T07:43:07.853283+0000","last_active":"2026-02-24T07:43:35.662574+0000","last_peered":"2026-02-24T07:43:35.662574+0000","last_clean":"2026-02-24T07:43:35.662574+0000","last_became_active":"2026-02-24T07:43:07.853069+0000","last_became_peered":"2026-02-24T07:43:07.853069+0000","last_unstale":"2026-02-24T07:43:35.662574+0000","last_undegraded":"2026-02-24T07:43:35.662574+0000","last_fullsized":"2026-02-24T07:43:35.662574+0000","mapping_epoch":30,"log_start":"38'11416","ondisk_log_start":"38'11416","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_clean_scrub_stamp":"2026-02-24T07:43:06.832892+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:44:32.581403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":748355584,"num_objects":11419,"num_object_clones":0,"num_object_copies":34257,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11419,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11419,"num_write_kb":730816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":11911233536,"num_objects":181751,"num_object_clones":0,"num_object_copies":545253,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":181751,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":181751,"num_write_kb":11632064,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":35734421504,"data_stored":35734421504,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":48,"ondisk_log_size":48,"up":48,"acting":48,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":240,"num_write":135,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":3,"up_from":25,"seq":107374182411,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":10745396,"kb_used_data":10140832,"kb_used_omap":12,"kb_used_meta":604467,"kb_avail":723257804,"statfs":{"total":751619276800,"available":740615991296,"internally_reserved":0,"allocated":10384211968,"data_stored":10383977086,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12690,"internal_metadata":618974830},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":19,"seq":81604378637,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":8241128,"kb_used_data":7781012,"kb_used_omap":15,"kb_used_meta":460080,"kb_avail":725762072,"statfs":{"total":751619276800,"available":743180361728,"internally_reserved":0,"allocated":7967756288,"data_stored":7967529285,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":15947,"internal_metadata":471122357},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542158,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":9153840,"kb_used_data":8621532,"kb_used_omap":18,"kb_used_meta":532269,"kb_avail":724849360,"statfs":{"total":751619276800,"available":742245744640,"internally_reserved":0,"allocated":8828448768,"data_stored":8828217438,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":18551,"internal_metadata":545044361},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[8,0,4],"upper_bound":8},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705681,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":8307636,"kb_used_data":7847520,"kb_used_omap":22,"kb_used_meta":460073,"kb_avail":725695564,"statfs":{"total":751619276800,"available":743112257536,"internally_reserved":0,"allocated":8035860480,"data_stored":8035625598,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":23108,"internal_metadata":471115196},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8178434048,"data_stored":8178434048,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8946909184,"data_stored":8946909184,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8183742464,"data_stored":8183742464,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":10425335808,"data_stored":10425335808,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-24T07:43:40.099 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph pg dump --format=json 2026-02-24T07:43:40.226 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:43:40.782 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:43:40.541+0000 7fb95ed02640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:43:40.782 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:43:40.542+0000 7fb95ed02640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:43:40.782 INFO:teuthology.orchestra.run.trial035.stderr:osd.0: osd_enable_op_tracker = '' 2026-02-24T07:43:40.782 INFO:teuthology.orchestra.run.trial035.stderr:osd.1: osd_enable_op_tracker = '' 2026-02-24T07:43:40.783 INFO:teuthology.orchestra.run.trial035.stdout:osd.0: {} 2026-02-24T07:43:40.783 INFO:teuthology.orchestra.run.trial035.stdout:osd.1: {} 2026-02-24T07:43:40.783 INFO:teuthology.orchestra.run.trial035.stdout:osd.2: {} 2026-02-24T07:43:40.783 INFO:teuthology.orchestra.run.trial035.stdout:osd.3: {} 2026-02-24T07:43:40.783 INFO:teuthology.orchestra.run.trial035.stderr:osd.2: osd_enable_op_tracker = '' 2026-02-24T07:43:40.784 INFO:teuthology.orchestra.run.trial035.stderr:osd.3: osd_enable_op_tracker = '' 2026-02-24T07:43:40.932 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 32 16 209529 209513 409.169 310.438 0.00371243 0.00244136 2026-02-24T07:43:40.954 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:43:40 trial035 ceph-mon[39284]: from='client.14395 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2026-02-24T07:43:40.954 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:43:40 trial035 ceph-mon[39284]: pgmap v34: 17 pgs: 17 active+clean; 11 GiB data, 35 GiB used, 2.7 TiB / 2.7 TiB avail; 346 MiB/s wr, 5.53k op/s 2026-02-24T07:43:40.954 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:43:40 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/3102170324' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2026-02-24T07:43:41.222 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:43:41.220+0000 7fa3856a0640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:43:41.222 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:43:41.220+0000 7fa3856a0640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:43:41.357 INFO:teuthology.orchestra.run.trial035.stdout: 2026-02-24T07:43:41.358 INFO:teuthology.orchestra.run.trial035.stderr:dumped all 2026-02-24T07:43:41.402 INFO:teuthology.orchestra.run.trial035.stdout:{"pg_ready":true,"pg_map":{"version":34,"stamp":"2026-02-24T07:43:39.664111+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":11911823904,"num_objects":181753,"num_object_clones":0,"num_object_copies":545259,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":181753,"num_whiteouts":0,"num_read":136,"num_read_kb":240,"num_write":181886,"num_write_kb":11634214,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":51,"ondisk_log_size":51,"up":51,"acting":51,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":51,"num_osds":4,"num_per_pool_osds":4,"num_per_pool_omap_osds":4,"kb":2936012800,"kb_used":36448000,"kb_used_data":34390896,"kb_used_omap":68,"kb_used_meta":2056891,"kb_avail":2899564800,"statfs":{"total":3006477107200,"available":2969154355200,"internally_reserved":0,"allocated":35216277504,"data_stored":35215349407,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":70296,"internal_metadata":2106256744},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[10,0,4],"upper_bound":8},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2142437376,"num_objects":32691,"num_object_clones":0,"num_object_copies":98073,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32691,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":32691,"num_write_kb":2092224,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.907693"},"pg_stats":[{"pgid":"2.f","version":"38'11131","reported_seq":11157,"reported_epoch":38,"state":"active+clean","last_fresh":"2026-02-24T07:43:35.655469+0000","last_change":"2026-02-24T07:43:07.853034+0000","last_active":"2026-02-24T07:43:35.655469+0000","last_peered":"2026-02-24T07:43:35.655469+0000","last_clean":"2026-02-24T07:43:35.655469+0000","last_became_active":"2026-02-24T07:43:07.852899+0000","last_became_peered":"2026-02-24T07:43:07.852899+0000","last_unstale":"2026-02-24T07:43:35.655469+0000","last_undegraded":"2026-02-24T07:43:35.655469+0000","last_fullsized":"2026-02-24T07:43:35.655469+0000","mapping_epoch":30,"log_start":"38'11128","ondisk_log_start":"38'11128","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_clean_scrub_stamp":"2026-02-24T07:43:06.832892+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:44:18.843618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":729481216,"num_objects":11131,"num_object_clones":0,"num_object_copies":33393,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11131,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11131,"num_write_kb":712384,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.e","version":"38'11437","reported_seq":11463,"reported_epoch":38,"state":"active+clean","last_fresh":"2026-02-24T07:43:35.661075+0000","last_change":"2026-02-24T07:43:07.853009+0000","last_active":"2026-02-24T07:43:35.661075+0000","last_peered":"2026-02-24T07:43:35.661075+0000","last_clean":"2026-02-24T07:43:35.661075+0000","last_became_active":"2026-02-24T07:43:07.852815+0000","last_became_peered":"2026-02-24T07:43:07.852815+0000","last_unstale":"2026-02-24T07:43:35.661075+0000","last_undegraded":"2026-02-24T07:43:35.661075+0000","last_fullsized":"2026-02-24T07:43:35.661075+0000","mapping_epoch":30,"log_start":"38'11434","ondisk_log_start":"38'11434","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_clean_scrub_stamp":"2026-02-24T07:43:06.832892+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:44:11.750665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":749535232,"num_objects":11437,"num_object_clones":0,"num_object_copies":34311,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11437,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11437,"num_write_kb":731968,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.d","version":"38'11494","reported_seq":11520,"reported_epoch":38,"state":"active+clean","last_fresh":"2026-02-24T07:43:35.658803+0000","last_change":"2026-02-24T07:43:07.854403+0000","last_active":"2026-02-24T07:43:35.658803+0000","last_peered":"2026-02-24T07:43:35.658803+0000","last_clean":"2026-02-24T07:43:35.658803+0000","last_became_active":"2026-02-24T07:43:07.853975+0000","last_became_peered":"2026-02-24T07:43:07.853975+0000","last_unstale":"2026-02-24T07:43:35.658803+0000","last_undegraded":"2026-02-24T07:43:35.658803+0000","last_fullsized":"2026-02-24T07:43:35.658803+0000","mapping_epoch":30,"log_start":"38'11491","ondisk_log_start":"38'11491","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_clean_scrub_stamp":"2026-02-24T07:43:06.832892+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:44:14.520704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":753270784,"num_objects":11494,"num_object_clones":0,"num_object_copies":34482,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11494,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11494,"num_write_kb":735616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"38'11334","reported_seq":11359,"reported_epoch":38,"state":"active+clean","last_fresh":"2026-02-24T07:43:35.663918+0000","last_change":"2026-02-24T07:43:07.854780+0000","last_active":"2026-02-24T07:43:35.663918+0000","last_peered":"2026-02-24T07:43:35.663918+0000","last_clean":"2026-02-24T07:43:35.663918+0000","last_became_active":"2026-02-24T07:43:07.854428+0000","last_became_peered":"2026-02-24T07:43:07.854428+0000","last_unstale":"2026-02-24T07:43:35.663918+0000","last_undegraded":"2026-02-24T07:43:35.663918+0000","last_fullsized":"2026-02-24T07:43:35.663918+0000","mapping_epoch":30,"log_start":"38'11331","ondisk_log_start":"38'11331","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_clean_scrub_stamp":"2026-02-24T07:43:06.832892+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:44:16.718883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":742785024,"num_objects":11334,"num_object_clones":0,"num_object_copies":34002,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11334,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11334,"num_write_kb":725376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"29'99","reported_seq":197,"reported_epoch":38,"state":"active+clean","last_fresh":"2026-02-24T07:43:34.757645+0000","last_change":"2026-02-24T07:42:54.186862+0000","last_active":"2026-02-24T07:43:34.757645+0000","last_peered":"2026-02-24T07:43:34.757645+0000","last_clean":"2026-02-24T07:43:34.757645+0000","last_became_active":"2026-02-24T07:42:54.186719+0000","last_became_peered":"2026-02-24T07:42:54.186719+0000","last_unstale":"2026-02-24T07:43:34.757645+0000","last_undegraded":"2026-02-24T07:43:34.757645+0000","last_fullsized":"2026-02-24T07:43:34.757645+0000","mapping_epoch":27,"log_start":"29'96","ondisk_log_start":"29'96","created":20,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:42:43.640776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:42:43.640776+0000","last_clean_scrub_stamp":"2026-02-24T07:42:43.640776+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":96,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:43:57.783028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":240,"num_write":135,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"38'11407","reported_seq":11433,"reported_epoch":38,"state":"active+clean","last_fresh":"2026-02-24T07:43:35.663121+0000","last_change":"2026-02-24T07:43:07.842967+0000","last_active":"2026-02-24T07:43:35.663121+0000","last_peered":"2026-02-24T07:43:35.663121+0000","last_clean":"2026-02-24T07:43:35.663121+0000","last_became_active":"2026-02-24T07:43:07.842530+0000","last_became_peered":"2026-02-24T07:43:07.842530+0000","last_unstale":"2026-02-24T07:43:35.663121+0000","last_undegraded":"2026-02-24T07:43:35.663121+0000","last_fullsized":"2026-02-24T07:43:35.663121+0000","mapping_epoch":30,"log_start":"38'11404","ondisk_log_start":"38'11404","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_clean_scrub_stamp":"2026-02-24T07:43:06.832892+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:44:14.900069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":747569152,"num_objects":11407,"num_object_clones":0,"num_object_copies":34221,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11407,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11407,"num_write_kb":730048,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"38'11434","reported_seq":11460,"reported_epoch":38,"state":"active+clean","last_fresh":"2026-02-24T07:43:35.661359+0000","last_change":"2026-02-24T07:43:07.854082+0000","last_active":"2026-02-24T07:43:35.661359+0000","last_peered":"2026-02-24T07:43:35.661359+0000","last_clean":"2026-02-24T07:43:35.661359+0000","last_became_active":"2026-02-24T07:43:07.853885+0000","last_became_peered":"2026-02-24T07:43:07.853885+0000","last_unstale":"2026-02-24T07:43:35.661359+0000","last_undegraded":"2026-02-24T07:43:35.661359+0000","last_fullsized":"2026-02-24T07:43:35.661359+0000","mapping_epoch":30,"log_start":"38'11431","ondisk_log_start":"38'11431","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_clean_scrub_stamp":"2026-02-24T07:43:06.832892+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:44:20.512438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":749338624,"num_objects":11434,"num_object_clones":0,"num_object_copies":34302,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11434,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11434,"num_write_kb":731776,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"38'11365","reported_seq":11391,"reported_epoch":38,"state":"active+clean","last_fresh":"2026-02-24T07:43:35.660812+0000","last_change":"2026-02-24T07:43:07.854497+0000","last_active":"2026-02-24T07:43:35.660812+0000","last_peered":"2026-02-24T07:43:35.660812+0000","last_clean":"2026-02-24T07:43:35.660812+0000","last_became_active":"2026-02-24T07:43:07.854212+0000","last_became_peered":"2026-02-24T07:43:07.854212+0000","last_unstale":"2026-02-24T07:43:35.660812+0000","last_undegraded":"2026-02-24T07:43:35.660812+0000","last_fullsized":"2026-02-24T07:43:35.660812+0000","mapping_epoch":30,"log_start":"38'11362","ondisk_log_start":"38'11362","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_clean_scrub_stamp":"2026-02-24T07:43:06.832892+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:44:21.531086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":744816640,"num_objects":11365,"num_object_clones":0,"num_object_copies":34095,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11365,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11365,"num_write_kb":727360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[1,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"38'11563","reported_seq":11589,"reported_epoch":38,"state":"active+clean","last_fresh":"2026-02-24T07:43:35.652634+0000","last_change":"2026-02-24T07:43:07.843123+0000","last_active":"2026-02-24T07:43:35.652634+0000","last_peered":"2026-02-24T07:43:35.652634+0000","last_clean":"2026-02-24T07:43:35.652634+0000","last_became_active":"2026-02-24T07:43:07.842864+0000","last_became_peered":"2026-02-24T07:43:07.842864+0000","last_unstale":"2026-02-24T07:43:35.652634+0000","last_undegraded":"2026-02-24T07:43:35.652634+0000","last_fullsized":"2026-02-24T07:43:35.652634+0000","mapping_epoch":30,"log_start":"38'11560","ondisk_log_start":"38'11560","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_clean_scrub_stamp":"2026-02-24T07:43:06.832892+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:44:34.876596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":757792768,"num_objects":11563,"num_object_clones":0,"num_object_copies":34689,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11563,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11563,"num_write_kb":740032,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.4","version":"38'11245","reported_seq":11271,"reported_epoch":38,"state":"active+clean","last_fresh":"2026-02-24T07:43:35.658277+0000","last_change":"2026-02-24T07:43:07.854462+0000","last_active":"2026-02-24T07:43:35.658277+0000","last_peered":"2026-02-24T07:43:35.658277+0000","last_clean":"2026-02-24T07:43:35.658277+0000","last_became_active":"2026-02-24T07:43:07.853966+0000","last_became_peered":"2026-02-24T07:43:07.853966+0000","last_unstale":"2026-02-24T07:43:35.658277+0000","last_undegraded":"2026-02-24T07:43:35.658277+0000","last_fullsized":"2026-02-24T07:43:35.658277+0000","mapping_epoch":30,"log_start":"38'11242","ondisk_log_start":"38'11242","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_clean_scrub_stamp":"2026-02-24T07:43:06.832892+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:44:13.031431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":736952320,"num_objects":11245,"num_object_clones":0,"num_object_copies":33735,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11245,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11245,"num_write_kb":719680,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"38'11430","reported_seq":11456,"reported_epoch":38,"state":"active+clean","last_fresh":"2026-02-24T07:43:35.652962+0000","last_change":"2026-02-24T07:43:07.854184+0000","last_active":"2026-02-24T07:43:35.652962+0000","last_peered":"2026-02-24T07:43:35.652962+0000","last_clean":"2026-02-24T07:43:35.652962+0000","last_became_active":"2026-02-24T07:43:07.853878+0000","last_became_peered":"2026-02-24T07:43:07.853878+0000","last_unstale":"2026-02-24T07:43:35.652962+0000","last_undegraded":"2026-02-24T07:43:35.652962+0000","last_fullsized":"2026-02-24T07:43:35.652962+0000","mapping_epoch":30,"log_start":"38'11427","ondisk_log_start":"38'11427","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_clean_scrub_stamp":"2026-02-24T07:43:06.832892+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:44:31.507398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":749076480,"num_objects":11430,"num_object_clones":0,"num_object_copies":34290,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11430,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11430,"num_write_kb":731520,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"38'11444","reported_seq":11470,"reported_epoch":38,"state":"active+clean","last_fresh":"2026-02-24T07:43:35.660446+0000","last_change":"2026-02-24T07:43:07.842180+0000","last_active":"2026-02-24T07:43:35.660446+0000","last_peered":"2026-02-24T07:43:35.660446+0000","last_clean":"2026-02-24T07:43:35.660446+0000","last_became_active":"2026-02-24T07:43:07.841993+0000","last_became_peered":"2026-02-24T07:43:07.841993+0000","last_unstale":"2026-02-24T07:43:35.660446+0000","last_undegraded":"2026-02-24T07:43:35.660446+0000","last_fullsized":"2026-02-24T07:43:35.660446+0000","mapping_epoch":30,"log_start":"38'11441","ondisk_log_start":"38'11441","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_clean_scrub_stamp":"2026-02-24T07:43:06.832892+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:44:19.519502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":749993984,"num_objects":11444,"num_object_clones":0,"num_object_copies":34332,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11444,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11444,"num_write_kb":732416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"38'11085","reported_seq":11111,"reported_epoch":38,"state":"active+clean","last_fresh":"2026-02-24T07:43:35.659003+0000","last_change":"2026-02-24T07:43:07.842375+0000","last_active":"2026-02-24T07:43:35.659003+0000","last_peered":"2026-02-24T07:43:35.659003+0000","last_clean":"2026-02-24T07:43:35.659003+0000","last_became_active":"2026-02-24T07:43:07.841920+0000","last_became_peered":"2026-02-24T07:43:07.841920+0000","last_unstale":"2026-02-24T07:43:35.659003+0000","last_undegraded":"2026-02-24T07:43:35.659003+0000","last_fullsized":"2026-02-24T07:43:35.659003+0000","mapping_epoch":30,"log_start":"38'11082","ondisk_log_start":"38'11082","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_clean_scrub_stamp":"2026-02-24T07:43:06.832892+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:44:14.681944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":726466560,"num_objects":11085,"num_object_clones":0,"num_object_copies":33255,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11085,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11085,"num_write_kb":709440,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.8","version":"38'11258","reported_seq":11284,"reported_epoch":38,"state":"active+clean","last_fresh":"2026-02-24T07:43:35.658362+0000","last_change":"2026-02-24T07:43:07.853992+0000","last_active":"2026-02-24T07:43:35.658362+0000","last_peered":"2026-02-24T07:43:35.658362+0000","last_clean":"2026-02-24T07:43:35.658362+0000","last_became_active":"2026-02-24T07:43:07.853758+0000","last_became_peered":"2026-02-24T07:43:07.853758+0000","last_unstale":"2026-02-24T07:43:35.658362+0000","last_undegraded":"2026-02-24T07:43:35.658362+0000","last_fullsized":"2026-02-24T07:43:35.658362+0000","mapping_epoch":30,"log_start":"38'11255","ondisk_log_start":"38'11255","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_clean_scrub_stamp":"2026-02-24T07:43:06.832892+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:44:27.436119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":737804288,"num_objects":11258,"num_object_clones":0,"num_object_copies":33774,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11258,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11258,"num_write_kb":720512,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"38'11250","reported_seq":11276,"reported_epoch":38,"state":"active+clean","last_fresh":"2026-02-24T07:43:35.650870+0000","last_change":"2026-02-24T07:43:07.854687+0000","last_active":"2026-02-24T07:43:35.650870+0000","last_peered":"2026-02-24T07:43:35.650870+0000","last_clean":"2026-02-24T07:43:35.650870+0000","last_became_active":"2026-02-24T07:43:07.854377+0000","last_became_peered":"2026-02-24T07:43:07.854377+0000","last_unstale":"2026-02-24T07:43:35.650870+0000","last_undegraded":"2026-02-24T07:43:35.650870+0000","last_fullsized":"2026-02-24T07:43:35.650870+0000","mapping_epoch":30,"log_start":"38'11247","ondisk_log_start":"38'11247","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_clean_scrub_stamp":"2026-02-24T07:43:06.832892+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:44:34.215480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":737280000,"num_objects":11250,"num_object_clones":0,"num_object_copies":33750,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11250,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11250,"num_write_kb":720000,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.a","version":"38'11455","reported_seq":11481,"reported_epoch":38,"state":"active+clean","last_fresh":"2026-02-24T07:43:35.657171+0000","last_change":"2026-02-24T07:43:07.841933+0000","last_active":"2026-02-24T07:43:35.657171+0000","last_peered":"2026-02-24T07:43:35.657171+0000","last_clean":"2026-02-24T07:43:35.657171+0000","last_became_active":"2026-02-24T07:43:07.841721+0000","last_became_peered":"2026-02-24T07:43:07.841721+0000","last_unstale":"2026-02-24T07:43:35.657171+0000","last_undegraded":"2026-02-24T07:43:35.657171+0000","last_fullsized":"2026-02-24T07:43:35.657171+0000","mapping_epoch":30,"log_start":"38'11452","ondisk_log_start":"38'11452","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_clean_scrub_stamp":"2026-02-24T07:43:06.832892+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:44:14.991691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":750714880,"num_objects":11455,"num_object_clones":0,"num_object_copies":34365,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11455,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11455,"num_write_kb":733120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.b","version":"38'11419","reported_seq":11445,"reported_epoch":38,"state":"active+clean","last_fresh":"2026-02-24T07:43:35.662574+0000","last_change":"2026-02-24T07:43:07.853283+0000","last_active":"2026-02-24T07:43:35.662574+0000","last_peered":"2026-02-24T07:43:35.662574+0000","last_clean":"2026-02-24T07:43:35.662574+0000","last_became_active":"2026-02-24T07:43:07.853069+0000","last_became_peered":"2026-02-24T07:43:07.853069+0000","last_unstale":"2026-02-24T07:43:35.662574+0000","last_undegraded":"2026-02-24T07:43:35.662574+0000","last_fullsized":"2026-02-24T07:43:35.662574+0000","mapping_epoch":30,"log_start":"38'11416","ondisk_log_start":"38'11416","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_clean_scrub_stamp":"2026-02-24T07:43:06.832892+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:44:32.581403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":748355584,"num_objects":11419,"num_object_clones":0,"num_object_copies":34257,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11419,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11419,"num_write_kb":730816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":11911233536,"num_objects":181751,"num_object_clones":0,"num_object_copies":545253,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":181751,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":181751,"num_write_kb":11632064,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":35734421504,"data_stored":35734421504,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":48,"ondisk_log_size":48,"up":48,"acting":48,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":240,"num_write":135,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":3,"up_from":25,"seq":107374182411,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":10745396,"kb_used_data":10140832,"kb_used_omap":12,"kb_used_meta":604467,"kb_avail":723257804,"statfs":{"total":751619276800,"available":740615991296,"internally_reserved":0,"allocated":10384211968,"data_stored":10383977086,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12690,"internal_metadata":618974830},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":19,"seq":81604378637,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":8241128,"kb_used_data":7781012,"kb_used_omap":15,"kb_used_meta":460080,"kb_avail":725762072,"statfs":{"total":751619276800,"available":743180361728,"internally_reserved":0,"allocated":7967756288,"data_stored":7967529285,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":15947,"internal_metadata":471122357},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542158,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":9153840,"kb_used_data":8621532,"kb_used_omap":18,"kb_used_meta":532269,"kb_avail":724849360,"statfs":{"total":751619276800,"available":742245744640,"internally_reserved":0,"allocated":8828448768,"data_stored":8828217438,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":18551,"internal_metadata":545044361},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[8,0,4],"upper_bound":8},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705681,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":8307636,"kb_used_data":7847520,"kb_used_omap":22,"kb_used_meta":460073,"kb_avail":725695564,"statfs":{"total":751619276800,"available":743112257536,"internally_reserved":0,"allocated":8035860480,"data_stored":8035625598,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":23108,"internal_metadata":471115196},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8178434048,"data_stored":8178434048,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8946909184,"data_stored":8946909184,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8183742464,"data_stored":8183742464,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":10425335808,"data_stored":10425335808,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-24T07:43:41.404 INFO:tasks.cephadm.ceph_manager.ceph:recovered! 2026-02-24T07:43:41.404 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2026-02-24T07:43:41.405 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph pg dump --format=json 2026-02-24T07:43:41.545 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:43:41.670 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:43:41.667+0000 7f090aec0640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:43:41.670 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:43:41.668+0000 7f090aec0640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:43:41.808 INFO:teuthology.orchestra.run.trial035.stdout: 2026-02-24T07:43:41.808 INFO:teuthology.orchestra.run.trial035.stderr:dumped all 2026-02-24T07:43:41.848 INFO:teuthology.orchestra.run.trial035.stdout:{"pg_ready":true,"pg_map":{"version":35,"stamp":"2026-02-24T07:43:41.664356+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":13646955040,"num_objects":208229,"num_object_clones":0,"num_object_copies":624687,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":208229,"num_whiteouts":0,"num_read":136,"num_read_kb":240,"num_write":208362,"num_write_kb":13328678,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":52,"ondisk_log_size":52,"up":51,"acting":51,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":51,"num_osds":4,"num_per_pool_osds":4,"num_per_pool_omap_osds":4,"kb":2936012800,"kb_used":41766700,"kb_used_data":39438940,"kb_used_omap":74,"kb_used_meta":2327605,"kb_avail":2894246100,"statfs":{"total":3006477107200,"available":2963708006400,"internally_reserved":0,"allocated":40385474560,"data_stored":40384475552,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":76155,"internal_metadata":2383468165},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":3877568512,"num_objects":59167,"num_object_clones":0,"num_object_copies":177501,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":59167,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":59167,"num_write_kb":3786688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.907319"},"pg_stats":[{"pgid":"2.f","version":"40'12768","reported_seq":12798,"reported_epoch":40,"state":"active+clean","last_fresh":"2026-02-24T07:43:40.658768+0000","last_change":"2026-02-24T07:43:07.853034+0000","last_active":"2026-02-24T07:43:40.658768+0000","last_peered":"2026-02-24T07:43:40.658768+0000","last_clean":"2026-02-24T07:43:40.658768+0000","last_became_active":"2026-02-24T07:43:07.852899+0000","last_became_peered":"2026-02-24T07:43:07.852899+0000","last_unstale":"2026-02-24T07:43:40.658768+0000","last_undegraded":"2026-02-24T07:43:40.658768+0000","last_fullsized":"2026-02-24T07:43:40.658768+0000","mapping_epoch":30,"log_start":"40'12765","ondisk_log_start":"40'12765","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_clean_scrub_stamp":"2026-02-24T07:43:06.832892+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:44:18.843618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":836763648,"num_objects":12768,"num_object_clones":0,"num_object_copies":38304,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12768,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12768,"num_write_kb":817152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.e","version":"40'13124","reported_seq":13154,"reported_epoch":40,"state":"active+clean","last_fresh":"2026-02-24T07:43:40.657069+0000","last_change":"2026-02-24T07:43:07.853009+0000","last_active":"2026-02-24T07:43:40.657069+0000","last_peered":"2026-02-24T07:43:40.657069+0000","last_clean":"2026-02-24T07:43:40.657069+0000","last_became_active":"2026-02-24T07:43:07.852815+0000","last_became_peered":"2026-02-24T07:43:07.852815+0000","last_unstale":"2026-02-24T07:43:40.657069+0000","last_undegraded":"2026-02-24T07:43:40.657069+0000","last_fullsized":"2026-02-24T07:43:40.657069+0000","mapping_epoch":30,"log_start":"40'13121","ondisk_log_start":"40'13121","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_clean_scrub_stamp":"2026-02-24T07:43:06.832892+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:44:11.750665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":860094464,"num_objects":13124,"num_object_clones":0,"num_object_copies":39372,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13124,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13124,"num_write_kb":839936,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.d","version":"40'13173","reported_seq":13203,"reported_epoch":40,"state":"active+clean","last_fresh":"2026-02-24T07:43:40.663206+0000","last_change":"2026-02-24T07:43:07.854403+0000","last_active":"2026-02-24T07:43:40.663206+0000","last_peered":"2026-02-24T07:43:40.663206+0000","last_clean":"2026-02-24T07:43:40.663206+0000","last_became_active":"2026-02-24T07:43:07.853975+0000","last_became_peered":"2026-02-24T07:43:07.853975+0000","last_unstale":"2026-02-24T07:43:40.663206+0000","last_undegraded":"2026-02-24T07:43:40.663206+0000","last_fullsized":"2026-02-24T07:43:40.663206+0000","mapping_epoch":30,"log_start":"40'13170","ondisk_log_start":"40'13170","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_clean_scrub_stamp":"2026-02-24T07:43:06.832892+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:44:14.520704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":863305728,"num_objects":13173,"num_object_clones":0,"num_object_copies":39519,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13173,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13173,"num_write_kb":843072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"40'13026","reported_seq":13056,"reported_epoch":40,"state":"active+clean","last_fresh":"2026-02-24T07:43:40.657278+0000","last_change":"2026-02-24T07:43:07.854780+0000","last_active":"2026-02-24T07:43:40.657278+0000","last_peered":"2026-02-24T07:43:40.657278+0000","last_clean":"2026-02-24T07:43:40.657278+0000","last_became_active":"2026-02-24T07:43:07.854428+0000","last_became_peered":"2026-02-24T07:43:07.854428+0000","last_unstale":"2026-02-24T07:43:40.657278+0000","last_undegraded":"2026-02-24T07:43:40.657278+0000","last_fullsized":"2026-02-24T07:43:40.657278+0000","mapping_epoch":30,"log_start":"40'13023","ondisk_log_start":"40'13023","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_clean_scrub_stamp":"2026-02-24T07:43:06.832892+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:44:16.718883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":853671936,"num_objects":13026,"num_object_clones":0,"num_object_copies":39078,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13026,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13026,"num_write_kb":833664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"29'99","reported_seq":201,"reported_epoch":40,"state":"active+clean","last_fresh":"2026-02-24T07:43:37.766077+0000","last_change":"2026-02-24T07:42:54.186862+0000","last_active":"2026-02-24T07:43:37.766077+0000","last_peered":"2026-02-24T07:43:37.766077+0000","last_clean":"2026-02-24T07:43:37.766077+0000","last_became_active":"2026-02-24T07:42:54.186719+0000","last_became_peered":"2026-02-24T07:42:54.186719+0000","last_unstale":"2026-02-24T07:43:37.766077+0000","last_undegraded":"2026-02-24T07:43:37.766077+0000","last_fullsized":"2026-02-24T07:43:37.766077+0000","mapping_epoch":27,"log_start":"29'96","ondisk_log_start":"29'96","created":20,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:42:43.640776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:42:43.640776+0000","last_clean_scrub_stamp":"2026-02-24T07:42:43.640776+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":96,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:43:57.783028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":240,"num_write":135,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"40'13070","reported_seq":13100,"reported_epoch":40,"state":"active+clean","last_fresh":"2026-02-24T07:43:40.657461+0000","last_change":"2026-02-24T07:43:07.842967+0000","last_active":"2026-02-24T07:43:40.657461+0000","last_peered":"2026-02-24T07:43:40.657461+0000","last_clean":"2026-02-24T07:43:40.657461+0000","last_became_active":"2026-02-24T07:43:07.842530+0000","last_became_peered":"2026-02-24T07:43:07.842530+0000","last_unstale":"2026-02-24T07:43:40.657461+0000","last_undegraded":"2026-02-24T07:43:40.657461+0000","last_fullsized":"2026-02-24T07:43:40.657461+0000","mapping_epoch":30,"log_start":"40'13067","ondisk_log_start":"40'13067","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_clean_scrub_stamp":"2026-02-24T07:43:06.832892+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:44:14.900069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":856555520,"num_objects":13070,"num_object_clones":0,"num_object_copies":39210,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13070,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13070,"num_write_kb":836480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"40'13085","reported_seq":13115,"reported_epoch":40,"state":"active+clean","last_fresh":"2026-02-24T07:43:40.663504+0000","last_change":"2026-02-24T07:43:07.854082+0000","last_active":"2026-02-24T07:43:40.663504+0000","last_peered":"2026-02-24T07:43:40.663504+0000","last_clean":"2026-02-24T07:43:40.663504+0000","last_became_active":"2026-02-24T07:43:07.853885+0000","last_became_peered":"2026-02-24T07:43:07.853885+0000","last_unstale":"2026-02-24T07:43:40.663504+0000","last_undegraded":"2026-02-24T07:43:40.663504+0000","last_fullsized":"2026-02-24T07:43:40.663504+0000","mapping_epoch":30,"log_start":"40'13082","ondisk_log_start":"40'13082","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_clean_scrub_stamp":"2026-02-24T07:43:06.832892+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:44:20.512438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":857538560,"num_objects":13085,"num_object_clones":0,"num_object_copies":39255,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13085,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13085,"num_write_kb":837440,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"40'12977","reported_seq":13007,"reported_epoch":40,"state":"active+clean","last_fresh":"2026-02-24T07:43:40.660958+0000","last_change":"2026-02-24T07:43:07.854497+0000","last_active":"2026-02-24T07:43:40.660958+0000","last_peered":"2026-02-24T07:43:40.660958+0000","last_clean":"2026-02-24T07:43:40.660958+0000","last_became_active":"2026-02-24T07:43:07.854212+0000","last_became_peered":"2026-02-24T07:43:07.854212+0000","last_unstale":"2026-02-24T07:43:40.660958+0000","last_undegraded":"2026-02-24T07:43:40.660958+0000","last_fullsized":"2026-02-24T07:43:40.660958+0000","mapping_epoch":30,"log_start":"40'12974","ondisk_log_start":"40'12974","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_clean_scrub_stamp":"2026-02-24T07:43:06.832892+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:44:21.531086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":850460672,"num_objects":12977,"num_object_clones":0,"num_object_copies":38931,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12977,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12977,"num_write_kb":830528,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[1,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"40'13271","reported_seq":13301,"reported_epoch":40,"state":"active+clean","last_fresh":"2026-02-24T07:43:40.663430+0000","last_change":"2026-02-24T07:43:07.843123+0000","last_active":"2026-02-24T07:43:40.663430+0000","last_peered":"2026-02-24T07:43:40.663430+0000","last_clean":"2026-02-24T07:43:40.663430+0000","last_became_active":"2026-02-24T07:43:07.842864+0000","last_became_peered":"2026-02-24T07:43:07.842864+0000","last_unstale":"2026-02-24T07:43:40.663430+0000","last_undegraded":"2026-02-24T07:43:40.663430+0000","last_fullsized":"2026-02-24T07:43:40.663430+0000","mapping_epoch":30,"log_start":"40'13268","ondisk_log_start":"40'13268","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_clean_scrub_stamp":"2026-02-24T07:43:06.832892+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:44:34.876596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":869728256,"num_objects":13271,"num_object_clones":0,"num_object_copies":39813,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13271,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13271,"num_write_kb":849344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.4","version":"40'12847","reported_seq":12876,"reported_epoch":40,"state":"active+clean","last_fresh":"2026-02-24T07:43:40.664662+0000","last_change":"2026-02-24T07:43:07.854462+0000","last_active":"2026-02-24T07:43:40.664662+0000","last_peered":"2026-02-24T07:43:40.664662+0000","last_clean":"2026-02-24T07:43:40.664662+0000","last_became_active":"2026-02-24T07:43:07.853966+0000","last_became_peered":"2026-02-24T07:43:07.853966+0000","last_unstale":"2026-02-24T07:43:40.664662+0000","last_undegraded":"2026-02-24T07:43:40.664662+0000","last_fullsized":"2026-02-24T07:43:40.664662+0000","mapping_epoch":30,"log_start":"40'12844","ondisk_log_start":"40'12844","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_clean_scrub_stamp":"2026-02-24T07:43:06.832892+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:44:13.031431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":841940992,"num_objects":12847,"num_object_clones":0,"num_object_copies":38541,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12847,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12847,"num_write_kb":822208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"40'13040","reported_seq":13070,"reported_epoch":40,"state":"active+clean","last_fresh":"2026-02-24T07:43:40.653117+0000","last_change":"2026-02-24T07:43:07.854184+0000","last_active":"2026-02-24T07:43:40.653117+0000","last_peered":"2026-02-24T07:43:40.653117+0000","last_clean":"2026-02-24T07:43:40.653117+0000","last_became_active":"2026-02-24T07:43:07.853878+0000","last_became_peered":"2026-02-24T07:43:07.853878+0000","last_unstale":"2026-02-24T07:43:40.653117+0000","last_undegraded":"2026-02-24T07:43:40.653117+0000","last_fullsized":"2026-02-24T07:43:40.653117+0000","mapping_epoch":30,"log_start":"40'13037","ondisk_log_start":"40'13037","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_clean_scrub_stamp":"2026-02-24T07:43:06.832892+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:44:31.507398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":854589440,"num_objects":13040,"num_object_clones":0,"num_object_copies":39120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13040,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13040,"num_write_kb":834560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"40'13091","reported_seq":13118,"reported_epoch":40,"state":"active+clean","last_fresh":"2026-02-24T07:43:40.663033+0000","last_change":"2026-02-24T07:43:07.842180+0000","last_active":"2026-02-24T07:43:40.663033+0000","last_peered":"2026-02-24T07:43:40.663033+0000","last_clean":"2026-02-24T07:43:40.663033+0000","last_became_active":"2026-02-24T07:43:07.841993+0000","last_became_peered":"2026-02-24T07:43:07.841993+0000","last_unstale":"2026-02-24T07:43:40.663033+0000","last_undegraded":"2026-02-24T07:43:40.663033+0000","last_fullsized":"2026-02-24T07:43:40.663033+0000","mapping_epoch":30,"log_start":"40'13087","ondisk_log_start":"40'13087","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_clean_scrub_stamp":"2026-02-24T07:43:06.832892+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":3000,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:44:19.519502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":857931776,"num_objects":13091,"num_object_clones":0,"num_object_copies":39273,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13091,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13091,"num_write_kb":837824,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"40'12720","reported_seq":12750,"reported_epoch":40,"state":"active+clean","last_fresh":"2026-02-24T07:43:40.646722+0000","last_change":"2026-02-24T07:43:07.842375+0000","last_active":"2026-02-24T07:43:40.646722+0000","last_peered":"2026-02-24T07:43:40.646722+0000","last_clean":"2026-02-24T07:43:40.646722+0000","last_became_active":"2026-02-24T07:43:07.841920+0000","last_became_peered":"2026-02-24T07:43:07.841920+0000","last_unstale":"2026-02-24T07:43:40.646722+0000","last_undegraded":"2026-02-24T07:43:40.646722+0000","last_fullsized":"2026-02-24T07:43:40.646722+0000","mapping_epoch":30,"log_start":"40'12717","ondisk_log_start":"40'12717","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_clean_scrub_stamp":"2026-02-24T07:43:06.832892+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:44:14.681944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":833617920,"num_objects":12720,"num_object_clones":0,"num_object_copies":38160,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12720,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12720,"num_write_kb":814080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.8","version":"40'12924","reported_seq":12954,"reported_epoch":40,"state":"active+clean","last_fresh":"2026-02-24T07:43:40.652247+0000","last_change":"2026-02-24T07:43:07.853992+0000","last_active":"2026-02-24T07:43:40.652247+0000","last_peered":"2026-02-24T07:43:40.652247+0000","last_clean":"2026-02-24T07:43:40.652247+0000","last_became_active":"2026-02-24T07:43:07.853758+0000","last_became_peered":"2026-02-24T07:43:07.853758+0000","last_unstale":"2026-02-24T07:43:40.652247+0000","last_undegraded":"2026-02-24T07:43:40.652247+0000","last_fullsized":"2026-02-24T07:43:40.652247+0000","mapping_epoch":30,"log_start":"40'12921","ondisk_log_start":"40'12921","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_clean_scrub_stamp":"2026-02-24T07:43:06.832892+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:44:27.436119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":846987264,"num_objects":12924,"num_object_clones":0,"num_object_copies":38772,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12924,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12924,"num_write_kb":827136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"40'12919","reported_seq":12949,"reported_epoch":40,"state":"active+clean","last_fresh":"2026-02-24T07:43:40.651452+0000","last_change":"2026-02-24T07:43:07.854687+0000","last_active":"2026-02-24T07:43:40.651452+0000","last_peered":"2026-02-24T07:43:40.651452+0000","last_clean":"2026-02-24T07:43:40.651452+0000","last_became_active":"2026-02-24T07:43:07.854377+0000","last_became_peered":"2026-02-24T07:43:07.854377+0000","last_unstale":"2026-02-24T07:43:40.651452+0000","last_undegraded":"2026-02-24T07:43:40.651452+0000","last_fullsized":"2026-02-24T07:43:40.651452+0000","mapping_epoch":30,"log_start":"40'12916","ondisk_log_start":"40'12916","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_clean_scrub_stamp":"2026-02-24T07:43:06.832892+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:44:34.215480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":846659584,"num_objects":12919,"num_object_clones":0,"num_object_copies":38757,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12919,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12919,"num_write_kb":826816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.a","version":"40'13117","reported_seq":13147,"reported_epoch":40,"state":"active+clean","last_fresh":"2026-02-24T07:43:40.660363+0000","last_change":"2026-02-24T07:43:07.841933+0000","last_active":"2026-02-24T07:43:40.660363+0000","last_peered":"2026-02-24T07:43:40.660363+0000","last_clean":"2026-02-24T07:43:40.660363+0000","last_became_active":"2026-02-24T07:43:07.841721+0000","last_became_peered":"2026-02-24T07:43:07.841721+0000","last_unstale":"2026-02-24T07:43:40.660363+0000","last_undegraded":"2026-02-24T07:43:40.660363+0000","last_fullsized":"2026-02-24T07:43:40.660363+0000","mapping_epoch":30,"log_start":"40'13114","ondisk_log_start":"40'13114","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_clean_scrub_stamp":"2026-02-24T07:43:06.832892+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:44:14.991691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":859635712,"num_objects":13117,"num_object_clones":0,"num_object_copies":39351,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13117,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13117,"num_write_kb":839488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.b","version":"40'13075","reported_seq":13105,"reported_epoch":40,"state":"active+clean","last_fresh":"2026-02-24T07:43:40.659070+0000","last_change":"2026-02-24T07:43:07.853283+0000","last_active":"2026-02-24T07:43:40.659070+0000","last_peered":"2026-02-24T07:43:40.659070+0000","last_clean":"2026-02-24T07:43:40.659070+0000","last_became_active":"2026-02-24T07:43:07.853069+0000","last_became_peered":"2026-02-24T07:43:07.853069+0000","last_unstale":"2026-02-24T07:43:40.659070+0000","last_undegraded":"2026-02-24T07:43:40.659070+0000","last_fullsized":"2026-02-24T07:43:40.659070+0000","mapping_epoch":30,"log_start":"40'13072","ondisk_log_start":"40'13072","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_clean_scrub_stamp":"2026-02-24T07:43:06.832892+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:44:32.581403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":856883200,"num_objects":13075,"num_object_clones":0,"num_object_copies":39225,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13075,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13075,"num_write_kb":836800,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":13646364672,"num_objects":208227,"num_object_clones":0,"num_object_copies":624681,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":208227,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":208227,"num_write_kb":13326528,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":40939225088,"data_stored":40939225088,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":49,"ondisk_log_size":49,"up":48,"acting":48,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":240,"num_write":135,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":3,"up_from":25,"seq":107374182412,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":12277516,"kb_used_data":11600824,"kb_used_omap":13,"kb_used_meta":676658,"kb_avail":721725684,"statfs":{"total":751619276800,"available":739047100416,"internally_reserved":0,"allocated":11879243776,"data_stored":11878993136,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13992,"internal_metadata":692898136},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":19,"seq":81604378638,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":9483276,"kb_used_data":8950968,"kb_used_omap":17,"kb_used_meta":532270,"kb_avail":724519924,"statfs":{"total":751619276800,"available":741908402176,"internally_reserved":0,"allocated":9165791232,"data_stored":9165540592,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":17900,"internal_metadata":545045012},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542159,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":10477576,"kb_used_data":9891124,"kb_used_omap":19,"kb_used_meta":586412,"kb_avail":723525624,"statfs":{"total":751619276800,"available":740890238976,"internally_reserved":0,"allocated":10128510976,"data_stored":10128263888,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19853,"internal_metadata":600486515},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705682,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":9528332,"kb_used_data":8996024,"kb_used_omap":23,"kb_used_meta":532264,"kb_avail":724474868,"statfs":{"total":751619276800,"available":741862264832,"internally_reserved":0,"allocated":9211928576,"data_stored":9211677936,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":24410,"internal_metadata":545038502},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":9368698880,"data_stored":9368698880,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":10251403264,"data_stored":10251403264,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":9374334976,"data_stored":9374334976,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":11944787968,"data_stored":11944787968,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-24T07:43:41.933 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 33 16 216014 215998 409.051 405.312 0.00190608 0.00244356 2026-02-24T07:43:42.932 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 34 16 223181 223165 410.194 447.938 0.00248754 0.00243674 2026-02-24T07:43:43.140 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2026-02-24T07:43:43.165 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:43:42 trial035 ceph-mon[39284]: from='client.14399 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2026-02-24T07:43:43.165 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:43:42 trial035 ceph-mon[39284]: pgmap v35: 17 pgs: 17 active+clean; 13 GiB data, 40 GiB used, 2.7 TiB / 2.7 TiB avail; 535 MiB/s wr, 8.56k op/s 2026-02-24T07:43:43.165 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:43:42 trial035 ceph-mon[39284]: from='client.14401 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2026-02-24T07:43:43.277 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:43:43.396 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:43:43.394+0000 7f55ae2a5640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:43:43.396 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:43:43.394+0000 7f55ae2a5640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:43:43.410 INFO:teuthology.orchestra.run.trial035.stdout:osd.0: {} 2026-02-24T07:43:43.411 INFO:teuthology.orchestra.run.trial035.stderr:osd.0: osd_enable_op_tracker = '' 2026-02-24T07:43:43.423 INFO:teuthology.orchestra.run.trial035.stdout:osd.1: {} 2026-02-24T07:43:43.423 INFO:teuthology.orchestra.run.trial035.stderr:osd.1: osd_enable_op_tracker = '' 2026-02-24T07:43:43.433 INFO:teuthology.orchestra.run.trial035.stdout:osd.2: {} 2026-02-24T07:43:43.434 INFO:teuthology.orchestra.run.trial035.stderr:osd.2: osd_enable_op_tracker = '' 2026-02-24T07:43:43.445 INFO:teuthology.orchestra.run.trial035.stdout:osd.3: {} 2026-02-24T07:43:43.446 INFO:teuthology.orchestra.run.trial035.stderr:osd.3: osd_enable_op_tracker = '' 2026-02-24T07:43:43.933 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 35 16 230205 230189 411.016 439 0.0019516 0.00243192 2026-02-24T07:43:44.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:43:43 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/231607951' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2026-02-24T07:43:44.933 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 36 16 237071 237055 411.518 429.125 0.00371671 0.00242892 2026-02-24T07:43:45.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:43:44 trial035 ceph-mon[39284]: pgmap v36: 17 pgs: 17 active+clean; 13 GiB data, 40 GiB used, 2.7 TiB / 2.7 TiB avail; 447 MiB/s wr, 7.15k op/s 2026-02-24T07:43:45.488 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2026-02-24T07:43:45.632 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:43:45.788 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:43:45.786+0000 7fc9a5e30640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:43:45.788 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:43:45.786+0000 7fc9a5e30640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:43:45.803 INFO:teuthology.orchestra.run.trial035.stdout:osd.0: {} 2026-02-24T07:43:45.803 INFO:teuthology.orchestra.run.trial035.stderr:osd.0: osd_enable_op_tracker = '' 2026-02-24T07:43:45.819 INFO:teuthology.orchestra.run.trial035.stdout:osd.1: {} 2026-02-24T07:43:45.819 INFO:teuthology.orchestra.run.trial035.stderr:osd.1: osd_enable_op_tracker = '' 2026-02-24T07:43:45.835 INFO:teuthology.orchestra.run.trial035.stdout:osd.2: {} 2026-02-24T07:43:45.835 INFO:teuthology.orchestra.run.trial035.stderr:osd.2: osd_enable_op_tracker = '' 2026-02-24T07:43:45.847 INFO:teuthology.orchestra.run.trial035.stdout:osd.3: {} 2026-02-24T07:43:45.848 INFO:teuthology.orchestra.run.trial035.stderr:osd.3: osd_enable_op_tracker = '' 2026-02-24T07:43:45.933 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 37 16 243309 243293 410.932 389.875 0.00113092 0.00243235 2026-02-24T07:43:46.130 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:43:45 trial035 ceph-mon[39284]: from='mgr.14284 10.20.193.35:0/3681373222' entity='mgr.x' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2026-02-24T07:43:46.130 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:43:45 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/3819619615' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2026-02-24T07:43:46.849 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2026-02-24T07:43:46.849 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2026-02-24T07:43:46.849 INFO:tasks.thrashosds.thrasher:Growing pool .mgr 2026-02-24T07:43:46.849 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph pg dump --format=json 2026-02-24T07:43:46.933 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 38 16 249543 249527 410.371 389.625 0.00182599 0.00243573 2026-02-24T07:43:46.988 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:43:47.105 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:43:46 trial035 ceph-mon[39284]: pgmap v37: 17 pgs: 17 active+clean; 13 GiB data, 43 GiB used, 2.7 TiB / 2.7 TiB avail; 180 MiB/s wr, 2.88k op/s 2026-02-24T07:43:47.105 INFO:journalctl@ceph.mgr.x.trial035.stdout:Feb 24 07:43:46 trial035 ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-mgr-x[39621]: ::ffff:10.20.193.91 - - [24/Feb/2026:07:43:46] "GET /metrics HTTP/1.1" 200 4214 "" "Prometheus/3.6.0" 2026-02-24T07:43:47.265 INFO:journalctl@ceph.prometheus.a.trial091.stdout:Feb 24 07:43:46 trial091 ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-prometheus-a[66007]: time=2026-02-24T07:43:46.973Z level=ERROR source=scrape.go:1630 msg="Failed to determine correct type of scrape target." component="scrape manager" scrape_pool=ceph target=http://trial035:9283/metrics content_type="text/html;charset=utf-8" fallback_media_type="" err="received unsupported Content-Type \"text/html;charset=utf-8\" and no fallback_scrape_protocol specified for target" 2026-02-24T07:43:47.481 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:43:47.468+0000 7f677d301640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:43:47.481 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:43:47.468+0000 7f677d301640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:43:47.599 INFO:teuthology.orchestra.run.trial035.stdout: 2026-02-24T07:43:47.599 INFO:teuthology.orchestra.run.trial035.stderr:dumped all 2026-02-24T07:43:47.642 INFO:teuthology.orchestra.run.trial035.stdout:{"pg_ready":true,"pg_map":{"version":37,"stamp":"2026-02-24T07:43:45.664879+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":13783466528,"num_objects":210312,"num_object_clones":0,"num_object_copies":630936,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":210312,"num_whiteouts":0,"num_read":136,"num_read_kb":240,"num_write":210445,"num_write_kb":13461990,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":52,"ondisk_log_size":52,"up":51,"acting":51,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":51,"num_osds":4,"num_per_pool_osds":4,"num_per_pool_omap_osds":4,"kb":2936012800,"kb_used":45023404,"kb_used_data":42515164,"kb_used_omap":74,"kb_used_meta":2508085,"kb_avail":2890989396,"statfs":{"total":3006477107200,"available":2960373141504,"internally_reserved":0,"allocated":43535527936,"data_stored":43534528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":76155,"internal_metadata":2568279685},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[4],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1871642624,"num_objects":28559,"num_object_clones":0,"num_object_copies":85677,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28559,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":28559,"num_write_kb":1827776,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.903568"},"pg_stats":[{"pgid":"2.f","version":"40'14851","reported_seq":14881,"reported_epoch":40,"state":"active+clean","last_fresh":"2026-02-24T07:43:45.661279+0000","last_change":"2026-02-24T07:43:07.853034+0000","last_active":"2026-02-24T07:43:45.661279+0000","last_peered":"2026-02-24T07:43:45.661279+0000","last_clean":"2026-02-24T07:43:45.661279+0000","last_became_active":"2026-02-24T07:43:07.852899+0000","last_became_peered":"2026-02-24T07:43:07.852899+0000","last_unstale":"2026-02-24T07:43:45.661279+0000","last_undegraded":"2026-02-24T07:43:45.661279+0000","last_fullsized":"2026-02-24T07:43:45.661279+0000","mapping_epoch":30,"log_start":"40'14848","ondisk_log_start":"40'14848","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_clean_scrub_stamp":"2026-02-24T07:43:06.832892+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:44:18.843618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":973275136,"num_objects":14851,"num_object_clones":0,"num_object_copies":44553,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14851,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14851,"num_write_kb":950464,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.e","version":"40'13124","reported_seq":13154,"reported_epoch":40,"state":"active+clean","last_fresh":"2026-02-24T07:43:40.657069+0000","last_change":"2026-02-24T07:43:07.853009+0000","last_active":"2026-02-24T07:43:40.657069+0000","last_peered":"2026-02-24T07:43:40.657069+0000","last_clean":"2026-02-24T07:43:40.657069+0000","last_became_active":"2026-02-24T07:43:07.852815+0000","last_became_peered":"2026-02-24T07:43:07.852815+0000","last_unstale":"2026-02-24T07:43:40.657069+0000","last_undegraded":"2026-02-24T07:43:40.657069+0000","last_fullsized":"2026-02-24T07:43:40.657069+0000","mapping_epoch":30,"log_start":"40'13121","ondisk_log_start":"40'13121","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_clean_scrub_stamp":"2026-02-24T07:43:06.832892+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:44:11.750665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":860094464,"num_objects":13124,"num_object_clones":0,"num_object_copies":39372,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13124,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13124,"num_write_kb":839936,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.d","version":"40'13173","reported_seq":13203,"reported_epoch":40,"state":"active+clean","last_fresh":"2026-02-24T07:43:40.663206+0000","last_change":"2026-02-24T07:43:07.854403+0000","last_active":"2026-02-24T07:43:40.663206+0000","last_peered":"2026-02-24T07:43:40.663206+0000","last_clean":"2026-02-24T07:43:40.663206+0000","last_became_active":"2026-02-24T07:43:07.853975+0000","last_became_peered":"2026-02-24T07:43:07.853975+0000","last_unstale":"2026-02-24T07:43:40.663206+0000","last_undegraded":"2026-02-24T07:43:40.663206+0000","last_fullsized":"2026-02-24T07:43:40.663206+0000","mapping_epoch":30,"log_start":"40'13170","ondisk_log_start":"40'13170","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_clean_scrub_stamp":"2026-02-24T07:43:06.832892+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:44:14.520704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":863305728,"num_objects":13173,"num_object_clones":0,"num_object_copies":39519,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13173,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13173,"num_write_kb":843072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"40'13026","reported_seq":13056,"reported_epoch":40,"state":"active+clean","last_fresh":"2026-02-24T07:43:40.657278+0000","last_change":"2026-02-24T07:43:07.854780+0000","last_active":"2026-02-24T07:43:40.657278+0000","last_peered":"2026-02-24T07:43:40.657278+0000","last_clean":"2026-02-24T07:43:40.657278+0000","last_became_active":"2026-02-24T07:43:07.854428+0000","last_became_peered":"2026-02-24T07:43:07.854428+0000","last_unstale":"2026-02-24T07:43:40.657278+0000","last_undegraded":"2026-02-24T07:43:40.657278+0000","last_fullsized":"2026-02-24T07:43:40.657278+0000","mapping_epoch":30,"log_start":"40'13023","ondisk_log_start":"40'13023","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_clean_scrub_stamp":"2026-02-24T07:43:06.832892+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:44:16.718883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":853671936,"num_objects":13026,"num_object_clones":0,"num_object_copies":39078,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13026,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13026,"num_write_kb":833664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"29'99","reported_seq":201,"reported_epoch":40,"state":"active+clean","last_fresh":"2026-02-24T07:43:37.766077+0000","last_change":"2026-02-24T07:42:54.186862+0000","last_active":"2026-02-24T07:43:37.766077+0000","last_peered":"2026-02-24T07:43:37.766077+0000","last_clean":"2026-02-24T07:43:37.766077+0000","last_became_active":"2026-02-24T07:42:54.186719+0000","last_became_peered":"2026-02-24T07:42:54.186719+0000","last_unstale":"2026-02-24T07:43:37.766077+0000","last_undegraded":"2026-02-24T07:43:37.766077+0000","last_fullsized":"2026-02-24T07:43:37.766077+0000","mapping_epoch":27,"log_start":"29'96","ondisk_log_start":"29'96","created":20,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:42:43.640776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:42:43.640776+0000","last_clean_scrub_stamp":"2026-02-24T07:42:43.640776+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":96,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:43:57.783028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":240,"num_write":135,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"40'13070","reported_seq":13100,"reported_epoch":40,"state":"active+clean","last_fresh":"2026-02-24T07:43:40.657461+0000","last_change":"2026-02-24T07:43:07.842967+0000","last_active":"2026-02-24T07:43:40.657461+0000","last_peered":"2026-02-24T07:43:40.657461+0000","last_clean":"2026-02-24T07:43:40.657461+0000","last_became_active":"2026-02-24T07:43:07.842530+0000","last_became_peered":"2026-02-24T07:43:07.842530+0000","last_unstale":"2026-02-24T07:43:40.657461+0000","last_undegraded":"2026-02-24T07:43:40.657461+0000","last_fullsized":"2026-02-24T07:43:40.657461+0000","mapping_epoch":30,"log_start":"40'13067","ondisk_log_start":"40'13067","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_clean_scrub_stamp":"2026-02-24T07:43:06.832892+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:44:14.900069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":856555520,"num_objects":13070,"num_object_clones":0,"num_object_copies":39210,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13070,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13070,"num_write_kb":836480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"40'13085","reported_seq":13115,"reported_epoch":40,"state":"active+clean","last_fresh":"2026-02-24T07:43:40.663504+0000","last_change":"2026-02-24T07:43:07.854082+0000","last_active":"2026-02-24T07:43:40.663504+0000","last_peered":"2026-02-24T07:43:40.663504+0000","last_clean":"2026-02-24T07:43:40.663504+0000","last_became_active":"2026-02-24T07:43:07.853885+0000","last_became_peered":"2026-02-24T07:43:07.853885+0000","last_unstale":"2026-02-24T07:43:40.663504+0000","last_undegraded":"2026-02-24T07:43:40.663504+0000","last_fullsized":"2026-02-24T07:43:40.663504+0000","mapping_epoch":30,"log_start":"40'13082","ondisk_log_start":"40'13082","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_clean_scrub_stamp":"2026-02-24T07:43:06.832892+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:44:20.512438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":857538560,"num_objects":13085,"num_object_clones":0,"num_object_copies":39255,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13085,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13085,"num_write_kb":837440,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"40'12977","reported_seq":13007,"reported_epoch":40,"state":"active+clean","last_fresh":"2026-02-24T07:43:40.660958+0000","last_change":"2026-02-24T07:43:07.854497+0000","last_active":"2026-02-24T07:43:40.660958+0000","last_peered":"2026-02-24T07:43:40.660958+0000","last_clean":"2026-02-24T07:43:40.660958+0000","last_became_active":"2026-02-24T07:43:07.854212+0000","last_became_peered":"2026-02-24T07:43:07.854212+0000","last_unstale":"2026-02-24T07:43:40.660958+0000","last_undegraded":"2026-02-24T07:43:40.660958+0000","last_fullsized":"2026-02-24T07:43:40.660958+0000","mapping_epoch":30,"log_start":"40'12974","ondisk_log_start":"40'12974","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_clean_scrub_stamp":"2026-02-24T07:43:06.832892+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:44:21.531086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":850460672,"num_objects":12977,"num_object_clones":0,"num_object_copies":38931,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12977,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12977,"num_write_kb":830528,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[1,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"40'13271","reported_seq":13301,"reported_epoch":40,"state":"active+clean","last_fresh":"2026-02-24T07:43:40.663430+0000","last_change":"2026-02-24T07:43:07.843123+0000","last_active":"2026-02-24T07:43:40.663430+0000","last_peered":"2026-02-24T07:43:40.663430+0000","last_clean":"2026-02-24T07:43:40.663430+0000","last_became_active":"2026-02-24T07:43:07.842864+0000","last_became_peered":"2026-02-24T07:43:07.842864+0000","last_unstale":"2026-02-24T07:43:40.663430+0000","last_undegraded":"2026-02-24T07:43:40.663430+0000","last_fullsized":"2026-02-24T07:43:40.663430+0000","mapping_epoch":30,"log_start":"40'13268","ondisk_log_start":"40'13268","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_clean_scrub_stamp":"2026-02-24T07:43:06.832892+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:44:34.876596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":869728256,"num_objects":13271,"num_object_clones":0,"num_object_copies":39813,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13271,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13271,"num_write_kb":849344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.4","version":"40'12847","reported_seq":12876,"reported_epoch":40,"state":"active+clean","last_fresh":"2026-02-24T07:43:40.664662+0000","last_change":"2026-02-24T07:43:07.854462+0000","last_active":"2026-02-24T07:43:40.664662+0000","last_peered":"2026-02-24T07:43:40.664662+0000","last_clean":"2026-02-24T07:43:40.664662+0000","last_became_active":"2026-02-24T07:43:07.853966+0000","last_became_peered":"2026-02-24T07:43:07.853966+0000","last_unstale":"2026-02-24T07:43:40.664662+0000","last_undegraded":"2026-02-24T07:43:40.664662+0000","last_fullsized":"2026-02-24T07:43:40.664662+0000","mapping_epoch":30,"log_start":"40'12844","ondisk_log_start":"40'12844","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_clean_scrub_stamp":"2026-02-24T07:43:06.832892+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:44:13.031431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":841940992,"num_objects":12847,"num_object_clones":0,"num_object_copies":38541,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12847,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12847,"num_write_kb":822208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"40'13040","reported_seq":13070,"reported_epoch":40,"state":"active+clean","last_fresh":"2026-02-24T07:43:40.653117+0000","last_change":"2026-02-24T07:43:07.854184+0000","last_active":"2026-02-24T07:43:40.653117+0000","last_peered":"2026-02-24T07:43:40.653117+0000","last_clean":"2026-02-24T07:43:40.653117+0000","last_became_active":"2026-02-24T07:43:07.853878+0000","last_became_peered":"2026-02-24T07:43:07.853878+0000","last_unstale":"2026-02-24T07:43:40.653117+0000","last_undegraded":"2026-02-24T07:43:40.653117+0000","last_fullsized":"2026-02-24T07:43:40.653117+0000","mapping_epoch":30,"log_start":"40'13037","ondisk_log_start":"40'13037","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_clean_scrub_stamp":"2026-02-24T07:43:06.832892+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:44:31.507398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":854589440,"num_objects":13040,"num_object_clones":0,"num_object_copies":39120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13040,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13040,"num_write_kb":834560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"40'13091","reported_seq":13118,"reported_epoch":40,"state":"active+clean","last_fresh":"2026-02-24T07:43:40.663033+0000","last_change":"2026-02-24T07:43:07.842180+0000","last_active":"2026-02-24T07:43:40.663033+0000","last_peered":"2026-02-24T07:43:40.663033+0000","last_clean":"2026-02-24T07:43:40.663033+0000","last_became_active":"2026-02-24T07:43:07.841993+0000","last_became_peered":"2026-02-24T07:43:07.841993+0000","last_unstale":"2026-02-24T07:43:40.663033+0000","last_undegraded":"2026-02-24T07:43:40.663033+0000","last_fullsized":"2026-02-24T07:43:40.663033+0000","mapping_epoch":30,"log_start":"40'13087","ondisk_log_start":"40'13087","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_clean_scrub_stamp":"2026-02-24T07:43:06.832892+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":3000,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:44:19.519502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":857931776,"num_objects":13091,"num_object_clones":0,"num_object_copies":39273,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13091,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13091,"num_write_kb":837824,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"40'12720","reported_seq":12750,"reported_epoch":40,"state":"active+clean","last_fresh":"2026-02-24T07:43:40.646722+0000","last_change":"2026-02-24T07:43:07.842375+0000","last_active":"2026-02-24T07:43:40.646722+0000","last_peered":"2026-02-24T07:43:40.646722+0000","last_clean":"2026-02-24T07:43:40.646722+0000","last_became_active":"2026-02-24T07:43:07.841920+0000","last_became_peered":"2026-02-24T07:43:07.841920+0000","last_unstale":"2026-02-24T07:43:40.646722+0000","last_undegraded":"2026-02-24T07:43:40.646722+0000","last_fullsized":"2026-02-24T07:43:40.646722+0000","mapping_epoch":30,"log_start":"40'12717","ondisk_log_start":"40'12717","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_clean_scrub_stamp":"2026-02-24T07:43:06.832892+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:44:14.681944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":833617920,"num_objects":12720,"num_object_clones":0,"num_object_copies":38160,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12720,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12720,"num_write_kb":814080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.8","version":"40'12924","reported_seq":12954,"reported_epoch":40,"state":"active+clean","last_fresh":"2026-02-24T07:43:40.652247+0000","last_change":"2026-02-24T07:43:07.853992+0000","last_active":"2026-02-24T07:43:40.652247+0000","last_peered":"2026-02-24T07:43:40.652247+0000","last_clean":"2026-02-24T07:43:40.652247+0000","last_became_active":"2026-02-24T07:43:07.853758+0000","last_became_peered":"2026-02-24T07:43:07.853758+0000","last_unstale":"2026-02-24T07:43:40.652247+0000","last_undegraded":"2026-02-24T07:43:40.652247+0000","last_fullsized":"2026-02-24T07:43:40.652247+0000","mapping_epoch":30,"log_start":"40'12921","ondisk_log_start":"40'12921","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_clean_scrub_stamp":"2026-02-24T07:43:06.832892+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:44:27.436119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":846987264,"num_objects":12924,"num_object_clones":0,"num_object_copies":38772,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12924,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12924,"num_write_kb":827136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"40'12919","reported_seq":12949,"reported_epoch":40,"state":"active+clean","last_fresh":"2026-02-24T07:43:40.651452+0000","last_change":"2026-02-24T07:43:07.854687+0000","last_active":"2026-02-24T07:43:40.651452+0000","last_peered":"2026-02-24T07:43:40.651452+0000","last_clean":"2026-02-24T07:43:40.651452+0000","last_became_active":"2026-02-24T07:43:07.854377+0000","last_became_peered":"2026-02-24T07:43:07.854377+0000","last_unstale":"2026-02-24T07:43:40.651452+0000","last_undegraded":"2026-02-24T07:43:40.651452+0000","last_fullsized":"2026-02-24T07:43:40.651452+0000","mapping_epoch":30,"log_start":"40'12916","ondisk_log_start":"40'12916","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_clean_scrub_stamp":"2026-02-24T07:43:06.832892+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:44:34.215480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":846659584,"num_objects":12919,"num_object_clones":0,"num_object_copies":38757,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12919,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12919,"num_write_kb":826816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.a","version":"40'13117","reported_seq":13147,"reported_epoch":40,"state":"active+clean","last_fresh":"2026-02-24T07:43:40.660363+0000","last_change":"2026-02-24T07:43:07.841933+0000","last_active":"2026-02-24T07:43:40.660363+0000","last_peered":"2026-02-24T07:43:40.660363+0000","last_clean":"2026-02-24T07:43:40.660363+0000","last_became_active":"2026-02-24T07:43:07.841721+0000","last_became_peered":"2026-02-24T07:43:07.841721+0000","last_unstale":"2026-02-24T07:43:40.660363+0000","last_undegraded":"2026-02-24T07:43:40.660363+0000","last_fullsized":"2026-02-24T07:43:40.660363+0000","mapping_epoch":30,"log_start":"40'13114","ondisk_log_start":"40'13114","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_clean_scrub_stamp":"2026-02-24T07:43:06.832892+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:44:14.991691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":859635712,"num_objects":13117,"num_object_clones":0,"num_object_copies":39351,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13117,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13117,"num_write_kb":839488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.b","version":"40'13075","reported_seq":13105,"reported_epoch":40,"state":"active+clean","last_fresh":"2026-02-24T07:43:40.659070+0000","last_change":"2026-02-24T07:43:07.853283+0000","last_active":"2026-02-24T07:43:40.659070+0000","last_peered":"2026-02-24T07:43:40.659070+0000","last_clean":"2026-02-24T07:43:40.659070+0000","last_became_active":"2026-02-24T07:43:07.853069+0000","last_became_peered":"2026-02-24T07:43:07.853069+0000","last_unstale":"2026-02-24T07:43:40.659070+0000","last_undegraded":"2026-02-24T07:43:40.659070+0000","last_fullsized":"2026-02-24T07:43:40.659070+0000","mapping_epoch":30,"log_start":"40'13072","ondisk_log_start":"40'13072","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_clean_scrub_stamp":"2026-02-24T07:43:06.832892+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:44:32.581403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":856883200,"num_objects":13075,"num_object_clones":0,"num_object_copies":39225,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13075,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13075,"num_write_kb":836800,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":13782876160,"num_objects":210310,"num_object_clones":0,"num_object_copies":630930,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":210310,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":210310,"num_write_kb":13459840,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":43970789376,"data_stored":43970789376,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":49,"ondisk_log_size":49,"up":48,"acting":48,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":240,"num_write":135,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":3,"up_from":25,"seq":107374182412,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":12277516,"kb_used_data":11600824,"kb_used_omap":13,"kb_used_meta":676658,"kb_avail":721725684,"statfs":{"total":751619276800,"available":739047100416,"internally_reserved":0,"allocated":11879243776,"data_stored":11878993136,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13992,"internal_metadata":692898136},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":19,"seq":81604378639,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":11251468,"kb_used_data":10610872,"kb_used_omap":17,"kb_used_meta":640558,"kb_avail":722751732,"statfs":{"total":751619276800,"available":740097773568,"internally_reserved":0,"allocated":10865532928,"data_stored":10865282288,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":17900,"internal_metadata":655931924},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542159,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":10477576,"kb_used_data":9891124,"kb_used_omap":19,"kb_used_meta":586412,"kb_avail":723525624,"statfs":{"total":751619276800,"available":740890238976,"internally_reserved":0,"allocated":10128510976,"data_stored":10128263888,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19853,"internal_metadata":600486515},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705683,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":11016844,"kb_used_data":10412344,"kb_used_omap":23,"kb_used_meta":604456,"kb_avail":722986356,"statfs":{"total":751619276800,"available":740338028544,"internally_reserved":0,"allocated":10662240256,"data_stored":10661989616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":24410,"internal_metadata":618963110},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":10885005312,"data_stored":10885005312,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":10251403264,"data_stored":10251403264,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":10889592832,"data_stored":10889592832,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":11944787968,"data_stored":11944787968,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-24T07:43:47.644 INFO:tasks.cephadm.ceph_manager.ceph:increase pool size by 10 2026-02-24T07:43:47.645 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph osd pool set .mgr pg_num 31 2026-02-24T07:43:47.777 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:43:47.898 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:43:47.896+0000 7f4890a7b640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:43:47.899 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:43:47.897+0000 7f4890a7b640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:43:47.933 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 39 16 255719 255703 409.745 386 0.00242067 0.00243943 2026-02-24T07:43:48.035 INFO:teuthology.orchestra.run.trial035.stderr:Error EPERM: crimson_allow_pg_split is false; pg_num increase denied for crimson pool 2026-02-24T07:43:48.072 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-24T07:43:48.181 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2026-02-24T07:43:48.316 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:43:48.444 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:43:48.442+0000 7f4a9eb63640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:43:48.444 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:43:48.442+0000 7f4a9eb63640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:43:48.459 INFO:teuthology.orchestra.run.trial035.stdout:osd.0: {} 2026-02-24T07:43:48.460 INFO:teuthology.orchestra.run.trial035.stderr:osd.0: osd_enable_op_tracker = '' 2026-02-24T07:43:48.474 INFO:teuthology.orchestra.run.trial035.stdout:osd.1: {} 2026-02-24T07:43:48.474 INFO:teuthology.orchestra.run.trial035.stderr:osd.1: osd_enable_op_tracker = '' 2026-02-24T07:43:48.487 INFO:teuthology.orchestra.run.trial035.stdout:osd.2: {} 2026-02-24T07:43:48.487 INFO:teuthology.orchestra.run.trial035.stderr:osd.2: osd_enable_op_tracker = '' 2026-02-24T07:43:48.501 INFO:teuthology.orchestra.run.trial035.stdout:osd.3: {} 2026-02-24T07:43:48.501 INFO:teuthology.orchestra.run.trial035.stderr:osd.3: osd_enable_op_tracker = '' 2026-02-24T07:43:48.933 INFO:tasks.radosbench.radosbench.0.trial035.stdout:2026-02-24T07:43:48.932806+0000 min lat: 0.000685593 max lat: 0.066631 avg lat: 0.0024357 2026-02-24T07:43:48.933 INFO:tasks.radosbench.radosbench.0.trial035.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2026-02-24T07:43:48.933 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 40 16 262675 262659 410.369 434.75 0.00190754 0.0024357 2026-02-24T07:43:49.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:43:48 trial035 ceph-mon[39284]: from='client.14407 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2026-02-24T07:43:49.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:43:48 trial035 ceph-mon[39284]: pgmap v38: 17 pgs: 17 active+clean; 15 GiB data, 46 GiB used, 2.7 TiB / 2.7 TiB avail; 376 MiB/s wr, 6.02k op/s 2026-02-24T07:43:49.205 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:43:48 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/1428195449' entity='client.admin' cmd={"prefix": "osd pool set", "pool": ".mgr", "var": "pg_num", "val": "31"} : dispatch 2026-02-24T07:43:49.205 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:43:48 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/385405599' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2026-02-24T07:43:49.933 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 41 16 270172 270156 411.788 468.562 0.00179668 0.00242731 2026-02-24T07:43:50.542 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2026-02-24T07:43:50.685 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:43:50.809 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:43:50.806+0000 7fd641d70640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:43:50.809 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:43:50.806+0000 7fd641d70640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:43:50.823 INFO:teuthology.orchestra.run.trial035.stdout:osd.0: {} 2026-02-24T07:43:50.823 INFO:teuthology.orchestra.run.trial035.stderr:osd.0: osd_enable_op_tracker = '' 2026-02-24T07:43:50.835 INFO:teuthology.orchestra.run.trial035.stdout:osd.1: {} 2026-02-24T07:43:50.836 INFO:teuthology.orchestra.run.trial035.stderr:osd.1: osd_enable_op_tracker = '' 2026-02-24T07:43:50.851 INFO:teuthology.orchestra.run.trial035.stdout:osd.2: {} 2026-02-24T07:43:50.852 INFO:teuthology.orchestra.run.trial035.stderr:osd.2: osd_enable_op_tracker = '' 2026-02-24T07:43:50.864 INFO:teuthology.orchestra.run.trial035.stdout:osd.3: {} 2026-02-24T07:43:50.864 INFO:teuthology.orchestra.run.trial035.stderr:osd.3: osd_enable_op_tracker = '' 2026-02-24T07:43:50.933 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 42 16 277148 277132 412.363 436 0.00185691 0.00242399 2026-02-24T07:43:51.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:43:50 trial035 ceph-mon[39284]: pgmap v39: 17 pgs: 17 active+clean; 15 GiB data, 46 GiB used, 2.7 TiB / 2.7 TiB avail; 316 MiB/s wr, 5.06k op/s 2026-02-24T07:43:51.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:43:50 trial035 ceph-mon[39284]: purged_snaps scrub starts 2026-02-24T07:43:51.205 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:43:50 trial035 ceph-mon[39284]: purged_snaps scrub ok 2026-02-24T07:43:51.205 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:43:50 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/49237053' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2026-02-24T07:43:51.933 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 43 15 284285 284270 413.148 446.125 0.00224921 0.00241937 2026-02-24T07:43:52.906 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2026-02-24T07:43:52.933 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 44 15 291052 291037 413.369 422.938 0.00212325 0.00241804 2026-02-24T07:43:53.044 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:43:53.073 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2026-02-24T07:43:53.074 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2026-02-24T07:43:53.074 INFO:tasks.thrashosds.thrasher:Growing pool unique_pool_0 2026-02-24T07:43:53.074 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph pg dump --format=json 2026-02-24T07:43:53.089 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:43:52 trial035 ceph-mon[39284]: pgmap v40: 17 pgs: 17 active+clean; 17 GiB data, 53 GiB used, 2.7 TiB / 2.7 TiB avail; 488 MiB/s wr, 7.80k op/s 2026-02-24T07:43:53.089 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:43:52 trial035 ceph-mon[39284]: purged_snaps scrub starts 2026-02-24T07:43:53.089 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:43:52 trial035 ceph-mon[39284]: purged_snaps scrub ok 2026-02-24T07:43:53.154 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:43:53.151+0000 7f13d219d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:43:53.154 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:43:53.152+0000 7f13d219d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:43:53.169 INFO:teuthology.orchestra.run.trial035.stdout:osd.0: {} 2026-02-24T07:43:53.170 INFO:teuthology.orchestra.run.trial035.stderr:osd.0: osd_enable_op_tracker = '' 2026-02-24T07:43:53.187 INFO:teuthology.orchestra.run.trial035.stdout:osd.1: {} 2026-02-24T07:43:53.187 INFO:teuthology.orchestra.run.trial035.stderr:osd.1: osd_enable_op_tracker = '' 2026-02-24T07:43:53.201 INFO:teuthology.orchestra.run.trial035.stdout:osd.2: {} 2026-02-24T07:43:53.201 INFO:teuthology.orchestra.run.trial035.stderr:osd.2: osd_enable_op_tracker = '' 2026-02-24T07:43:53.212 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:43:53.214 INFO:teuthology.orchestra.run.trial035.stdout:osd.3: {} 2026-02-24T07:43:53.214 INFO:teuthology.orchestra.run.trial035.stderr:osd.3: osd_enable_op_tracker = '' 2026-02-24T07:43:53.331 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:43:53.329+0000 7f65f9a75640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:43:53.331 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:43:53.329+0000 7f65f9a75640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:43:53.465 INFO:teuthology.orchestra.run.trial035.stdout: 2026-02-24T07:43:53.465 INFO:teuthology.orchestra.run.trial035.stderr:dumped all 2026-02-24T07:43:53.511 INFO:teuthology.orchestra.run.trial035.stdout:{"pg_ready":true,"pg_map":{"version":40,"stamp":"2026-02-24T07:43:51.665659+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":18047959584,"num_objects":275383,"num_object_clones":0,"num_object_copies":826149,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":275383,"num_whiteouts":0,"num_read":136,"num_read_kb":240,"num_write":275516,"num_write_kb":17626534,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":51,"ondisk_log_size":51,"up":51,"acting":51,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":51,"num_osds":4,"num_per_pool_osds":4,"num_per_pool_omap_osds":4,"kb":2936012800,"kb_used":55576620,"kb_used_data":52418588,"kb_used_omap":74,"kb_used_meta":3157813,"kb_avail":2880436180,"statfs":{"total":3006477107200,"available":2949566648320,"internally_reserved":0,"allocated":53676634112,"data_stored":53675635104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":76155,"internal_metadata":3233601157},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":6136135680,"num_objects":93630,"num_object_clones":0,"num_object_copies":280890,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":93630,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":93630,"num_write_kb":5992320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001547"},"pg_stats":[{"pgid":"2.f","version":"40'16989","reported_seq":17019,"reported_epoch":40,"state":"active+clean","last_fresh":"2026-02-24T07:43:50.659201+0000","last_change":"2026-02-24T07:43:07.853034+0000","last_active":"2026-02-24T07:43:50.659201+0000","last_peered":"2026-02-24T07:43:50.659201+0000","last_clean":"2026-02-24T07:43:50.659201+0000","last_became_active":"2026-02-24T07:43:07.852899+0000","last_became_peered":"2026-02-24T07:43:07.852899+0000","last_unstale":"2026-02-24T07:43:50.659201+0000","last_undegraded":"2026-02-24T07:43:50.659201+0000","last_fullsized":"2026-02-24T07:43:50.659201+0000","mapping_epoch":30,"log_start":"40'16986","ondisk_log_start":"40'16986","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_clean_scrub_stamp":"2026-02-24T07:43:06.832892+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:44:18.843618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1113391104,"num_objects":16989,"num_object_clones":0,"num_object_copies":50967,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16989,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16989,"num_write_kb":1087296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.e","version":"40'17379","reported_seq":17409,"reported_epoch":40,"state":"active+clean","last_fresh":"2026-02-24T07:43:50.657405+0000","last_change":"2026-02-24T07:43:07.853009+0000","last_active":"2026-02-24T07:43:50.657405+0000","last_peered":"2026-02-24T07:43:50.657405+0000","last_clean":"2026-02-24T07:43:50.657405+0000","last_became_active":"2026-02-24T07:43:07.852815+0000","last_became_peered":"2026-02-24T07:43:07.852815+0000","last_unstale":"2026-02-24T07:43:50.657405+0000","last_undegraded":"2026-02-24T07:43:50.657405+0000","last_fullsized":"2026-02-24T07:43:50.657405+0000","mapping_epoch":30,"log_start":"40'17376","ondisk_log_start":"40'17376","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_clean_scrub_stamp":"2026-02-24T07:43:06.832892+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:44:11.750665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1138950144,"num_objects":17379,"num_object_clones":0,"num_object_copies":52137,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17379,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17379,"num_write_kb":1112256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.d","version":"40'17360","reported_seq":17389,"reported_epoch":40,"state":"active+clean","last_fresh":"2026-02-24T07:43:50.664631+0000","last_change":"2026-02-24T07:43:07.854403+0000","last_active":"2026-02-24T07:43:50.664631+0000","last_peered":"2026-02-24T07:43:50.664631+0000","last_clean":"2026-02-24T07:43:50.664631+0000","last_became_active":"2026-02-24T07:43:07.853975+0000","last_became_peered":"2026-02-24T07:43:07.853975+0000","last_unstale":"2026-02-24T07:43:50.664631+0000","last_undegraded":"2026-02-24T07:43:50.664631+0000","last_fullsized":"2026-02-24T07:43:50.664631+0000","mapping_epoch":30,"log_start":"40'17357","ondisk_log_start":"40'17357","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_clean_scrub_stamp":"2026-02-24T07:43:06.832892+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:44:14.520704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1137704960,"num_objects":17360,"num_object_clones":0,"num_object_copies":52080,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17360,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17360,"num_write_kb":1111040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"40'17326","reported_seq":17355,"reported_epoch":40,"state":"active+clean","last_fresh":"2026-02-24T07:43:50.663137+0000","last_change":"2026-02-24T07:43:07.854780+0000","last_active":"2026-02-24T07:43:50.663137+0000","last_peered":"2026-02-24T07:43:50.663137+0000","last_clean":"2026-02-24T07:43:50.663137+0000","last_became_active":"2026-02-24T07:43:07.854428+0000","last_became_peered":"2026-02-24T07:43:07.854428+0000","last_unstale":"2026-02-24T07:43:50.663137+0000","last_undegraded":"2026-02-24T07:43:50.663137+0000","last_fullsized":"2026-02-24T07:43:50.663137+0000","mapping_epoch":30,"log_start":"40'17323","ondisk_log_start":"40'17323","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_clean_scrub_stamp":"2026-02-24T07:43:06.832892+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:44:16.718883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1135476736,"num_objects":17326,"num_object_clones":0,"num_object_copies":51978,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17326,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17326,"num_write_kb":1108864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"29'99","reported_seq":201,"reported_epoch":40,"state":"active+clean","last_fresh":"2026-02-24T07:43:37.766077+0000","last_change":"2026-02-24T07:42:54.186862+0000","last_active":"2026-02-24T07:43:37.766077+0000","last_peered":"2026-02-24T07:43:37.766077+0000","last_clean":"2026-02-24T07:43:37.766077+0000","last_became_active":"2026-02-24T07:42:54.186719+0000","last_became_peered":"2026-02-24T07:42:54.186719+0000","last_unstale":"2026-02-24T07:43:37.766077+0000","last_undegraded":"2026-02-24T07:43:37.766077+0000","last_fullsized":"2026-02-24T07:43:37.766077+0000","mapping_epoch":27,"log_start":"29'96","ondisk_log_start":"29'96","created":20,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:42:43.640776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:42:43.640776+0000","last_clean_scrub_stamp":"2026-02-24T07:42:43.640776+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":96,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:43:57.783028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":240,"num_write":135,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"40'17268","reported_seq":17298,"reported_epoch":40,"state":"active+clean","last_fresh":"2026-02-24T07:43:50.657306+0000","last_change":"2026-02-24T07:43:07.842967+0000","last_active":"2026-02-24T07:43:50.657306+0000","last_peered":"2026-02-24T07:43:50.657306+0000","last_clean":"2026-02-24T07:43:50.657306+0000","last_became_active":"2026-02-24T07:43:07.842530+0000","last_became_peered":"2026-02-24T07:43:07.842530+0000","last_unstale":"2026-02-24T07:43:50.657306+0000","last_undegraded":"2026-02-24T07:43:50.657306+0000","last_fullsized":"2026-02-24T07:43:50.657306+0000","mapping_epoch":30,"log_start":"40'17265","ondisk_log_start":"40'17265","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_clean_scrub_stamp":"2026-02-24T07:43:06.832892+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:44:14.900069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1131675648,"num_objects":17268,"num_object_clones":0,"num_object_copies":51804,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17268,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17268,"num_write_kb":1105152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"40'17158","reported_seq":17188,"reported_epoch":40,"state":"active+clean","last_fresh":"2026-02-24T07:43:50.659670+0000","last_change":"2026-02-24T07:43:07.854082+0000","last_active":"2026-02-24T07:43:50.659670+0000","last_peered":"2026-02-24T07:43:50.659670+0000","last_clean":"2026-02-24T07:43:50.659670+0000","last_became_active":"2026-02-24T07:43:07.853885+0000","last_became_peered":"2026-02-24T07:43:07.853885+0000","last_unstale":"2026-02-24T07:43:50.659670+0000","last_undegraded":"2026-02-24T07:43:50.659670+0000","last_fullsized":"2026-02-24T07:43:50.659670+0000","mapping_epoch":30,"log_start":"40'17155","ondisk_log_start":"40'17155","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_clean_scrub_stamp":"2026-02-24T07:43:06.832892+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:44:20.512438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1124466688,"num_objects":17158,"num_object_clones":0,"num_object_copies":51474,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17158,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17158,"num_write_kb":1098112,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"40'17219","reported_seq":17249,"reported_epoch":40,"state":"active+clean","last_fresh":"2026-02-24T07:43:50.660953+0000","last_change":"2026-02-24T07:43:07.854497+0000","last_active":"2026-02-24T07:43:50.660953+0000","last_peered":"2026-02-24T07:43:50.660953+0000","last_clean":"2026-02-24T07:43:50.660953+0000","last_became_active":"2026-02-24T07:43:07.854212+0000","last_became_peered":"2026-02-24T07:43:07.854212+0000","last_unstale":"2026-02-24T07:43:50.660953+0000","last_undegraded":"2026-02-24T07:43:50.660953+0000","last_fullsized":"2026-02-24T07:43:50.660953+0000","mapping_epoch":30,"log_start":"40'17216","ondisk_log_start":"40'17216","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_clean_scrub_stamp":"2026-02-24T07:43:06.832892+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:44:21.531086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1128464384,"num_objects":17219,"num_object_clones":0,"num_object_copies":51657,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17219,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17219,"num_write_kb":1102016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[1,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"40'17468","reported_seq":17498,"reported_epoch":40,"state":"active+clean","last_fresh":"2026-02-24T07:43:50.663973+0000","last_change":"2026-02-24T07:43:07.843123+0000","last_active":"2026-02-24T07:43:50.663973+0000","last_peered":"2026-02-24T07:43:50.663973+0000","last_clean":"2026-02-24T07:43:50.663973+0000","last_became_active":"2026-02-24T07:43:07.842864+0000","last_became_peered":"2026-02-24T07:43:07.842864+0000","last_unstale":"2026-02-24T07:43:50.663973+0000","last_undegraded":"2026-02-24T07:43:50.663973+0000","last_fullsized":"2026-02-24T07:43:50.663973+0000","mapping_epoch":30,"log_start":"40'17465","ondisk_log_start":"40'17465","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_clean_scrub_stamp":"2026-02-24T07:43:06.832892+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:44:34.876596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1144782848,"num_objects":17468,"num_object_clones":0,"num_object_copies":52404,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17468,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17468,"num_write_kb":1117952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.4","version":"40'17083","reported_seq":17113,"reported_epoch":40,"state":"active+clean","last_fresh":"2026-02-24T07:43:50.663605+0000","last_change":"2026-02-24T07:43:07.854462+0000","last_active":"2026-02-24T07:43:50.663605+0000","last_peered":"2026-02-24T07:43:50.663605+0000","last_clean":"2026-02-24T07:43:50.663605+0000","last_became_active":"2026-02-24T07:43:07.853966+0000","last_became_peered":"2026-02-24T07:43:07.853966+0000","last_unstale":"2026-02-24T07:43:50.663605+0000","last_undegraded":"2026-02-24T07:43:50.663605+0000","last_fullsized":"2026-02-24T07:43:50.663605+0000","mapping_epoch":30,"log_start":"40'17080","ondisk_log_start":"40'17080","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_clean_scrub_stamp":"2026-02-24T07:43:06.832892+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:44:13.031431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1119551488,"num_objects":17083,"num_object_clones":0,"num_object_copies":51249,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17083,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17083,"num_write_kb":1093312,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"40'17152","reported_seq":17182,"reported_epoch":40,"state":"active+clean","last_fresh":"2026-02-24T07:43:50.661390+0000","last_change":"2026-02-24T07:43:07.854184+0000","last_active":"2026-02-24T07:43:50.661390+0000","last_peered":"2026-02-24T07:43:50.661390+0000","last_clean":"2026-02-24T07:43:50.661390+0000","last_became_active":"2026-02-24T07:43:07.853878+0000","last_became_peered":"2026-02-24T07:43:07.853878+0000","last_unstale":"2026-02-24T07:43:50.661390+0000","last_undegraded":"2026-02-24T07:43:50.661390+0000","last_fullsized":"2026-02-24T07:43:50.661390+0000","mapping_epoch":30,"log_start":"40'17149","ondisk_log_start":"40'17149","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_clean_scrub_stamp":"2026-02-24T07:43:06.832892+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:44:31.507398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1124073472,"num_objects":17152,"num_object_clones":0,"num_object_copies":51456,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17152,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17152,"num_write_kb":1097728,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"40'17260","reported_seq":17290,"reported_epoch":40,"state":"active+clean","last_fresh":"2026-02-24T07:43:50.658851+0000","last_change":"2026-02-24T07:43:07.842180+0000","last_active":"2026-02-24T07:43:50.658851+0000","last_peered":"2026-02-24T07:43:50.658851+0000","last_clean":"2026-02-24T07:43:50.658851+0000","last_became_active":"2026-02-24T07:43:07.841993+0000","last_became_peered":"2026-02-24T07:43:07.841993+0000","last_unstale":"2026-02-24T07:43:50.658851+0000","last_undegraded":"2026-02-24T07:43:50.658851+0000","last_fullsized":"2026-02-24T07:43:50.658851+0000","mapping_epoch":30,"log_start":"40'17257","ondisk_log_start":"40'17257","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_clean_scrub_stamp":"2026-02-24T07:43:06.832892+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:44:19.519502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1131151360,"num_objects":17260,"num_object_clones":0,"num_object_copies":51780,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17260,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17260,"num_write_kb":1104640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"40'16882","reported_seq":16912,"reported_epoch":40,"state":"active+clean","last_fresh":"2026-02-24T07:43:50.656622+0000","last_change":"2026-02-24T07:43:07.842375+0000","last_active":"2026-02-24T07:43:50.656622+0000","last_peered":"2026-02-24T07:43:50.656622+0000","last_clean":"2026-02-24T07:43:50.656622+0000","last_became_active":"2026-02-24T07:43:07.841920+0000","last_became_peered":"2026-02-24T07:43:07.841920+0000","last_unstale":"2026-02-24T07:43:50.656622+0000","last_undegraded":"2026-02-24T07:43:50.656622+0000","last_fullsized":"2026-02-24T07:43:50.656622+0000","mapping_epoch":30,"log_start":"40'16879","ondisk_log_start":"40'16879","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_clean_scrub_stamp":"2026-02-24T07:43:06.832892+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:44:14.681944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1106378752,"num_objects":16882,"num_object_clones":0,"num_object_copies":50646,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16882,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16882,"num_write_kb":1080448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.8","version":"40'17078","reported_seq":17108,"reported_epoch":40,"state":"active+clean","last_fresh":"2026-02-24T07:43:50.666625+0000","last_change":"2026-02-24T07:43:07.853992+0000","last_active":"2026-02-24T07:43:50.666625+0000","last_peered":"2026-02-24T07:43:50.666625+0000","last_clean":"2026-02-24T07:43:50.666625+0000","last_became_active":"2026-02-24T07:43:07.853758+0000","last_became_peered":"2026-02-24T07:43:07.853758+0000","last_unstale":"2026-02-24T07:43:50.666625+0000","last_undegraded":"2026-02-24T07:43:50.666625+0000","last_fullsized":"2026-02-24T07:43:50.666625+0000","mapping_epoch":30,"log_start":"40'17075","ondisk_log_start":"40'17075","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_clean_scrub_stamp":"2026-02-24T07:43:06.832892+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:44:27.436119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1119223808,"num_objects":17078,"num_object_clones":0,"num_object_copies":51234,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17078,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17078,"num_write_kb":1092992,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"40'17162","reported_seq":17192,"reported_epoch":40,"state":"active+clean","last_fresh":"2026-02-24T07:43:50.657009+0000","last_change":"2026-02-24T07:43:07.854687+0000","last_active":"2026-02-24T07:43:50.657009+0000","last_peered":"2026-02-24T07:43:50.657009+0000","last_clean":"2026-02-24T07:43:50.657009+0000","last_became_active":"2026-02-24T07:43:07.854377+0000","last_became_peered":"2026-02-24T07:43:07.854377+0000","last_unstale":"2026-02-24T07:43:50.657009+0000","last_undegraded":"2026-02-24T07:43:50.657009+0000","last_fullsized":"2026-02-24T07:43:50.657009+0000","mapping_epoch":30,"log_start":"40'17159","ondisk_log_start":"40'17159","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_clean_scrub_stamp":"2026-02-24T07:43:06.832892+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:44:34.215480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1124728832,"num_objects":17162,"num_object_clones":0,"num_object_copies":51486,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17162,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17162,"num_write_kb":1098368,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.a","version":"40'17296","reported_seq":17326,"reported_epoch":40,"state":"active+clean","last_fresh":"2026-02-24T07:43:50.663204+0000","last_change":"2026-02-24T07:43:07.841933+0000","last_active":"2026-02-24T07:43:50.663204+0000","last_peered":"2026-02-24T07:43:50.663204+0000","last_clean":"2026-02-24T07:43:50.663204+0000","last_became_active":"2026-02-24T07:43:07.841721+0000","last_became_peered":"2026-02-24T07:43:07.841721+0000","last_unstale":"2026-02-24T07:43:50.663204+0000","last_undegraded":"2026-02-24T07:43:50.663204+0000","last_fullsized":"2026-02-24T07:43:50.663204+0000","mapping_epoch":30,"log_start":"40'17293","ondisk_log_start":"40'17293","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_clean_scrub_stamp":"2026-02-24T07:43:06.832892+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:44:14.991691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1133510656,"num_objects":17296,"num_object_clones":0,"num_object_copies":51888,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17296,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17296,"num_write_kb":1106944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.b","version":"40'17301","reported_seq":17331,"reported_epoch":40,"state":"active+clean","last_fresh":"2026-02-24T07:43:50.666574+0000","last_change":"2026-02-24T07:43:07.853283+0000","last_active":"2026-02-24T07:43:50.666574+0000","last_peered":"2026-02-24T07:43:50.666574+0000","last_clean":"2026-02-24T07:43:50.666574+0000","last_became_active":"2026-02-24T07:43:07.853069+0000","last_became_peered":"2026-02-24T07:43:07.853069+0000","last_unstale":"2026-02-24T07:43:50.666574+0000","last_undegraded":"2026-02-24T07:43:50.666574+0000","last_fullsized":"2026-02-24T07:43:50.666574+0000","mapping_epoch":30,"log_start":"40'17298","ondisk_log_start":"40'17298","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_clean_scrub_stamp":"2026-02-24T07:43:06.832892+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:44:32.581403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1133838336,"num_objects":17301,"num_object_clones":0,"num_object_copies":51903,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17301,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17301,"num_write_kb":1107264,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":18047369216,"num_objects":275381,"num_object_clones":0,"num_object_copies":826143,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":275381,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":275381,"num_write_kb":17624384,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":54140862464,"data_stored":54140862464,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":48,"ondisk_log_size":48,"up":48,"acting":48,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":240,"num_write":135,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":3,"up_from":25,"seq":107374182414,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":16260876,"kb_used_data":15331512,"kb_used_omap":13,"kb_used_meta":929330,"kb_avail":717742324,"statfs":{"total":751619276800,"available":734968139776,"internally_reserved":0,"allocated":15699468288,"data_stored":15699217648,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13992,"internal_metadata":951634264},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":19,"seq":81604378640,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":12812108,"kb_used_data":12099320,"kb_used_omap":17,"kb_used_meta":712750,"kb_avail":721191092,"statfs":{"total":751619276800,"available":738499678208,"internally_reserved":0,"allocated":12389703680,"data_stored":12389453040,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":17900,"internal_metadata":729856532},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542161,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":13908616,"kb_used_data":13105524,"kb_used_omap":19,"kb_used_meta":802988,"kb_avail":720094584,"statfs":{"total":751619276800,"available":737376854016,"internally_reserved":0,"allocated":13420056576,"data_stored":13419809488,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19853,"internal_metadata":822260339},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705684,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":12595020,"kb_used_data":11882232,"kb_used_omap":23,"kb_used_meta":712744,"kb_avail":721408180,"statfs":{"total":751619276800,"available":738721976320,"internally_reserved":0,"allocated":12167405568,"data_stored":12167154928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":24410,"internal_metadata":729850022},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12398886912,"data_stored":12398886912,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":13556973568,"data_stored":13556973568,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12395806720,"data_stored":12395806720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":15789195264,"data_stored":15789195264,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-24T07:43:53.513 INFO:tasks.cephadm.ceph_manager.ceph:increase pool size by 10 2026-02-24T07:43:53.513 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph osd pool set unique_pool_0 pg_num 26 2026-02-24T07:43:53.646 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:43:53.776 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:43:53.773+0000 7f99e9244640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:43:53.776 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:43:53.773+0000 7f99e9244640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:43:53.913 INFO:teuthology.orchestra.run.trial035.stderr:Error EPERM: crimson_allow_pg_split is false; pg_num increase denied for crimson pool 2026-02-24T07:43:53.933 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 45 15 297991 297976 413.82 433.688 0.00159922 0.00241542 2026-02-24T07:43:53.952 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-24T07:43:54.073 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:43:53 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/3779297449' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2026-02-24T07:43:54.934 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 46 16 305232 305216 414.66 452.5 0.00316029 0.00241054 2026-02-24T07:43:55.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:43:54 trial035 ceph-mon[39284]: from='client.14417 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2026-02-24T07:43:55.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:43:54 trial035 ceph-mon[39284]: pgmap v41: 17 pgs: 17 active+clean; 17 GiB data, 53 GiB used, 2.7 TiB / 2.7 TiB avail; 350 MiB/s wr, 5.59k op/s 2026-02-24T07:43:55.205 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:43:54 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/4129475265' entity='client.admin' cmd={"prefix": "osd pool set", "pool": "unique_pool_0", "var": "pg_num", "val": "26"} : dispatch 2026-02-24T07:43:55.259 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2026-02-24T07:43:55.402 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:43:55.519 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:43:55.516+0000 7f3031a06640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:43:55.519 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:43:55.517+0000 7f3031a06640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:43:55.532 INFO:teuthology.orchestra.run.trial035.stdout:osd.0: {} 2026-02-24T07:43:55.533 INFO:teuthology.orchestra.run.trial035.stderr:osd.0: osd_enable_op_tracker = '' 2026-02-24T07:43:55.547 INFO:teuthology.orchestra.run.trial035.stdout:osd.1: {} 2026-02-24T07:43:55.547 INFO:teuthology.orchestra.run.trial035.stderr:osd.1: osd_enable_op_tracker = '' 2026-02-24T07:43:55.558 INFO:teuthology.orchestra.run.trial035.stdout:osd.2: {} 2026-02-24T07:43:55.559 INFO:teuthology.orchestra.run.trial035.stderr:osd.2: osd_enable_op_tracker = '' 2026-02-24T07:43:55.572 INFO:teuthology.orchestra.run.trial035.stdout:osd.3: {} 2026-02-24T07:43:55.572 INFO:teuthology.orchestra.run.trial035.stderr:osd.3: osd_enable_op_tracker = '' 2026-02-24T07:43:55.933 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 47 16 312244 312228 415.161 438.25 0.00117634 0.00240762 2026-02-24T07:43:56.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:43:55 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/3796247799' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2026-02-24T07:43:56.933 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 48 15 319309 319294 415.712 441.625 0.00227554 0.00240442 2026-02-24T07:43:57.105 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:43:56 trial035 ceph-mon[39284]: pgmap v42: 17 pgs: 17 active+clean; 17 GiB data, 56 GiB used, 2.7 TiB / 2.7 TiB avail; 361 MiB/s wr, 5.78k op/s 2026-02-24T07:43:57.106 INFO:journalctl@ceph.mgr.x.trial035.stdout:Feb 24 07:43:56 trial035 ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-mgr-x[39621]: ::ffff:10.20.193.91 - - [24/Feb/2026:07:43:56] "GET /metrics HTTP/1.1" 200 4214 "" "Prometheus/3.6.0" 2026-02-24T07:43:57.356 INFO:journalctl@ceph.prometheus.a.trial091.stdout:Feb 24 07:43:56 trial091 ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-prometheus-a[66007]: time=2026-02-24T07:43:56.973Z level=ERROR source=scrape.go:1630 msg="Failed to determine correct type of scrape target." component="scrape manager" scrape_pool=ceph target=http://trial035:9283/metrics content_type="text/html;charset=utf-8" fallback_media_type="" err="received unsupported Content-Type \"text/html;charset=utf-8\" and no fallback_scrape_protocol specified for target" 2026-02-24T07:43:57.614 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2026-02-24T07:43:57.752 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:43:57.871 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:43:57.869+0000 7f1abb4b8640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:43:57.872 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:43:57.870+0000 7f1abb4b8640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:43:57.885 INFO:teuthology.orchestra.run.trial035.stdout:osd.0: {} 2026-02-24T07:43:57.885 INFO:teuthology.orchestra.run.trial035.stderr:osd.0: osd_enable_op_tracker = '' 2026-02-24T07:43:57.898 INFO:teuthology.orchestra.run.trial035.stdout:osd.1: {} 2026-02-24T07:43:57.898 INFO:teuthology.orchestra.run.trial035.stderr:osd.1: osd_enable_op_tracker = '' 2026-02-24T07:43:57.911 INFO:teuthology.orchestra.run.trial035.stdout:osd.2: {} 2026-02-24T07:43:57.911 INFO:teuthology.orchestra.run.trial035.stderr:osd.2: osd_enable_op_tracker = '' 2026-02-24T07:43:57.924 INFO:teuthology.orchestra.run.trial035.stdout:osd.3: {} 2026-02-24T07:43:57.924 INFO:teuthology.orchestra.run.trial035.stderr:osd.3: osd_enable_op_tracker = '' 2026-02-24T07:43:57.933 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 49 15 326517 326502 416.421 450.5 0.00154784 0.00240034 2026-02-24T07:43:58.934 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 50 15 333888 333873 417.306 460.688 0.00267067 0.00239524 2026-02-24T07:43:58.954 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2026-02-24T07:43:58.954 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2026-02-24T07:43:58.954 INFO:tasks.thrashosds.thrasher:fixing pg num pool .mgr 2026-02-24T07:43:58.954 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph pg dump --format=json 2026-02-24T07:43:59.092 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:43:59.115 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:43:58 trial035 ceph-mon[39284]: pgmap v43: 17 pgs: 17 active+clean; 19 GiB data, 60 GiB used, 2.7 TiB / 2.7 TiB avail; 521 MiB/s wr, 8.33k op/s 2026-02-24T07:43:59.115 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:43:58 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/17334942' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2026-02-24T07:43:59.206 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:43:59.205+0000 7fb9e2b4a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:43:59.206 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:43:59.205+0000 7fb9e2b4a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:43:59.350 INFO:teuthology.orchestra.run.trial035.stdout: 2026-02-24T07:43:59.350 INFO:teuthology.orchestra.run.trial035.stderr:dumped all 2026-02-24T07:43:59.394 INFO:teuthology.orchestra.run.trial035.stdout:{"pg_ready":true,"pg_map":{"version":43,"stamp":"2026-02-24T07:43:57.666471+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":20337066528,"num_objects":310312,"num_object_clones":0,"num_object_copies":930936,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":310312,"num_whiteouts":0,"num_read":136,"num_read_kb":240,"num_write":310445,"num_write_kb":19861990,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":51,"ondisk_log_size":51,"up":51,"acting":51,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":51,"num_osds":4,"num_per_pool_osds":4,"num_per_pool_omap_osds":4,"kb":2936012800,"kb_used":62692780,"kb_used_data":59119708,"kb_used_omap":75,"kb_used_meta":3572916,"kb_avail":2873320020,"statfs":{"total":3006477107200,"available":2942279700480,"internally_reserved":0,"allocated":60538580992,"data_stored":60537581984,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":77457,"internal_metadata":3658666351},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":6553600000,"num_objects":100000,"num_object_clones":0,"num_object_copies":300000,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":100000,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":100000,"num_write_kb":6400000,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001592"},"pg_stats":[{"pgid":"2.f","version":"40'19251","reported_seq":19281,"reported_epoch":40,"state":"active+clean","last_fresh":"2026-02-24T07:43:55.645615+0000","last_change":"2026-02-24T07:43:07.853034+0000","last_active":"2026-02-24T07:43:55.645615+0000","last_peered":"2026-02-24T07:43:55.645615+0000","last_clean":"2026-02-24T07:43:55.645615+0000","last_became_active":"2026-02-24T07:43:07.852899+0000","last_became_peered":"2026-02-24T07:43:07.852899+0000","last_unstale":"2026-02-24T07:43:55.645615+0000","last_undegraded":"2026-02-24T07:43:55.645615+0000","last_fullsized":"2026-02-24T07:43:55.645615+0000","mapping_epoch":30,"log_start":"40'19248","ondisk_log_start":"40'19248","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_clean_scrub_stamp":"2026-02-24T07:43:06.832892+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:44:18.843618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1261633536,"num_objects":19251,"num_object_clones":0,"num_object_copies":57753,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19251,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19251,"num_write_kb":1232064,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.e","version":"40'19566","reported_seq":19596,"reported_epoch":40,"state":"active+clean","last_fresh":"2026-02-24T07:43:55.663161+0000","last_change":"2026-02-24T07:43:07.853009+0000","last_active":"2026-02-24T07:43:55.663161+0000","last_peered":"2026-02-24T07:43:55.663161+0000","last_clean":"2026-02-24T07:43:55.663161+0000","last_became_active":"2026-02-24T07:43:07.852815+0000","last_became_peered":"2026-02-24T07:43:07.852815+0000","last_unstale":"2026-02-24T07:43:55.663161+0000","last_undegraded":"2026-02-24T07:43:55.663161+0000","last_fullsized":"2026-02-24T07:43:55.663161+0000","mapping_epoch":30,"log_start":"40'19563","ondisk_log_start":"40'19563","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_clean_scrub_stamp":"2026-02-24T07:43:06.832892+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:44:11.750665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1282277376,"num_objects":19566,"num_object_clones":0,"num_object_copies":58698,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19566,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19566,"num_write_kb":1252224,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.d","version":"40'19546","reported_seq":19576,"reported_epoch":40,"state":"active+clean","last_fresh":"2026-02-24T07:43:55.665730+0000","last_change":"2026-02-24T07:43:07.854403+0000","last_active":"2026-02-24T07:43:55.665730+0000","last_peered":"2026-02-24T07:43:55.665730+0000","last_clean":"2026-02-24T07:43:55.665730+0000","last_became_active":"2026-02-24T07:43:07.853975+0000","last_became_peered":"2026-02-24T07:43:07.853975+0000","last_unstale":"2026-02-24T07:43:55.665730+0000","last_undegraded":"2026-02-24T07:43:55.665730+0000","last_fullsized":"2026-02-24T07:43:55.665730+0000","mapping_epoch":30,"log_start":"40'19543","ondisk_log_start":"40'19543","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_clean_scrub_stamp":"2026-02-24T07:43:06.832892+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:44:14.520704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1280966656,"num_objects":19546,"num_object_clones":0,"num_object_copies":58638,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19546,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19546,"num_write_kb":1250944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"40'19501","reported_seq":19531,"reported_epoch":40,"state":"active+clean","last_fresh":"2026-02-24T07:43:55.657737+0000","last_change":"2026-02-24T07:43:07.854780+0000","last_active":"2026-02-24T07:43:55.657737+0000","last_peered":"2026-02-24T07:43:55.657737+0000","last_clean":"2026-02-24T07:43:55.657737+0000","last_became_active":"2026-02-24T07:43:07.854428+0000","last_became_peered":"2026-02-24T07:43:07.854428+0000","last_unstale":"2026-02-24T07:43:55.657737+0000","last_undegraded":"2026-02-24T07:43:55.657737+0000","last_fullsized":"2026-02-24T07:43:55.657737+0000","mapping_epoch":30,"log_start":"40'19498","ondisk_log_start":"40'19498","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_clean_scrub_stamp":"2026-02-24T07:43:06.832892+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:44:16.718883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1278017536,"num_objects":19501,"num_object_clones":0,"num_object_copies":58503,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19501,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19501,"num_write_kb":1248064,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"29'99","reported_seq":201,"reported_epoch":40,"state":"active+clean","last_fresh":"2026-02-24T07:43:37.766077+0000","last_change":"2026-02-24T07:42:54.186862+0000","last_active":"2026-02-24T07:43:37.766077+0000","last_peered":"2026-02-24T07:43:37.766077+0000","last_clean":"2026-02-24T07:43:37.766077+0000","last_became_active":"2026-02-24T07:42:54.186719+0000","last_became_peered":"2026-02-24T07:42:54.186719+0000","last_unstale":"2026-02-24T07:43:37.766077+0000","last_undegraded":"2026-02-24T07:43:37.766077+0000","last_fullsized":"2026-02-24T07:43:37.766077+0000","mapping_epoch":27,"log_start":"29'96","ondisk_log_start":"29'96","created":20,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:42:43.640776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:42:43.640776+0000","last_clean_scrub_stamp":"2026-02-24T07:42:43.640776+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":96,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:43:57.783028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":240,"num_write":135,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"40'19372","reported_seq":19402,"reported_epoch":40,"state":"active+clean","last_fresh":"2026-02-24T07:43:55.663043+0000","last_change":"2026-02-24T07:43:07.842967+0000","last_active":"2026-02-24T07:43:55.663043+0000","last_peered":"2026-02-24T07:43:55.663043+0000","last_clean":"2026-02-24T07:43:55.663043+0000","last_became_active":"2026-02-24T07:43:07.842530+0000","last_became_peered":"2026-02-24T07:43:07.842530+0000","last_unstale":"2026-02-24T07:43:55.663043+0000","last_undegraded":"2026-02-24T07:43:55.663043+0000","last_fullsized":"2026-02-24T07:43:55.663043+0000","mapping_epoch":30,"log_start":"40'19369","ondisk_log_start":"40'19369","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_clean_scrub_stamp":"2026-02-24T07:43:06.832892+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:44:14.900069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1269563392,"num_objects":19372,"num_object_clones":0,"num_object_copies":58116,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19372,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19372,"num_write_kb":1239808,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"40'19282","reported_seq":19312,"reported_epoch":40,"state":"active+clean","last_fresh":"2026-02-24T07:43:55.661659+0000","last_change":"2026-02-24T07:43:07.854082+0000","last_active":"2026-02-24T07:43:55.661659+0000","last_peered":"2026-02-24T07:43:55.661659+0000","last_clean":"2026-02-24T07:43:55.661659+0000","last_became_active":"2026-02-24T07:43:07.853885+0000","last_became_peered":"2026-02-24T07:43:07.853885+0000","last_unstale":"2026-02-24T07:43:55.661659+0000","last_undegraded":"2026-02-24T07:43:55.661659+0000","last_fullsized":"2026-02-24T07:43:55.661659+0000","mapping_epoch":30,"log_start":"40'19279","ondisk_log_start":"40'19279","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_clean_scrub_stamp":"2026-02-24T07:43:06.832892+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:44:20.512438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1263665152,"num_objects":19282,"num_object_clones":0,"num_object_copies":57846,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19282,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19282,"num_write_kb":1234048,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"40'19471","reported_seq":19501,"reported_epoch":40,"state":"active+clean","last_fresh":"2026-02-24T07:43:55.660687+0000","last_change":"2026-02-24T07:43:07.854497+0000","last_active":"2026-02-24T07:43:55.660687+0000","last_peered":"2026-02-24T07:43:55.660687+0000","last_clean":"2026-02-24T07:43:55.660687+0000","last_became_active":"2026-02-24T07:43:07.854212+0000","last_became_peered":"2026-02-24T07:43:07.854212+0000","last_unstale":"2026-02-24T07:43:55.660687+0000","last_undegraded":"2026-02-24T07:43:55.660687+0000","last_fullsized":"2026-02-24T07:43:55.660687+0000","mapping_epoch":30,"log_start":"40'19468","ondisk_log_start":"40'19468","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_clean_scrub_stamp":"2026-02-24T07:43:06.832892+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:44:21.531086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1276051456,"num_objects":19471,"num_object_clones":0,"num_object_copies":58413,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19471,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19471,"num_write_kb":1246144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[1,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"40'19581","reported_seq":19611,"reported_epoch":40,"state":"active+clean","last_fresh":"2026-02-24T07:43:55.658176+0000","last_change":"2026-02-24T07:43:07.843123+0000","last_active":"2026-02-24T07:43:55.658176+0000","last_peered":"2026-02-24T07:43:55.658176+0000","last_clean":"2026-02-24T07:43:55.658176+0000","last_became_active":"2026-02-24T07:43:07.842864+0000","last_became_peered":"2026-02-24T07:43:07.842864+0000","last_unstale":"2026-02-24T07:43:55.658176+0000","last_undegraded":"2026-02-24T07:43:55.658176+0000","last_fullsized":"2026-02-24T07:43:55.658176+0000","mapping_epoch":30,"log_start":"40'19578","ondisk_log_start":"40'19578","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_clean_scrub_stamp":"2026-02-24T07:43:06.832892+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:44:34.876596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1283260416,"num_objects":19581,"num_object_clones":0,"num_object_copies":58743,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19581,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19581,"num_write_kb":1253184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.4","version":"40'19295","reported_seq":19325,"reported_epoch":40,"state":"active+clean","last_fresh":"2026-02-24T07:43:55.659753+0000","last_change":"2026-02-24T07:43:07.854462+0000","last_active":"2026-02-24T07:43:55.659753+0000","last_peered":"2026-02-24T07:43:55.659753+0000","last_clean":"2026-02-24T07:43:55.659753+0000","last_became_active":"2026-02-24T07:43:07.853966+0000","last_became_peered":"2026-02-24T07:43:07.853966+0000","last_unstale":"2026-02-24T07:43:55.659753+0000","last_undegraded":"2026-02-24T07:43:55.659753+0000","last_fullsized":"2026-02-24T07:43:55.659753+0000","mapping_epoch":30,"log_start":"40'19292","ondisk_log_start":"40'19292","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_clean_scrub_stamp":"2026-02-24T07:43:06.832892+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:44:13.031431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1264517120,"num_objects":19295,"num_object_clones":0,"num_object_copies":57885,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19295,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19295,"num_write_kb":1234880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"40'19378","reported_seq":19408,"reported_epoch":40,"state":"active+clean","last_fresh":"2026-02-24T07:43:55.662709+0000","last_change":"2026-02-24T07:43:07.854184+0000","last_active":"2026-02-24T07:43:55.662709+0000","last_peered":"2026-02-24T07:43:55.662709+0000","last_clean":"2026-02-24T07:43:55.662709+0000","last_became_active":"2026-02-24T07:43:07.853878+0000","last_became_peered":"2026-02-24T07:43:07.853878+0000","last_unstale":"2026-02-24T07:43:55.662709+0000","last_undegraded":"2026-02-24T07:43:55.662709+0000","last_fullsized":"2026-02-24T07:43:55.662709+0000","mapping_epoch":30,"log_start":"40'19375","ondisk_log_start":"40'19375","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_clean_scrub_stamp":"2026-02-24T07:43:06.832892+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:44:31.507398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1269956608,"num_objects":19378,"num_object_clones":0,"num_object_copies":58134,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19378,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19378,"num_write_kb":1240192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"40'19489","reported_seq":19519,"reported_epoch":40,"state":"active+clean","last_fresh":"2026-02-24T07:43:55.660629+0000","last_change":"2026-02-24T07:43:07.842180+0000","last_active":"2026-02-24T07:43:55.660629+0000","last_peered":"2026-02-24T07:43:55.660629+0000","last_clean":"2026-02-24T07:43:55.660629+0000","last_became_active":"2026-02-24T07:43:07.841993+0000","last_became_peered":"2026-02-24T07:43:07.841993+0000","last_unstale":"2026-02-24T07:43:55.660629+0000","last_undegraded":"2026-02-24T07:43:55.660629+0000","last_fullsized":"2026-02-24T07:43:55.660629+0000","mapping_epoch":30,"log_start":"40'19486","ondisk_log_start":"40'19486","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_clean_scrub_stamp":"2026-02-24T07:43:06.832892+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:44:19.519502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1277231104,"num_objects":19489,"num_object_clones":0,"num_object_copies":58467,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19489,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19489,"num_write_kb":1247296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"40'19015","reported_seq":19044,"reported_epoch":40,"state":"active+clean","last_fresh":"2026-02-24T07:43:55.665805+0000","last_change":"2026-02-24T07:43:07.842375+0000","last_active":"2026-02-24T07:43:55.665805+0000","last_peered":"2026-02-24T07:43:55.665805+0000","last_clean":"2026-02-24T07:43:55.665805+0000","last_became_active":"2026-02-24T07:43:07.841920+0000","last_became_peered":"2026-02-24T07:43:07.841920+0000","last_unstale":"2026-02-24T07:43:55.665805+0000","last_undegraded":"2026-02-24T07:43:55.665805+0000","last_fullsized":"2026-02-24T07:43:55.665805+0000","mapping_epoch":30,"log_start":"40'19012","ondisk_log_start":"40'19012","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_clean_scrub_stamp":"2026-02-24T07:43:06.832892+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:44:14.681944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1246167040,"num_objects":19015,"num_object_clones":0,"num_object_copies":57045,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19015,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19015,"num_write_kb":1216960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.8","version":"40'19198","reported_seq":19228,"reported_epoch":40,"state":"active+clean","last_fresh":"2026-02-24T07:43:55.659839+0000","last_change":"2026-02-24T07:43:07.853992+0000","last_active":"2026-02-24T07:43:55.659839+0000","last_peered":"2026-02-24T07:43:55.659839+0000","last_clean":"2026-02-24T07:43:55.659839+0000","last_became_active":"2026-02-24T07:43:07.853758+0000","last_became_peered":"2026-02-24T07:43:07.853758+0000","last_unstale":"2026-02-24T07:43:55.659839+0000","last_undegraded":"2026-02-24T07:43:55.659839+0000","last_fullsized":"2026-02-24T07:43:55.659839+0000","mapping_epoch":30,"log_start":"40'19195","ondisk_log_start":"40'19195","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_clean_scrub_stamp":"2026-02-24T07:43:06.832892+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:44:27.436119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1258160128,"num_objects":19198,"num_object_clones":0,"num_object_copies":57594,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19198,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19198,"num_write_kb":1228672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"40'19383","reported_seq":19413,"reported_epoch":40,"state":"active+clean","last_fresh":"2026-02-24T07:43:55.665661+0000","last_change":"2026-02-24T07:43:07.854687+0000","last_active":"2026-02-24T07:43:55.665661+0000","last_peered":"2026-02-24T07:43:55.665661+0000","last_clean":"2026-02-24T07:43:55.665661+0000","last_became_active":"2026-02-24T07:43:07.854377+0000","last_became_peered":"2026-02-24T07:43:07.854377+0000","last_unstale":"2026-02-24T07:43:55.665661+0000","last_undegraded":"2026-02-24T07:43:55.665661+0000","last_fullsized":"2026-02-24T07:43:55.665661+0000","mapping_epoch":30,"log_start":"40'19380","ondisk_log_start":"40'19380","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_clean_scrub_stamp":"2026-02-24T07:43:06.832892+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:44:34.215480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1270284288,"num_objects":19383,"num_object_clones":0,"num_object_copies":58149,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19383,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19383,"num_write_kb":1240512,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.a","version":"40'19484","reported_seq":19514,"reported_epoch":40,"state":"active+clean","last_fresh":"2026-02-24T07:43:55.660168+0000","last_change":"2026-02-24T07:43:07.841933+0000","last_active":"2026-02-24T07:43:55.660168+0000","last_peered":"2026-02-24T07:43:55.660168+0000","last_clean":"2026-02-24T07:43:55.660168+0000","last_became_active":"2026-02-24T07:43:07.841721+0000","last_became_peered":"2026-02-24T07:43:07.841721+0000","last_unstale":"2026-02-24T07:43:55.660168+0000","last_undegraded":"2026-02-24T07:43:55.660168+0000","last_fullsized":"2026-02-24T07:43:55.660168+0000","mapping_epoch":30,"log_start":"40'19481","ondisk_log_start":"40'19481","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_clean_scrub_stamp":"2026-02-24T07:43:06.832892+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:44:14.991691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1276903424,"num_objects":19484,"num_object_clones":0,"num_object_copies":58452,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19484,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19484,"num_write_kb":1246976,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.b","version":"40'19498","reported_seq":19527,"reported_epoch":40,"state":"active+clean","last_fresh":"2026-02-24T07:43:55.667455+0000","last_change":"2026-02-24T07:43:07.853283+0000","last_active":"2026-02-24T07:43:55.667455+0000","last_peered":"2026-02-24T07:43:55.667455+0000","last_clean":"2026-02-24T07:43:55.667455+0000","last_became_active":"2026-02-24T07:43:07.853069+0000","last_became_peered":"2026-02-24T07:43:07.853069+0000","last_unstale":"2026-02-24T07:43:55.667455+0000","last_undegraded":"2026-02-24T07:43:55.667455+0000","last_fullsized":"2026-02-24T07:43:55.667455+0000","mapping_epoch":30,"log_start":"40'19495","ondisk_log_start":"40'19495","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_clean_scrub_stamp":"2026-02-24T07:43:06.832892+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:44:32.581403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1277820928,"num_objects":19498,"num_object_clones":0,"num_object_copies":58494,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19498,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19498,"num_write_kb":1247872,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":20336476160,"num_objects":310310,"num_object_clones":0,"num_object_copies":930930,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":310310,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":310310,"num_write_kb":19859840,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":61009297408,"data_stored":61009297408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":48,"ondisk_log_size":48,"up":48,"acting":48,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":240,"num_write":135,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":3,"up_from":25,"seq":107374182415,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":18328908,"kb_used_data":17291256,"kb_used_omap":13,"kb_used_meta":1037618,"kb_avail":715674292,"statfs":{"total":751619276800,"available":732850475008,"internally_reserved":0,"allocated":17706246144,"data_stored":17705995504,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13992,"internal_metadata":1062521176},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":19,"seq":81604378641,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":14434636,"kb_used_data":13613560,"kb_used_omap":18,"kb_used_meta":821037,"kb_avail":719568564,"statfs":{"total":751619276800,"available":736838209536,"internally_reserved":0,"allocated":13940285440,"data_stored":13940034800,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":18551,"internal_metadata":840742793},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542162,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":15671496,"kb_used_data":14778228,"kb_used_omap":20,"kb_used_meta":893227,"kb_avail":718331704,"statfs":{"total":751619276800,"available":735571664896,"internally_reserved":0,"allocated":15132905472,"data_stored":15132658384,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":20504,"internal_metadata":914665448},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705685,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":14257740,"kb_used_data":13436664,"kb_used_omap":23,"kb_used_meta":821032,"kb_avail":719745460,"statfs":{"total":751619276800,"available":737019351040,"internally_reserved":0,"allocated":13759143936,"data_stored":13758893296,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":24410,"internal_metadata":840736934},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":13982629888,"data_stored":13982629888,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":15269429248,"data_stored":15269429248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":13966966784,"data_stored":13966966784,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":17790271488,"data_stored":17790271488,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-24T07:43:59.397 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph osd pool set .mgr pgp_num 31 2026-02-24T07:43:59.530 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:43:59.660 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:43:59.659+0000 7fb223d8b640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:43:59.661 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:43:59.659+0000 7fb223d8b640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:43:59.794 INFO:teuthology.orchestra.run.trial035.stderr:Error EPERM: crimson_allow_pg_split is false; pgp_num increase denied 2026-02-24T07:43:59.833 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-24T07:43:59.934 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 51 16 341481 341465 418.427 474.5 0.00150627 0.00238882 2026-02-24T07:43:59.954 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:43:59 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/1245302471' entity='client.admin' cmd={"prefix": "osd pool set", "pool": ".mgr", "var": "pgp_num", "val": "31"} : dispatch 2026-02-24T07:43:59.968 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2026-02-24T07:44:00.102 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:44:00.219 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:44:00.217+0000 7f6226595640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:44:00.219 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:44:00.217+0000 7f6226595640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:44:00.236 INFO:teuthology.orchestra.run.trial035.stdout:osd.0: {} 2026-02-24T07:44:00.236 INFO:teuthology.orchestra.run.trial035.stderr:osd.0: osd_enable_op_tracker = '' 2026-02-24T07:44:00.251 INFO:teuthology.orchestra.run.trial035.stdout:osd.1: {} 2026-02-24T07:44:00.251 INFO:teuthology.orchestra.run.trial035.stderr:osd.1: osd_enable_op_tracker = '' 2026-02-24T07:44:00.264 INFO:teuthology.orchestra.run.trial035.stdout:osd.2: {} 2026-02-24T07:44:00.264 INFO:teuthology.orchestra.run.trial035.stderr:osd.2: osd_enable_op_tracker = '' 2026-02-24T07:44:00.276 INFO:teuthology.orchestra.run.trial035.stdout:osd.3: {} 2026-02-24T07:44:00.277 INFO:teuthology.orchestra.run.trial035.stderr:osd.3: osd_enable_op_tracker = '' 2026-02-24T07:44:00.934 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 52 15 348730 348715 419.093 453.125 0.00471667 0.00238502 2026-02-24T07:44:01.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:44:00 trial035 ceph-mon[39284]: from='client.14425 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2026-02-24T07:44:01.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:44:00 trial035 ceph-mon[39284]: 1.0 scrub starts 2026-02-24T07:44:01.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:44:00 trial035 ceph-mon[39284]: 1.0 scrub ok 2026-02-24T07:44:01.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:44:00 trial035 ceph-mon[39284]: pgmap v44: 17 pgs: 17 active+clean; 19 GiB data, 60 GiB used, 2.7 TiB / 2.7 TiB avail; 356 MiB/s wr, 5.70k op/s 2026-02-24T07:44:01.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:44:00 trial035 ceph-mon[39284]: from='mgr.14284 10.20.193.35:0/3681373222' entity='mgr.x' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2026-02-24T07:44:01.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:44:00 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/377043491' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2026-02-24T07:44:01.934 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 53 16 355710 355694 419.415 436.188 0.00221732 0.0023832 2026-02-24T07:44:02.317 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2026-02-24T07:44:02.455 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:44:02.577 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:44:02.575+0000 7ff061acf640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:44:02.577 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:44:02.575+0000 7ff061acf640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:44:02.591 INFO:teuthology.orchestra.run.trial035.stdout:osd.0: {} 2026-02-24T07:44:02.591 INFO:teuthology.orchestra.run.trial035.stderr:osd.0: osd_enable_op_tracker = '' 2026-02-24T07:44:02.603 INFO:teuthology.orchestra.run.trial035.stdout:osd.1: {} 2026-02-24T07:44:02.603 INFO:teuthology.orchestra.run.trial035.stderr:osd.1: osd_enable_op_tracker = '' 2026-02-24T07:44:02.615 INFO:teuthology.orchestra.run.trial035.stdout:osd.2: {} 2026-02-24T07:44:02.615 INFO:teuthology.orchestra.run.trial035.stderr:osd.2: osd_enable_op_tracker = '' 2026-02-24T07:44:02.627 INFO:teuthology.orchestra.run.trial035.stdout:osd.3: {} 2026-02-24T07:44:02.627 INFO:teuthology.orchestra.run.trial035.stderr:osd.3: osd_enable_op_tracker = '' 2026-02-24T07:44:02.934 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 54 16 362781 362765 419.832 441.938 0.000950795 0.00238079 2026-02-24T07:44:02.954 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:44:02 trial035 ceph-mon[39284]: pgmap v45: 17 pgs: 17 active+clean; 21 GiB data, 67 GiB used, 2.7 TiB / 2.7 TiB avail; 546 MiB/s wr, 8.74k op/s 2026-02-24T07:44:02.955 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:44:02 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/1500653966' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2026-02-24T07:44:03.934 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 55 15 369810 369795 420.186 439.375 0.00129014 0.00237882 2026-02-24T07:44:04.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:44:03 trial035 ceph-mon[39284]: purged_snaps scrub starts 2026-02-24T07:44:04.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:44:03 trial035 ceph-mon[39284]: purged_snaps scrub ok 2026-02-24T07:44:04.666 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2026-02-24T07:44:04.800 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:44:04.835 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2026-02-24T07:44:04.835 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2026-02-24T07:44:04.835 INFO:tasks.thrashosds.thrasher:Growing pool .mgr 2026-02-24T07:44:04.835 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph pg dump --format=json 2026-02-24T07:44:04.918 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:44:04.916+0000 7f085ef54640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:44:04.918 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:44:04.916+0000 7f085ef54640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:44:04.933 INFO:teuthology.orchestra.run.trial035.stdout:osd.0: {} 2026-02-24T07:44:04.933 INFO:teuthology.orchestra.run.trial035.stderr:osd.0: osd_enable_op_tracker = '' 2026-02-24T07:44:04.934 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 56 16 376651 376635 420.316 427.5 0.00212134 0.00237808 2026-02-24T07:44:04.949 INFO:teuthology.orchestra.run.trial035.stdout:osd.1: {} 2026-02-24T07:44:04.949 INFO:teuthology.orchestra.run.trial035.stderr:osd.1: osd_enable_op_tracker = '' 2026-02-24T07:44:04.949 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:44:04 trial035 ceph-mon[39284]: pgmap v46: 17 pgs: 17 active+clean; 21 GiB data, 67 GiB used, 2.7 TiB / 2.7 TiB avail; 372 MiB/s wr, 5.96k op/s 2026-02-24T07:44:04.963 INFO:teuthology.orchestra.run.trial035.stdout:osd.2: {} 2026-02-24T07:44:04.963 INFO:teuthology.orchestra.run.trial035.stderr:osd.2: osd_enable_op_tracker = '' 2026-02-24T07:44:04.977 INFO:teuthology.orchestra.run.trial035.stdout:osd.3: {} 2026-02-24T07:44:04.978 INFO:teuthology.orchestra.run.trial035.stderr:osd.3: osd_enable_op_tracker = '' 2026-02-24T07:44:04.982 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:44:05.105 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:44:05.104+0000 7f690d8c8640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:44:05.105 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:44:05.104+0000 7f690d8c8640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:44:05.243 INFO:teuthology.orchestra.run.trial035.stdout: 2026-02-24T07:44:05.244 INFO:teuthology.orchestra.run.trial035.stderr:dumped all 2026-02-24T07:44:05.287 INFO:teuthology.orchestra.run.trial035.stdout:{"pg_ready":true,"pg_map":{"version":46,"stamp":"2026-02-24T07:44:03.667275+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":22732800544,"num_objects":346868,"num_object_clones":0,"num_object_copies":1040604,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":346868,"num_whiteouts":0,"num_read":136,"num_read_kb":240,"num_write":347001,"num_write_kb":22201574,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":52,"ondisk_log_size":52,"up":51,"acting":51,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":51,"num_osds":4,"num_per_pool_osds":4,"num_per_pool_omap_osds":4,"kb":2936012800,"kb_used":70175020,"kb_used_data":66150748,"kb_used_omap":75,"kb_used_meta":4024116,"kb_avail":2865837780,"statfs":{"total":3006477107200,"available":2934617886720,"internally_reserved":0,"allocated":67738365952,"data_stored":67737366944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":77457,"internal_metadata":4120695151},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":4684840960,"num_objects":71485,"num_object_clones":0,"num_object_copies":214455,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":71485,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":71485,"num_write_kb":4575040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001615"},"pg_stats":[{"pgid":"2.f","version":"40'21494","reported_seq":21524,"reported_epoch":40,"state":"active+clean","last_fresh":"2026-02-24T07:44:00.661675+0000","last_change":"2026-02-24T07:43:07.853034+0000","last_active":"2026-02-24T07:44:00.661675+0000","last_peered":"2026-02-24T07:44:00.661675+0000","last_clean":"2026-02-24T07:44:00.661675+0000","last_became_active":"2026-02-24T07:43:07.852899+0000","last_became_peered":"2026-02-24T07:43:07.852899+0000","last_unstale":"2026-02-24T07:44:00.661675+0000","last_undegraded":"2026-02-24T07:44:00.661675+0000","last_fullsized":"2026-02-24T07:44:00.661675+0000","mapping_epoch":30,"log_start":"40'21491","ondisk_log_start":"40'21491","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_clean_scrub_stamp":"2026-02-24T07:43:06.832892+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:44:18.843618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1408630784,"num_objects":21494,"num_object_clones":0,"num_object_copies":64482,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21494,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21494,"num_write_kb":1375616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.e","version":"40'21827","reported_seq":21857,"reported_epoch":40,"state":"active+clean","last_fresh":"2026-02-24T07:44:00.665506+0000","last_change":"2026-02-24T07:43:07.853009+0000","last_active":"2026-02-24T07:44:00.665506+0000","last_peered":"2026-02-24T07:44:00.665506+0000","last_clean":"2026-02-24T07:44:00.665506+0000","last_became_active":"2026-02-24T07:43:07.852815+0000","last_became_peered":"2026-02-24T07:43:07.852815+0000","last_unstale":"2026-02-24T07:44:00.665506+0000","last_undegraded":"2026-02-24T07:44:00.665506+0000","last_fullsized":"2026-02-24T07:44:00.665506+0000","mapping_epoch":30,"log_start":"40'21824","ondisk_log_start":"40'21824","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_clean_scrub_stamp":"2026-02-24T07:43:06.832892+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:44:11.750665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1430454272,"num_objects":21827,"num_object_clones":0,"num_object_copies":65481,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21827,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21827,"num_write_kb":1396928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.d","version":"40'21780","reported_seq":21810,"reported_epoch":40,"state":"active+clean","last_fresh":"2026-02-24T07:44:00.667318+0000","last_change":"2026-02-24T07:43:07.854403+0000","last_active":"2026-02-24T07:44:00.667318+0000","last_peered":"2026-02-24T07:44:00.667318+0000","last_clean":"2026-02-24T07:44:00.667318+0000","last_became_active":"2026-02-24T07:43:07.853975+0000","last_became_peered":"2026-02-24T07:43:07.853975+0000","last_unstale":"2026-02-24T07:44:00.667318+0000","last_undegraded":"2026-02-24T07:44:00.667318+0000","last_fullsized":"2026-02-24T07:44:00.667318+0000","mapping_epoch":30,"log_start":"40'21776","ondisk_log_start":"40'21776","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_clean_scrub_stamp":"2026-02-24T07:43:06.832892+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":3000,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:44:14.520704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1427374080,"num_objects":21780,"num_object_clones":0,"num_object_copies":65340,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21780,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21780,"num_write_kb":1393920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"40'21791","reported_seq":21821,"reported_epoch":40,"state":"active+clean","last_fresh":"2026-02-24T07:44:00.651930+0000","last_change":"2026-02-24T07:43:07.854780+0000","last_active":"2026-02-24T07:44:00.651930+0000","last_peered":"2026-02-24T07:44:00.651930+0000","last_clean":"2026-02-24T07:44:00.651930+0000","last_became_active":"2026-02-24T07:43:07.854428+0000","last_became_peered":"2026-02-24T07:43:07.854428+0000","last_unstale":"2026-02-24T07:44:00.651930+0000","last_undegraded":"2026-02-24T07:44:00.651930+0000","last_fullsized":"2026-02-24T07:44:00.651930+0000","mapping_epoch":30,"log_start":"40'21788","ondisk_log_start":"40'21788","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_clean_scrub_stamp":"2026-02-24T07:43:06.832892+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:44:16.718883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1428094976,"num_objects":21791,"num_object_clones":0,"num_object_copies":65373,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21791,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21791,"num_write_kb":1394624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"29'99","reported_seq":209,"reported_epoch":40,"state":"active+clean","last_fresh":"2026-02-24T07:43:59.386353+0000","last_change":"2026-02-24T07:43:59.386353+0000","last_active":"2026-02-24T07:43:59.386353+0000","last_peered":"2026-02-24T07:43:59.386353+0000","last_clean":"2026-02-24T07:43:59.386353+0000","last_became_active":"2026-02-24T07:42:54.186719+0000","last_became_peered":"2026-02-24T07:42:54.186719+0000","last_unstale":"2026-02-24T07:43:59.386353+0000","last_undegraded":"2026-02-24T07:43:59.386353+0000","last_fullsized":"2026-02-24T07:43:59.386353+0000","mapping_epoch":27,"log_start":"29'96","ondisk_log_start":"29'96","created":20,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'99","last_scrub_stamp":"2026-02-24T07:43:59.386324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:42:43.640776+0000","last_clean_scrub_stamp":"2026-02-24T07:43:59.386324+0000","objects_scrubbed":2,"log_size":3,"log_dups_size":96,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:45:24.339819+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":240,"num_write":135,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"40'21713","reported_seq":21743,"reported_epoch":40,"state":"active+clean","last_fresh":"2026-02-24T07:44:00.665597+0000","last_change":"2026-02-24T07:43:07.842967+0000","last_active":"2026-02-24T07:44:00.665597+0000","last_peered":"2026-02-24T07:44:00.665597+0000","last_clean":"2026-02-24T07:44:00.665597+0000","last_became_active":"2026-02-24T07:43:07.842530+0000","last_became_peered":"2026-02-24T07:43:07.842530+0000","last_unstale":"2026-02-24T07:44:00.665597+0000","last_undegraded":"2026-02-24T07:44:00.665597+0000","last_fullsized":"2026-02-24T07:44:00.665597+0000","mapping_epoch":30,"log_start":"40'21710","ondisk_log_start":"40'21710","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_clean_scrub_stamp":"2026-02-24T07:43:06.832892+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:44:14.900069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1422983168,"num_objects":21713,"num_object_clones":0,"num_object_copies":65139,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21713,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21713,"num_write_kb":1389632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"40'21475","reported_seq":21505,"reported_epoch":40,"state":"active+clean","last_fresh":"2026-02-24T07:44:00.668004+0000","last_change":"2026-02-24T07:43:07.854082+0000","last_active":"2026-02-24T07:44:00.668004+0000","last_peered":"2026-02-24T07:44:00.668004+0000","last_clean":"2026-02-24T07:44:00.668004+0000","last_became_active":"2026-02-24T07:43:07.853885+0000","last_became_peered":"2026-02-24T07:43:07.853885+0000","last_unstale":"2026-02-24T07:44:00.668004+0000","last_undegraded":"2026-02-24T07:44:00.668004+0000","last_fullsized":"2026-02-24T07:44:00.668004+0000","mapping_epoch":30,"log_start":"40'21472","ondisk_log_start":"40'21472","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_clean_scrub_stamp":"2026-02-24T07:43:06.832892+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:44:20.512438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1407385600,"num_objects":21475,"num_object_clones":0,"num_object_copies":64425,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21475,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21475,"num_write_kb":1374400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"40'21735","reported_seq":21765,"reported_epoch":40,"state":"active+clean","last_fresh":"2026-02-24T07:44:00.662455+0000","last_change":"2026-02-24T07:43:07.854497+0000","last_active":"2026-02-24T07:44:00.662455+0000","last_peered":"2026-02-24T07:44:00.662455+0000","last_clean":"2026-02-24T07:44:00.662455+0000","last_became_active":"2026-02-24T07:43:07.854212+0000","last_became_peered":"2026-02-24T07:43:07.854212+0000","last_unstale":"2026-02-24T07:44:00.662455+0000","last_undegraded":"2026-02-24T07:44:00.662455+0000","last_fullsized":"2026-02-24T07:44:00.662455+0000","mapping_epoch":30,"log_start":"40'21732","ondisk_log_start":"40'21732","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_clean_scrub_stamp":"2026-02-24T07:43:06.832892+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:44:21.531086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1424424960,"num_objects":21735,"num_object_clones":0,"num_object_copies":65205,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21735,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21735,"num_write_kb":1391040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[1,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"40'21853","reported_seq":21883,"reported_epoch":40,"state":"active+clean","last_fresh":"2026-02-24T07:44:00.663781+0000","last_change":"2026-02-24T07:43:07.843123+0000","last_active":"2026-02-24T07:44:00.663781+0000","last_peered":"2026-02-24T07:44:00.663781+0000","last_clean":"2026-02-24T07:44:00.663781+0000","last_became_active":"2026-02-24T07:43:07.842864+0000","last_became_peered":"2026-02-24T07:43:07.842864+0000","last_unstale":"2026-02-24T07:44:00.663781+0000","last_undegraded":"2026-02-24T07:44:00.663781+0000","last_fullsized":"2026-02-24T07:44:00.663781+0000","mapping_epoch":30,"log_start":"40'21850","ondisk_log_start":"40'21850","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_clean_scrub_stamp":"2026-02-24T07:43:06.832892+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:44:34.876596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1432158208,"num_objects":21853,"num_object_clones":0,"num_object_copies":65559,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21853,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21853,"num_write_kb":1398592,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.4","version":"40'21640","reported_seq":21670,"reported_epoch":40,"state":"active+clean","last_fresh":"2026-02-24T07:44:00.662857+0000","last_change":"2026-02-24T07:43:07.854462+0000","last_active":"2026-02-24T07:44:00.662857+0000","last_peered":"2026-02-24T07:44:00.662857+0000","last_clean":"2026-02-24T07:44:00.662857+0000","last_became_active":"2026-02-24T07:43:07.853966+0000","last_became_peered":"2026-02-24T07:43:07.853966+0000","last_unstale":"2026-02-24T07:44:00.662857+0000","last_undegraded":"2026-02-24T07:44:00.662857+0000","last_fullsized":"2026-02-24T07:44:00.662857+0000","mapping_epoch":30,"log_start":"40'21637","ondisk_log_start":"40'21637","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_clean_scrub_stamp":"2026-02-24T07:43:06.832892+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:44:13.031431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1418199040,"num_objects":21640,"num_object_clones":0,"num_object_copies":64920,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21640,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21640,"num_write_kb":1384960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"40'21711","reported_seq":21741,"reported_epoch":40,"state":"active+clean","last_fresh":"2026-02-24T07:44:00.667266+0000","last_change":"2026-02-24T07:43:07.854184+0000","last_active":"2026-02-24T07:44:00.667266+0000","last_peered":"2026-02-24T07:44:00.667266+0000","last_clean":"2026-02-24T07:44:00.667266+0000","last_became_active":"2026-02-24T07:43:07.853878+0000","last_became_peered":"2026-02-24T07:43:07.853878+0000","last_unstale":"2026-02-24T07:44:00.667266+0000","last_undegraded":"2026-02-24T07:44:00.667266+0000","last_fullsized":"2026-02-24T07:44:00.667266+0000","mapping_epoch":30,"log_start":"40'21708","ondisk_log_start":"40'21708","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_clean_scrub_stamp":"2026-02-24T07:43:06.832892+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:44:31.507398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1422852096,"num_objects":21711,"num_object_clones":0,"num_object_copies":65133,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21711,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21711,"num_write_kb":1389504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"40'21818","reported_seq":21848,"reported_epoch":40,"state":"active+clean","last_fresh":"2026-02-24T07:44:00.663042+0000","last_change":"2026-02-24T07:43:07.842180+0000","last_active":"2026-02-24T07:44:00.663042+0000","last_peered":"2026-02-24T07:44:00.663042+0000","last_clean":"2026-02-24T07:44:00.663042+0000","last_became_active":"2026-02-24T07:43:07.841993+0000","last_became_peered":"2026-02-24T07:43:07.841993+0000","last_unstale":"2026-02-24T07:44:00.663042+0000","last_undegraded":"2026-02-24T07:44:00.663042+0000","last_fullsized":"2026-02-24T07:44:00.663042+0000","mapping_epoch":30,"log_start":"40'21815","ondisk_log_start":"40'21815","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_clean_scrub_stamp":"2026-02-24T07:43:06.832892+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:44:19.519502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1429864448,"num_objects":21818,"num_object_clones":0,"num_object_copies":65454,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21818,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21818,"num_write_kb":1396352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"40'21320","reported_seq":21350,"reported_epoch":40,"state":"active+clean","last_fresh":"2026-02-24T07:44:00.657376+0000","last_change":"2026-02-24T07:43:07.842375+0000","last_active":"2026-02-24T07:44:00.657376+0000","last_peered":"2026-02-24T07:44:00.657376+0000","last_clean":"2026-02-24T07:44:00.657376+0000","last_became_active":"2026-02-24T07:43:07.841920+0000","last_became_peered":"2026-02-24T07:43:07.841920+0000","last_unstale":"2026-02-24T07:44:00.657376+0000","last_undegraded":"2026-02-24T07:44:00.657376+0000","last_fullsized":"2026-02-24T07:44:00.657376+0000","mapping_epoch":30,"log_start":"40'21317","ondisk_log_start":"40'21317","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_clean_scrub_stamp":"2026-02-24T07:43:06.832892+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:44:14.681944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1397227520,"num_objects":21320,"num_object_clones":0,"num_object_copies":63960,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21320,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21320,"num_write_kb":1364480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.8","version":"40'21429","reported_seq":21459,"reported_epoch":40,"state":"active+clean","last_fresh":"2026-02-24T07:44:00.660431+0000","last_change":"2026-02-24T07:43:07.853992+0000","last_active":"2026-02-24T07:44:00.660431+0000","last_peered":"2026-02-24T07:44:00.660431+0000","last_clean":"2026-02-24T07:44:00.660431+0000","last_became_active":"2026-02-24T07:43:07.853758+0000","last_became_peered":"2026-02-24T07:43:07.853758+0000","last_unstale":"2026-02-24T07:44:00.660431+0000","last_undegraded":"2026-02-24T07:44:00.660431+0000","last_fullsized":"2026-02-24T07:44:00.660431+0000","mapping_epoch":30,"log_start":"40'21426","ondisk_log_start":"40'21426","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_clean_scrub_stamp":"2026-02-24T07:43:06.832892+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:44:27.436119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1404370944,"num_objects":21429,"num_object_clones":0,"num_object_copies":64287,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21429,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21429,"num_write_kb":1371456,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"40'21693","reported_seq":21723,"reported_epoch":40,"state":"active+clean","last_fresh":"2026-02-24T07:44:00.667212+0000","last_change":"2026-02-24T07:43:07.854687+0000","last_active":"2026-02-24T07:44:00.667212+0000","last_peered":"2026-02-24T07:44:00.667212+0000","last_clean":"2026-02-24T07:44:00.667212+0000","last_became_active":"2026-02-24T07:43:07.854377+0000","last_became_peered":"2026-02-24T07:43:07.854377+0000","last_unstale":"2026-02-24T07:44:00.667212+0000","last_undegraded":"2026-02-24T07:44:00.667212+0000","last_fullsized":"2026-02-24T07:44:00.667212+0000","mapping_epoch":30,"log_start":"40'21690","ondisk_log_start":"40'21690","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_clean_scrub_stamp":"2026-02-24T07:43:06.832892+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:44:34.215480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1421672448,"num_objects":21693,"num_object_clones":0,"num_object_copies":65079,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21693,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21693,"num_write_kb":1388352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.a","version":"40'21820","reported_seq":21850,"reported_epoch":40,"state":"active+clean","last_fresh":"2026-02-24T07:44:00.667173+0000","last_change":"2026-02-24T07:43:07.841933+0000","last_active":"2026-02-24T07:44:00.667173+0000","last_peered":"2026-02-24T07:44:00.667173+0000","last_clean":"2026-02-24T07:44:00.667173+0000","last_became_active":"2026-02-24T07:43:07.841721+0000","last_became_peered":"2026-02-24T07:43:07.841721+0000","last_unstale":"2026-02-24T07:44:00.667173+0000","last_undegraded":"2026-02-24T07:44:00.667173+0000","last_fullsized":"2026-02-24T07:44:00.667173+0000","mapping_epoch":30,"log_start":"40'21817","ondisk_log_start":"40'21817","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_clean_scrub_stamp":"2026-02-24T07:43:06.832892+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:44:14.991691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1429995520,"num_objects":21820,"num_object_clones":0,"num_object_copies":65460,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21820,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21820,"num_write_kb":1396480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.b","version":"40'21767","reported_seq":21797,"reported_epoch":40,"state":"active+clean","last_fresh":"2026-02-24T07:44:00.665590+0000","last_change":"2026-02-24T07:43:07.853283+0000","last_active":"2026-02-24T07:44:00.665590+0000","last_peered":"2026-02-24T07:44:00.665590+0000","last_clean":"2026-02-24T07:44:00.665590+0000","last_became_active":"2026-02-24T07:43:07.853069+0000","last_became_peered":"2026-02-24T07:43:07.853069+0000","last_unstale":"2026-02-24T07:44:00.665590+0000","last_undegraded":"2026-02-24T07:44:00.665590+0000","last_fullsized":"2026-02-24T07:44:00.665590+0000","mapping_epoch":30,"log_start":"40'21764","ondisk_log_start":"40'21764","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_clean_scrub_stamp":"2026-02-24T07:43:06.832892+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:44:32.581403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1426522112,"num_objects":21767,"num_object_clones":0,"num_object_copies":65301,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21767,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21767,"num_write_kb":1393088,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":22732210176,"num_objects":346866,"num_object_clones":0,"num_object_copies":1040598,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":346866,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":346866,"num_write_kb":22199424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":68195975168,"data_stored":68195975168,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":49,"ondisk_log_size":49,"up":48,"acting":48,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":240,"num_write":135,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":3,"up_from":25,"seq":107374182416,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":20500876,"kb_used_data":19318840,"kb_used_omap":13,"kb_used_meta":1182002,"kb_avail":713502324,"statfs":{"total":751619276800,"available":730626379776,"internally_reserved":0,"allocated":19782492160,"data_stored":19782241520,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13992,"internal_metadata":1210370392},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":19,"seq":81604378642,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":16173772,"kb_used_data":15244408,"kb_used_omap":18,"kb_used_meta":929325,"kb_avail":717829428,"statfs":{"total":751619276800,"available":735057334272,"internally_reserved":0,"allocated":15610273792,"data_stored":15610023152,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":18551,"internal_metadata":951629705},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542163,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":17519432,"kb_used_data":16517876,"kb_used_omap":20,"kb_used_meta":1001515,"kb_avail":716483768,"statfs":{"total":751619276800,"available":733679378432,"internally_reserved":0,"allocated":16914305024,"data_stored":16914057936,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":20504,"internal_metadata":1025552360},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705686,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":15980940,"kb_used_data":15069624,"kb_used_omap":23,"kb_used_meta":911272,"kb_avail":718022260,"statfs":{"total":751619276800,"available":735254794240,"internally_reserved":0,"allocated":15431294976,"data_stored":15431044336,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":24410,"internal_metadata":933142694},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":15618867200,"data_stored":15618867200,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":17070424064,"data_stored":17070424064,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":15622864896,"data_stored":15622864896,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":19883819008,"data_stored":19883819008,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-24T07:44:05.291 INFO:tasks.cephadm.ceph_manager.ceph:increase pool size by 10 2026-02-24T07:44:05.292 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph osd pool set .mgr pg_num 41 2026-02-24T07:44:05.424 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:44:05.537 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:44:05.536+0000 7f2cc40d6640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:44:05.537 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:44:05.536+0000 7f2cc40d6640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:44:05.673 INFO:teuthology.orchestra.run.trial035.stderr:Error EPERM: crimson_allow_pg_split is false; pg_num increase denied for crimson pool 2026-02-24T07:44:05.710 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-24T07:44:05.934 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 57 15 383296 383281 420.229 415.375 0.00423759 0.00237857 2026-02-24T07:44:06.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:44:05 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/3974108513' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2026-02-24T07:44:06.205 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:44:05 trial035 ceph-mon[39284]: from='mgr.14284 10.20.193.35:0/3681373222' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2026-02-24T07:44:06.205 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:44:05 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/3940019374' entity='client.admin' cmd={"prefix": "osd pool set", "pool": ".mgr", "var": "pg_num", "val": "41"} : dispatch 2026-02-24T07:44:06.205 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:44:05 trial035 ceph-mon[39284]: from='mgr.14284 10.20.193.35:0/3681373222' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-24T07:44:06.205 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:44:05 trial035 ceph-mon[39284]: from='mgr.14284 10.20.193.35:0/3681373222' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2026-02-24T07:44:06.205 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:44:05 trial035 ceph-mon[39284]: from='mgr.14284 10.20.193.35:0/3681373222' entity='mgr.x' 2026-02-24T07:44:06.205 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:44:05 trial035 ceph-mon[39284]: from='mgr.14284 10.20.193.35:0/3681373222' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2026-02-24T07:44:06.934 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 58 15 390336 390321 420.569 440 0.00250089 0.00237664 2026-02-24T07:44:07.037 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2026-02-24T07:44:07.169 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:44:07.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:44:06 trial035 ceph-mon[39284]: from='client.14435 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2026-02-24T07:44:07.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:44:06 trial035 ceph-mon[39284]: pgmap v47: 17 pgs: 17 active+clean; 21 GiB data, 70 GiB used, 2.7 TiB / 2.7 TiB avail; 383 MiB/s wr, 6.13k op/s 2026-02-24T07:44:07.204 INFO:journalctl@ceph.mgr.x.trial035.stdout:Feb 24 07:44:06 trial035 ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-mgr-x[39621]: ::ffff:10.20.193.91 - - [24/Feb/2026:07:44:06] "GET /metrics HTTP/1.1" 200 4208 "" "Prometheus/3.6.0" 2026-02-24T07:44:07.356 INFO:journalctl@ceph.prometheus.a.trial091.stdout:Feb 24 07:44:06 trial091 ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-prometheus-a[66007]: time=2026-02-24T07:44:06.973Z level=ERROR source=scrape.go:1630 msg="Failed to determine correct type of scrape target." component="scrape manager" scrape_pool=ceph target=http://trial035:9283/metrics content_type="text/html;charset=utf-8" fallback_media_type="" err="received unsupported Content-Type \"text/html;charset=utf-8\" and no fallback_scrape_protocol specified for target" 2026-02-24T07:44:07.628 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:44:07.615+0000 7f8bb180b640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:44:07.629 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:44:07.616+0000 7f8bb180b640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:44:07.629 INFO:teuthology.orchestra.run.trial035.stdout:osd.0: {} 2026-02-24T07:44:07.630 INFO:teuthology.orchestra.run.trial035.stderr:osd.0: osd_enable_op_tracker = '' 2026-02-24T07:44:07.644 INFO:teuthology.orchestra.run.trial035.stdout:osd.1: {} 2026-02-24T07:44:07.644 INFO:teuthology.orchestra.run.trial035.stderr:osd.1: osd_enable_op_tracker = '' 2026-02-24T07:44:07.656 INFO:teuthology.orchestra.run.trial035.stdout:osd.2: {} 2026-02-24T07:44:07.657 INFO:teuthology.orchestra.run.trial035.stderr:osd.2: osd_enable_op_tracker = '' 2026-02-24T07:44:07.668 INFO:teuthology.orchestra.run.trial035.stdout:osd.3: {} 2026-02-24T07:44:07.669 INFO:teuthology.orchestra.run.trial035.stderr:osd.3: osd_enable_op_tracker = '' 2026-02-24T07:44:07.934 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 59 16 395839 395823 419.269 343.875 0.00157955 0.00238329 2026-02-24T07:44:08.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:44:07 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/827021697' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2026-02-24T07:44:08.935 INFO:tasks.radosbench.radosbench.0.trial035.stdout:2026-02-24T07:44:08.934343+0000 min lat: 0.000685593 max lat: 0.066631 avg lat: 0.00238684 2026-02-24T07:44:08.935 INFO:tasks.radosbench.radosbench.0.trial035.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2026-02-24T07:44:08.935 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 60 16 402074 402058 418.776 389.688 0.00178123 0.00238684 2026-02-24T07:44:09.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:44:08 trial035 ceph-mon[39284]: pgmap v48: 17 pgs: 17 active+clean; 23 GiB data, 74 GiB used, 2.7 TiB / 2.7 TiB avail; 541 MiB/s wr, 8.65k op/s 2026-02-24T07:44:09.935 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 61 16 409333 409317 419.347 453.688 0.00160572 0.00238357 2026-02-24T07:44:10.240 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2026-02-24T07:44:10.370 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:44:10.711 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2026-02-24T07:44:10.711 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2026-02-24T07:44:10.711 INFO:tasks.thrashosds.thrasher:Growing pool unique_pool_0 2026-02-24T07:44:10.711 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph pg dump --format=json 2026-02-24T07:44:10.875 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:44:10.812+0000 7fd9df337640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:44:10.875 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:44:10.812+0000 7fd9df337640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:44:10.875 INFO:teuthology.orchestra.run.trial035.stderr:osd.0: osd_enable_op_tracker = '' 2026-02-24T07:44:10.875 INFO:teuthology.orchestra.run.trial035.stderr:osd.1: osd_enable_op_tracker = '' 2026-02-24T07:44:10.876 INFO:teuthology.orchestra.run.trial035.stdout:osd.0: {} 2026-02-24T07:44:10.876 INFO:teuthology.orchestra.run.trial035.stdout:osd.1: {} 2026-02-24T07:44:10.876 INFO:teuthology.orchestra.run.trial035.stdout:osd.2: {} 2026-02-24T07:44:10.876 INFO:teuthology.orchestra.run.trial035.stderr:osd.2: osd_enable_op_tracker = '' 2026-02-24T07:44:10.877 INFO:teuthology.orchestra.run.trial035.stdout:osd.3: {} 2026-02-24T07:44:10.877 INFO:teuthology.orchestra.run.trial035.stderr:osd.3: osd_enable_op_tracker = '' 2026-02-24T07:44:10.879 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:44:10.935 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 62 16 415338 415322 418.637 375.312 0.000852373 0.00238694 2026-02-24T07:44:11.169 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:44:10 trial035 ceph-mon[39284]: pgmap v49: 17 pgs: 17 active+clean; 23 GiB data, 74 GiB used, 2.7 TiB / 2.7 TiB avail; 371 MiB/s wr, 5.93k op/s 2026-02-24T07:44:11.169 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:44:10 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/805483220' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2026-02-24T07:44:11.935 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 63 16 420571 420555 417.183 327.062 0.00118494 0.00239596 2026-02-24T07:44:12.007 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:44:11.984+0000 7fb3da204640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:44:12.007 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:44:11.984+0000 7fb3da204640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:44:12.126 INFO:teuthology.orchestra.run.trial035.stdout: 2026-02-24T07:44:12.127 INFO:teuthology.orchestra.run.trial035.stderr:dumped all 2026-02-24T07:44:12.785 INFO:teuthology.orchestra.run.trial035.stdout:{"pg_ready":true,"pg_map":{"version":50,"stamp":"2026-02-24T07:44:11.668194+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":27135902240,"num_objects":414054,"num_object_clones":0,"num_object_copies":1242162,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":414054,"num_whiteouts":0,"num_read":136,"num_read_kb":240,"num_write":414187,"num_write_kb":26501478,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":51,"ondisk_log_size":51,"up":51,"acting":51,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":51,"num_osds":4,"num_per_pool_osds":4,"num_per_pool_omap_osds":4,"kb":2936012800,"kb_used":83603692,"kb_used_data":78814108,"kb_used_omap":76,"kb_used_meta":4789363,"kb_avail":2852409108,"statfs":{"total":3006477107200,"available":2920866926592,"internally_reserved":0,"allocated":80705646592,"data_stored":80704647584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":78108,"internal_metadata":4904308452},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[5,3],"upper_bound":4},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":6798835712,"num_objects":103742,"num_object_clones":0,"num_object_copies":311226,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":103742,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":103742,"num_write_kb":6639488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001436"},"pg_stats":[{"pgid":"2.f","version":"40'25583","reported_seq":25613,"reported_epoch":40,"state":"active+clean","last_fresh":"2026-02-24T07:44:10.664505+0000","last_change":"2026-02-24T07:43:07.853034+0000","last_active":"2026-02-24T07:44:10.664505+0000","last_peered":"2026-02-24T07:44:10.664505+0000","last_clean":"2026-02-24T07:44:10.664505+0000","last_became_active":"2026-02-24T07:43:07.852899+0000","last_became_peered":"2026-02-24T07:43:07.852899+0000","last_unstale":"2026-02-24T07:44:10.664505+0000","last_undegraded":"2026-02-24T07:44:10.664505+0000","last_fullsized":"2026-02-24T07:44:10.664505+0000","mapping_epoch":30,"log_start":"40'25580","ondisk_log_start":"40'25580","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_clean_scrub_stamp":"2026-02-24T07:43:06.832892+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:44:18.843618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1676607488,"num_objects":25583,"num_object_clones":0,"num_object_copies":76749,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25583,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":25583,"num_write_kb":1637312,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.e","version":"40'25929","reported_seq":25959,"reported_epoch":40,"state":"active+clean","last_fresh":"2026-02-24T07:44:10.669290+0000","last_change":"2026-02-24T07:43:07.853009+0000","last_active":"2026-02-24T07:44:10.669290+0000","last_peered":"2026-02-24T07:44:10.669290+0000","last_clean":"2026-02-24T07:44:10.669290+0000","last_became_active":"2026-02-24T07:43:07.852815+0000","last_became_peered":"2026-02-24T07:43:07.852815+0000","last_unstale":"2026-02-24T07:44:10.669290+0000","last_undegraded":"2026-02-24T07:44:10.669290+0000","last_fullsized":"2026-02-24T07:44:10.669290+0000","mapping_epoch":30,"log_start":"40'25926","ondisk_log_start":"40'25926","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_clean_scrub_stamp":"2026-02-24T07:43:06.832892+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:44:11.750665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1699282944,"num_objects":25929,"num_object_clones":0,"num_object_copies":77787,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25929,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":25929,"num_write_kb":1659456,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.d","version":"40'26054","reported_seq":26084,"reported_epoch":40,"state":"active+clean","last_fresh":"2026-02-24T07:44:10.668060+0000","last_change":"2026-02-24T07:43:07.854403+0000","last_active":"2026-02-24T07:44:10.668060+0000","last_peered":"2026-02-24T07:44:10.668060+0000","last_clean":"2026-02-24T07:44:10.668060+0000","last_became_active":"2026-02-24T07:43:07.853975+0000","last_became_peered":"2026-02-24T07:43:07.853975+0000","last_unstale":"2026-02-24T07:44:10.668060+0000","last_undegraded":"2026-02-24T07:44:10.668060+0000","last_fullsized":"2026-02-24T07:44:10.668060+0000","mapping_epoch":30,"log_start":"40'26051","ondisk_log_start":"40'26051","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_clean_scrub_stamp":"2026-02-24T07:43:06.832892+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:44:14.520704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1707474944,"num_objects":26054,"num_object_clones":0,"num_object_copies":78162,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26054,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":26054,"num_write_kb":1667456,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"40'25993","reported_seq":26022,"reported_epoch":40,"state":"active+clean","last_fresh":"2026-02-24T07:44:10.667937+0000","last_change":"2026-02-24T07:43:07.854780+0000","last_active":"2026-02-24T07:44:10.667937+0000","last_peered":"2026-02-24T07:44:10.667937+0000","last_clean":"2026-02-24T07:44:10.667937+0000","last_became_active":"2026-02-24T07:43:07.854428+0000","last_became_peered":"2026-02-24T07:43:07.854428+0000","last_unstale":"2026-02-24T07:44:10.667937+0000","last_undegraded":"2026-02-24T07:44:10.667937+0000","last_fullsized":"2026-02-24T07:44:10.667937+0000","mapping_epoch":30,"log_start":"40'25990","ondisk_log_start":"40'25990","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_clean_scrub_stamp":"2026-02-24T07:43:06.832892+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:44:16.718883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1703477248,"num_objects":25993,"num_object_clones":0,"num_object_copies":77979,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25993,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":25993,"num_write_kb":1663552,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"29'99","reported_seq":209,"reported_epoch":40,"state":"active+clean","last_fresh":"2026-02-24T07:43:59.386353+0000","last_change":"2026-02-24T07:43:59.386353+0000","last_active":"2026-02-24T07:43:59.386353+0000","last_peered":"2026-02-24T07:43:59.386353+0000","last_clean":"2026-02-24T07:43:59.386353+0000","last_became_active":"2026-02-24T07:42:54.186719+0000","last_became_peered":"2026-02-24T07:42:54.186719+0000","last_unstale":"2026-02-24T07:43:59.386353+0000","last_undegraded":"2026-02-24T07:43:59.386353+0000","last_fullsized":"2026-02-24T07:43:59.386353+0000","mapping_epoch":27,"log_start":"29'96","ondisk_log_start":"29'96","created":20,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'99","last_scrub_stamp":"2026-02-24T07:43:59.386324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:42:43.640776+0000","last_clean_scrub_stamp":"2026-02-24T07:43:59.386324+0000","objects_scrubbed":2,"log_size":3,"log_dups_size":96,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:45:24.339819+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":240,"num_write":135,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"40'26005","reported_seq":26035,"reported_epoch":40,"state":"active+clean","last_fresh":"2026-02-24T07:44:10.666288+0000","last_change":"2026-02-24T07:43:07.842967+0000","last_active":"2026-02-24T07:44:10.666288+0000","last_peered":"2026-02-24T07:44:10.666288+0000","last_clean":"2026-02-24T07:44:10.666288+0000","last_became_active":"2026-02-24T07:43:07.842530+0000","last_became_peered":"2026-02-24T07:43:07.842530+0000","last_unstale":"2026-02-24T07:44:10.666288+0000","last_undegraded":"2026-02-24T07:44:10.666288+0000","last_fullsized":"2026-02-24T07:44:10.666288+0000","mapping_epoch":30,"log_start":"40'26002","ondisk_log_start":"40'26002","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_clean_scrub_stamp":"2026-02-24T07:43:06.832892+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:44:14.900069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1704263680,"num_objects":26005,"num_object_clones":0,"num_object_copies":78015,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26005,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":26005,"num_write_kb":1664320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"40'25668","reported_seq":25698,"reported_epoch":40,"state":"active+clean","last_fresh":"2026-02-24T07:44:10.668409+0000","last_change":"2026-02-24T07:43:07.854082+0000","last_active":"2026-02-24T07:44:10.668409+0000","last_peered":"2026-02-24T07:44:10.668409+0000","last_clean":"2026-02-24T07:44:10.668409+0000","last_became_active":"2026-02-24T07:43:07.853885+0000","last_became_peered":"2026-02-24T07:43:07.853885+0000","last_unstale":"2026-02-24T07:44:10.668409+0000","last_undegraded":"2026-02-24T07:44:10.668409+0000","last_fullsized":"2026-02-24T07:44:10.668409+0000","mapping_epoch":30,"log_start":"40'25665","ondisk_log_start":"40'25665","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_clean_scrub_stamp":"2026-02-24T07:43:06.832892+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:44:20.512438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1682178048,"num_objects":25668,"num_object_clones":0,"num_object_copies":77004,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25668,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":25668,"num_write_kb":1642752,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"40'25969","reported_seq":25999,"reported_epoch":40,"state":"active+clean","last_fresh":"2026-02-24T07:44:10.665789+0000","last_change":"2026-02-24T07:43:07.854497+0000","last_active":"2026-02-24T07:44:10.665789+0000","last_peered":"2026-02-24T07:44:10.665789+0000","last_clean":"2026-02-24T07:44:10.665789+0000","last_became_active":"2026-02-24T07:43:07.854212+0000","last_became_peered":"2026-02-24T07:43:07.854212+0000","last_unstale":"2026-02-24T07:44:10.665789+0000","last_undegraded":"2026-02-24T07:44:10.665789+0000","last_fullsized":"2026-02-24T07:44:10.665789+0000","mapping_epoch":30,"log_start":"40'25966","ondisk_log_start":"40'25966","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_clean_scrub_stamp":"2026-02-24T07:43:06.832892+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:44:21.531086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1701904384,"num_objects":25969,"num_object_clones":0,"num_object_copies":77907,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25969,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":25969,"num_write_kb":1662016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[1,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"40'26060","reported_seq":26090,"reported_epoch":40,"state":"active+clean","last_fresh":"2026-02-24T07:44:10.668354+0000","last_change":"2026-02-24T07:43:07.843123+0000","last_active":"2026-02-24T07:44:10.668354+0000","last_peered":"2026-02-24T07:44:10.668354+0000","last_clean":"2026-02-24T07:44:10.668354+0000","last_became_active":"2026-02-24T07:43:07.842864+0000","last_became_peered":"2026-02-24T07:43:07.842864+0000","last_unstale":"2026-02-24T07:44:10.668354+0000","last_undegraded":"2026-02-24T07:44:10.668354+0000","last_fullsized":"2026-02-24T07:44:10.668354+0000","mapping_epoch":30,"log_start":"40'26057","ondisk_log_start":"40'26057","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_clean_scrub_stamp":"2026-02-24T07:43:06.832892+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:44:34.876596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1707868160,"num_objects":26060,"num_object_clones":0,"num_object_copies":78180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26060,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":26060,"num_write_kb":1667840,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.4","version":"40'25795","reported_seq":25825,"reported_epoch":40,"state":"active+clean","last_fresh":"2026-02-24T07:44:10.663575+0000","last_change":"2026-02-24T07:43:07.854462+0000","last_active":"2026-02-24T07:44:10.663575+0000","last_peered":"2026-02-24T07:44:10.663575+0000","last_clean":"2026-02-24T07:44:10.663575+0000","last_became_active":"2026-02-24T07:43:07.853966+0000","last_became_peered":"2026-02-24T07:43:07.853966+0000","last_unstale":"2026-02-24T07:44:10.663575+0000","last_undegraded":"2026-02-24T07:44:10.663575+0000","last_fullsized":"2026-02-24T07:44:10.663575+0000","mapping_epoch":30,"log_start":"40'25792","ondisk_log_start":"40'25792","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_clean_scrub_stamp":"2026-02-24T07:43:06.832892+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:44:13.031431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1690501120,"num_objects":25795,"num_object_clones":0,"num_object_copies":77385,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25795,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":25795,"num_write_kb":1650880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"40'25914","reported_seq":25944,"reported_epoch":40,"state":"active+clean","last_fresh":"2026-02-24T07:44:10.663220+0000","last_change":"2026-02-24T07:43:07.854184+0000","last_active":"2026-02-24T07:44:10.663220+0000","last_peered":"2026-02-24T07:44:10.663220+0000","last_clean":"2026-02-24T07:44:10.663220+0000","last_became_active":"2026-02-24T07:43:07.853878+0000","last_became_peered":"2026-02-24T07:43:07.853878+0000","last_unstale":"2026-02-24T07:44:10.663220+0000","last_undegraded":"2026-02-24T07:44:10.663220+0000","last_fullsized":"2026-02-24T07:44:10.663220+0000","mapping_epoch":30,"log_start":"40'25911","ondisk_log_start":"40'25911","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_clean_scrub_stamp":"2026-02-24T07:43:06.832892+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:44:31.507398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1698299904,"num_objects":25914,"num_object_clones":0,"num_object_copies":77742,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25914,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":25914,"num_write_kb":1658496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"40'26054","reported_seq":26084,"reported_epoch":40,"state":"active+clean","last_fresh":"2026-02-24T07:44:10.667720+0000","last_change":"2026-02-24T07:43:07.842180+0000","last_active":"2026-02-24T07:44:10.667720+0000","last_peered":"2026-02-24T07:44:10.667720+0000","last_clean":"2026-02-24T07:44:10.667720+0000","last_became_active":"2026-02-24T07:43:07.841993+0000","last_became_peered":"2026-02-24T07:43:07.841993+0000","last_unstale":"2026-02-24T07:44:10.667720+0000","last_undegraded":"2026-02-24T07:44:10.667720+0000","last_fullsized":"2026-02-24T07:44:10.667720+0000","mapping_epoch":30,"log_start":"40'26051","ondisk_log_start":"40'26051","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_clean_scrub_stamp":"2026-02-24T07:43:06.832892+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:44:19.519502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1707474944,"num_objects":26054,"num_object_clones":0,"num_object_copies":78162,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26054,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":26054,"num_write_kb":1667456,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"40'25553","reported_seq":25583,"reported_epoch":40,"state":"active+clean","last_fresh":"2026-02-24T07:44:10.658392+0000","last_change":"2026-02-24T07:43:07.842375+0000","last_active":"2026-02-24T07:44:10.658392+0000","last_peered":"2026-02-24T07:44:10.658392+0000","last_clean":"2026-02-24T07:44:10.658392+0000","last_became_active":"2026-02-24T07:43:07.841920+0000","last_became_peered":"2026-02-24T07:43:07.841920+0000","last_unstale":"2026-02-24T07:44:10.658392+0000","last_undegraded":"2026-02-24T07:44:10.658392+0000","last_fullsized":"2026-02-24T07:44:10.658392+0000","mapping_epoch":30,"log_start":"40'25550","ondisk_log_start":"40'25550","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_clean_scrub_stamp":"2026-02-24T07:43:06.832892+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:44:14.681944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1674641408,"num_objects":25553,"num_object_clones":0,"num_object_copies":76659,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25553,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":25553,"num_write_kb":1635392,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.8","version":"40'25578","reported_seq":25608,"reported_epoch":40,"state":"active+clean","last_fresh":"2026-02-24T07:44:10.661409+0000","last_change":"2026-02-24T07:43:07.853992+0000","last_active":"2026-02-24T07:44:10.661409+0000","last_peered":"2026-02-24T07:44:10.661409+0000","last_clean":"2026-02-24T07:44:10.661409+0000","last_became_active":"2026-02-24T07:43:07.853758+0000","last_became_peered":"2026-02-24T07:43:07.853758+0000","last_unstale":"2026-02-24T07:44:10.661409+0000","last_undegraded":"2026-02-24T07:44:10.661409+0000","last_fullsized":"2026-02-24T07:44:10.661409+0000","mapping_epoch":30,"log_start":"40'25575","ondisk_log_start":"40'25575","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_clean_scrub_stamp":"2026-02-24T07:43:06.832892+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:44:27.436119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1676279808,"num_objects":25578,"num_object_clones":0,"num_object_copies":76734,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25578,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":25578,"num_write_kb":1636992,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"40'25957","reported_seq":25987,"reported_epoch":40,"state":"active+clean","last_fresh":"2026-02-24T07:44:10.663646+0000","last_change":"2026-02-24T07:43:07.854687+0000","last_active":"2026-02-24T07:44:10.663646+0000","last_peered":"2026-02-24T07:44:10.663646+0000","last_clean":"2026-02-24T07:44:10.663646+0000","last_became_active":"2026-02-24T07:43:07.854377+0000","last_became_peered":"2026-02-24T07:43:07.854377+0000","last_unstale":"2026-02-24T07:44:10.663646+0000","last_undegraded":"2026-02-24T07:44:10.663646+0000","last_fullsized":"2026-02-24T07:44:10.663646+0000","mapping_epoch":30,"log_start":"40'25954","ondisk_log_start":"40'25954","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_clean_scrub_stamp":"2026-02-24T07:43:06.832892+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:44:34.215480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1701117952,"num_objects":25957,"num_object_clones":0,"num_object_copies":77871,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25957,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":25957,"num_write_kb":1661248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.a","version":"40'25965","reported_seq":25995,"reported_epoch":40,"state":"active+clean","last_fresh":"2026-02-24T07:44:10.667802+0000","last_change":"2026-02-24T07:43:07.841933+0000","last_active":"2026-02-24T07:44:10.667802+0000","last_peered":"2026-02-24T07:44:10.667802+0000","last_clean":"2026-02-24T07:44:10.667802+0000","last_became_active":"2026-02-24T07:43:07.841721+0000","last_became_peered":"2026-02-24T07:43:07.841721+0000","last_unstale":"2026-02-24T07:44:10.667802+0000","last_undegraded":"2026-02-24T07:44:10.667802+0000","last_fullsized":"2026-02-24T07:44:10.667802+0000","mapping_epoch":30,"log_start":"40'25962","ondisk_log_start":"40'25962","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_clean_scrub_stamp":"2026-02-24T07:43:06.832892+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:44:14.991691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1701642240,"num_objects":25965,"num_object_clones":0,"num_object_copies":77895,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25965,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":25965,"num_write_kb":1661760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.b","version":"40'25975","reported_seq":26005,"reported_epoch":40,"state":"active+clean","last_fresh":"2026-02-24T07:44:10.665788+0000","last_change":"2026-02-24T07:43:07.853283+0000","last_active":"2026-02-24T07:44:10.665788+0000","last_peered":"2026-02-24T07:44:10.665788+0000","last_clean":"2026-02-24T07:44:10.665788+0000","last_became_active":"2026-02-24T07:43:07.853069+0000","last_became_peered":"2026-02-24T07:43:07.853069+0000","last_unstale":"2026-02-24T07:44:10.665788+0000","last_undegraded":"2026-02-24T07:44:10.665788+0000","last_fullsized":"2026-02-24T07:44:10.665788+0000","mapping_epoch":30,"log_start":"40'25972","ondisk_log_start":"40'25972","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_clean_scrub_stamp":"2026-02-24T07:43:06.832892+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:44:32.581403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1702297600,"num_objects":25975,"num_object_clones":0,"num_object_copies":77925,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25975,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":25975,"num_write_kb":1662400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":27135311872,"num_objects":414052,"num_object_clones":0,"num_object_copies":1242156,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":414052,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":414052,"num_write_kb":26499328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":81404690432,"data_stored":81404690432,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":48,"ondisk_log_size":48,"up":48,"acting":48,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":240,"num_write":135,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":3,"up_from":25,"seq":107374182418,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":24518156,"kb_used_data":23112312,"kb_used_omap":14,"kb_used_meta":1405809,"kb_avail":709485044,"statfs":{"total":751619276800,"available":726512685056,"internally_reserved":0,"allocated":23667007488,"data_stored":23666756848,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":14643,"internal_metadata":1439549133},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":19,"seq":81604378644,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":18982412,"kb_used_data":17908600,"kb_used_omap":18,"kb_used_meta":1073709,"kb_avail":715020788,"statfs":{"total":751619276800,"available":732181286912,"internally_reserved":0,"allocated":18338406400,"data_stored":18338155760,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":18551,"internal_metadata":1099478921},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,2],"upper_bound":4},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542165,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":20995208,"kb_used_data":19759028,"kb_used_omap":20,"kb_used_meta":1236139,"kb_avail":713007992,"statfs":{"total":751619276800,"available":730120183808,"internally_reserved":0,"allocated":20233244672,"data_stored":20232997584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":20504,"internal_metadata":1265807336},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,1],"upper_bound":4},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705688,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":19107916,"kb_used_data":18034168,"kb_used_omap":23,"kb_used_meta":1073704,"kb_avail":714895284,"statfs":{"total":751619276800,"available":732052770816,"internally_reserved":0,"allocated":18466988032,"data_stored":18466737392,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":24410,"internal_metadata":1099473062},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":18638110720,"data_stored":18638110720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":20382089216,"data_stored":20382089216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":18638438400,"data_stored":18638438400,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":23746052096,"data_stored":23746052096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-24T07:44:12.786 INFO:tasks.cephadm.ceph_manager.ceph:increase pool size by 10 2026-02-24T07:44:12.786 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph osd pool set unique_pool_0 pg_num 36 2026-02-24T07:44:12.929 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:44:12.935 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 64 15 426142 426127 416.105 348.25 0.00162546 0.00240216 2026-02-24T07:44:13.047 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:44:12 trial035 ceph-mon[39284]: pgmap v50: 17 pgs: 17 active+clean; 25 GiB data, 80 GiB used, 2.7 TiB / 2.7 TiB avail; 540 MiB/s wr, 8.64k op/s 2026-02-24T07:44:13.364 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:44:13.313+0000 7f72c904b640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:44:13.364 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:44:13.314+0000 7f72c904b640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:44:13.444 INFO:teuthology.orchestra.run.trial035.stderr:Error EPERM: crimson_allow_pg_split is false; pg_num increase denied for crimson pool 2026-02-24T07:44:13.479 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-24T07:44:13.572 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2026-02-24T07:44:13.710 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:44:13.838 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:44:13.836+0000 7f36bdc66640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:44:13.838 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:44:13.836+0000 7f36bdc66640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:44:13.853 INFO:teuthology.orchestra.run.trial035.stdout:osd.0: {} 2026-02-24T07:44:13.853 INFO:teuthology.orchestra.run.trial035.stderr:osd.0: osd_enable_op_tracker = '' 2026-02-24T07:44:13.866 INFO:teuthology.orchestra.run.trial035.stdout:osd.1: {} 2026-02-24T07:44:13.866 INFO:teuthology.orchestra.run.trial035.stderr:osd.1: osd_enable_op_tracker = '' 2026-02-24T07:44:13.879 INFO:teuthology.orchestra.run.trial035.stdout:osd.2: {} 2026-02-24T07:44:13.879 INFO:teuthology.orchestra.run.trial035.stderr:osd.2: osd_enable_op_tracker = '' 2026-02-24T07:44:13.891 INFO:teuthology.orchestra.run.trial035.stdout:osd.3: {} 2026-02-24T07:44:13.891 INFO:teuthology.orchestra.run.trial035.stderr:osd.3: osd_enable_op_tracker = '' 2026-02-24T07:44:13.935 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 65 16 431730 431714 415.075 349.188 0.00200196 0.00240808 2026-02-24T07:44:14.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:44:13 trial035 ceph-mon[39284]: from='client.14443 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2026-02-24T07:44:14.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:44:13 trial035 ceph-mon[39284]: 2.e scrub starts 2026-02-24T07:44:14.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:44:13 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/3626923439' entity='client.admin' cmd={"prefix": "osd pool set", "pool": "unique_pool_0", "var": "pg_num", "val": "36"} : dispatch 2026-02-24T07:44:14.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:44:13 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/3570048394' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2026-02-24T07:44:14.935 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 66 16 437645 437629 414.387 369.688 0.00175348 0.00241211 2026-02-24T07:44:15.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:44:14 trial035 ceph-mon[39284]: pgmap v51: 17 pgs: 17 active+clean; 25 GiB data, 80 GiB used, 2.7 TiB / 2.7 TiB avail; 350 MiB/s wr, 5.60k op/s 2026-02-24T07:44:15.205 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:44:14 trial035 ceph-mon[39284]: 2.4 scrub starts 2026-02-24T07:44:15.931 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2026-02-24T07:44:15.935 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 67 16 443006 442990 413.203 335.062 0.00187735 0.00241899 2026-02-24T07:44:16.071 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:44:16.131 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:44:15 trial035 ceph-mon[39284]: from='mgr.14284 10.20.193.35:0/3681373222' entity='mgr.x' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2026-02-24T07:44:16.200 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:44:16.198+0000 7fc3113eb640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:44:16.200 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:44:16.198+0000 7fc3113eb640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:44:16.215 INFO:teuthology.orchestra.run.trial035.stdout:osd.0: {} 2026-02-24T07:44:16.216 INFO:teuthology.orchestra.run.trial035.stderr:osd.0: osd_enable_op_tracker = '' 2026-02-24T07:44:16.228 INFO:teuthology.orchestra.run.trial035.stdout:osd.1: {} 2026-02-24T07:44:16.229 INFO:teuthology.orchestra.run.trial035.stderr:osd.1: osd_enable_op_tracker = '' 2026-02-24T07:44:16.241 INFO:teuthology.orchestra.run.trial035.stdout:osd.2: {} 2026-02-24T07:44:16.242 INFO:teuthology.orchestra.run.trial035.stderr:osd.2: osd_enable_op_tracker = '' 2026-02-24T07:44:16.253 INFO:teuthology.orchestra.run.trial035.stdout:osd.3: {} 2026-02-24T07:44:16.253 INFO:teuthology.orchestra.run.trial035.stderr:osd.3: osd_enable_op_tracker = '' 2026-02-24T07:44:16.935 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 68 16 448689 448673 412.349 355.188 0.00221948 0.00242402 2026-02-24T07:44:17.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:44:16 trial035 ceph-mon[39284]: 2.7 scrub starts 2026-02-24T07:44:17.205 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:44:16 trial035 ceph-mon[39284]: pgmap v52: 17 pgs: 17 active+clean; 25 GiB data, 82 GiB used, 2.7 TiB / 2.7 TiB avail; 359 MiB/s wr, 5.74k op/s 2026-02-24T07:44:17.205 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:44:16 trial035 ceph-mon[39284]: 2.e scrub ok 2026-02-24T07:44:17.205 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:44:16 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/315247162' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2026-02-24T07:44:17.205 INFO:journalctl@ceph.mgr.x.trial035.stdout:Feb 24 07:44:16 trial035 ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-mgr-x[39621]: ::ffff:10.20.193.91 - - [24/Feb/2026:07:44:16] "GET /metrics HTTP/1.1" 200 4221 "" "Prometheus/3.6.0" 2026-02-24T07:44:17.266 INFO:journalctl@ceph.prometheus.a.trial091.stdout:Feb 24 07:44:16 trial091 ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-prometheus-a[66007]: time=2026-02-24T07:44:16.973Z level=ERROR source=scrape.go:1630 msg="Failed to determine correct type of scrape target." component="scrape manager" scrape_pool=ceph target=http://trial035:9283/metrics content_type="text/html;charset=utf-8" fallback_media_type="" err="received unsupported Content-Type \"text/html;charset=utf-8\" and no fallback_scrape_protocol specified for target" 2026-02-24T07:44:17.935 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 69 16 455240 455224 412.307 409.438 0.0049418 0.00242427 2026-02-24T07:44:18.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:44:17 trial035 ceph-mon[39284]: 2.4 scrub ok 2026-02-24T07:44:18.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:44:17 trial035 ceph-mon[39284]: pgmap v53: 17 pgs: 3 active+clean+scrubbing, 14 active+clean; 27 GiB data, 85 GiB used, 2.7 TiB / 2.7 TiB avail; 483 MiB/s wr, 7.74k op/s 2026-02-24T07:44:18.294 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2026-02-24T07:44:18.431 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:44:18.479 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2026-02-24T07:44:18.479 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2026-02-24T07:44:18.479 INFO:tasks.thrashosds.thrasher:Growing pool .mgr 2026-02-24T07:44:18.479 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph pg dump --format=json 2026-02-24T07:44:18.623 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:44:18.597+0000 7f9c2a3f0640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:44:18.623 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:44:18.597+0000 7f9c2a3f0640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:44:18.623 INFO:teuthology.orchestra.run.trial035.stderr:osd.0: osd_enable_op_tracker = '' 2026-02-24T07:44:18.624 INFO:teuthology.orchestra.run.trial035.stdout:osd.0: {} 2026-02-24T07:44:18.627 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:44:18.628 INFO:teuthology.orchestra.run.trial035.stdout:osd.1: {} 2026-02-24T07:44:18.628 INFO:teuthology.orchestra.run.trial035.stderr:osd.1: osd_enable_op_tracker = '' 2026-02-24T07:44:18.644 INFO:teuthology.orchestra.run.trial035.stdout:osd.2: {} 2026-02-24T07:44:18.644 INFO:teuthology.orchestra.run.trial035.stderr:osd.2: osd_enable_op_tracker = '' 2026-02-24T07:44:18.657 INFO:teuthology.orchestra.run.trial035.stdout:osd.3: {} 2026-02-24T07:44:18.657 INFO:teuthology.orchestra.run.trial035.stderr:osd.3: osd_enable_op_tracker = '' 2026-02-24T07:44:18.935 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 70 15 461382 461367 411.901 383.938 0.0319527 0.00242656 2026-02-24T07:44:19.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:44:18 trial035 ceph-mon[39284]: 2.7 scrub ok 2026-02-24T07:44:19.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:44:18 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/3115346225' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2026-02-24T07:44:19.933 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:44:19.881+0000 7fb9f3bd0640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:44:19.933 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:44:19.882+0000 7fb9f3bd0640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:44:19.935 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 71 16 466749 466733 410.823 335.375 0.00108756 0.00243301 2026-02-24T07:44:19.954 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:44:19 trial035 ceph-mon[39284]: pgmap v54: 17 pgs: 3 active+clean+scrubbing, 14 active+clean; 27 GiB data, 85 GiB used, 2.7 TiB / 2.7 TiB avail; 314 MiB/s wr, 5.03k op/s 2026-02-24T07:44:20.010 INFO:teuthology.orchestra.run.trial035.stdout: 2026-02-24T07:44:20.010 INFO:teuthology.orchestra.run.trial035.stderr:dumped all 2026-02-24T07:44:20.737 INFO:teuthology.orchestra.run.trial035.stdout:{"pg_ready":true,"pg_map":{"version":54,"stamp":"2026-02-24T07:44:19.669224+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":28954853920,"num_objects":441809,"num_object_clones":0,"num_object_copies":1325427,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":441809,"num_whiteouts":0,"num_read":136,"num_read_kb":240,"num_write":441942,"num_write_kb":28277798,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":52,"ondisk_log_size":52,"up":51,"acting":51,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":51,"num_osds":4,"num_per_pool_osds":4,"num_per_pool_omap_osds":4,"kb":2936012800,"kb_used":89432300,"kb_used_data":84188380,"kb_used_omap":76,"kb_used_meta":5243699,"kb_avail":2846580500,"statfs":{"total":3006477107200,"available":2914898432000,"internally_reserved":0,"allocated":86208901120,"data_stored":86207902112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":78108,"internal_metadata":5369548516},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[7,0,2,6],"upper_bound":16},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":3952607232,"num_objects":60312,"num_object_clones":0,"num_object_copies":180936,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60312,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":60312,"num_write_kb":3859968,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001499"},"pg_stats":[{"pgid":"2.f","version":"40'27303","reported_seq":27332,"reported_epoch":40,"state":"active+clean","last_fresh":"2026-02-24T07:44:15.665422+0000","last_change":"2026-02-24T07:43:07.853034+0000","last_active":"2026-02-24T07:44:15.665422+0000","last_peered":"2026-02-24T07:44:15.665422+0000","last_clean":"2026-02-24T07:44:15.665422+0000","last_became_active":"2026-02-24T07:43:07.852899+0000","last_became_peered":"2026-02-24T07:43:07.852899+0000","last_unstale":"2026-02-24T07:44:15.665422+0000","last_undegraded":"2026-02-24T07:44:15.665422+0000","last_fullsized":"2026-02-24T07:44:15.665422+0000","mapping_epoch":30,"log_start":"40'27300","ondisk_log_start":"40'27300","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_clean_scrub_stamp":"2026-02-24T07:43:06.832892+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:44:18.843618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1789329408,"num_objects":27303,"num_object_clones":0,"num_object_copies":81909,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27303,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":27303,"num_write_kb":1747392,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.e","version":"40'27656","reported_seq":27930,"reported_epoch":40,"state":"active+clean+scrubbing","last_fresh":"2026-02-24T07:44:15.684580+0000","last_change":"2026-02-24T07:44:13.139313+0000","last_active":"2026-02-24T07:44:15.684580+0000","last_peered":"2026-02-24T07:44:15.684580+0000","last_clean":"2026-02-24T07:44:15.684580+0000","last_became_active":"2026-02-24T07:43:07.852815+0000","last_became_peered":"2026-02-24T07:43:07.852815+0000","last_unstale":"2026-02-24T07:44:15.684580+0000","last_undegraded":"2026-02-24T07:44:15.684580+0000","last_fullsized":"2026-02-24T07:44:15.684580+0000","mapping_epoch":30,"log_start":"40'27653","ondisk_log_start":"40'27653","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_clean_scrub_stamp":"2026-02-24T07:43:06.832892+0000","objects_scrubbed":23849,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"scrubbing for 3s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1812463616,"num_objects":27656,"num_object_clones":0,"num_object_copies":82968,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27656,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":27656,"num_write_kb":1769984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.d","version":"40'27821","reported_seq":27851,"reported_epoch":40,"state":"active+clean","last_fresh":"2026-02-24T07:44:15.680777+0000","last_change":"2026-02-24T07:43:07.854403+0000","last_active":"2026-02-24T07:44:15.680777+0000","last_peered":"2026-02-24T07:44:15.680777+0000","last_clean":"2026-02-24T07:44:15.680777+0000","last_became_active":"2026-02-24T07:43:07.853975+0000","last_became_peered":"2026-02-24T07:43:07.853975+0000","last_unstale":"2026-02-24T07:44:15.680777+0000","last_undegraded":"2026-02-24T07:44:15.680777+0000","last_fullsized":"2026-02-24T07:44:15.680777+0000","mapping_epoch":30,"log_start":"40'27818","ondisk_log_start":"40'27818","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_clean_scrub_stamp":"2026-02-24T07:43:06.832892+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1823277056,"num_objects":27821,"num_object_clones":0,"num_object_copies":83463,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27821,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":27821,"num_write_kb":1780544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"40'27773","reported_seq":27803,"reported_epoch":40,"state":"active+clean","last_fresh":"2026-02-24T07:44:15.680568+0000","last_change":"2026-02-24T07:43:07.854780+0000","last_active":"2026-02-24T07:44:15.680568+0000","last_peered":"2026-02-24T07:44:15.680568+0000","last_clean":"2026-02-24T07:44:15.680568+0000","last_became_active":"2026-02-24T07:43:07.854428+0000","last_became_peered":"2026-02-24T07:43:07.854428+0000","last_unstale":"2026-02-24T07:44:15.680568+0000","last_undegraded":"2026-02-24T07:44:15.680568+0000","last_fullsized":"2026-02-24T07:44:15.680568+0000","mapping_epoch":30,"log_start":"40'27770","ondisk_log_start":"40'27770","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_clean_scrub_stamp":"2026-02-24T07:43:06.832892+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:44:16.718883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1820131328,"num_objects":27773,"num_object_clones":0,"num_object_copies":83319,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27773,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":27773,"num_write_kb":1777472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"29'99","reported_seq":209,"reported_epoch":40,"state":"active+clean","last_fresh":"2026-02-24T07:43:59.386353+0000","last_change":"2026-02-24T07:43:59.386353+0000","last_active":"2026-02-24T07:43:59.386353+0000","last_peered":"2026-02-24T07:43:59.386353+0000","last_clean":"2026-02-24T07:43:59.386353+0000","last_became_active":"2026-02-24T07:42:54.186719+0000","last_became_peered":"2026-02-24T07:42:54.186719+0000","last_unstale":"2026-02-24T07:43:59.386353+0000","last_undegraded":"2026-02-24T07:43:59.386353+0000","last_fullsized":"2026-02-24T07:43:59.386353+0000","mapping_epoch":27,"log_start":"29'96","ondisk_log_start":"29'96","created":20,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'99","last_scrub_stamp":"2026-02-24T07:43:59.386324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:42:43.640776+0000","last_clean_scrub_stamp":"2026-02-24T07:43:59.386324+0000","objects_scrubbed":2,"log_size":3,"log_dups_size":96,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:45:24.339819+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":240,"num_write":135,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"40'27739","reported_seq":27769,"reported_epoch":40,"state":"active+clean","last_fresh":"2026-02-24T07:44:15.672518+0000","last_change":"2026-02-24T07:43:07.842967+0000","last_active":"2026-02-24T07:44:15.672518+0000","last_peered":"2026-02-24T07:44:15.672518+0000","last_clean":"2026-02-24T07:44:15.672518+0000","last_became_active":"2026-02-24T07:43:07.842530+0000","last_became_peered":"2026-02-24T07:43:07.842530+0000","last_unstale":"2026-02-24T07:44:15.672518+0000","last_undegraded":"2026-02-24T07:44:15.672518+0000","last_fullsized":"2026-02-24T07:44:15.672518+0000","mapping_epoch":30,"log_start":"40'27736","ondisk_log_start":"40'27736","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_clean_scrub_stamp":"2026-02-24T07:43:06.832892+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1817903104,"num_objects":27739,"num_object_clones":0,"num_object_copies":83217,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27739,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":27739,"num_write_kb":1775296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"40'27375","reported_seq":27405,"reported_epoch":40,"state":"active+clean","last_fresh":"2026-02-24T07:44:15.664194+0000","last_change":"2026-02-24T07:43:07.854082+0000","last_active":"2026-02-24T07:44:15.664194+0000","last_peered":"2026-02-24T07:44:15.664194+0000","last_clean":"2026-02-24T07:44:15.664194+0000","last_became_active":"2026-02-24T07:43:07.853885+0000","last_became_peered":"2026-02-24T07:43:07.853885+0000","last_unstale":"2026-02-24T07:44:15.664194+0000","last_undegraded":"2026-02-24T07:44:15.664194+0000","last_fullsized":"2026-02-24T07:44:15.664194+0000","mapping_epoch":30,"log_start":"40'27372","ondisk_log_start":"40'27372","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_clean_scrub_stamp":"2026-02-24T07:43:06.832892+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:44:20.512438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1794048000,"num_objects":27375,"num_object_clones":0,"num_object_copies":82125,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27375,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":27375,"num_write_kb":1752000,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"40'27730","reported_seq":27760,"reported_epoch":40,"state":"active+clean","last_fresh":"2026-02-24T07:44:15.675658+0000","last_change":"2026-02-24T07:43:07.854497+0000","last_active":"2026-02-24T07:44:15.675658+0000","last_peered":"2026-02-24T07:44:15.675658+0000","last_clean":"2026-02-24T07:44:15.675658+0000","last_became_active":"2026-02-24T07:43:07.854212+0000","last_became_peered":"2026-02-24T07:43:07.854212+0000","last_unstale":"2026-02-24T07:44:15.675658+0000","last_undegraded":"2026-02-24T07:44:15.675658+0000","last_fullsized":"2026-02-24T07:44:15.675658+0000","mapping_epoch":30,"log_start":"40'27727","ondisk_log_start":"40'27727","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_clean_scrub_stamp":"2026-02-24T07:43:06.832892+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:44:21.531086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1817313280,"num_objects":27730,"num_object_clones":0,"num_object_copies":83190,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27730,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":27730,"num_write_kb":1774720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[1,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"40'27810","reported_seq":27840,"reported_epoch":40,"state":"active+clean","last_fresh":"2026-02-24T07:44:15.670748+0000","last_change":"2026-02-24T07:43:07.843123+0000","last_active":"2026-02-24T07:44:15.670748+0000","last_peered":"2026-02-24T07:44:15.670748+0000","last_clean":"2026-02-24T07:44:15.670748+0000","last_became_active":"2026-02-24T07:43:07.842864+0000","last_became_peered":"2026-02-24T07:43:07.842864+0000","last_unstale":"2026-02-24T07:44:15.670748+0000","last_undegraded":"2026-02-24T07:44:15.670748+0000","last_fullsized":"2026-02-24T07:44:15.670748+0000","mapping_epoch":30,"log_start":"40'27807","ondisk_log_start":"40'27807","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_clean_scrub_stamp":"2026-02-24T07:43:06.832892+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:44:34.876596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1822556160,"num_objects":27810,"num_object_clones":0,"num_object_copies":83430,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27810,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":27810,"num_write_kb":1779840,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.4","version":"40'27521","reported_seq":27698,"reported_epoch":40,"state":"active+clean+scrubbing","last_fresh":"2026-02-24T07:44:15.681725+0000","last_change":"2026-02-24T07:44:14.135314+0000","last_active":"2026-02-24T07:44:15.681725+0000","last_peered":"2026-02-24T07:44:15.681725+0000","last_clean":"2026-02-24T07:44:15.681725+0000","last_became_active":"2026-02-24T07:43:07.853966+0000","last_became_peered":"2026-02-24T07:43:07.853966+0000","last_unstale":"2026-02-24T07:44:15.681725+0000","last_undegraded":"2026-02-24T07:44:15.681725+0000","last_fullsized":"2026-02-24T07:44:15.681725+0000","mapping_epoch":30,"log_start":"40'27518","ondisk_log_start":"40'27518","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_clean_scrub_stamp":"2026-02-24T07:43:06.832892+0000","objects_scrubbed":14299,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"scrubbing for 2s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1803616256,"num_objects":27521,"num_object_clones":0,"num_object_copies":82563,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27521,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":27521,"num_write_kb":1761344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"40'27618","reported_seq":27648,"reported_epoch":40,"state":"active+clean","last_fresh":"2026-02-24T07:44:15.667099+0000","last_change":"2026-02-24T07:43:07.854184+0000","last_active":"2026-02-24T07:44:15.667099+0000","last_peered":"2026-02-24T07:44:15.667099+0000","last_clean":"2026-02-24T07:44:15.667099+0000","last_became_active":"2026-02-24T07:43:07.853878+0000","last_became_peered":"2026-02-24T07:43:07.853878+0000","last_unstale":"2026-02-24T07:44:15.667099+0000","last_undegraded":"2026-02-24T07:44:15.667099+0000","last_fullsized":"2026-02-24T07:44:15.667099+0000","mapping_epoch":30,"log_start":"40'27615","ondisk_log_start":"40'27615","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_clean_scrub_stamp":"2026-02-24T07:43:06.832892+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:44:31.507398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1809973248,"num_objects":27618,"num_object_clones":0,"num_object_copies":82854,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27618,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":27618,"num_write_kb":1767552,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"40'27728","reported_seq":27758,"reported_epoch":40,"state":"active+clean","last_fresh":"2026-02-24T07:44:15.663395+0000","last_change":"2026-02-24T07:43:07.842180+0000","last_active":"2026-02-24T07:44:15.663395+0000","last_peered":"2026-02-24T07:44:15.663395+0000","last_clean":"2026-02-24T07:44:15.663395+0000","last_became_active":"2026-02-24T07:43:07.841993+0000","last_became_peered":"2026-02-24T07:43:07.841993+0000","last_unstale":"2026-02-24T07:44:15.663395+0000","last_undegraded":"2026-02-24T07:44:15.663395+0000","last_fullsized":"2026-02-24T07:44:15.663395+0000","mapping_epoch":30,"log_start":"40'27725","ondisk_log_start":"40'27725","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_clean_scrub_stamp":"2026-02-24T07:43:06.832892+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:44:19.519502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1817182208,"num_objects":27728,"num_object_clones":0,"num_object_copies":83184,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27728,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":27728,"num_write_kb":1774592,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"40'27358","reported_seq":27422,"reported_epoch":40,"state":"active+clean+scrubbing","last_fresh":"2026-02-24T07:44:15.673105+0000","last_change":"2026-02-24T07:44:15.419815+0000","last_active":"2026-02-24T07:44:15.673105+0000","last_peered":"2026-02-24T07:44:15.673105+0000","last_clean":"2026-02-24T07:44:15.673105+0000","last_became_active":"2026-02-24T07:43:07.841920+0000","last_became_peered":"2026-02-24T07:43:07.841920+0000","last_unstale":"2026-02-24T07:44:15.673105+0000","last_undegraded":"2026-02-24T07:44:15.673105+0000","last_fullsized":"2026-02-24T07:44:15.673105+0000","mapping_epoch":30,"log_start":"40'27355","ondisk_log_start":"40'27355","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_clean_scrub_stamp":"2026-02-24T07:43:06.832892+0000","objects_scrubbed":3199,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"scrubbing for 1s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1792933888,"num_objects":27358,"num_object_clones":0,"num_object_copies":82074,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27358,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":27358,"num_write_kb":1750912,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.8","version":"40'27290","reported_seq":27320,"reported_epoch":40,"state":"active+clean","last_fresh":"2026-02-24T07:44:15.668063+0000","last_change":"2026-02-24T07:43:07.853992+0000","last_active":"2026-02-24T07:44:15.668063+0000","last_peered":"2026-02-24T07:44:15.668063+0000","last_clean":"2026-02-24T07:44:15.668063+0000","last_became_active":"2026-02-24T07:43:07.853758+0000","last_became_peered":"2026-02-24T07:43:07.853758+0000","last_unstale":"2026-02-24T07:44:15.668063+0000","last_undegraded":"2026-02-24T07:44:15.668063+0000","last_fullsized":"2026-02-24T07:44:15.668063+0000","mapping_epoch":30,"log_start":"40'27287","ondisk_log_start":"40'27287","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_clean_scrub_stamp":"2026-02-24T07:43:06.832892+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:44:27.436119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1788477440,"num_objects":27290,"num_object_clones":0,"num_object_copies":81870,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27290,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":27290,"num_write_kb":1746560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"40'27727","reported_seq":27756,"reported_epoch":40,"state":"active+clean","last_fresh":"2026-02-24T07:44:15.681264+0000","last_change":"2026-02-24T07:43:07.854687+0000","last_active":"2026-02-24T07:44:15.681264+0000","last_peered":"2026-02-24T07:44:15.681264+0000","last_clean":"2026-02-24T07:44:15.681264+0000","last_became_active":"2026-02-24T07:43:07.854377+0000","last_became_peered":"2026-02-24T07:43:07.854377+0000","last_unstale":"2026-02-24T07:44:15.681264+0000","last_undegraded":"2026-02-24T07:44:15.681264+0000","last_fullsized":"2026-02-24T07:44:15.681264+0000","mapping_epoch":30,"log_start":"40'27723","ondisk_log_start":"40'27723","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_clean_scrub_stamp":"2026-02-24T07:43:06.832892+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":3000,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:44:34.215480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1817116672,"num_objects":27727,"num_object_clones":0,"num_object_copies":83181,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27727,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":27727,"num_write_kb":1774528,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.a","version":"40'27619","reported_seq":27649,"reported_epoch":40,"state":"active+clean","last_fresh":"2026-02-24T07:44:15.671583+0000","last_change":"2026-02-24T07:43:07.841933+0000","last_active":"2026-02-24T07:44:15.671583+0000","last_peered":"2026-02-24T07:44:15.671583+0000","last_clean":"2026-02-24T07:44:15.671583+0000","last_became_active":"2026-02-24T07:43:07.841721+0000","last_became_peered":"2026-02-24T07:43:07.841721+0000","last_unstale":"2026-02-24T07:44:15.671583+0000","last_undegraded":"2026-02-24T07:44:15.671583+0000","last_fullsized":"2026-02-24T07:44:15.671583+0000","mapping_epoch":30,"log_start":"40'27616","ondisk_log_start":"40'27616","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_clean_scrub_stamp":"2026-02-24T07:43:06.832892+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1810038784,"num_objects":27619,"num_object_clones":0,"num_object_copies":82857,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27619,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":27619,"num_write_kb":1767616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.b","version":"40'27739","reported_seq":27769,"reported_epoch":40,"state":"active+clean","last_fresh":"2026-02-24T07:44:15.661746+0000","last_change":"2026-02-24T07:43:07.853283+0000","last_active":"2026-02-24T07:44:15.661746+0000","last_peered":"2026-02-24T07:44:15.661746+0000","last_clean":"2026-02-24T07:44:15.661746+0000","last_became_active":"2026-02-24T07:43:07.853069+0000","last_became_peered":"2026-02-24T07:43:07.853069+0000","last_unstale":"2026-02-24T07:44:15.661746+0000","last_undegraded":"2026-02-24T07:44:15.661746+0000","last_fullsized":"2026-02-24T07:44:15.661746+0000","mapping_epoch":30,"log_start":"40'27736","ondisk_log_start":"40'27736","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_clean_scrub_stamp":"2026-02-24T07:43:06.832892+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:44:32.581403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1817903104,"num_objects":27739,"num_object_clones":0,"num_object_copies":83217,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27739,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":27739,"num_write_kb":1775296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":28954263552,"num_objects":441807,"num_object_clones":0,"num_object_copies":1325421,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":441807,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":441807,"num_write_kb":28275648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":86858661888,"data_stored":86858661888,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":49,"ondisk_log_size":49,"up":48,"acting":48,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":240,"num_write":135,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":3,"up_from":25,"seq":107374182419,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":26198028,"kb_used_data":24665784,"kb_used_omap":14,"kb_used_meta":1532145,"kb_avail":707805172,"statfs":{"total":751619276800,"available":724792496128,"internally_reserved":0,"allocated":25257762816,"data_stored":25257512176,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":14643,"internal_metadata":1568917197},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":19,"seq":81604378645,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":20370828,"kb_used_data":19170744,"kb_used_omap":18,"kb_used_meta":1200045,"kb_avail":713632372,"statfs":{"total":751619276800,"available":730759548928,"internally_reserved":0,"allocated":19630841856,"data_stored":19630591216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":18551,"internal_metadata":1228846985},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542166,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":22394888,"kb_used_data":21083380,"kb_used_omap":20,"kb_used_meta":1311467,"kb_avail":711608312,"statfs":{"total":751619276800,"available":728686911488,"internally_reserved":0,"allocated":21589381120,"data_stored":21589134032,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":20504,"internal_metadata":1342943208},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[7,0,2,6],"upper_bound":16},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705689,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":20468556,"kb_used_data":19268472,"kb_used_omap":23,"kb_used_meta":1200040,"kb_avail":713534644,"statfs":{"total":751619276800,"available":730659475456,"internally_reserved":0,"allocated":19730915328,"data_stored":19730664688,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":24410,"internal_metadata":1228841126},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":19890700288,"data_stored":19890700288,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":21750546432,"data_stored":21750546432,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":19880280064,"data_stored":19880280064,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":25337135104,"data_stored":25337135104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-24T07:44:20.739 INFO:tasks.cephadm.ceph_manager.ceph:increase pool size by 10 2026-02-24T07:44:20.739 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph osd pool set .mgr pg_num 51 2026-02-24T07:44:20.872 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:44:20.935 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 72 16 471840 471824 409.536 318.188 0.000980723 0.00244067 2026-02-24T07:44:20.989 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:44:20 trial035 ceph-mon[39284]: 2.3 scrub starts 2026-02-24T07:44:20.989 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:44:20 trial035 ceph-mon[39284]: from='client.14453 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2026-02-24T07:44:20.989 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:44:20 trial035 ceph-mon[39284]: 2.f scrub starts 2026-02-24T07:44:21.353 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:44:21.285+0000 7f675cd69640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:44:21.353 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:44:21.285+0000 7f675cd69640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:44:21.415 INFO:teuthology.orchestra.run.trial035.stderr:Error EPERM: crimson_allow_pg_split is false; pg_num increase denied for crimson pool 2026-02-24T07:44:21.433 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2026-02-24T07:44:21.669 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:44:21.708 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-24T07:44:21.784 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:44:21.781+0000 7fe0aa95c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:44:21.784 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:44:21.782+0000 7fe0aa95c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:44:21.798 INFO:teuthology.orchestra.run.trial035.stdout:osd.0: {} 2026-02-24T07:44:21.798 INFO:teuthology.orchestra.run.trial035.stderr:osd.0: osd_enable_op_tracker = '' 2026-02-24T07:44:21.811 INFO:teuthology.orchestra.run.trial035.stdout:osd.1: {} 2026-02-24T07:44:21.811 INFO:teuthology.orchestra.run.trial035.stderr:osd.1: osd_enable_op_tracker = '' 2026-02-24T07:44:21.824 INFO:teuthology.orchestra.run.trial035.stdout:osd.2: {} 2026-02-24T07:44:21.824 INFO:teuthology.orchestra.run.trial035.stderr:osd.2: osd_enable_op_tracker = '' 2026-02-24T07:44:21.836 INFO:teuthology.orchestra.run.trial035.stdout:osd.3: {} 2026-02-24T07:44:21.836 INFO:teuthology.orchestra.run.trial035.stderr:osd.3: osd_enable_op_tracker = '' 2026-02-24T07:44:21.936 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 73 16 476742 476726 408.122 306.375 0.00110722 0.00244912 2026-02-24T07:44:21.954 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:44:21 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/3749802379' entity='client.admin' cmd={"prefix": "osd pool set", "pool": ".mgr", "var": "pg_num", "val": "51"} : dispatch 2026-02-24T07:44:21.954 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:44:21 trial035 ceph-mon[39284]: pgmap v55: 17 pgs: 1 active+clean+scrubbing, 16 active+clean; 29 GiB data, 91 GiB used, 2.6 TiB / 2.7 TiB avail; 463 MiB/s wr, 7.41k op/s 2026-02-24T07:44:21.954 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:44:21 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/1606252801' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2026-02-24T07:44:22.935 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 74 16 482736 482720 407.669 374.625 0.00699307 0.00245186 2026-02-24T07:44:23.704 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:44:23 trial035 ceph-mon[39284]: 2.3 scrub ok 2026-02-24T07:44:23.875 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2026-02-24T07:44:23.936 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 75 16 489028 489012 407.477 393.25 0.00124509 0.00245305 2026-02-24T07:44:24.016 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:44:24.133 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:44:24.131+0000 7f4c9cc59640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:44:24.133 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:44:24.131+0000 7f4c9cc59640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:44:24.151 INFO:teuthology.orchestra.run.trial035.stdout:osd.0: {} 2026-02-24T07:44:24.151 INFO:teuthology.orchestra.run.trial035.stderr:osd.0: osd_enable_op_tracker = '' 2026-02-24T07:44:24.166 INFO:teuthology.orchestra.run.trial035.stdout:osd.1: {} 2026-02-24T07:44:24.166 INFO:teuthology.orchestra.run.trial035.stderr:osd.1: osd_enable_op_tracker = '' 2026-02-24T07:44:24.178 INFO:teuthology.orchestra.run.trial035.stdout:osd.2: {} 2026-02-24T07:44:24.178 INFO:teuthology.orchestra.run.trial035.stderr:osd.2: osd_enable_op_tracker = '' 2026-02-24T07:44:24.191 INFO:teuthology.orchestra.run.trial035.stdout:osd.3: {} 2026-02-24T07:44:24.191 INFO:teuthology.orchestra.run.trial035.stderr:osd.3: osd_enable_op_tracker = '' 2026-02-24T07:44:24.454 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:44:24 trial035 ceph-mon[39284]: 2.d scrub starts 2026-02-24T07:44:24.454 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:44:24 trial035 ceph-mon[39284]: 2.f scrub ok 2026-02-24T07:44:24.454 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:44:24 trial035 ceph-mon[39284]: pgmap v56: 17 pgs: 1 active+clean+scrubbing, 16 active+clean; 29 GiB data, 91 GiB used, 2.6 TiB / 2.7 TiB avail; 294 MiB/s wr, 4.70k op/s 2026-02-24T07:44:24.454 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:44:24 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/2461192067' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2026-02-24T07:44:24.936 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 76 16 494698 494682 406.778 354.375 0.00135004 0.00245725 2026-02-24T07:44:25.704 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:44:25 trial035 ceph-mon[39284]: 2.a scrub starts 2026-02-24T07:44:25.705 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:44:25 trial035 ceph-mon[39284]: 2.0 scrub starts 2026-02-24T07:44:25.936 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 77 16 500088 500072 405.87 336.875 0.00312306 0.00246273 2026-02-24T07:44:26.254 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2026-02-24T07:44:26.390 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:44:26.515 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:44:26.513+0000 7f393f856640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:44:26.515 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:44:26.513+0000 7f393f856640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:44:26.529 INFO:teuthology.orchestra.run.trial035.stdout:osd.0: {} 2026-02-24T07:44:26.529 INFO:teuthology.orchestra.run.trial035.stderr:osd.0: osd_enable_op_tracker = '' 2026-02-24T07:44:26.529 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:44:26 trial035 ceph-mon[39284]: pgmap v57: 17 pgs: 1 active+clean+scrubbing, 16 active+clean; 29 GiB data, 94 GiB used, 2.6 TiB / 2.7 TiB avail; 303 MiB/s wr, 4.84k op/s 2026-02-24T07:44:26.542 INFO:teuthology.orchestra.run.trial035.stdout:osd.1: {} 2026-02-24T07:44:26.542 INFO:teuthology.orchestra.run.trial035.stderr:osd.1: osd_enable_op_tracker = '' 2026-02-24T07:44:26.554 INFO:teuthology.orchestra.run.trial035.stdout:osd.2: {} 2026-02-24T07:44:26.554 INFO:teuthology.orchestra.run.trial035.stderr:osd.2: osd_enable_op_tracker = '' 2026-02-24T07:44:26.566 INFO:teuthology.orchestra.run.trial035.stdout:osd.3: {} 2026-02-24T07:44:26.566 INFO:teuthology.orchestra.run.trial035.stderr:osd.3: osd_enable_op_tracker = '' 2026-02-24T07:44:26.709 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2026-02-24T07:44:26.709 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2026-02-24T07:44:26.709 INFO:tasks.thrashosds.thrasher:fixing pg num pool unique_pool_0 2026-02-24T07:44:26.710 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph pg dump --format=json 2026-02-24T07:44:26.839 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:44:26.936 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 78 16 504828 504812 404.464 296.25 0.000806219 0.00247108 2026-02-24T07:44:27.269 INFO:journalctl@ceph.mgr.x.trial035.stdout:Feb 24 07:44:26 trial035 ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-mgr-x[39621]: ::ffff:10.20.193.91 - - [24/Feb/2026:07:44:26] "GET /metrics HTTP/1.1" 200 4221 "" "Prometheus/3.6.0" 2026-02-24T07:44:27.353 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:44:27.319+0000 7f191d00c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:44:27.354 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:44:27.319+0000 7f191d00c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:44:27.356 INFO:journalctl@ceph.prometheus.a.trial091.stdout:Feb 24 07:44:26 trial091 ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-prometheus-a[66007]: time=2026-02-24T07:44:26.973Z level=ERROR source=scrape.go:1630 msg="Failed to determine correct type of scrape target." component="scrape manager" scrape_pool=ceph target=http://trial035:9283/metrics content_type="text/html;charset=utf-8" fallback_media_type="" err="received unsupported Content-Type \"text/html;charset=utf-8\" and no fallback_scrape_protocol specified for target" 2026-02-24T07:44:27.458 INFO:teuthology.orchestra.run.trial035.stdout: 2026-02-24T07:44:27.459 INFO:teuthology.orchestra.run.trial035.stderr:dumped all 2026-02-24T07:44:27.704 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:44:27 trial035 ceph-mon[39284]: 2.c scrub starts 2026-02-24T07:44:27.705 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:44:27 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/2644087562' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2026-02-24T07:44:27.705 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:44:27 trial035 ceph-mon[39284]: 2.d scrub ok 2026-02-24T07:44:27.936 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 79 15 508098 508083 401.932 204.438 0.00217978 0.00248686 2026-02-24T07:44:28.127 INFO:teuthology.orchestra.run.trial035.stdout:{"pg_ready":true,"pg_map":{"version":57,"stamp":"2026-02-24T07:44:25.669921+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":30945706528,"num_objects":472187,"num_object_clones":0,"num_object_copies":1416561,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":472187,"num_whiteouts":0,"num_read":136,"num_read_kb":240,"num_write":472320,"num_write_kb":30221990,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":52,"ondisk_log_size":52,"up":51,"acting":51,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":51,"num_osds":4,"num_per_pool_osds":4,"num_per_pool_omap_osds":4,"kb":2936012800,"kb_used":98123564,"kb_used_data":92428508,"kb_used_omap":76,"kb_used_meta":5694899,"kb_avail":2837889236,"statfs":{"total":3006477107200,"available":2905998577664,"internally_reserved":0,"allocated":94646792192,"data_stored":94645793184,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":78108,"internal_metadata":5831577316},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":3809804288,"num_objects":58133,"num_object_clones":0,"num_object_copies":174399,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58133,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":58133,"num_write_kb":3720512,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001432"},"pg_stats":[{"pgid":"2.f","version":"40'30858","reported_seq":31192,"reported_epoch":40,"state":"active+clean","last_fresh":"2026-02-24T07:44:25.664791+0000","last_change":"2026-02-24T07:44:23.320237+0000","last_active":"2026-02-24T07:44:25.664791+0000","last_peered":"2026-02-24T07:44:25.664791+0000","last_clean":"2026-02-24T07:44:25.664791+0000","last_became_active":"2026-02-24T07:43:07.852899+0000","last_became_peered":"2026-02-24T07:43:07.852899+0000","last_unstale":"2026-02-24T07:44:25.664791+0000","last_undegraded":"2026-02-24T07:44:25.664791+0000","last_fullsized":"2026-02-24T07:44:25.664791+0000","mapping_epoch":30,"log_start":"40'30855","ondisk_log_start":"40'30855","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'30039","last_scrub_stamp":"2026-02-24T07:44:23.320202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_clean_scrub_stamp":"2026-02-24T07:44:23.320202+0000","objects_scrubbed":29558,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:45:24.615266+0000","scrub_duration":2598,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2022309888,"num_objects":30858,"num_object_clones":0,"num_object_copies":92574,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30858,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":30858,"num_write_kb":1974912,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.e","version":"40'29425","reported_seq":29739,"reported_epoch":40,"state":"active+clean","last_fresh":"2026-02-24T07:44:20.682471+0000","last_change":"2026-02-24T07:44:16.083399+0000","last_active":"2026-02-24T07:44:20.682471+0000","last_peered":"2026-02-24T07:44:20.682471+0000","last_clean":"2026-02-24T07:44:20.682471+0000","last_became_active":"2026-02-24T07:43:07.852815+0000","last_became_peered":"2026-02-24T07:43:07.852815+0000","last_unstale":"2026-02-24T07:44:20.682471+0000","last_undegraded":"2026-02-24T07:44:20.682471+0000","last_fullsized":"2026-02-24T07:44:20.682471+0000","mapping_epoch":30,"log_start":"40'29422","ondisk_log_start":"40'29422","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'27767","last_scrub_stamp":"2026-02-24T07:44:16.083320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_clean_scrub_stamp":"2026-02-24T07:44:16.083320+0000","objects_scrubbed":27311,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:45:28.111749+0000","scrub_duration":2943,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1928396800,"num_objects":29425,"num_object_clones":0,"num_object_copies":88275,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29425,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":29425,"num_write_kb":1883200,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.d","version":"40'29670","reported_seq":29699,"reported_epoch":40,"state":"active+clean","last_fresh":"2026-02-24T07:44:20.689255+0000","last_change":"2026-02-24T07:43:07.854403+0000","last_active":"2026-02-24T07:44:20.689255+0000","last_peered":"2026-02-24T07:44:20.689255+0000","last_clean":"2026-02-24T07:44:20.689255+0000","last_became_active":"2026-02-24T07:43:07.853975+0000","last_became_peered":"2026-02-24T07:43:07.853975+0000","last_unstale":"2026-02-24T07:44:20.689255+0000","last_undegraded":"2026-02-24T07:44:20.689255+0000","last_fullsized":"2026-02-24T07:44:20.689255+0000","mapping_epoch":30,"log_start":"40'29667","ondisk_log_start":"40'29667","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_clean_scrub_stamp":"2026-02-24T07:43:06.832892+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1944453120,"num_objects":29670,"num_object_clones":0,"num_object_copies":89010,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29670,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":29670,"num_write_kb":1898880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"40'29565","reported_seq":29595,"reported_epoch":40,"state":"active+clean","last_fresh":"2026-02-24T07:44:20.684859+0000","last_change":"2026-02-24T07:43:07.854780+0000","last_active":"2026-02-24T07:44:20.684859+0000","last_peered":"2026-02-24T07:44:20.684859+0000","last_clean":"2026-02-24T07:44:20.684859+0000","last_became_active":"2026-02-24T07:43:07.854428+0000","last_became_peered":"2026-02-24T07:43:07.854428+0000","last_unstale":"2026-02-24T07:44:20.684859+0000","last_undegraded":"2026-02-24T07:44:20.684859+0000","last_fullsized":"2026-02-24T07:44:20.684859+0000","mapping_epoch":30,"log_start":"40'29562","ondisk_log_start":"40'29562","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_clean_scrub_stamp":"2026-02-24T07:43:06.832892+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1937571840,"num_objects":29565,"num_object_clones":0,"num_object_copies":88695,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29565,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":29565,"num_write_kb":1892160,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"29'99","reported_seq":209,"reported_epoch":40,"state":"active+clean","last_fresh":"2026-02-24T07:43:59.386353+0000","last_change":"2026-02-24T07:43:59.386353+0000","last_active":"2026-02-24T07:43:59.386353+0000","last_peered":"2026-02-24T07:43:59.386353+0000","last_clean":"2026-02-24T07:43:59.386353+0000","last_became_active":"2026-02-24T07:42:54.186719+0000","last_became_peered":"2026-02-24T07:42:54.186719+0000","last_unstale":"2026-02-24T07:43:59.386353+0000","last_undegraded":"2026-02-24T07:43:59.386353+0000","last_fullsized":"2026-02-24T07:43:59.386353+0000","mapping_epoch":27,"log_start":"29'96","ondisk_log_start":"29'96","created":20,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'99","last_scrub_stamp":"2026-02-24T07:43:59.386324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:42:43.640776+0000","last_clean_scrub_stamp":"2026-02-24T07:43:59.386324+0000","objects_scrubbed":2,"log_size":3,"log_dups_size":96,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:45:24.339819+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":240,"num_write":135,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"40'29549","reported_seq":29692,"reported_epoch":40,"state":"active+clean+scrubbing","last_fresh":"2026-02-24T07:44:20.673139+0000","last_change":"2026-02-24T07:44:19.439445+0000","last_active":"2026-02-24T07:44:20.673139+0000","last_peered":"2026-02-24T07:44:20.673139+0000","last_clean":"2026-02-24T07:44:20.673139+0000","last_became_active":"2026-02-24T07:43:07.842530+0000","last_became_peered":"2026-02-24T07:43:07.842530+0000","last_unstale":"2026-02-24T07:44:20.673139+0000","last_undegraded":"2026-02-24T07:44:20.673139+0000","last_fullsized":"2026-02-24T07:44:20.673139+0000","mapping_epoch":30,"log_start":"40'29546","ondisk_log_start":"40'29546","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_clean_scrub_stamp":"2026-02-24T07:43:06.832892+0000","objects_scrubbed":10949,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"scrubbing for 2s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1936523264,"num_objects":29549,"num_object_clones":0,"num_object_copies":88647,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29549,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":29549,"num_write_kb":1891136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"40'29171","reported_seq":29201,"reported_epoch":40,"state":"active+clean","last_fresh":"2026-02-24T07:44:20.675073+0000","last_change":"2026-02-24T07:43:07.854082+0000","last_active":"2026-02-24T07:44:20.675073+0000","last_peered":"2026-02-24T07:44:20.675073+0000","last_clean":"2026-02-24T07:44:20.675073+0000","last_became_active":"2026-02-24T07:43:07.853885+0000","last_became_peered":"2026-02-24T07:43:07.853885+0000","last_unstale":"2026-02-24T07:44:20.675073+0000","last_undegraded":"2026-02-24T07:44:20.675073+0000","last_fullsized":"2026-02-24T07:44:20.675073+0000","mapping_epoch":30,"log_start":"40'29168","ondisk_log_start":"40'29168","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_clean_scrub_stamp":"2026-02-24T07:43:06.832892+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1911750656,"num_objects":29171,"num_object_clones":0,"num_object_copies":87513,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29171,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":29171,"num_write_kb":1866944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"40'29548","reported_seq":29578,"reported_epoch":40,"state":"active+clean","last_fresh":"2026-02-24T07:44:20.689360+0000","last_change":"2026-02-24T07:43:07.854497+0000","last_active":"2026-02-24T07:44:20.689360+0000","last_peered":"2026-02-24T07:44:20.689360+0000","last_clean":"2026-02-24T07:44:20.689360+0000","last_became_active":"2026-02-24T07:43:07.854212+0000","last_became_peered":"2026-02-24T07:43:07.854212+0000","last_unstale":"2026-02-24T07:44:20.689360+0000","last_undegraded":"2026-02-24T07:44:20.689360+0000","last_fullsized":"2026-02-24T07:44:20.689360+0000","mapping_epoch":30,"log_start":"40'29545","ondisk_log_start":"40'29545","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_clean_scrub_stamp":"2026-02-24T07:43:06.832892+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:44:21.531086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1936457728,"num_objects":29548,"num_object_clones":0,"num_object_copies":88644,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29548,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":29548,"num_write_kb":1891072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[1,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"40'29601","reported_seq":29630,"reported_epoch":40,"state":"active+clean","last_fresh":"2026-02-24T07:44:20.679096+0000","last_change":"2026-02-24T07:43:07.843123+0000","last_active":"2026-02-24T07:44:20.679096+0000","last_peered":"2026-02-24T07:44:20.679096+0000","last_clean":"2026-02-24T07:44:20.679096+0000","last_became_active":"2026-02-24T07:43:07.842864+0000","last_became_peered":"2026-02-24T07:43:07.842864+0000","last_unstale":"2026-02-24T07:44:20.679096+0000","last_undegraded":"2026-02-24T07:44:20.679096+0000","last_fullsized":"2026-02-24T07:44:20.679096+0000","mapping_epoch":30,"log_start":"40'29597","ondisk_log_start":"40'29597","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_clean_scrub_stamp":"2026-02-24T07:43:06.832892+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":3000,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:44:34.876596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1939931136,"num_objects":29601,"num_object_clones":0,"num_object_copies":88803,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29601,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":29601,"num_write_kb":1894464,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.4","version":"40'29311","reported_seq":29626,"reported_epoch":40,"state":"active+clean","last_fresh":"2026-02-24T07:44:20.688382+0000","last_change":"2026-02-24T07:44:16.909673+0000","last_active":"2026-02-24T07:44:20.688382+0000","last_peered":"2026-02-24T07:44:20.688382+0000","last_clean":"2026-02-24T07:44:20.688382+0000","last_became_active":"2026-02-24T07:43:07.853966+0000","last_became_peered":"2026-02-24T07:43:07.853966+0000","last_unstale":"2026-02-24T07:44:20.688382+0000","last_undegraded":"2026-02-24T07:44:20.688382+0000","last_fullsized":"2026-02-24T07:44:20.688382+0000","mapping_epoch":30,"log_start":"40'29308","ondisk_log_start":"40'29308","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'27932","last_scrub_stamp":"2026-02-24T07:44:16.909624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_clean_scrub_stamp":"2026-02-24T07:44:16.909624+0000","objects_scrubbed":27480,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:45:43.624405+0000","scrub_duration":2774,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1920925696,"num_objects":29311,"num_object_clones":0,"num_object_copies":87933,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29311,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":29311,"num_write_kb":1875904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"40'29327","reported_seq":29357,"reported_epoch":40,"state":"active+clean","last_fresh":"2026-02-24T07:44:20.668756+0000","last_change":"2026-02-24T07:43:07.854184+0000","last_active":"2026-02-24T07:44:20.668756+0000","last_peered":"2026-02-24T07:44:20.668756+0000","last_clean":"2026-02-24T07:44:20.668756+0000","last_became_active":"2026-02-24T07:43:07.853878+0000","last_became_peered":"2026-02-24T07:43:07.853878+0000","last_unstale":"2026-02-24T07:44:20.668756+0000","last_undegraded":"2026-02-24T07:44:20.668756+0000","last_fullsized":"2026-02-24T07:44:20.668756+0000","mapping_epoch":30,"log_start":"40'29324","ondisk_log_start":"40'29324","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_clean_scrub_stamp":"2026-02-24T07:43:06.832892+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:44:31.507398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1921974272,"num_objects":29327,"num_object_clones":0,"num_object_copies":87981,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29327,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":29327,"num_write_kb":1876928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"40'29532","reported_seq":29562,"reported_epoch":40,"state":"active+clean","last_fresh":"2026-02-24T07:44:20.685608+0000","last_change":"2026-02-24T07:43:07.842180+0000","last_active":"2026-02-24T07:44:20.685608+0000","last_peered":"2026-02-24T07:44:20.685608+0000","last_clean":"2026-02-24T07:44:20.685608+0000","last_became_active":"2026-02-24T07:43:07.841993+0000","last_became_peered":"2026-02-24T07:43:07.841993+0000","last_unstale":"2026-02-24T07:44:20.685608+0000","last_undegraded":"2026-02-24T07:44:20.685608+0000","last_fullsized":"2026-02-24T07:44:20.685608+0000","mapping_epoch":30,"log_start":"40'29529","ondisk_log_start":"40'29529","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_clean_scrub_stamp":"2026-02-24T07:43:06.832892+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1935409152,"num_objects":29532,"num_object_clones":0,"num_object_copies":88596,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29532,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":29532,"num_write_kb":1890048,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"40'29085","reported_seq":29401,"reported_epoch":40,"state":"active+clean","last_fresh":"2026-02-24T07:44:20.671539+0000","last_change":"2026-02-24T07:44:18.020443+0000","last_active":"2026-02-24T07:44:20.671539+0000","last_peered":"2026-02-24T07:44:20.671539+0000","last_clean":"2026-02-24T07:44:20.671539+0000","last_became_active":"2026-02-24T07:43:07.841920+0000","last_became_peered":"2026-02-24T07:43:07.841920+0000","last_unstale":"2026-02-24T07:44:20.671539+0000","last_undegraded":"2026-02-24T07:44:20.671539+0000","last_fullsized":"2026-02-24T07:44:20.671539+0000","mapping_epoch":30,"log_start":"40'29082","ondisk_log_start":"40'29082","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'28226","last_scrub_stamp":"2026-02-24T07:44:18.020379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_clean_scrub_stamp":"2026-02-24T07:44:18.020379+0000","objects_scrubbed":27705,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:45:27.051564+0000","scrub_duration":2601,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1906114560,"num_objects":29085,"num_object_clones":0,"num_object_copies":87255,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29085,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":29085,"num_write_kb":1861440,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.8","version":"40'29125","reported_seq":29155,"reported_epoch":40,"state":"active+clean","last_fresh":"2026-02-24T07:44:20.676943+0000","last_change":"2026-02-24T07:43:07.853992+0000","last_active":"2026-02-24T07:44:20.676943+0000","last_peered":"2026-02-24T07:44:20.676943+0000","last_clean":"2026-02-24T07:44:20.676943+0000","last_became_active":"2026-02-24T07:43:07.853758+0000","last_became_peered":"2026-02-24T07:43:07.853758+0000","last_unstale":"2026-02-24T07:44:20.676943+0000","last_undegraded":"2026-02-24T07:44:20.676943+0000","last_fullsized":"2026-02-24T07:44:20.676943+0000","mapping_epoch":30,"log_start":"40'29122","ondisk_log_start":"40'29122","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_clean_scrub_stamp":"2026-02-24T07:43:06.832892+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:44:27.436119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1908736000,"num_objects":29125,"num_object_clones":0,"num_object_copies":87375,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29125,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":29125,"num_write_kb":1864000,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"40'29574","reported_seq":29604,"reported_epoch":40,"state":"active+clean","last_fresh":"2026-02-24T07:44:20.683017+0000","last_change":"2026-02-24T07:43:07.854687+0000","last_active":"2026-02-24T07:44:20.683017+0000","last_peered":"2026-02-24T07:44:20.683017+0000","last_clean":"2026-02-24T07:44:20.683017+0000","last_became_active":"2026-02-24T07:43:07.854377+0000","last_became_peered":"2026-02-24T07:43:07.854377+0000","last_unstale":"2026-02-24T07:44:20.683017+0000","last_undegraded":"2026-02-24T07:44:20.683017+0000","last_fullsized":"2026-02-24T07:44:20.683017+0000","mapping_epoch":30,"log_start":"40'29571","ondisk_log_start":"40'29571","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_clean_scrub_stamp":"2026-02-24T07:43:06.832892+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:44:34.215480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1938161664,"num_objects":29574,"num_object_clones":0,"num_object_copies":88722,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29574,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":29574,"num_write_kb":1892736,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.a","version":"40'29376","reported_seq":29406,"reported_epoch":40,"state":"active+clean","last_fresh":"2026-02-24T07:44:20.689095+0000","last_change":"2026-02-24T07:43:07.841933+0000","last_active":"2026-02-24T07:44:20.689095+0000","last_peered":"2026-02-24T07:44:20.689095+0000","last_clean":"2026-02-24T07:44:20.689095+0000","last_became_active":"2026-02-24T07:43:07.841721+0000","last_became_peered":"2026-02-24T07:43:07.841721+0000","last_unstale":"2026-02-24T07:44:20.689095+0000","last_undegraded":"2026-02-24T07:44:20.689095+0000","last_fullsized":"2026-02-24T07:44:20.689095+0000","mapping_epoch":30,"log_start":"40'29373","ondisk_log_start":"40'29373","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_clean_scrub_stamp":"2026-02-24T07:43:06.832892+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1925185536,"num_objects":29376,"num_object_clones":0,"num_object_copies":88128,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29376,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":29376,"num_write_kb":1880064,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.b","version":"40'29468","reported_seq":29498,"reported_epoch":40,"state":"active+clean","last_fresh":"2026-02-24T07:44:20.672760+0000","last_change":"2026-02-24T07:43:07.853283+0000","last_active":"2026-02-24T07:44:20.672760+0000","last_peered":"2026-02-24T07:44:20.672760+0000","last_clean":"2026-02-24T07:44:20.672760+0000","last_became_active":"2026-02-24T07:43:07.853069+0000","last_became_peered":"2026-02-24T07:43:07.853069+0000","last_unstale":"2026-02-24T07:44:20.672760+0000","last_undegraded":"2026-02-24T07:44:20.672760+0000","last_fullsized":"2026-02-24T07:44:20.672760+0000","mapping_epoch":30,"log_start":"40'29465","ondisk_log_start":"40'29465","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_clean_scrub_stamp":"2026-02-24T07:43:06.832892+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:44:32.581403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1931214848,"num_objects":29468,"num_object_clones":0,"num_object_copies":88404,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29468,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":29468,"num_write_kb":1885952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":30945116160,"num_objects":472185,"num_object_clones":0,"num_object_copies":1416555,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":472185,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":472185,"num_write_kb":30219840,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":95039782912,"data_stored":95039782912,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":49,"ondisk_log_size":49,"up":48,"acting":48,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":240,"num_write":135,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":3,"up_from":25,"seq":107374182420,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":27880780,"kb_used_data":26276408,"kb_used_omap":14,"kb_used_meta":1604337,"kb_avail":706122420,"statfs":{"total":751619276800,"available":723069358080,"internally_reserved":0,"allocated":26907041792,"data_stored":26906791152,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":14643,"internal_metadata":1642841805},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":19,"seq":81604378647,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":23256268,"kb_used_data":21904760,"kb_used_omap":18,"kb_used_meta":1351469,"kb_avail":710746932,"statfs":{"total":751619276800,"available":727804858368,"internally_reserved":0,"allocated":22430474240,"data_stored":22430223600,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":18551,"internal_metadata":1383905161},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542167,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":23879816,"kb_used_data":22492148,"kb_used_omap":20,"kb_used_meta":1387627,"kb_avail":710123384,"statfs":{"total":751619276800,"available":727166345216,"internally_reserved":0,"allocated":23031959552,"data_stored":23031712464,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":20504,"internal_metadata":1420931048},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705691,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":23106700,"kb_used_data":21755192,"kb_used_omap":23,"kb_used_meta":1351464,"kb_avail":710896500,"statfs":{"total":751619276800,"available":727958016000,"internally_reserved":0,"allocated":22277316608,"data_stored":22277065968,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":24410,"internal_metadata":1383899302},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":22458138624,"data_stored":22458138624,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":23162126336,"data_stored":23162126336,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":22440378368,"data_stored":22440378368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":26979139584,"data_stored":26979139584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-24T07:44:28.128 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph osd pool set unique_pool_0 pgp_num 36 2026-02-24T07:44:28.273 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:44:28.408 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:44:28.405+0000 7efe558c7640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:44:28.408 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:44:28.406+0000 7efe558c7640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:44:28.536 INFO:teuthology.orchestra.run.trial035.stderr:Error EPERM: crimson_allow_pg_split is false; pgp_num increase denied 2026-02-24T07:44:28.576 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-24T07:44:28.607 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2026-02-24T07:44:28.704 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:44:28 trial035 ceph-mon[39284]: 2.6 scrub starts 2026-02-24T07:44:28.705 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:44:28 trial035 ceph-mon[39284]: from='client.14463 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2026-02-24T07:44:28.705 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:44:28 trial035 ceph-mon[39284]: pgmap v58: 17 pgs: 3 active+clean+scrubbing, 14 active+clean; 30 GiB data, 97 GiB used, 2.6 TiB / 2.7 TiB avail; 432 MiB/s wr, 6.92k op/s 2026-02-24T07:44:28.705 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:44:28 trial035 ceph-mon[39284]: 2.8 scrub starts 2026-02-24T07:44:28.740 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:44:28.862 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:44:28.860+0000 7f3046888640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:44:28.862 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:44:28.860+0000 7f3046888640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:44:28.877 INFO:teuthology.orchestra.run.trial035.stdout:osd.0: {} 2026-02-24T07:44:28.877 INFO:teuthology.orchestra.run.trial035.stderr:osd.0: osd_enable_op_tracker = '' 2026-02-24T07:44:28.891 INFO:teuthology.orchestra.run.trial035.stdout:osd.1: {} 2026-02-24T07:44:28.891 INFO:teuthology.orchestra.run.trial035.stderr:osd.1: osd_enable_op_tracker = '' 2026-02-24T07:44:28.903 INFO:teuthology.orchestra.run.trial035.stdout:osd.2: {} 2026-02-24T07:44:28.903 INFO:teuthology.orchestra.run.trial035.stderr:osd.2: osd_enable_op_tracker = '' 2026-02-24T07:44:28.918 INFO:teuthology.orchestra.run.trial035.stdout:osd.3: {} 2026-02-24T07:44:28.918 INFO:teuthology.orchestra.run.trial035.stderr:osd.3: osd_enable_op_tracker = '' 2026-02-24T07:44:28.936 INFO:tasks.radosbench.radosbench.0.trial035.stdout:2026-02-24T07:44:28.935847+0000 min lat: 0.000663722 max lat: 0.066631 avg lat: 0.00249424 2026-02-24T07:44:28.936 INFO:tasks.radosbench.radosbench.0.trial035.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2026-02-24T07:44:28.936 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 80 15 513009 512994 400.744 306.938 0.00277928 0.00249424 2026-02-24T07:44:29.704 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:44:29 trial035 ceph-mon[39284]: 2.a scrub ok 2026-02-24T07:44:29.704 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:44:29 trial035 ceph-mon[39284]: 2.0 scrub ok 2026-02-24T07:44:29.704 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:44:29 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/3825732378' entity='client.admin' cmd={"prefix": "osd pool set", "pool": "unique_pool_0", "var": "pgp_num", "val": "36"} : dispatch 2026-02-24T07:44:29.704 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:44:29 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/2729986471' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2026-02-24T07:44:29.936 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 81 16 518722 518706 400.204 357 0.00235604 0.00249762 2026-02-24T07:44:30.704 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:44:30 trial035 ceph-mon[39284]: 2.c scrub ok 2026-02-24T07:44:30.704 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:44:30 trial035 ceph-mon[39284]: pgmap v59: 17 pgs: 3 active+clean+scrubbing, 14 active+clean; 30 GiB data, 97 GiB used, 2.6 TiB / 2.7 TiB avail; 297 MiB/s wr, 4.75k op/s 2026-02-24T07:44:30.704 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:44:30 trial035 ceph-mon[39284]: from='mgr.14284 10.20.193.35:0/3681373222' entity='mgr.x' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2026-02-24T07:44:30.936 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 82 16 524138 524122 399.451 338.5 0.00176875 0.00250232 2026-02-24T07:44:30.959 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2026-02-24T07:44:31.095 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:44:31.213 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:44:31.212+0000 7f34cfc3c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:44:31.213 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:44:31.212+0000 7f34cfc3c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:44:31.228 INFO:teuthology.orchestra.run.trial035.stdout:osd.0: {} 2026-02-24T07:44:31.228 INFO:teuthology.orchestra.run.trial035.stderr:osd.0: osd_enable_op_tracker = '' 2026-02-24T07:44:31.241 INFO:teuthology.orchestra.run.trial035.stdout:osd.1: {} 2026-02-24T07:44:31.241 INFO:teuthology.orchestra.run.trial035.stderr:osd.1: osd_enable_op_tracker = '' 2026-02-24T07:44:31.257 INFO:teuthology.orchestra.run.trial035.stdout:osd.2: {} 2026-02-24T07:44:31.258 INFO:teuthology.orchestra.run.trial035.stderr:osd.2: osd_enable_op_tracker = '' 2026-02-24T07:44:31.270 INFO:teuthology.orchestra.run.trial035.stdout:osd.3: {} 2026-02-24T07:44:31.270 INFO:teuthology.orchestra.run.trial035.stderr:osd.3: osd_enable_op_tracker = '' 2026-02-24T07:44:31.705 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:44:31 trial035 ceph-mon[39284]: 2.1 scrub starts 2026-02-24T07:44:31.705 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:44:31 trial035 ceph-mon[39284]: 2.6 scrub ok 2026-02-24T07:44:31.705 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:44:31 trial035 ceph-mon[39284]: 2.8 scrub ok 2026-02-24T07:44:31.706 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:44:31 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/747856007' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2026-02-24T07:44:31.936 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 83 16 530033 530017 399.077 368.438 0.00199282 0.00250467 2026-02-24T07:44:32.705 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:44:32 trial035 ceph-mon[39284]: pgmap v60: 17 pgs: 2 active+clean+scrubbing, 15 active+clean; 32 GiB data, 101 GiB used, 2.6 TiB / 2.7 TiB avail; 422 MiB/s wr, 6.75k op/s 2026-02-24T07:44:32.705 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:44:32 trial035 ceph-mon[39284]: 2.5 scrub starts 2026-02-24T07:44:32.937 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 84 16 535992 535976 398.759 372.438 0.00208248 0.00250667 2026-02-24T07:44:33.352 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2026-02-24T07:44:33.493 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:44:33.578 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2026-02-24T07:44:33.578 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2026-02-24T07:44:33.578 INFO:tasks.thrashosds.thrasher:Growing pool .mgr 2026-02-24T07:44:33.579 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph pg dump --format=json 2026-02-24T07:44:33.608 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:44:33.606+0000 7f3e93052640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:44:33.608 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:44:33.607+0000 7f3e93052640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:44:33.622 INFO:teuthology.orchestra.run.trial035.stdout:osd.0: {} 2026-02-24T07:44:33.622 INFO:teuthology.orchestra.run.trial035.stderr:osd.0: osd_enable_op_tracker = '' 2026-02-24T07:44:33.638 INFO:teuthology.orchestra.run.trial035.stdout:osd.1: {} 2026-02-24T07:44:33.638 INFO:teuthology.orchestra.run.trial035.stderr:osd.1: osd_enable_op_tracker = '' 2026-02-24T07:44:33.649 INFO:teuthology.orchestra.run.trial035.stdout:osd.2: {} 2026-02-24T07:44:33.649 INFO:teuthology.orchestra.run.trial035.stderr:osd.2: osd_enable_op_tracker = '' 2026-02-24T07:44:33.660 INFO:teuthology.orchestra.run.trial035.stdout:osd.3: {} 2026-02-24T07:44:33.661 INFO:teuthology.orchestra.run.trial035.stderr:osd.3: osd_enable_op_tracker = '' 2026-02-24T07:44:33.715 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:44:33.838 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:44:33.837+0000 7fdda7640640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:44:33.838 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:44:33.837+0000 7fdda7640640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:44:33.936 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 85 16 542006 541990 398.49 375.875 0.00152636 0.00250837 2026-02-24T07:44:33.954 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:44:33 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/2187512572' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2026-02-24T07:44:33.975 INFO:teuthology.orchestra.run.trial035.stdout: 2026-02-24T07:44:33.976 INFO:teuthology.orchestra.run.trial035.stderr:dumped all 2026-02-24T07:44:34.020 INFO:teuthology.orchestra.run.trial035.stdout:{"pg_ready":true,"pg_map":{"version":61,"stamp":"2026-02-24T07:44:33.670951+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":34261041696,"num_objects":522775,"num_object_clones":0,"num_object_copies":1568325,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":522775,"num_whiteouts":0,"num_read":136,"num_read_kb":240,"num_write":522908,"num_write_kb":33459622,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":52,"ondisk_log_size":52,"up":51,"acting":51,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":51,"num_osds":4,"num_per_pool_osds":4,"num_per_pool_omap_osds":4,"kb":2936012800,"kb_used":105837740,"kb_used_data":99727580,"kb_used_omap":76,"kb_used_meta":6110003,"kb_avail":2830175060,"statfs":{"total":3006477107200,"available":2898099261440,"internally_reserved":0,"allocated":102121041920,"data_stored":102120042912,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":78108,"internal_metadata":6256643812},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,1,2,1],"upper_bound":16},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":3430809600,"num_objects":52350,"num_object_clones":0,"num_object_copies":157050,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52350,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":52350,"num_write_kb":3350400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001531"},"pg_stats":[{"pgid":"2.f","version":"40'32354","reported_seq":32685,"reported_epoch":40,"state":"active+clean","last_fresh":"2026-02-24T07:44:30.656916+0000","last_change":"2026-02-24T07:44:23.320237+0000","last_active":"2026-02-24T07:44:30.656916+0000","last_peered":"2026-02-24T07:44:30.656916+0000","last_clean":"2026-02-24T07:44:30.656916+0000","last_became_active":"2026-02-24T07:43:07.852899+0000","last_became_peered":"2026-02-24T07:43:07.852899+0000","last_unstale":"2026-02-24T07:44:30.656916+0000","last_undegraded":"2026-02-24T07:44:30.656916+0000","last_fullsized":"2026-02-24T07:44:30.656916+0000","mapping_epoch":30,"log_start":"40'32350","ondisk_log_start":"40'32350","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'30039","last_scrub_stamp":"2026-02-24T07:44:23.320202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_clean_scrub_stamp":"2026-02-24T07:44:23.320202+0000","objects_scrubbed":29558,"log_size":4,"log_dups_size":3000,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:45:24.615266+0000","scrub_duration":2598,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2120351744,"num_objects":32354,"num_object_clones":0,"num_object_copies":97062,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32354,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":32354,"num_write_kb":2070656,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.e","version":"40'32727","reported_seq":33041,"reported_epoch":40,"state":"active+clean","last_fresh":"2026-02-24T07:44:30.686456+0000","last_change":"2026-02-24T07:44:16.083399+0000","last_active":"2026-02-24T07:44:30.686456+0000","last_peered":"2026-02-24T07:44:30.686456+0000","last_clean":"2026-02-24T07:44:30.686456+0000","last_became_active":"2026-02-24T07:43:07.852815+0000","last_became_peered":"2026-02-24T07:43:07.852815+0000","last_unstale":"2026-02-24T07:44:30.686456+0000","last_undegraded":"2026-02-24T07:44:30.686456+0000","last_fullsized":"2026-02-24T07:44:30.686456+0000","mapping_epoch":30,"log_start":"40'32724","ondisk_log_start":"40'32724","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'27767","last_scrub_stamp":"2026-02-24T07:44:16.083320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_clean_scrub_stamp":"2026-02-24T07:44:16.083320+0000","objects_scrubbed":27311,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:45:28.111749+0000","scrub_duration":2943,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2144796672,"num_objects":32727,"num_object_clones":0,"num_object_copies":98181,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32727,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":32727,"num_write_kb":2094528,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.d","version":"40'32906","reported_seq":33257,"reported_epoch":40,"state":"active+clean","last_fresh":"2026-02-24T07:44:30.689336+0000","last_change":"2026-02-24T07:44:26.596366+0000","last_active":"2026-02-24T07:44:30.689336+0000","last_peered":"2026-02-24T07:44:30.689336+0000","last_clean":"2026-02-24T07:44:30.689336+0000","last_became_active":"2026-02-24T07:43:07.853975+0000","last_became_peered":"2026-02-24T07:43:07.853975+0000","last_unstale":"2026-02-24T07:44:30.689336+0000","last_undegraded":"2026-02-24T07:44:30.689336+0000","last_fullsized":"2026-02-24T07:44:30.689336+0000","mapping_epoch":30,"log_start":"40'32903","ondisk_log_start":"40'32903","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'31656","last_scrub_stamp":"2026-02-24T07:44:26.596292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_clean_scrub_stamp":"2026-02-24T07:44:26.596292+0000","objects_scrubbed":31097,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:45:55.416603+0000","scrub_duration":3380,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2156527616,"num_objects":32906,"num_object_clones":0,"num_object_copies":98718,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32906,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":32906,"num_write_kb":2105984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"40'32804","reported_seq":33162,"reported_epoch":40,"state":"active+clean","last_fresh":"2026-02-24T07:44:30.688577+0000","last_change":"2026-02-24T07:44:29.667393+0000","last_active":"2026-02-24T07:44:30.688577+0000","last_peered":"2026-02-24T07:44:30.688577+0000","last_clean":"2026-02-24T07:44:30.688577+0000","last_became_active":"2026-02-24T07:43:07.854428+0000","last_became_peered":"2026-02-24T07:43:07.854428+0000","last_unstale":"2026-02-24T07:44:30.688577+0000","last_undegraded":"2026-02-24T07:44:30.688577+0000","last_fullsized":"2026-02-24T07:44:30.688577+0000","mapping_epoch":30,"log_start":"40'32801","ondisk_log_start":"40'32801","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'32466","last_scrub_stamp":"2026-02-24T07:44:29.667348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_clean_scrub_stamp":"2026-02-24T07:44:29.667348+0000","objects_scrubbed":31970,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:45:43.092916+0000","scrub_duration":3552,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2149842944,"num_objects":32804,"num_object_clones":0,"num_object_copies":98412,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32804,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":32804,"num_write_kb":2099456,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"29'99","reported_seq":209,"reported_epoch":40,"state":"active+clean","last_fresh":"2026-02-24T07:43:59.386353+0000","last_change":"2026-02-24T07:43:59.386353+0000","last_active":"2026-02-24T07:43:59.386353+0000","last_peered":"2026-02-24T07:43:59.386353+0000","last_clean":"2026-02-24T07:43:59.386353+0000","last_became_active":"2026-02-24T07:42:54.186719+0000","last_became_peered":"2026-02-24T07:42:54.186719+0000","last_unstale":"2026-02-24T07:43:59.386353+0000","last_undegraded":"2026-02-24T07:43:59.386353+0000","last_fullsized":"2026-02-24T07:43:59.386353+0000","mapping_epoch":27,"log_start":"29'96","ondisk_log_start":"29'96","created":20,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'99","last_scrub_stamp":"2026-02-24T07:43:59.386324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:42:43.640776+0000","last_clean_scrub_stamp":"2026-02-24T07:43:59.386324+0000","objects_scrubbed":2,"log_size":3,"log_dups_size":96,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:45:24.339819+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":240,"num_write":135,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"40'32842","reported_seq":33178,"reported_epoch":40,"state":"active+clean","last_fresh":"2026-02-24T07:44:30.683918+0000","last_change":"2026-02-24T07:44:22.665897+0000","last_active":"2026-02-24T07:44:30.683918+0000","last_peered":"2026-02-24T07:44:30.683918+0000","last_clean":"2026-02-24T07:44:30.683918+0000","last_became_active":"2026-02-24T07:43:07.842530+0000","last_became_peered":"2026-02-24T07:43:07.842530+0000","last_unstale":"2026-02-24T07:44:30.683918+0000","last_undegraded":"2026-02-24T07:44:30.683918+0000","last_fullsized":"2026-02-24T07:44:30.683918+0000","mapping_epoch":30,"log_start":"40'32839","ondisk_log_start":"40'32839","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'30210","last_scrub_stamp":"2026-02-24T07:44:22.665762+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_clean_scrub_stamp":"2026-02-24T07:44:22.665762+0000","objects_scrubbed":29682,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:45:26.371453+0000","scrub_duration":3226,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2152333312,"num_objects":32842,"num_object_clones":0,"num_object_copies":98526,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32842,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":32842,"num_write_kb":2101888,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"40'32425","reported_seq":32777,"reported_epoch":40,"state":"active+clean","last_fresh":"2026-02-24T07:44:30.679701+0000","last_change":"2026-02-24T07:44:28.446165+0000","last_active":"2026-02-24T07:44:30.679701+0000","last_peered":"2026-02-24T07:44:30.679701+0000","last_clean":"2026-02-24T07:44:30.679701+0000","last_became_active":"2026-02-24T07:43:07.853885+0000","last_became_peered":"2026-02-24T07:43:07.853885+0000","last_unstale":"2026-02-24T07:44:30.679701+0000","last_undegraded":"2026-02-24T07:44:30.679701+0000","last_fullsized":"2026-02-24T07:44:30.679701+0000","mapping_epoch":30,"log_start":"40'32422","ondisk_log_start":"40'32422","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'31685","last_scrub_stamp":"2026-02-24T07:44:28.446064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_clean_scrub_stamp":"2026-02-24T07:44:28.446064+0000","objects_scrubbed":31280,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:45:52.594601+0000","scrub_duration":3117,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2125004800,"num_objects":32425,"num_object_clones":0,"num_object_copies":97275,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32425,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":32425,"num_write_kb":2075200,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"40'32780","reported_seq":32874,"reported_epoch":40,"state":"active+clean+scrubbing","last_fresh":"2026-02-24T07:44:30.691006+0000","last_change":"2026-02-24T07:44:30.111203+0000","last_active":"2026-02-24T07:44:30.691006+0000","last_peered":"2026-02-24T07:44:30.691006+0000","last_clean":"2026-02-24T07:44:30.691006+0000","last_became_active":"2026-02-24T07:43:07.854212+0000","last_became_peered":"2026-02-24T07:43:07.854212+0000","last_unstale":"2026-02-24T07:44:30.691006+0000","last_undegraded":"2026-02-24T07:44:30.691006+0000","last_fullsized":"2026-02-24T07:44:30.691006+0000","mapping_epoch":30,"log_start":"40'32777","ondisk_log_start":"40'32777","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_clean_scrub_stamp":"2026-02-24T07:43:06.832892+0000","objects_scrubbed":5999,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"scrubbing for 1s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2148270080,"num_objects":32780,"num_object_clones":0,"num_object_copies":98340,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32780,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":32780,"num_write_kb":2097920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[1,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"40'32821","reported_seq":32851,"reported_epoch":40,"state":"active+clean","last_fresh":"2026-02-24T07:44:30.683766+0000","last_change":"2026-02-24T07:43:07.843123+0000","last_active":"2026-02-24T07:44:30.683766+0000","last_peered":"2026-02-24T07:44:30.683766+0000","last_clean":"2026-02-24T07:44:30.683766+0000","last_became_active":"2026-02-24T07:43:07.842864+0000","last_became_peered":"2026-02-24T07:43:07.842864+0000","last_unstale":"2026-02-24T07:44:30.683766+0000","last_undegraded":"2026-02-24T07:44:30.683766+0000","last_fullsized":"2026-02-24T07:44:30.683766+0000","mapping_epoch":30,"log_start":"40'32818","ondisk_log_start":"40'32818","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_clean_scrub_stamp":"2026-02-24T07:43:06.832892+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:44:34.876596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2150957056,"num_objects":32821,"num_object_clones":0,"num_object_copies":98463,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32821,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":32821,"num_write_kb":2100544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.4","version":"40'32646","reported_seq":32959,"reported_epoch":40,"state":"active+clean","last_fresh":"2026-02-24T07:44:30.692266+0000","last_change":"2026-02-24T07:44:16.909673+0000","last_active":"2026-02-24T07:44:30.692266+0000","last_peered":"2026-02-24T07:44:30.692266+0000","last_clean":"2026-02-24T07:44:30.692266+0000","last_became_active":"2026-02-24T07:43:07.853966+0000","last_became_peered":"2026-02-24T07:43:07.853966+0000","last_unstale":"2026-02-24T07:44:30.692266+0000","last_undegraded":"2026-02-24T07:44:30.692266+0000","last_fullsized":"2026-02-24T07:44:30.692266+0000","mapping_epoch":30,"log_start":"40'32643","ondisk_log_start":"40'32643","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'27932","last_scrub_stamp":"2026-02-24T07:44:16.909624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_clean_scrub_stamp":"2026-02-24T07:44:16.909624+0000","objects_scrubbed":27480,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:45:43.624405+0000","scrub_duration":2774,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2139488256,"num_objects":32646,"num_object_clones":0,"num_object_copies":97938,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32646,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":32646,"num_write_kb":2089344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"40'32647","reported_seq":32676,"reported_epoch":40,"state":"active+clean","last_fresh":"2026-02-24T07:44:30.676739+0000","last_change":"2026-02-24T07:43:07.854184+0000","last_active":"2026-02-24T07:44:30.676739+0000","last_peered":"2026-02-24T07:44:30.676739+0000","last_clean":"2026-02-24T07:44:30.676739+0000","last_became_active":"2026-02-24T07:43:07.853878+0000","last_became_peered":"2026-02-24T07:43:07.853878+0000","last_unstale":"2026-02-24T07:44:30.676739+0000","last_undegraded":"2026-02-24T07:44:30.676739+0000","last_fullsized":"2026-02-24T07:44:30.676739+0000","mapping_epoch":30,"log_start":"40'32644","ondisk_log_start":"40'32644","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_clean_scrub_stamp":"2026-02-24T07:43:06.832892+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:44:31.507398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2139553792,"num_objects":32647,"num_object_clones":0,"num_object_copies":97941,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32647,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":32647,"num_write_kb":2089408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"40'32687","reported_seq":33047,"reported_epoch":40,"state":"active+clean","last_fresh":"2026-02-24T07:44:30.690957+0000","last_change":"2026-02-24T07:44:30.553008+0000","last_active":"2026-02-24T07:44:30.690957+0000","last_peered":"2026-02-24T07:44:30.690957+0000","last_clean":"2026-02-24T07:44:30.690957+0000","last_became_active":"2026-02-24T07:43:07.841993+0000","last_became_peered":"2026-02-24T07:43:07.841993+0000","last_unstale":"2026-02-24T07:44:30.690957+0000","last_undegraded":"2026-02-24T07:44:30.690957+0000","last_fullsized":"2026-02-24T07:44:30.690957+0000","mapping_epoch":30,"log_start":"40'32684","ondisk_log_start":"40'32684","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'32640","last_scrub_stamp":"2026-02-24T07:44:30.552965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_clean_scrub_stamp":"2026-02-24T07:44:30.552965+0000","objects_scrubbed":32121,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:45:56.537381+0000","scrub_duration":3414,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2142175232,"num_objects":32687,"num_object_clones":0,"num_object_copies":98061,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32687,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":32687,"num_write_kb":2091968,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"40'32391","reported_seq":32708,"reported_epoch":40,"state":"active+clean","last_fresh":"2026-02-24T07:44:30.686593+0000","last_change":"2026-02-24T07:44:18.020443+0000","last_active":"2026-02-24T07:44:30.686593+0000","last_peered":"2026-02-24T07:44:30.686593+0000","last_clean":"2026-02-24T07:44:30.686593+0000","last_became_active":"2026-02-24T07:43:07.841920+0000","last_became_peered":"2026-02-24T07:43:07.841920+0000","last_unstale":"2026-02-24T07:44:30.686593+0000","last_undegraded":"2026-02-24T07:44:30.686593+0000","last_fullsized":"2026-02-24T07:44:30.686593+0000","mapping_epoch":30,"log_start":"40'32388","ondisk_log_start":"40'32388","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'28226","last_scrub_stamp":"2026-02-24T07:44:18.020379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_clean_scrub_stamp":"2026-02-24T07:44:18.020379+0000","objects_scrubbed":27705,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:45:27.051564+0000","scrub_duration":2601,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2122776576,"num_objects":32391,"num_object_clones":0,"num_object_copies":97173,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32391,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":32391,"num_write_kb":2073024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.8","version":"40'32406","reported_seq":32716,"reported_epoch":40,"state":"active+clean+scrubbing","last_fresh":"2026-02-24T07:44:30.688086+0000","last_change":"2026-02-24T07:44:28.343012+0000","last_active":"2026-02-24T07:44:30.688086+0000","last_peered":"2026-02-24T07:44:30.688086+0000","last_clean":"2026-02-24T07:44:30.688086+0000","last_became_active":"2026-02-24T07:43:07.853758+0000","last_became_peered":"2026-02-24T07:43:07.853758+0000","last_unstale":"2026-02-24T07:44:30.688086+0000","last_undegraded":"2026-02-24T07:44:30.688086+0000","last_fullsized":"2026-02-24T07:44:30.688086+0000","mapping_epoch":30,"log_start":"40'32403","ondisk_log_start":"40'32403","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_clean_scrub_stamp":"2026-02-24T07:43:06.832892+0000","objects_scrubbed":27599,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"scrubbing for 3s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2123759616,"num_objects":32406,"num_object_clones":0,"num_object_copies":97218,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32406,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":32406,"num_write_kb":2073984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"40'32808","reported_seq":32838,"reported_epoch":40,"state":"active+clean","last_fresh":"2026-02-24T07:44:30.692177+0000","last_change":"2026-02-24T07:43:07.854687+0000","last_active":"2026-02-24T07:44:30.692177+0000","last_peered":"2026-02-24T07:44:30.692177+0000","last_clean":"2026-02-24T07:44:30.692177+0000","last_became_active":"2026-02-24T07:43:07.854377+0000","last_became_peered":"2026-02-24T07:43:07.854377+0000","last_unstale":"2026-02-24T07:44:30.692177+0000","last_undegraded":"2026-02-24T07:44:30.692177+0000","last_fullsized":"2026-02-24T07:44:30.692177+0000","mapping_epoch":30,"log_start":"40'32805","ondisk_log_start":"40'32805","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_clean_scrub_stamp":"2026-02-24T07:43:06.832892+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:44:34.215480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2150105088,"num_objects":32808,"num_object_clones":0,"num_object_copies":98424,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32808,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":32808,"num_write_kb":2099712,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.a","version":"40'32694","reported_seq":33049,"reported_epoch":40,"state":"active+clean","last_fresh":"2026-02-24T07:44:30.695772+0000","last_change":"2026-02-24T07:44:27.902144+0000","last_active":"2026-02-24T07:44:30.695772+0000","last_peered":"2026-02-24T07:44:30.695772+0000","last_clean":"2026-02-24T07:44:30.695772+0000","last_became_active":"2026-02-24T07:43:07.841721+0000","last_became_peered":"2026-02-24T07:43:07.841721+0000","last_unstale":"2026-02-24T07:44:30.695772+0000","last_undegraded":"2026-02-24T07:44:30.695772+0000","last_fullsized":"2026-02-24T07:44:30.695772+0000","mapping_epoch":30,"log_start":"40'32691","ondisk_log_start":"40'32691","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'31794","last_scrub_stamp":"2026-02-24T07:44:27.902040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_clean_scrub_stamp":"2026-02-24T07:44:27.902040+0000","objects_scrubbed":31292,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:45:39.265782+0000","scrub_duration":3732,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2142633984,"num_objects":32694,"num_object_clones":0,"num_object_copies":98082,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32694,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":32694,"num_write_kb":2092416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.b","version":"40'32835","reported_seq":32865,"reported_epoch":40,"state":"active+clean","last_fresh":"2026-02-24T07:44:30.687826+0000","last_change":"2026-02-24T07:43:07.853283+0000","last_active":"2026-02-24T07:44:30.687826+0000","last_peered":"2026-02-24T07:44:30.687826+0000","last_clean":"2026-02-24T07:44:30.687826+0000","last_became_active":"2026-02-24T07:43:07.853069+0000","last_became_peered":"2026-02-24T07:43:07.853069+0000","last_unstale":"2026-02-24T07:44:30.687826+0000","last_undegraded":"2026-02-24T07:44:30.687826+0000","last_fullsized":"2026-02-24T07:44:30.687826+0000","mapping_epoch":30,"log_start":"40'32832","ondisk_log_start":"40'32832","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_clean_scrub_stamp":"2026-02-24T07:43:06.832892+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:44:32.581403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2151874560,"num_objects":32835,"num_object_clones":0,"num_object_copies":98505,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32835,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":32835,"num_write_kb":2101440,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":34260451328,"num_objects":522773,"num_object_clones":0,"num_object_copies":1568319,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":522773,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":522773,"num_write_kb":33457472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":102769688576,"data_stored":102769688576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":49,"ondisk_log_size":49,"up":48,"acting":48,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":240,"num_write":135,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":3,"up_from":25,"seq":107374182422,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":30956684,"kb_used_data":29189880,"kb_used_omap":14,"kb_used_meta":1766769,"kb_avail":703046516,"statfs":{"total":751619276800,"available":719919632384,"internally_reserved":0,"allocated":29890437120,"data_stored":29890186480,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":14643,"internal_metadata":1809172173},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":19,"seq":81604378648,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":24125772,"kb_used_data":22738168,"kb_used_omap":18,"kb_used_meta":1387565,"kb_avail":709877428,"statfs":{"total":751619276800,"available":726914486272,"internally_reserved":0,"allocated":23283884032,"data_stored":23283633392,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":18551,"internal_metadata":1420867465},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542169,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":26547848,"kb_used_data":24979700,"kb_used_omap":20,"kb_used_meta":1568107,"kb_avail":707455352,"statfs":{"total":751619276800,"available":724434280448,"internally_reserved":0,"allocated":25579212800,"data_stored":25578965712,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":20504,"internal_metadata":1605742568},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,1,2,1],"upper_bound":16},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705692,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":24207436,"kb_used_data":22819832,"kb_used_omap":23,"kb_used_meta":1387560,"kb_avail":709795764,"statfs":{"total":751619276800,"available":726830862336,"internally_reserved":0,"allocated":23367507968,"data_stored":23367257328,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":24410,"internal_metadata":1420861606},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":23542824960,"data_stored":23542824960,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":25725698048,"data_stored":25725698048,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":23525064704,"data_stored":23525064704,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":29976100864,"data_stored":29976100864,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-24T07:44:34.022 INFO:tasks.cephadm.ceph_manager.ceph:increase pool size by 10 2026-02-24T07:44:34.022 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph osd pool set .mgr pg_num 61 2026-02-24T07:44:34.193 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:44:34.316 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:44:34.315+0000 7f94a17a9640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:44:34.316 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:44:34.315+0000 7f94a17a9640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:44:34.456 INFO:teuthology.orchestra.run.trial035.stderr:Error EPERM: crimson_allow_pg_split is false; pg_num increase denied for crimson pool 2026-02-24T07:44:34.493 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-24T07:44:34.936 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 86 16 548447 548431 398.537 402.562 0.00133511 0.00250807 2026-02-24T07:44:34.954 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:44:34 trial035 ceph-mon[39284]: 2.1 scrub ok 2026-02-24T07:44:34.954 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:44:34 trial035 ceph-mon[39284]: pgmap v61: 17 pgs: 2 active+clean+scrubbing, 15 active+clean; 32 GiB data, 101 GiB used, 2.6 TiB / 2.7 TiB avail; 273 MiB/s wr, 4.36k op/s 2026-02-24T07:44:34.954 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:44:34 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/2441488475' entity='client.admin' cmd={"prefix": "osd pool set", "pool": ".mgr", "var": "pg_num", "val": "61"} : dispatch 2026-02-24T07:44:35.705 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2026-02-24T07:44:35.845 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:44:35.937 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 87 15 555189 555174 398.8 421.438 0.00382368 0.00250643 2026-02-24T07:44:35.954 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:44:35 trial035 ceph-mon[39284]: from='client.14473 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2026-02-24T07:44:35.957 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:44:35.956+0000 7f123c4a2640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:44:35.957 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:44:35.956+0000 7f123c4a2640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:44:35.971 INFO:teuthology.orchestra.run.trial035.stdout:osd.0: {} 2026-02-24T07:44:35.971 INFO:teuthology.orchestra.run.trial035.stderr:osd.0: osd_enable_op_tracker = '' 2026-02-24T07:44:35.986 INFO:teuthology.orchestra.run.trial035.stdout:osd.1: {} 2026-02-24T07:44:35.986 INFO:teuthology.orchestra.run.trial035.stderr:osd.1: osd_enable_op_tracker = '' 2026-02-24T07:44:35.998 INFO:teuthology.orchestra.run.trial035.stdout:osd.2: {} 2026-02-24T07:44:35.998 INFO:teuthology.orchestra.run.trial035.stderr:osd.2: osd_enable_op_tracker = '' 2026-02-24T07:44:36.010 INFO:teuthology.orchestra.run.trial035.stdout:osd.3: {} 2026-02-24T07:44:36.010 INFO:teuthology.orchestra.run.trial035.stderr:osd.3: osd_enable_op_tracker = '' 2026-02-24T07:44:36.937 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 88 16 561834 561818 398.986 415.25 0.00221939 0.00250526 2026-02-24T07:44:36.954 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:44:36 trial035 ceph-mon[39284]: pgmap v62: 17 pgs: 2 active+clean+scrubbing, 15 active+clean; 32 GiB data, 104 GiB used, 2.6 TiB / 2.7 TiB avail; 282 MiB/s wr, 4.51k op/s 2026-02-24T07:44:36.954 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:44:36 trial035 ceph-mon[39284]: 2.5 scrub ok 2026-02-24T07:44:36.954 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:44:36 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/2105118230' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2026-02-24T07:44:37.356 INFO:journalctl@ceph.prometheus.a.trial091.stdout:Feb 24 07:44:36 trial091 ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-prometheus-a[66007]: time=2026-02-24T07:44:36.973Z level=ERROR source=scrape.go:1630 msg="Failed to determine correct type of scrape target." component="scrape manager" scrape_pool=ceph target=http://trial035:9283/metrics content_type="text/html;charset=utf-8" fallback_media_type="" err="received unsupported Content-Type \"text/html;charset=utf-8\" and no fallback_scrape_protocol specified for target" 2026-02-24T07:44:37.454 INFO:journalctl@ceph.mgr.x.trial035.stdout:Feb 24 07:44:36 trial035 ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-mgr-x[39621]: ::ffff:10.20.193.91 - - [24/Feb/2026:07:44:36] "GET /metrics HTTP/1.1" 200 4222 "" "Prometheus/3.6.0" 2026-02-24T07:44:37.937 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 89 16 568071 568055 398.883 389.812 0.00359756 0.00250591 2026-02-24T07:44:37.954 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:44:37 trial035 ceph-mon[39284]: 2.b scrub starts 2026-02-24T07:44:38.051 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2026-02-24T07:44:38.188 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:44:38.310 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:44:38.308+0000 7f0bec346640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:44:38.310 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:44:38.308+0000 7f0bec346640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:44:38.324 INFO:teuthology.orchestra.run.trial035.stdout:osd.0: {} 2026-02-24T07:44:38.324 INFO:teuthology.orchestra.run.trial035.stderr:osd.0: osd_enable_op_tracker = '' 2026-02-24T07:44:38.337 INFO:teuthology.orchestra.run.trial035.stdout:osd.1: {} 2026-02-24T07:44:38.337 INFO:teuthology.orchestra.run.trial035.stderr:osd.1: osd_enable_op_tracker = '' 2026-02-24T07:44:38.350 INFO:teuthology.orchestra.run.trial035.stdout:osd.2: {} 2026-02-24T07:44:38.351 INFO:teuthology.orchestra.run.trial035.stderr:osd.2: osd_enable_op_tracker = '' 2026-02-24T07:44:38.362 INFO:teuthology.orchestra.run.trial035.stdout:osd.3: {} 2026-02-24T07:44:38.362 INFO:teuthology.orchestra.run.trial035.stderr:osd.3: osd_enable_op_tracker = '' 2026-02-24T07:44:38.937 INFO:tasks.radosbench.radosbench.0.trial035.stdout:Total time run: 90.0034 2026-02-24T07:44:38.937 INFO:tasks.radosbench.radosbench.0.trial035.stdout:Total writes made: 574067 2026-02-24T07:44:38.937 INFO:tasks.radosbench.radosbench.0.trial035.stdout:Write size: 65536 2026-02-24T07:44:38.937 INFO:tasks.radosbench.radosbench.0.trial035.stdout:Object size: 65536 2026-02-24T07:44:38.937 INFO:tasks.radosbench.radosbench.0.trial035.stdout:Bandwidth (MB/sec): 398.642 2026-02-24T07:44:38.937 INFO:tasks.radosbench.radosbench.0.trial035.stdout:Stddev Bandwidth: 53.9334 2026-02-24T07:44:38.937 INFO:tasks.radosbench.radosbench.0.trial035.stdout:Max bandwidth (MB/sec): 490.438 2026-02-24T07:44:38.938 INFO:tasks.radosbench.radosbench.0.trial035.stdout:Min bandwidth (MB/sec): 204.438 2026-02-24T07:44:38.938 INFO:tasks.radosbench.radosbench.0.trial035.stdout:Average IOPS: 6378 2026-02-24T07:44:38.938 INFO:tasks.radosbench.radosbench.0.trial035.stdout:Stddev IOPS: 862.935 2026-02-24T07:44:38.938 INFO:tasks.radosbench.radosbench.0.trial035.stdout:Max IOPS: 7847 2026-02-24T07:44:38.938 INFO:tasks.radosbench.radosbench.0.trial035.stdout:Min IOPS: 3271 2026-02-24T07:44:38.938 INFO:tasks.radosbench.radosbench.0.trial035.stdout:Average Latency(s): 0.00250741 2026-02-24T07:44:38.938 INFO:tasks.radosbench.radosbench.0.trial035.stdout:Stddev Latency(s): 0.00221053 2026-02-24T07:44:38.938 INFO:tasks.radosbench.radosbench.0.trial035.stdout:Max latency(s): 0.066631 2026-02-24T07:44:38.938 INFO:tasks.radosbench.radosbench.0.trial035.stdout:Min latency(s): 0.000663722 2026-02-24T07:44:38.938 INFO:tasks.radosbench.radosbench.0.trial035.stdout:Cleaning up (deleting benchmark objects) 2026-02-24T07:44:38.954 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:44:38 trial035 ceph-mon[39284]: pgmap v63: 17 pgs: 1 active+clean+scrubbing, 16 active+clean; 34 GiB data, 107 GiB used, 2.6 TiB / 2.7 TiB avail; 422 MiB/s wr, 6.75k op/s 2026-02-24T07:44:38.954 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:44:38 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/2212312955' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2026-02-24T07:44:39.494 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2026-02-24T07:44:39.494 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2026-02-24T07:44:39.494 INFO:tasks.thrashosds.thrasher:fixing pg num pool unique_pool_0 2026-02-24T07:44:39.495 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph pg dump --format=json 2026-02-24T07:44:39.622 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:44:39.740 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:44:39.739+0000 7f80ccd38640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:44:39.740 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:44:39.739+0000 7f80ccd38640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:44:39.869 INFO:teuthology.orchestra.run.trial035.stdout: 2026-02-24T07:44:39.870 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:44:39 trial035 ceph-mon[39284]: 2.9 scrub starts 2026-02-24T07:44:39.871 INFO:teuthology.orchestra.run.trial035.stderr:dumped all 2026-02-24T07:44:39.911 INFO:teuthology.orchestra.run.trial035.stdout:{"pg_ready":true,"pg_map":{"version":64,"stamp":"2026-02-24T07:44:39.671702+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":36257464864,"num_objects":553238,"num_object_clones":0,"num_object_copies":1659714,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":553238,"num_whiteouts":0,"num_read":136,"num_read_kb":240,"num_write":553371,"num_write_kb":35409254,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":53,"ondisk_log_size":53,"up":51,"acting":51,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":51,"num_osds":4,"num_per_pool_osds":4,"num_per_pool_omap_osds":4,"kb":2936012800,"kb_used":112208364,"kb_used_data":105719196,"kb_used_omap":76,"kb_used_meta":6489011,"kb_avail":2823804436,"statfs":{"total":3006477107200,"available":2891575742464,"internally_reserved":0,"allocated":108256456704,"data_stored":108255457696,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":78108,"internal_metadata":6644748004},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,1,1,11],"upper_bound":16},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":3567386624,"num_objects":54434,"num_object_clones":0,"num_object_copies":163302,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54434,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":54434,"num_write_kb":3483776,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001537"},"pg_stats":[{"pgid":"2.f","version":"40'34157","reported_seq":34490,"reported_epoch":40,"state":"active+clean","last_fresh":"2026-02-24T07:44:35.662074+0000","last_change":"2026-02-24T07:44:23.320237+0000","last_active":"2026-02-24T07:44:35.662074+0000","last_peered":"2026-02-24T07:44:35.662074+0000","last_clean":"2026-02-24T07:44:35.662074+0000","last_became_active":"2026-02-24T07:43:07.852899+0000","last_became_peered":"2026-02-24T07:43:07.852899+0000","last_unstale":"2026-02-24T07:44:35.662074+0000","last_undegraded":"2026-02-24T07:44:35.662074+0000","last_fullsized":"2026-02-24T07:44:35.662074+0000","mapping_epoch":30,"log_start":"40'34154","ondisk_log_start":"40'34154","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'30039","last_scrub_stamp":"2026-02-24T07:44:23.320202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_clean_scrub_stamp":"2026-02-24T07:44:23.320202+0000","objects_scrubbed":29558,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:45:24.615266+0000","scrub_duration":2598,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2238513152,"num_objects":34157,"num_object_clones":0,"num_object_copies":102471,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34157,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":34157,"num_write_kb":2186048,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.e","version":"40'34710","reported_seq":35024,"reported_epoch":40,"state":"active+clean","last_fresh":"2026-02-24T07:44:35.692357+0000","last_change":"2026-02-24T07:44:16.083399+0000","last_active":"2026-02-24T07:44:35.692357+0000","last_peered":"2026-02-24T07:44:35.692357+0000","last_clean":"2026-02-24T07:44:35.692357+0000","last_became_active":"2026-02-24T07:43:07.852815+0000","last_became_peered":"2026-02-24T07:43:07.852815+0000","last_unstale":"2026-02-24T07:44:35.692357+0000","last_undegraded":"2026-02-24T07:44:35.692357+0000","last_fullsized":"2026-02-24T07:44:35.692357+0000","mapping_epoch":30,"log_start":"40'34707","ondisk_log_start":"40'34707","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'27767","last_scrub_stamp":"2026-02-24T07:44:16.083320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_clean_scrub_stamp":"2026-02-24T07:44:16.083320+0000","objects_scrubbed":27311,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:45:28.111749+0000","scrub_duration":2943,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2274754560,"num_objects":34710,"num_object_clones":0,"num_object_copies":104130,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34710,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":34710,"num_write_kb":2221440,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.d","version":"40'34799","reported_seq":35150,"reported_epoch":40,"state":"active+clean","last_fresh":"2026-02-24T07:44:35.693653+0000","last_change":"2026-02-24T07:44:26.596366+0000","last_active":"2026-02-24T07:44:35.693653+0000","last_peered":"2026-02-24T07:44:35.693653+0000","last_clean":"2026-02-24T07:44:35.693653+0000","last_became_active":"2026-02-24T07:43:07.853975+0000","last_became_peered":"2026-02-24T07:43:07.853975+0000","last_unstale":"2026-02-24T07:44:35.693653+0000","last_undegraded":"2026-02-24T07:44:35.693653+0000","last_fullsized":"2026-02-24T07:44:35.693653+0000","mapping_epoch":30,"log_start":"40'34795","ondisk_log_start":"40'34795","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'31656","last_scrub_stamp":"2026-02-24T07:44:26.596292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_clean_scrub_stamp":"2026-02-24T07:44:26.596292+0000","objects_scrubbed":31097,"log_size":4,"log_dups_size":3000,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:45:55.416603+0000","scrub_duration":3380,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2280587264,"num_objects":34799,"num_object_clones":0,"num_object_copies":104397,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34799,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":34799,"num_write_kb":2227136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"40'34793","reported_seq":35151,"reported_epoch":40,"state":"active+clean","last_fresh":"2026-02-24T07:44:35.695359+0000","last_change":"2026-02-24T07:44:29.667393+0000","last_active":"2026-02-24T07:44:35.695359+0000","last_peered":"2026-02-24T07:44:35.695359+0000","last_clean":"2026-02-24T07:44:35.695359+0000","last_became_active":"2026-02-24T07:43:07.854428+0000","last_became_peered":"2026-02-24T07:43:07.854428+0000","last_unstale":"2026-02-24T07:44:35.695359+0000","last_undegraded":"2026-02-24T07:44:35.695359+0000","last_fullsized":"2026-02-24T07:44:35.695359+0000","mapping_epoch":30,"log_start":"40'34790","ondisk_log_start":"40'34790","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'32466","last_scrub_stamp":"2026-02-24T07:44:29.667348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_clean_scrub_stamp":"2026-02-24T07:44:29.667348+0000","objects_scrubbed":31970,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:45:43.092916+0000","scrub_duration":3552,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2280194048,"num_objects":34793,"num_object_clones":0,"num_object_copies":104379,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34793,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":34793,"num_write_kb":2226752,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"29'99","reported_seq":209,"reported_epoch":40,"state":"active+clean","last_fresh":"2026-02-24T07:43:59.386353+0000","last_change":"2026-02-24T07:43:59.386353+0000","last_active":"2026-02-24T07:43:59.386353+0000","last_peered":"2026-02-24T07:43:59.386353+0000","last_clean":"2026-02-24T07:43:59.386353+0000","last_became_active":"2026-02-24T07:42:54.186719+0000","last_became_peered":"2026-02-24T07:42:54.186719+0000","last_unstale":"2026-02-24T07:43:59.386353+0000","last_undegraded":"2026-02-24T07:43:59.386353+0000","last_fullsized":"2026-02-24T07:43:59.386353+0000","mapping_epoch":27,"log_start":"29'96","ondisk_log_start":"29'96","created":20,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'99","last_scrub_stamp":"2026-02-24T07:43:59.386324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:42:43.640776+0000","last_clean_scrub_stamp":"2026-02-24T07:43:59.386324+0000","objects_scrubbed":2,"log_size":3,"log_dups_size":96,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:45:24.339819+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":240,"num_write":135,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"40'34736","reported_seq":35072,"reported_epoch":40,"state":"active+clean","last_fresh":"2026-02-24T07:44:35.686519+0000","last_change":"2026-02-24T07:44:22.665897+0000","last_active":"2026-02-24T07:44:35.686519+0000","last_peered":"2026-02-24T07:44:35.686519+0000","last_clean":"2026-02-24T07:44:35.686519+0000","last_became_active":"2026-02-24T07:43:07.842530+0000","last_became_peered":"2026-02-24T07:43:07.842530+0000","last_unstale":"2026-02-24T07:44:35.686519+0000","last_undegraded":"2026-02-24T07:44:35.686519+0000","last_fullsized":"2026-02-24T07:44:35.686519+0000","mapping_epoch":30,"log_start":"40'34733","ondisk_log_start":"40'34733","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'30210","last_scrub_stamp":"2026-02-24T07:44:22.665762+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_clean_scrub_stamp":"2026-02-24T07:44:22.665762+0000","objects_scrubbed":29682,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:45:26.371453+0000","scrub_duration":3226,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2276458496,"num_objects":34736,"num_object_clones":0,"num_object_copies":104208,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34736,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":34736,"num_write_kb":2223104,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"40'34312","reported_seq":34664,"reported_epoch":40,"state":"active+clean","last_fresh":"2026-02-24T07:44:35.690818+0000","last_change":"2026-02-24T07:44:28.446165+0000","last_active":"2026-02-24T07:44:35.690818+0000","last_peered":"2026-02-24T07:44:35.690818+0000","last_clean":"2026-02-24T07:44:35.690818+0000","last_became_active":"2026-02-24T07:43:07.853885+0000","last_became_peered":"2026-02-24T07:43:07.853885+0000","last_unstale":"2026-02-24T07:44:35.690818+0000","last_undegraded":"2026-02-24T07:44:35.690818+0000","last_fullsized":"2026-02-24T07:44:35.690818+0000","mapping_epoch":30,"log_start":"40'34309","ondisk_log_start":"40'34309","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'31685","last_scrub_stamp":"2026-02-24T07:44:28.446064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_clean_scrub_stamp":"2026-02-24T07:44:28.446064+0000","objects_scrubbed":31280,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:45:52.594601+0000","scrub_duration":3117,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2248671232,"num_objects":34312,"num_object_clones":0,"num_object_copies":102936,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34312,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":34312,"num_write_kb":2195968,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"40'34709","reported_seq":35080,"reported_epoch":40,"state":"active+clean","last_fresh":"2026-02-24T07:44:35.690929+0000","last_change":"2026-02-24T07:44:33.357960+0000","last_active":"2026-02-24T07:44:35.690929+0000","last_peered":"2026-02-24T07:44:35.690929+0000","last_clean":"2026-02-24T07:44:35.690929+0000","last_became_active":"2026-02-24T07:43:07.854212+0000","last_became_peered":"2026-02-24T07:43:07.854212+0000","last_unstale":"2026-02-24T07:44:35.690929+0000","last_undegraded":"2026-02-24T07:44:35.690929+0000","last_fullsized":"2026-02-24T07:44:35.690929+0000","mapping_epoch":30,"log_start":"40'34706","ondisk_log_start":"40'34706","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'33783","last_scrub_stamp":"2026-02-24T07:44:33.357894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_clean_scrub_stamp":"2026-02-24T07:44:33.357894+0000","objects_scrubbed":33200,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:45:38.914707+0000","scrub_duration":3247,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2274689024,"num_objects":34709,"num_object_clones":0,"num_object_copies":104127,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34709,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":34709,"num_write_kb":2221376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[1,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"40'34686","reported_seq":34716,"reported_epoch":40,"state":"active+clean","last_fresh":"2026-02-24T07:44:35.689041+0000","last_change":"2026-02-24T07:43:07.843123+0000","last_active":"2026-02-24T07:44:35.689041+0000","last_peered":"2026-02-24T07:44:35.689041+0000","last_clean":"2026-02-24T07:44:35.689041+0000","last_became_active":"2026-02-24T07:43:07.842864+0000","last_became_peered":"2026-02-24T07:43:07.842864+0000","last_unstale":"2026-02-24T07:44:35.689041+0000","last_undegraded":"2026-02-24T07:44:35.689041+0000","last_fullsized":"2026-02-24T07:44:35.689041+0000","mapping_epoch":30,"log_start":"40'34683","ondisk_log_start":"40'34683","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_clean_scrub_stamp":"2026-02-24T07:43:06.832892+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2273181696,"num_objects":34686,"num_object_clones":0,"num_object_copies":104058,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34686,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":34686,"num_write_kb":2219904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.4","version":"40'34518","reported_seq":34833,"reported_epoch":40,"state":"active+clean","last_fresh":"2026-02-24T07:44:35.693880+0000","last_change":"2026-02-24T07:44:16.909673+0000","last_active":"2026-02-24T07:44:35.693880+0000","last_peered":"2026-02-24T07:44:35.693880+0000","last_clean":"2026-02-24T07:44:35.693880+0000","last_became_active":"2026-02-24T07:43:07.853966+0000","last_became_peered":"2026-02-24T07:43:07.853966+0000","last_unstale":"2026-02-24T07:44:35.693880+0000","last_undegraded":"2026-02-24T07:44:35.693880+0000","last_fullsized":"2026-02-24T07:44:35.693880+0000","mapping_epoch":30,"log_start":"40'34515","ondisk_log_start":"40'34515","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'27932","last_scrub_stamp":"2026-02-24T07:44:16.909624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_clean_scrub_stamp":"2026-02-24T07:44:16.909624+0000","objects_scrubbed":27480,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:45:43.624405+0000","scrub_duration":2774,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2262171648,"num_objects":34518,"num_object_clones":0,"num_object_copies":103554,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34518,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":34518,"num_write_kb":2209152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"40'34564","reported_seq":34933,"reported_epoch":40,"state":"active+clean+scrubbing","last_fresh":"2026-02-24T07:44:35.687266+0000","last_change":"2026-02-24T07:44:32.341300+0000","last_active":"2026-02-24T07:44:35.687266+0000","last_peered":"2026-02-24T07:44:35.687266+0000","last_clean":"2026-02-24T07:44:35.687266+0000","last_became_active":"2026-02-24T07:43:07.853878+0000","last_became_peered":"2026-02-24T07:43:07.853878+0000","last_unstale":"2026-02-24T07:44:35.687266+0000","last_undegraded":"2026-02-24T07:44:35.687266+0000","last_fullsized":"2026-02-24T07:44:35.687266+0000","mapping_epoch":30,"log_start":"40'34561","ondisk_log_start":"40'34561","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_clean_scrub_stamp":"2026-02-24T07:43:06.832892+0000","objects_scrubbed":33649,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"scrubbing for 4s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2265186304,"num_objects":34564,"num_object_clones":0,"num_object_copies":103692,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34564,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":34564,"num_write_kb":2212096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"40'34548","reported_seq":34908,"reported_epoch":40,"state":"active+clean","last_fresh":"2026-02-24T07:44:35.695246+0000","last_change":"2026-02-24T07:44:30.553008+0000","last_active":"2026-02-24T07:44:35.695246+0000","last_peered":"2026-02-24T07:44:35.695246+0000","last_clean":"2026-02-24T07:44:35.695246+0000","last_became_active":"2026-02-24T07:43:07.841993+0000","last_became_peered":"2026-02-24T07:43:07.841993+0000","last_unstale":"2026-02-24T07:44:35.695246+0000","last_undegraded":"2026-02-24T07:44:35.695246+0000","last_fullsized":"2026-02-24T07:44:35.695246+0000","mapping_epoch":30,"log_start":"40'34544","ondisk_log_start":"40'34544","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'32640","last_scrub_stamp":"2026-02-24T07:44:30.552965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_clean_scrub_stamp":"2026-02-24T07:44:30.552965+0000","objects_scrubbed":32121,"log_size":4,"log_dups_size":3000,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:45:56.537381+0000","scrub_duration":3414,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2264137728,"num_objects":34548,"num_object_clones":0,"num_object_copies":103644,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34548,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":34548,"num_write_kb":2211072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"40'34265","reported_seq":34581,"reported_epoch":40,"state":"active+clean","last_fresh":"2026-02-24T07:44:35.690864+0000","last_change":"2026-02-24T07:44:18.020443+0000","last_active":"2026-02-24T07:44:35.690864+0000","last_peered":"2026-02-24T07:44:35.690864+0000","last_clean":"2026-02-24T07:44:35.690864+0000","last_became_active":"2026-02-24T07:43:07.841920+0000","last_became_peered":"2026-02-24T07:43:07.841920+0000","last_unstale":"2026-02-24T07:44:35.690864+0000","last_undegraded":"2026-02-24T07:44:35.690864+0000","last_fullsized":"2026-02-24T07:44:35.690864+0000","mapping_epoch":30,"log_start":"40'34262","ondisk_log_start":"40'34262","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'28226","last_scrub_stamp":"2026-02-24T07:44:18.020379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_clean_scrub_stamp":"2026-02-24T07:44:18.020379+0000","objects_scrubbed":27705,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:45:27.051564+0000","scrub_duration":2601,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2245591040,"num_objects":34265,"num_object_clones":0,"num_object_copies":102795,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34265,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":34265,"num_write_kb":2192960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.8","version":"40'34348","reported_seq":34707,"reported_epoch":40,"state":"active+clean","last_fresh":"2026-02-24T07:44:35.687292+0000","last_change":"2026-02-24T07:44:31.118780+0000","last_active":"2026-02-24T07:44:35.687292+0000","last_peered":"2026-02-24T07:44:35.687292+0000","last_clean":"2026-02-24T07:44:35.687292+0000","last_became_active":"2026-02-24T07:43:07.853758+0000","last_became_peered":"2026-02-24T07:43:07.853758+0000","last_unstale":"2026-02-24T07:44:35.687292+0000","last_undegraded":"2026-02-24T07:44:35.687292+0000","last_fullsized":"2026-02-24T07:44:35.687292+0000","mapping_epoch":30,"log_start":"40'34345","ondisk_log_start":"40'34345","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'32571","last_scrub_stamp":"2026-02-24T07:44:31.118720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_clean_scrub_stamp":"2026-02-24T07:44:31.118720+0000","objects_scrubbed":32077,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:45:44.647478+0000","scrub_duration":2775,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2251030528,"num_objects":34348,"num_object_clones":0,"num_object_copies":103044,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34348,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":34348,"num_write_kb":2198272,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"40'34717","reported_seq":34747,"reported_epoch":40,"state":"active+clean","last_fresh":"2026-02-24T07:44:35.686135+0000","last_change":"2026-02-24T07:43:07.854687+0000","last_active":"2026-02-24T07:44:35.686135+0000","last_peered":"2026-02-24T07:44:35.686135+0000","last_clean":"2026-02-24T07:44:35.686135+0000","last_became_active":"2026-02-24T07:43:07.854377+0000","last_became_peered":"2026-02-24T07:43:07.854377+0000","last_unstale":"2026-02-24T07:44:35.686135+0000","last_undegraded":"2026-02-24T07:44:35.686135+0000","last_fullsized":"2026-02-24T07:44:35.686135+0000","mapping_epoch":30,"log_start":"40'34714","ondisk_log_start":"40'34714","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_clean_scrub_stamp":"2026-02-24T07:43:06.832892+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2275213312,"num_objects":34717,"num_object_clones":0,"num_object_copies":104151,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34717,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":34717,"num_write_kb":2221888,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.a","version":"40'34624","reported_seq":34979,"reported_epoch":40,"state":"active+clean","last_fresh":"2026-02-24T07:44:35.691016+0000","last_change":"2026-02-24T07:44:27.902144+0000","last_active":"2026-02-24T07:44:35.691016+0000","last_peered":"2026-02-24T07:44:35.691016+0000","last_clean":"2026-02-24T07:44:35.691016+0000","last_became_active":"2026-02-24T07:43:07.841721+0000","last_became_peered":"2026-02-24T07:43:07.841721+0000","last_unstale":"2026-02-24T07:44:35.691016+0000","last_undegraded":"2026-02-24T07:44:35.691016+0000","last_fullsized":"2026-02-24T07:44:35.691016+0000","mapping_epoch":30,"log_start":"40'34621","ondisk_log_start":"40'34621","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'31794","last_scrub_stamp":"2026-02-24T07:44:27.902040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_clean_scrub_stamp":"2026-02-24T07:44:27.902040+0000","objects_scrubbed":31292,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:45:39.265782+0000","scrub_duration":3732,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2269118464,"num_objects":34624,"num_object_clones":0,"num_object_copies":103872,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34624,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":34624,"num_write_kb":2215936,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.b","version":"40'34750","reported_seq":34780,"reported_epoch":40,"state":"active+clean","last_fresh":"2026-02-24T07:44:35.687197+0000","last_change":"2026-02-24T07:43:07.853283+0000","last_active":"2026-02-24T07:44:35.687197+0000","last_peered":"2026-02-24T07:44:35.687197+0000","last_clean":"2026-02-24T07:44:35.687197+0000","last_became_active":"2026-02-24T07:43:07.853069+0000","last_became_peered":"2026-02-24T07:43:07.853069+0000","last_unstale":"2026-02-24T07:44:35.687197+0000","last_undegraded":"2026-02-24T07:44:35.687197+0000","last_fullsized":"2026-02-24T07:44:35.687197+0000","mapping_epoch":30,"log_start":"40'34747","ondisk_log_start":"40'34747","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_clean_scrub_stamp":"2026-02-24T07:43:06.832892+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2277376000,"num_objects":34750,"num_object_clones":0,"num_object_copies":104250,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34750,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":34750,"num_write_kb":2224000,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":36256874496,"num_objects":553236,"num_object_clones":0,"num_object_copies":1659708,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":553236,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":553236,"num_write_kb":35407104,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":108755812352,"data_stored":108755812352,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":50,"ondisk_log_size":50,"up":48,"acting":48,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":240,"num_write":135,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":3,"up_from":25,"seq":107374182423,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":32771148,"kb_used_data":30878008,"kb_used_omap":14,"kb_used_meta":1893105,"kb_avail":701232052,"statfs":{"total":751619276800,"available":718061621248,"internally_reserved":0,"allocated":31619080192,"data_stored":31618829552,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":14643,"internal_metadata":1938540237},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,1,1,11],"upper_bound":16},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":19,"seq":81604378649,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":25794636,"kb_used_data":24298744,"kb_used_omap":18,"kb_used_meta":1495853,"kb_avail":708208564,"statfs":{"total":751619276800,"available":725205569536,"internally_reserved":0,"allocated":24881913856,"data_stored":24881663216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":18551,"internal_metadata":1531754377},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542170,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":28052680,"kb_used_data":26412340,"kb_used_omap":20,"kb_used_meta":1640299,"kb_avail":705950520,"statfs":{"total":751619276800,"available":722893332480,"internally_reserved":0,"allocated":27046236160,"data_stored":27045989072,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":20504,"internal_metadata":1679667176},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705693,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":25589900,"kb_used_data":24130104,"kb_used_omap":23,"kb_used_meta":1459752,"kb_avail":708413300,"statfs":{"total":751619276800,"available":725415219200,"internally_reserved":0,"allocated":24709226496,"data_stored":24708975856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":24410,"internal_metadata":1494786214},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":24920719360,"data_stored":24920719360,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":27225292800,"data_stored":27225292800,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":24889524224,"data_stored":24889524224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":31720275968,"data_stored":31720275968,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-24T07:44:39.912 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph osd pool set unique_pool_0 pgp_num 36 2026-02-24T07:44:40.044 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:44:40.171 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:44:40.169+0000 7fe44d25f640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:44:40.171 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:44:40.170+0000 7fe44d25f640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:44:40.302 INFO:teuthology.orchestra.run.trial035.stderr:Error EPERM: crimson_allow_pg_split is false; pgp_num increase denied 2026-02-24T07:44:40.339 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-24T07:44:40.403 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2026-02-24T07:44:40.533 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:44:40.653 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:44:40.652+0000 7f84f54c8640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:44:40.653 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:44:40.652+0000 7f84f54c8640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:44:40.669 INFO:teuthology.orchestra.run.trial035.stdout:osd.0: {} 2026-02-24T07:44:40.669 INFO:teuthology.orchestra.run.trial035.stderr:osd.0: osd_enable_op_tracker = '' 2026-02-24T07:44:40.682 INFO:teuthology.orchestra.run.trial035.stdout:osd.1: {} 2026-02-24T07:44:40.682 INFO:teuthology.orchestra.run.trial035.stderr:osd.1: osd_enable_op_tracker = '' 2026-02-24T07:44:40.683 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:44:40 trial035 ceph-mon[39284]: 2.2 scrub starts 2026-02-24T07:44:40.683 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:44:40 trial035 ceph-mon[39284]: 2.b scrub ok 2026-02-24T07:44:40.683 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:44:40 trial035 ceph-mon[39284]: pgmap v64: 17 pgs: 1 active+clean+scrubbing, 16 active+clean; 34 GiB data, 107 GiB used, 2.6 TiB / 2.7 TiB avail; 283 MiB/s wr, 4.54k op/s 2026-02-24T07:44:40.683 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:44:40 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/3290952744' entity='client.admin' cmd={"prefix": "osd pool set", "pool": "unique_pool_0", "var": "pgp_num", "val": "36"} : dispatch 2026-02-24T07:44:40.683 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:44:40 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/904413537' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2026-02-24T07:44:40.697 INFO:teuthology.orchestra.run.trial035.stdout:osd.2: {} 2026-02-24T07:44:40.697 INFO:teuthology.orchestra.run.trial035.stderr:osd.2: osd_enable_op_tracker = '' 2026-02-24T07:44:40.709 INFO:teuthology.orchestra.run.trial035.stdout:osd.3: {} 2026-02-24T07:44:40.710 INFO:teuthology.orchestra.run.trial035.stderr:osd.3: osd_enable_op_tracker = '' 2026-02-24T07:44:41.954 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:44:41 trial035 ceph-mon[39284]: from='client.14481 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2026-02-24T07:44:42.749 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2026-02-24T07:44:42.879 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:44:42.941 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:44:42 trial035 ceph-mon[39284]: 2.9 scrub ok 2026-02-24T07:44:42.941 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:44:42 trial035 ceph-mon[39284]: pgmap v65: 17 pgs: 2 active+clean+scrubbing, 15 active+clean; 34 GiB data, 110 GiB used, 2.6 TiB / 2.7 TiB avail; 85 B/s rd, 392 MiB/s wr, 7.22k op/s 2026-02-24T07:44:43.005 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:44:43.003+0000 7f752c63d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:44:43.005 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:44:43.003+0000 7f752c63d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:44:43.019 INFO:teuthology.orchestra.run.trial035.stdout:osd.0: {} 2026-02-24T07:44:43.020 INFO:teuthology.orchestra.run.trial035.stderr:osd.0: osd_enable_op_tracker = '' 2026-02-24T07:44:43.032 INFO:teuthology.orchestra.run.trial035.stdout:osd.1: {} 2026-02-24T07:44:43.033 INFO:teuthology.orchestra.run.trial035.stderr:osd.1: osd_enable_op_tracker = '' 2026-02-24T07:44:43.047 INFO:teuthology.orchestra.run.trial035.stdout:osd.2: {} 2026-02-24T07:44:43.048 INFO:teuthology.orchestra.run.trial035.stderr:osd.2: osd_enable_op_tracker = '' 2026-02-24T07:44:43.063 INFO:teuthology.orchestra.run.trial035.stdout:osd.3: {} 2026-02-24T07:44:43.063 INFO:teuthology.orchestra.run.trial035.stderr:osd.3: osd_enable_op_tracker = '' 2026-02-24T07:44:43.954 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:44:43 trial035 ceph-mon[39284]: purged_snaps scrub starts 2026-02-24T07:44:43.955 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:44:43 trial035 ceph-mon[39284]: purged_snaps scrub ok 2026-02-24T07:44:43.955 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:44:43 trial035 ceph-mon[39284]: 2.2 scrub ok 2026-02-24T07:44:43.955 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:44:43 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/2556499531' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2026-02-24T07:44:45.101 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2026-02-24T07:44:45.124 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:44:44 trial035 ceph-mon[39284]: pgmap v66: 17 pgs: 2 active+clean+scrubbing, 15 active+clean; 34 GiB data, 110 GiB used, 2.6 TiB / 2.7 TiB avail; 85 B/s rd, 267 MiB/s wr, 5.22k op/s 2026-02-24T07:44:45.230 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:44:45.341 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2026-02-24T07:44:45.342 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2026-02-24T07:44:45.342 INFO:tasks.thrashosds.thrasher:Growing pool .mgr 2026-02-24T07:44:45.342 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph pg dump --format=json 2026-02-24T07:44:45.525 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:44:45.763 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:44:45.694+0000 7f801e9b7640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:44:45.764 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:44:45.695+0000 7f801e9b7640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:44:45.764 INFO:teuthology.orchestra.run.trial035.stderr:osd.0: osd_enable_op_tracker = '' 2026-02-24T07:44:45.764 INFO:teuthology.orchestra.run.trial035.stderr:osd.1: osd_enable_op_tracker = '' 2026-02-24T07:44:45.764 INFO:teuthology.orchestra.run.trial035.stdout:osd.0: {} 2026-02-24T07:44:45.764 INFO:teuthology.orchestra.run.trial035.stdout:osd.1: {} 2026-02-24T07:44:45.764 INFO:teuthology.orchestra.run.trial035.stdout:osd.2: {} 2026-02-24T07:44:45.765 INFO:teuthology.orchestra.run.trial035.stdout:osd.3: {} 2026-02-24T07:44:45.765 INFO:teuthology.orchestra.run.trial035.stderr:osd.2: osd_enable_op_tracker = '' 2026-02-24T07:44:45.765 INFO:teuthology.orchestra.run.trial035.stderr:osd.3: osd_enable_op_tracker = '' 2026-02-24T07:44:45.954 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:44:45 trial035 ceph-mon[39284]: from='mgr.14284 10.20.193.35:0/3681373222' entity='mgr.x' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2026-02-24T07:44:45.954 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:44:45 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/109636063' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2026-02-24T07:44:46.854 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:44:46.840+0000 7f8c7fad3640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:44:46.855 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:44:46.840+0000 7f8c7fad3640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:44:46.954 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:44:46 trial035 ceph-mon[39284]: pgmap v67: 17 pgs: 2 active+clean+scrubbing, 15 active+clean; 34 GiB data, 107 GiB used, 2.6 TiB / 2.7 TiB avail; 85 B/s rd, 267 MiB/s wr, 5.40k op/s 2026-02-24T07:44:46.970 INFO:teuthology.orchestra.run.trial035.stdout: 2026-02-24T07:44:46.970 INFO:teuthology.orchestra.run.trial035.stderr:dumped all 2026-02-24T07:44:47.264 INFO:journalctl@ceph.prometheus.a.trial091.stdout:Feb 24 07:44:46 trial091 ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-prometheus-a[66007]: time=2026-02-24T07:44:46.973Z level=ERROR source=scrape.go:1630 msg="Failed to determine correct type of scrape target." component="scrape manager" scrape_pool=ceph target=http://trial035:9283/metrics content_type="text/html;charset=utf-8" fallback_media_type="" err="received unsupported Content-Type \"text/html;charset=utf-8\" and no fallback_scrape_protocol specified for target" 2026-02-24T07:44:47.304 INFO:journalctl@ceph.mgr.x.trial035.stdout:Feb 24 07:44:46 trial035 ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-mgr-x[39621]: ::ffff:10.20.193.91 - - [24/Feb/2026:07:44:46] "GET /metrics HTTP/1.1" 200 4223 "" "Prometheus/3.6.0" 2026-02-24T07:44:47.585 INFO:teuthology.orchestra.run.trial035.stdout:{"pg_ready":true,"pg_map":{"version":67,"stamp":"2026-02-24T07:44:45.672414+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":36738105912,"num_objects":560573,"num_object_clones":0,"num_object_copies":1681719,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":560573,"num_whiteouts":0,"num_read":137,"num_read_kb":241,"num_write":587700,"num_write_kb":36742439,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":53,"ondisk_log_size":53,"up":51,"acting":51,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":51,"num_osds":4,"num_per_pool_osds":4,"num_per_pool_omap_osds":4,"kb":2936012800,"kb_used":112367672,"kb_used_data":105409000,"kb_used_omap":76,"kb_used_meta":6958387,"kb_avail":2823645128,"statfs":{"total":3006477107200,"available":2891412611072,"internally_reserved":0,"allocated":107938816000,"data_stored":107937804776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":78759,"internal_metadata":7125388377},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2477064216,"num_objects":37798,"num_object_clones":0,"num_object_copies":113394,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37798,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":64792,"num_write_kb":3282817,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001462"},"pg_stats":[{"pgid":"2.f","version":"40'38255","reported_seq":38589,"reported_epoch":40,"state":"active+clean","last_fresh":"2026-02-24T07:44:45.661621+0000","last_change":"2026-02-24T07:44:23.320237+0000","last_active":"2026-02-24T07:44:45.661621+0000","last_peered":"2026-02-24T07:44:45.661621+0000","last_clean":"2026-02-24T07:44:45.661621+0000","last_became_active":"2026-02-24T07:43:07.852899+0000","last_became_peered":"2026-02-24T07:43:07.852899+0000","last_unstale":"2026-02-24T07:44:45.661621+0000","last_undegraded":"2026-02-24T07:44:45.661621+0000","last_fullsized":"2026-02-24T07:44:45.661621+0000","mapping_epoch":30,"log_start":"40'38252","ondisk_log_start":"40'38252","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'30039","last_scrub_stamp":"2026-02-24T07:44:23.320202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_clean_scrub_stamp":"2026-02-24T07:44:23.320202+0000","objects_scrubbed":29558,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:45:24.615266+0000","scrub_duration":2598,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2133393408,"num_objects":32553,"num_object_clones":0,"num_object_copies":97659,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32553,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":38255,"num_write_kb":2265856,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.e","version":"40'36643","reported_seq":36957,"reported_epoch":40,"state":"active+clean","last_fresh":"2026-02-24T07:44:40.690691+0000","last_change":"2026-02-24T07:44:16.083399+0000","last_active":"2026-02-24T07:44:40.690691+0000","last_peered":"2026-02-24T07:44:40.690691+0000","last_clean":"2026-02-24T07:44:40.690691+0000","last_became_active":"2026-02-24T07:43:07.852815+0000","last_became_peered":"2026-02-24T07:43:07.852815+0000","last_unstale":"2026-02-24T07:44:40.690691+0000","last_undegraded":"2026-02-24T07:44:40.690691+0000","last_fullsized":"2026-02-24T07:44:40.690691+0000","mapping_epoch":30,"log_start":"40'36640","ondisk_log_start":"40'36640","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'27767","last_scrub_stamp":"2026-02-24T07:44:16.083320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_clean_scrub_stamp":"2026-02-24T07:44:16.083320+0000","objects_scrubbed":27311,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:45:28.111749+0000","scrub_duration":2943,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2313879552,"num_objects":35307,"num_object_clones":0,"num_object_copies":105921,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35307,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":36643,"num_write_kb":2302400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.d","version":"40'36822","reported_seq":37174,"reported_epoch":40,"state":"active+clean","last_fresh":"2026-02-24T07:44:40.693879+0000","last_change":"2026-02-24T07:44:26.596366+0000","last_active":"2026-02-24T07:44:40.693879+0000","last_peered":"2026-02-24T07:44:40.693879+0000","last_clean":"2026-02-24T07:44:40.693879+0000","last_became_active":"2026-02-24T07:43:07.853975+0000","last_became_peered":"2026-02-24T07:43:07.853975+0000","last_unstale":"2026-02-24T07:44:40.693879+0000","last_undegraded":"2026-02-24T07:44:40.693879+0000","last_fullsized":"2026-02-24T07:44:40.693879+0000","mapping_epoch":30,"log_start":"40'36819","ondisk_log_start":"40'36819","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'31656","last_scrub_stamp":"2026-02-24T07:44:26.596292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_clean_scrub_stamp":"2026-02-24T07:44:26.596292+0000","objects_scrubbed":31097,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:45:55.416603+0000","scrub_duration":3380,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2317287448,"num_objects":35360,"num_object_clones":0,"num_object_copies":106080,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35360,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":36822,"num_write_kb":2309761,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"40'36820","reported_seq":37176,"reported_epoch":40,"state":"active+clean","last_fresh":"2026-02-24T07:44:40.695331+0000","last_change":"2026-02-24T07:44:29.667393+0000","last_active":"2026-02-24T07:44:40.695331+0000","last_peered":"2026-02-24T07:44:40.695331+0000","last_clean":"2026-02-24T07:44:40.695331+0000","last_became_active":"2026-02-24T07:43:07.854428+0000","last_became_peered":"2026-02-24T07:43:07.854428+0000","last_unstale":"2026-02-24T07:44:40.695331+0000","last_undegraded":"2026-02-24T07:44:40.695331+0000","last_fullsized":"2026-02-24T07:44:40.695331+0000","mapping_epoch":30,"log_start":"40'36817","ondisk_log_start":"40'36817","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'32466","last_scrub_stamp":"2026-02-24T07:44:29.667348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_clean_scrub_stamp":"2026-02-24T07:44:29.667348+0000","objects_scrubbed":31970,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:45:43.092916+0000","scrub_duration":3552,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2320629760,"num_objects":35410,"num_object_clones":0,"num_object_copies":106230,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35410,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":36820,"num_write_kb":2311360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"29'99","reported_seq":209,"reported_epoch":40,"state":"active+clean","last_fresh":"2026-02-24T07:43:59.386353+0000","last_change":"2026-02-24T07:43:59.386353+0000","last_active":"2026-02-24T07:43:59.386353+0000","last_peered":"2026-02-24T07:43:59.386353+0000","last_clean":"2026-02-24T07:43:59.386353+0000","last_became_active":"2026-02-24T07:42:54.186719+0000","last_became_peered":"2026-02-24T07:42:54.186719+0000","last_unstale":"2026-02-24T07:43:59.386353+0000","last_undegraded":"2026-02-24T07:43:59.386353+0000","last_fullsized":"2026-02-24T07:43:59.386353+0000","mapping_epoch":27,"log_start":"29'96","ondisk_log_start":"29'96","created":20,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'99","last_scrub_stamp":"2026-02-24T07:43:59.386324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:42:43.640776+0000","last_clean_scrub_stamp":"2026-02-24T07:43:59.386324+0000","objects_scrubbed":2,"log_size":3,"log_dups_size":96,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:45:24.339819+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":240,"num_write":135,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"40'36765","reported_seq":37101,"reported_epoch":40,"state":"active+clean","last_fresh":"2026-02-24T07:44:40.688093+0000","last_change":"2026-02-24T07:44:22.665897+0000","last_active":"2026-02-24T07:44:40.688093+0000","last_peered":"2026-02-24T07:44:40.688093+0000","last_clean":"2026-02-24T07:44:40.688093+0000","last_became_active":"2026-02-24T07:43:07.842530+0000","last_became_peered":"2026-02-24T07:43:07.842530+0000","last_unstale":"2026-02-24T07:44:40.688093+0000","last_undegraded":"2026-02-24T07:44:40.688093+0000","last_fullsized":"2026-02-24T07:44:40.688093+0000","mapping_epoch":30,"log_start":"40'36762","ondisk_log_start":"40'36762","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'30210","last_scrub_stamp":"2026-02-24T07:44:22.665762+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_clean_scrub_stamp":"2026-02-24T07:44:22.665762+0000","objects_scrubbed":29682,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:45:26.371453+0000","scrub_duration":3226,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2316238848,"num_objects":35343,"num_object_clones":0,"num_object_copies":106029,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35343,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":36765,"num_write_kb":2307456,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"40'36312","reported_seq":36664,"reported_epoch":40,"state":"active+clean","last_fresh":"2026-02-24T07:44:40.689867+0000","last_change":"2026-02-24T07:44:28.446165+0000","last_active":"2026-02-24T07:44:40.689867+0000","last_peered":"2026-02-24T07:44:40.689867+0000","last_clean":"2026-02-24T07:44:40.689867+0000","last_became_active":"2026-02-24T07:43:07.853885+0000","last_became_peered":"2026-02-24T07:43:07.853885+0000","last_unstale":"2026-02-24T07:44:40.689867+0000","last_undegraded":"2026-02-24T07:44:40.689867+0000","last_fullsized":"2026-02-24T07:44:40.689867+0000","mapping_epoch":30,"log_start":"40'36309","ondisk_log_start":"40'36309","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'31685","last_scrub_stamp":"2026-02-24T07:44:28.446064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_clean_scrub_stamp":"2026-02-24T07:44:28.446064+0000","objects_scrubbed":31280,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:45:52.594601+0000","scrub_duration":3117,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2287992832,"num_objects":34912,"num_object_clones":0,"num_object_copies":104736,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34912,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":36312,"num_write_kb":2279168,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"40'36816","reported_seq":37187,"reported_epoch":40,"state":"active+clean","last_fresh":"2026-02-24T07:44:40.694134+0000","last_change":"2026-02-24T07:44:33.357960+0000","last_active":"2026-02-24T07:44:40.694134+0000","last_peered":"2026-02-24T07:44:40.694134+0000","last_clean":"2026-02-24T07:44:40.694134+0000","last_became_active":"2026-02-24T07:43:07.854212+0000","last_became_peered":"2026-02-24T07:43:07.854212+0000","last_unstale":"2026-02-24T07:44:40.694134+0000","last_undegraded":"2026-02-24T07:44:40.694134+0000","last_fullsized":"2026-02-24T07:44:40.694134+0000","mapping_epoch":30,"log_start":"40'36811","ondisk_log_start":"40'36811","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'33783","last_scrub_stamp":"2026-02-24T07:44:33.357894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_clean_scrub_stamp":"2026-02-24T07:44:33.357894+0000","objects_scrubbed":33200,"log_size":5,"log_dups_size":3000,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:45:38.914707+0000","scrub_duration":3247,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2314993664,"num_objects":35324,"num_object_clones":0,"num_object_copies":105972,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35324,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":36816,"num_write_kb":2308480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[1,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"40'36728","reported_seq":36893,"reported_epoch":40,"state":"active+clean+scrubbing","last_fresh":"2026-02-24T07:44:40.693998+0000","last_change":"2026-02-24T07:44:39.380731+0000","last_active":"2026-02-24T07:44:40.693998+0000","last_peered":"2026-02-24T07:44:40.693998+0000","last_clean":"2026-02-24T07:44:40.693998+0000","last_became_active":"2026-02-24T07:43:07.842864+0000","last_became_peered":"2026-02-24T07:43:07.842864+0000","last_unstale":"2026-02-24T07:44:40.693998+0000","last_undegraded":"2026-02-24T07:44:40.693998+0000","last_fullsized":"2026-02-24T07:44:40.693998+0000","mapping_epoch":30,"log_start":"40'36725","ondisk_log_start":"40'36725","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_clean_scrub_stamp":"2026-02-24T07:43:06.832892+0000","objects_scrubbed":12999,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"scrubbing for 2s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2315386880,"num_objects":35330,"num_object_clones":0,"num_object_copies":105990,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35330,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":36728,"num_write_kb":2305856,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.4","version":"40'36503","reported_seq":36818,"reported_epoch":40,"state":"active+clean","last_fresh":"2026-02-24T07:44:40.697273+0000","last_change":"2026-02-24T07:44:16.909673+0000","last_active":"2026-02-24T07:44:40.697273+0000","last_peered":"2026-02-24T07:44:40.697273+0000","last_clean":"2026-02-24T07:44:40.697273+0000","last_became_active":"2026-02-24T07:43:07.853966+0000","last_became_peered":"2026-02-24T07:43:07.853966+0000","last_unstale":"2026-02-24T07:44:40.697273+0000","last_undegraded":"2026-02-24T07:44:40.697273+0000","last_fullsized":"2026-02-24T07:44:40.697273+0000","mapping_epoch":30,"log_start":"40'36500","ondisk_log_start":"40'36500","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'27932","last_scrub_stamp":"2026-02-24T07:44:16.909624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_clean_scrub_stamp":"2026-02-24T07:44:16.909624+0000","objects_scrubbed":27480,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:45:43.624405+0000","scrub_duration":2774,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2299330560,"num_objects":35085,"num_object_clones":0,"num_object_copies":105255,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35085,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":36503,"num_write_kb":2290816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"40'36505","reported_seq":36884,"reported_epoch":40,"state":"active+clean","last_fresh":"2026-02-24T07:44:40.686666+0000","last_change":"2026-02-24T07:44:35.717081+0000","last_active":"2026-02-24T07:44:40.686666+0000","last_peered":"2026-02-24T07:44:40.686666+0000","last_clean":"2026-02-24T07:44:40.686666+0000","last_became_active":"2026-02-24T07:43:07.853878+0000","last_became_peered":"2026-02-24T07:43:07.853878+0000","last_unstale":"2026-02-24T07:44:40.686666+0000","last_undegraded":"2026-02-24T07:44:40.686666+0000","last_fullsized":"2026-02-24T07:44:40.686666+0000","mapping_epoch":30,"log_start":"40'36502","ondisk_log_start":"40'36502","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'34578","last_scrub_stamp":"2026-02-24T07:44:35.717022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_clean_scrub_stamp":"2026-02-24T07:44:35.717022+0000","objects_scrubbed":33884,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:46:00.809440+0000","scrub_duration":3376,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2302345216,"num_objects":35131,"num_object_clones":0,"num_object_copies":105393,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35131,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":36505,"num_write_kb":2292352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"40'36664","reported_seq":37023,"reported_epoch":40,"state":"active+clean","last_fresh":"2026-02-24T07:44:40.694894+0000","last_change":"2026-02-24T07:44:30.553008+0000","last_active":"2026-02-24T07:44:40.694894+0000","last_peered":"2026-02-24T07:44:40.694894+0000","last_clean":"2026-02-24T07:44:40.694894+0000","last_became_active":"2026-02-24T07:43:07.841993+0000","last_became_peered":"2026-02-24T07:43:07.841993+0000","last_unstale":"2026-02-24T07:44:40.694894+0000","last_undegraded":"2026-02-24T07:44:40.694894+0000","last_fullsized":"2026-02-24T07:44:40.694894+0000","mapping_epoch":30,"log_start":"40'36661","ondisk_log_start":"40'36661","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'32640","last_scrub_stamp":"2026-02-24T07:44:30.552965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_clean_scrub_stamp":"2026-02-24T07:44:30.552965+0000","objects_scrubbed":32121,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:45:56.537381+0000","scrub_duration":3414,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2303328256,"num_objects":35146,"num_object_clones":0,"num_object_copies":105438,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35146,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":36664,"num_write_kb":2297920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"40'36262","reported_seq":36579,"reported_epoch":40,"state":"active+clean","last_fresh":"2026-02-24T07:44:40.683980+0000","last_change":"2026-02-24T07:44:18.020443+0000","last_active":"2026-02-24T07:44:40.683980+0000","last_peered":"2026-02-24T07:44:40.683980+0000","last_clean":"2026-02-24T07:44:40.683980+0000","last_became_active":"2026-02-24T07:43:07.841920+0000","last_became_peered":"2026-02-24T07:43:07.841920+0000","last_unstale":"2026-02-24T07:44:40.683980+0000","last_undegraded":"2026-02-24T07:44:40.683980+0000","last_fullsized":"2026-02-24T07:44:40.683980+0000","mapping_epoch":30,"log_start":"40'36259","ondisk_log_start":"40'36259","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'28226","last_scrub_stamp":"2026-02-24T07:44:18.020379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_clean_scrub_stamp":"2026-02-24T07:44:18.020379+0000","objects_scrubbed":27705,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:45:27.051564+0000","scrub_duration":2601,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2284716032,"num_objects":34862,"num_object_clones":0,"num_object_copies":104586,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34862,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":36262,"num_write_kb":2275968,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.8","version":"40'36354","reported_seq":36713,"reported_epoch":40,"state":"active+clean","last_fresh":"2026-02-24T07:44:40.686602+0000","last_change":"2026-02-24T07:44:31.118780+0000","last_active":"2026-02-24T07:44:40.686602+0000","last_peered":"2026-02-24T07:44:40.686602+0000","last_clean":"2026-02-24T07:44:40.686602+0000","last_became_active":"2026-02-24T07:43:07.853758+0000","last_became_peered":"2026-02-24T07:43:07.853758+0000","last_unstale":"2026-02-24T07:44:40.686602+0000","last_undegraded":"2026-02-24T07:44:40.686602+0000","last_fullsized":"2026-02-24T07:44:40.686602+0000","mapping_epoch":30,"log_start":"40'36351","ondisk_log_start":"40'36351","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'32571","last_scrub_stamp":"2026-02-24T07:44:31.118720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_clean_scrub_stamp":"2026-02-24T07:44:31.118720+0000","objects_scrubbed":32077,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:45:44.647478+0000","scrub_duration":2775,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2293366784,"num_objects":34994,"num_object_clones":0,"num_object_copies":104982,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34994,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":36354,"num_write_kb":2283136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"40'36700","reported_seq":37013,"reported_epoch":40,"state":"active+clean+scrubbing","last_fresh":"2026-02-24T07:44:40.695913+0000","last_change":"2026-02-24T07:44:38.143814+0000","last_active":"2026-02-24T07:44:40.695913+0000","last_peered":"2026-02-24T07:44:40.695913+0000","last_clean":"2026-02-24T07:44:40.695913+0000","last_became_active":"2026-02-24T07:43:07.854377+0000","last_became_peered":"2026-02-24T07:43:07.854377+0000","last_unstale":"2026-02-24T07:44:40.695913+0000","last_undegraded":"2026-02-24T07:44:40.695913+0000","last_fullsized":"2026-02-24T07:44:40.695913+0000","mapping_epoch":30,"log_start":"40'36697","ondisk_log_start":"40'36697","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_clean_scrub_stamp":"2026-02-24T07:43:06.832892+0000","objects_scrubbed":27849,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"scrubbing for 3s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2313945088,"num_objects":35308,"num_object_clones":0,"num_object_copies":105924,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35308,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":36700,"num_write_kb":2304256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.a","version":"40'36658","reported_seq":37013,"reported_epoch":40,"state":"active+clean","last_fresh":"2026-02-24T07:44:40.691339+0000","last_change":"2026-02-24T07:44:27.902144+0000","last_active":"2026-02-24T07:44:40.691339+0000","last_peered":"2026-02-24T07:44:40.691339+0000","last_clean":"2026-02-24T07:44:40.691339+0000","last_became_active":"2026-02-24T07:43:07.841721+0000","last_became_peered":"2026-02-24T07:43:07.841721+0000","last_unstale":"2026-02-24T07:44:40.691339+0000","last_undegraded":"2026-02-24T07:44:40.691339+0000","last_fullsized":"2026-02-24T07:44:40.691339+0000","mapping_epoch":30,"log_start":"40'36655","ondisk_log_start":"40'36655","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'31794","last_scrub_stamp":"2026-02-24T07:44:27.902040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_clean_scrub_stamp":"2026-02-24T07:44:27.902040+0000","objects_scrubbed":31292,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:45:39.265782+0000","scrub_duration":3732,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2308833280,"num_objects":35230,"num_object_clones":0,"num_object_copies":105690,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35230,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":36658,"num_write_kb":2300416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.b","version":"40'36758","reported_seq":37153,"reported_epoch":40,"state":"active+clean","last_fresh":"2026-02-24T07:44:40.692783+0000","last_change":"2026-02-24T07:44:39.639056+0000","last_active":"2026-02-24T07:44:40.692783+0000","last_peered":"2026-02-24T07:44:40.692783+0000","last_clean":"2026-02-24T07:44:40.692783+0000","last_became_active":"2026-02-24T07:43:07.853069+0000","last_became_peered":"2026-02-24T07:43:07.853069+0000","last_unstale":"2026-02-24T07:44:40.692783+0000","last_undegraded":"2026-02-24T07:44:40.692783+0000","last_fullsized":"2026-02-24T07:44:40.692783+0000","mapping_epoch":30,"log_start":"40'36755","ondisk_log_start":"40'36755","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'36298","last_scrub_stamp":"2026-02-24T07:44:39.638986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_clean_scrub_stamp":"2026-02-24T07:44:39.638986+0000","objects_scrubbed":35656,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:46:09.128452+0000","scrub_duration":3228,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2311847936,"num_objects":35276,"num_object_clones":0,"num_object_copies":105828,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35276,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":36758,"num_write_kb":2305088,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":36737515544,"num_objects":560571,"num_object_clones":0,"num_object_copies":1681713,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":560571,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":587565,"num_write_kb":36740289,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":107427016704,"data_stored":107427004488,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":50,"ondisk_log_size":50,"up":48,"acting":48,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":240,"num_write":135,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":3,"up_from":25,"seq":107374182424,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":33623376,"kb_used_data":31639996,"kb_used_omap":14,"kb_used_meta":1983345,"kb_avail":700379824,"statfs":{"total":751619276800,"available":717188939776,"internally_reserved":0,"allocated":32399355904,"data_stored":32399101192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":14643,"internal_metadata":2030945997},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":19,"seq":81604378651,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":24806732,"kb_used_data":23184376,"kb_used_omap":18,"kb_used_meta":1622317,"kb_avail":709196468,"statfs":{"total":751619276800,"available":726217183232,"internally_reserved":0,"allocated":23740801024,"data_stored":23740550384,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":18551,"internal_metadata":1661253513},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542171,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":28914380,"kb_used_data":27201784,"kb_used_omap":20,"kb_used_meta":1712491,"kb_avail":705088820,"statfs":{"total":751619276800,"available":722010951680,"internally_reserved":0,"allocated":27854626816,"data_stored":27854375656,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":20504,"internal_metadata":1753591784},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705695,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":25023184,"kb_used_data":23382844,"kb_used_omap":24,"kb_used_meta":1640231,"kb_avail":708980016,"statfs":{"total":751619276800,"available":725995536384,"internally_reserved":0,"allocated":23944032256,"data_stored":23943777544,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":25061,"internal_metadata":1679597083},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":23751626752,"data_stored":23751622680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":27696173056,"data_stored":27696168984,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":23714529280,"data_stored":23714529280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":32264687616,"data_stored":32264683544,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-24T07:44:47.587 INFO:tasks.cephadm.ceph_manager.ceph:increase pool size by 10 2026-02-24T07:44:47.588 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph osd pool set .mgr pg_num 71 2026-02-24T07:44:47.729 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:44:48.154 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:44:48.134+0000 7f669dbe1640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:44:48.154 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:44:48.134+0000 7f669dbe1640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:44:48.264 INFO:teuthology.orchestra.run.trial035.stderr:Error EPERM: crimson_allow_pg_split is false; pg_num increase denied for crimson pool 2026-02-24T07:44:48.369 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2026-02-24T07:44:48.412 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-24T07:44:48.501 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:44:48.619 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:44:48.617+0000 7f286effb640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:44:48.619 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:44:48.618+0000 7f286effb640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:44:48.633 INFO:teuthology.orchestra.run.trial035.stdout:osd.0: {} 2026-02-24T07:44:48.633 INFO:teuthology.orchestra.run.trial035.stderr:osd.0: osd_enable_op_tracker = '' 2026-02-24T07:44:48.646 INFO:teuthology.orchestra.run.trial035.stdout:osd.1: {} 2026-02-24T07:44:48.646 INFO:teuthology.orchestra.run.trial035.stderr:osd.1: osd_enable_op_tracker = '' 2026-02-24T07:44:48.658 INFO:teuthology.orchestra.run.trial035.stdout:osd.2: {} 2026-02-24T07:44:48.658 INFO:teuthology.orchestra.run.trial035.stderr:osd.2: osd_enable_op_tracker = '' 2026-02-24T07:44:48.670 INFO:teuthology.orchestra.run.trial035.stdout:osd.3: {} 2026-02-24T07:44:48.670 INFO:teuthology.orchestra.run.trial035.stderr:osd.3: osd_enable_op_tracker = '' 2026-02-24T07:44:48.954 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:44:48 trial035 ceph-mon[39284]: from='client.14491 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2026-02-24T07:44:48.954 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:44:48 trial035 ceph-mon[39284]: pgmap v68: 17 pgs: 17 active+clean; 32 GiB data, 104 GiB used, 2.6 TiB / 2.7 TiB avail; 85 B/s rd, 258 MiB/s wr, 8.05k op/s 2026-02-24T07:44:48.954 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:44:48 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/1945645303' entity='client.admin' cmd={"prefix": "osd pool set", "pool": ".mgr", "var": "pg_num", "val": "71"} : dispatch 2026-02-24T07:44:48.954 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:44:48 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/3240858326' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2026-02-24T07:44:50.713 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2026-02-24T07:44:50.853 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:44:50.961 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:44:50.959+0000 7f9389457640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:44:50.961 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:44:50.959+0000 7f9389457640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:44:50.975 INFO:teuthology.orchestra.run.trial035.stdout:osd.0: {} 2026-02-24T07:44:50.975 INFO:teuthology.orchestra.run.trial035.stderr:osd.0: osd_enable_op_tracker = '' 2026-02-24T07:44:50.987 INFO:teuthology.orchestra.run.trial035.stdout:osd.1: {} 2026-02-24T07:44:50.987 INFO:teuthology.orchestra.run.trial035.stderr:osd.1: osd_enable_op_tracker = '' 2026-02-24T07:44:50.988 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:44:50 trial035 ceph-mon[39284]: pgmap v69: 17 pgs: 17 active+clean; 32 GiB data, 104 GiB used, 2.6 TiB / 2.7 TiB avail; 85 B/s rd, 108 MiB/s wr, 5.66k op/s 2026-02-24T07:44:50.998 INFO:teuthology.orchestra.run.trial035.stdout:osd.2: {} 2026-02-24T07:44:50.999 INFO:teuthology.orchestra.run.trial035.stderr:osd.2: osd_enable_op_tracker = '' 2026-02-24T07:44:51.009 INFO:teuthology.orchestra.run.trial035.stdout:osd.3: {} 2026-02-24T07:44:51.010 INFO:teuthology.orchestra.run.trial035.stderr:osd.3: osd_enable_op_tracker = '' 2026-02-24T07:44:52.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:44:51 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/2135587586' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2026-02-24T07:44:53.050 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2026-02-24T07:44:53.076 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:44:52 trial035 ceph-mon[39284]: pgmap v70: 17 pgs: 17 active+clean; 30 GiB data, 97 GiB used, 2.6 TiB / 2.7 TiB avail; 85 B/s rd, 108 MiB/s wr, 8.49k op/s 2026-02-24T07:44:53.200 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:44:53.319 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:44:53.317+0000 7fdab8b72640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:44:53.319 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:44:53.317+0000 7fdab8b72640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:44:53.334 INFO:teuthology.orchestra.run.trial035.stdout:osd.0: {} 2026-02-24T07:44:53.334 INFO:teuthology.orchestra.run.trial035.stderr:osd.0: osd_enable_op_tracker = '' 2026-02-24T07:44:53.346 INFO:teuthology.orchestra.run.trial035.stdout:osd.1: {} 2026-02-24T07:44:53.346 INFO:teuthology.orchestra.run.trial035.stderr:osd.1: osd_enable_op_tracker = '' 2026-02-24T07:44:53.357 INFO:teuthology.orchestra.run.trial035.stdout:osd.2: {} 2026-02-24T07:44:53.358 INFO:teuthology.orchestra.run.trial035.stderr:osd.2: osd_enable_op_tracker = '' 2026-02-24T07:44:53.369 INFO:teuthology.orchestra.run.trial035.stdout:osd.3: {} 2026-02-24T07:44:53.369 INFO:teuthology.orchestra.run.trial035.stderr:osd.3: osd_enable_op_tracker = '' 2026-02-24T07:44:53.413 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2026-02-24T07:44:53.413 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2026-02-24T07:44:53.413 INFO:tasks.thrashosds.thrasher:Growing pool .mgr 2026-02-24T07:44:53.414 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph pg dump --format=json 2026-02-24T07:44:53.580 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:44:53.687 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:44:53.684+0000 7f68244e2640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:44:53.687 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:44:53.685+0000 7f68244e2640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:44:53.818 INFO:teuthology.orchestra.run.trial035.stdout: 2026-02-24T07:44:53.818 INFO:teuthology.orchestra.run.trial035.stderr:dumped all 2026-02-24T07:44:53.954 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:44:53 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/2126864922' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2026-02-24T07:44:54.517 INFO:teuthology.orchestra.run.trial035.stdout:{"pg_ready":true,"pg_map":{"version":71,"stamp":"2026-02-24T07:44:53.673387+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":32309183032,"num_objects":492993,"num_object_clones":0,"num_object_copies":1478979,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":492993,"num_whiteouts":0,"num_read":137,"num_read_kb":241,"num_write":655280,"num_write_kb":36742439,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":52,"ondisk_log_size":52,"up":51,"acting":51,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":51,"num_osds":4,"num_per_pool_osds":4,"num_per_pool_omap_osds":4,"kb":2936012800,"kb_used":101296952,"kb_used_data":95245544,"kb_used_omap":66,"kb_used_meta":6051133,"kb_avail":2834715848,"statfs":{"total":3006477107200,"available":2902749028352,"internally_reserved":0,"allocated":97531437056,"data_stored":97530425832,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":68564,"internal_metadata":6196360236},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,5],"upper_bound":4},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-4569759744,"num_objects":-69729,"num_object_clones":0,"num_object_copies":-209187,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-69729,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":69729,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001474"},"pg_stats":[{"pgid":"2.f","version":"40'40264","reported_seq":40598,"reported_epoch":40,"state":"active+clean","last_fresh":"2026-02-24T07:44:50.667156+0000","last_change":"2026-02-24T07:44:23.320237+0000","last_active":"2026-02-24T07:44:50.667156+0000","last_peered":"2026-02-24T07:44:50.667156+0000","last_clean":"2026-02-24T07:44:50.667156+0000","last_became_active":"2026-02-24T07:43:07.852899+0000","last_became_peered":"2026-02-24T07:43:07.852899+0000","last_unstale":"2026-02-24T07:44:50.667156+0000","last_undegraded":"2026-02-24T07:44:50.667156+0000","last_fullsized":"2026-02-24T07:44:50.667156+0000","mapping_epoch":30,"log_start":"40'40261","ondisk_log_start":"40'40261","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'30039","last_scrub_stamp":"2026-02-24T07:44:23.320202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_clean_scrub_stamp":"2026-02-24T07:44:23.320202+0000","objects_scrubbed":29558,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:45:24.615266+0000","scrub_duration":2598,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2001731584,"num_objects":30544,"num_object_clones":0,"num_object_copies":91632,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30544,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":40264,"num_write_kb":2265856,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.e","version":"40'41034","reported_seq":41348,"reported_epoch":40,"state":"active+clean","last_fresh":"2026-02-24T07:44:50.709060+0000","last_change":"2026-02-24T07:44:16.083399+0000","last_active":"2026-02-24T07:44:50.709060+0000","last_peered":"2026-02-24T07:44:50.709060+0000","last_clean":"2026-02-24T07:44:50.709060+0000","last_became_active":"2026-02-24T07:43:07.852815+0000","last_became_peered":"2026-02-24T07:43:07.852815+0000","last_unstale":"2026-02-24T07:44:50.709060+0000","last_undegraded":"2026-02-24T07:44:50.709060+0000","last_fullsized":"2026-02-24T07:44:50.709060+0000","mapping_epoch":30,"log_start":"40'41031","ondisk_log_start":"40'41031","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'27767","last_scrub_stamp":"2026-02-24T07:44:16.083320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_clean_scrub_stamp":"2026-02-24T07:44:16.083320+0000","objects_scrubbed":27311,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:45:28.111749+0000","scrub_duration":2943,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2026110976,"num_objects":30916,"num_object_clones":0,"num_object_copies":92748,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30916,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":41034,"num_write_kb":2302400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.d","version":"40'41244","reported_seq":41594,"reported_epoch":40,"state":"active+clean","last_fresh":"2026-02-24T07:44:50.712078+0000","last_change":"2026-02-24T07:44:26.596366+0000","last_active":"2026-02-24T07:44:50.712078+0000","last_peered":"2026-02-24T07:44:50.712078+0000","last_clean":"2026-02-24T07:44:50.712078+0000","last_became_active":"2026-02-24T07:43:07.853975+0000","last_became_peered":"2026-02-24T07:43:07.853975+0000","last_unstale":"2026-02-24T07:44:50.712078+0000","last_undegraded":"2026-02-24T07:44:50.712078+0000","last_fullsized":"2026-02-24T07:44:50.712078+0000","mapping_epoch":30,"log_start":"40'41240","ondisk_log_start":"40'41240","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'31656","last_scrub_stamp":"2026-02-24T07:44:26.596292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_clean_scrub_stamp":"2026-02-24T07:44:26.596292+0000","objects_scrubbed":31097,"log_size":4,"log_dups_size":3000,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:45:55.416603+0000","scrub_duration":3380,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2027487256,"num_objects":30938,"num_object_clones":0,"num_object_copies":92814,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30938,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":41244,"num_write_kb":2309761,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"40'41083","reported_seq":41441,"reported_epoch":40,"state":"active+clean","last_fresh":"2026-02-24T07:44:50.704204+0000","last_change":"2026-02-24T07:44:29.667393+0000","last_active":"2026-02-24T07:44:50.704204+0000","last_peered":"2026-02-24T07:44:50.704204+0000","last_clean":"2026-02-24T07:44:50.704204+0000","last_became_active":"2026-02-24T07:43:07.854428+0000","last_became_peered":"2026-02-24T07:43:07.854428+0000","last_unstale":"2026-02-24T07:44:50.704204+0000","last_undegraded":"2026-02-24T07:44:50.704204+0000","last_fullsized":"2026-02-24T07:44:50.704204+0000","mapping_epoch":30,"log_start":"40'41080","ondisk_log_start":"40'41080","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'32466","last_scrub_stamp":"2026-02-24T07:44:29.667348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_clean_scrub_stamp":"2026-02-24T07:44:29.667348+0000","objects_scrubbed":31970,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:45:43.092916+0000","scrub_duration":3552,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2041249792,"num_objects":31147,"num_object_clones":0,"num_object_copies":93441,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31147,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":41083,"num_write_kb":2311360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"29'99","reported_seq":209,"reported_epoch":40,"state":"active+clean","last_fresh":"2026-02-24T07:43:59.386353+0000","last_change":"2026-02-24T07:43:59.386353+0000","last_active":"2026-02-24T07:43:59.386353+0000","last_peered":"2026-02-24T07:43:59.386353+0000","last_clean":"2026-02-24T07:43:59.386353+0000","last_became_active":"2026-02-24T07:42:54.186719+0000","last_became_peered":"2026-02-24T07:42:54.186719+0000","last_unstale":"2026-02-24T07:43:59.386353+0000","last_undegraded":"2026-02-24T07:43:59.386353+0000","last_fullsized":"2026-02-24T07:43:59.386353+0000","mapping_epoch":27,"log_start":"29'96","ondisk_log_start":"29'96","created":20,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'99","last_scrub_stamp":"2026-02-24T07:43:59.386324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:42:43.640776+0000","last_clean_scrub_stamp":"2026-02-24T07:43:59.386324+0000","objects_scrubbed":2,"log_size":3,"log_dups_size":96,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:45:24.339819+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":240,"num_write":135,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"40'41136","reported_seq":41472,"reported_epoch":40,"state":"active+clean","last_fresh":"2026-02-24T07:44:50.691679+0000","last_change":"2026-02-24T07:44:22.665897+0000","last_active":"2026-02-24T07:44:50.691679+0000","last_peered":"2026-02-24T07:44:50.691679+0000","last_clean":"2026-02-24T07:44:50.691679+0000","last_became_active":"2026-02-24T07:43:07.842530+0000","last_became_peered":"2026-02-24T07:43:07.842530+0000","last_unstale":"2026-02-24T07:44:50.691679+0000","last_undegraded":"2026-02-24T07:44:50.691679+0000","last_fullsized":"2026-02-24T07:44:50.691679+0000","mapping_epoch":30,"log_start":"40'41133","ondisk_log_start":"40'41133","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'30210","last_scrub_stamp":"2026-02-24T07:44:22.665762+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_clean_scrub_stamp":"2026-02-24T07:44:22.665762+0000","objects_scrubbed":29682,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:45:26.371453+0000","scrub_duration":3226,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2029780992,"num_objects":30972,"num_object_clones":0,"num_object_copies":92916,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30972,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":41136,"num_write_kb":2307456,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"40'40734","reported_seq":41086,"reported_epoch":40,"state":"active+clean","last_fresh":"2026-02-24T07:44:50.689528+0000","last_change":"2026-02-24T07:44:28.446165+0000","last_active":"2026-02-24T07:44:50.689528+0000","last_peered":"2026-02-24T07:44:50.689528+0000","last_clean":"2026-02-24T07:44:50.689528+0000","last_became_active":"2026-02-24T07:43:07.853885+0000","last_became_peered":"2026-02-24T07:43:07.853885+0000","last_unstale":"2026-02-24T07:44:50.689528+0000","last_undegraded":"2026-02-24T07:44:50.689528+0000","last_fullsized":"2026-02-24T07:44:50.689528+0000","mapping_epoch":30,"log_start":"40'40731","ondisk_log_start":"40'40731","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'31685","last_scrub_stamp":"2026-02-24T07:44:28.446064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_clean_scrub_stamp":"2026-02-24T07:44:28.446064+0000","objects_scrubbed":31280,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:45:52.594601+0000","scrub_duration":3117,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1998192640,"num_objects":30490,"num_object_clones":0,"num_object_copies":91470,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30490,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":40734,"num_write_kb":2279168,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"40'41210","reported_seq":41581,"reported_epoch":40,"state":"active+clean","last_fresh":"2026-02-24T07:44:50.707237+0000","last_change":"2026-02-24T07:44:33.357960+0000","last_active":"2026-02-24T07:44:50.707237+0000","last_peered":"2026-02-24T07:44:50.707237+0000","last_clean":"2026-02-24T07:44:50.707237+0000","last_became_active":"2026-02-24T07:43:07.854212+0000","last_became_peered":"2026-02-24T07:43:07.854212+0000","last_unstale":"2026-02-24T07:44:50.707237+0000","last_undegraded":"2026-02-24T07:44:50.707237+0000","last_fullsized":"2026-02-24T07:44:50.707237+0000","mapping_epoch":30,"log_start":"40'41207","ondisk_log_start":"40'41207","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'33783","last_scrub_stamp":"2026-02-24T07:44:33.357894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_clean_scrub_stamp":"2026-02-24T07:44:33.357894+0000","objects_scrubbed":33200,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:45:38.914707+0000","scrub_duration":3247,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2027028480,"num_objects":30930,"num_object_clones":0,"num_object_copies":92790,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30930,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":41210,"num_write_kb":2308480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[1,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"40'41263","reported_seq":41654,"reported_epoch":40,"state":"active+clean","last_fresh":"2026-02-24T07:44:50.693631+0000","last_change":"2026-02-24T07:44:42.974303+0000","last_active":"2026-02-24T07:44:50.693631+0000","last_peered":"2026-02-24T07:44:50.693631+0000","last_clean":"2026-02-24T07:44:50.693631+0000","last_became_active":"2026-02-24T07:43:07.842864+0000","last_became_peered":"2026-02-24T07:43:07.842864+0000","last_unstale":"2026-02-24T07:44:50.693631+0000","last_undegraded":"2026-02-24T07:44:50.693631+0000","last_fullsized":"2026-02-24T07:44:50.693631+0000","mapping_epoch":30,"log_start":"40'41260","ondisk_log_start":"40'41260","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'37783","last_scrub_stamp":"2026-02-24T07:44:42.974258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_clean_scrub_stamp":"2026-02-24T07:44:42.974258+0000","objects_scrubbed":35124,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:45:49.565319+0000","scrub_duration":3593,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2018181120,"num_objects":30795,"num_object_clones":0,"num_object_copies":92385,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30795,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":41263,"num_write_kb":2305856,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.4","version":"40'40744","reported_seq":41058,"reported_epoch":40,"state":"active+clean","last_fresh":"2026-02-24T07:44:50.709005+0000","last_change":"2026-02-24T07:44:16.909673+0000","last_active":"2026-02-24T07:44:50.709005+0000","last_peered":"2026-02-24T07:44:50.709005+0000","last_clean":"2026-02-24T07:44:50.709005+0000","last_became_active":"2026-02-24T07:43:07.853966+0000","last_became_peered":"2026-02-24T07:43:07.853966+0000","last_unstale":"2026-02-24T07:44:50.709005+0000","last_undegraded":"2026-02-24T07:44:50.709005+0000","last_fullsized":"2026-02-24T07:44:50.709005+0000","mapping_epoch":30,"log_start":"40'40741","ondisk_log_start":"40'40741","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'27932","last_scrub_stamp":"2026-02-24T07:44:16.909624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_clean_scrub_stamp":"2026-02-24T07:44:16.909624+0000","objects_scrubbed":27480,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:45:43.624405+0000","scrub_duration":2774,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2021392384,"num_objects":30844,"num_object_clones":0,"num_object_copies":92532,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30844,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":40744,"num_write_kb":2290816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"40'40917","reported_seq":41296,"reported_epoch":40,"state":"active+clean","last_fresh":"2026-02-24T07:44:50.693272+0000","last_change":"2026-02-24T07:44:35.717081+0000","last_active":"2026-02-24T07:44:50.693272+0000","last_peered":"2026-02-24T07:44:50.693272+0000","last_clean":"2026-02-24T07:44:50.693272+0000","last_became_active":"2026-02-24T07:43:07.853878+0000","last_became_peered":"2026-02-24T07:43:07.853878+0000","last_unstale":"2026-02-24T07:44:50.693272+0000","last_undegraded":"2026-02-24T07:44:50.693272+0000","last_fullsized":"2026-02-24T07:44:50.693272+0000","mapping_epoch":30,"log_start":"40'40914","ondisk_log_start":"40'40914","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'34578","last_scrub_stamp":"2026-02-24T07:44:35.717022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_clean_scrub_stamp":"2026-02-24T07:44:35.717022+0000","objects_scrubbed":33884,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:46:00.809440+0000","scrub_duration":3376,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2013200384,"num_objects":30719,"num_object_clones":0,"num_object_copies":92157,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30719,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":40917,"num_write_kb":2292352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"40'41032","reported_seq":41392,"reported_epoch":40,"state":"active+clean","last_fresh":"2026-02-24T07:44:50.710394+0000","last_change":"2026-02-24T07:44:30.553008+0000","last_active":"2026-02-24T07:44:50.710394+0000","last_peered":"2026-02-24T07:44:50.710394+0000","last_clean":"2026-02-24T07:44:50.710394+0000","last_became_active":"2026-02-24T07:43:07.841993+0000","last_became_peered":"2026-02-24T07:43:07.841993+0000","last_unstale":"2026-02-24T07:44:50.710394+0000","last_undegraded":"2026-02-24T07:44:50.710394+0000","last_fullsized":"2026-02-24T07:44:50.710394+0000","mapping_epoch":30,"log_start":"40'41029","ondisk_log_start":"40'41029","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'32640","last_scrub_stamp":"2026-02-24T07:44:30.552965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_clean_scrub_stamp":"2026-02-24T07:44:30.552965+0000","objects_scrubbed":32121,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:45:56.537381+0000","scrub_duration":3414,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2017067008,"num_objects":30778,"num_object_clones":0,"num_object_copies":92334,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30778,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":41032,"num_write_kb":2297920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"40'40490","reported_seq":40807,"reported_epoch":40,"state":"active+clean","last_fresh":"2026-02-24T07:44:50.688698+0000","last_change":"2026-02-24T07:44:18.020443+0000","last_active":"2026-02-24T07:44:50.688698+0000","last_peered":"2026-02-24T07:44:50.688698+0000","last_clean":"2026-02-24T07:44:50.688698+0000","last_became_active":"2026-02-24T07:43:07.841920+0000","last_became_peered":"2026-02-24T07:43:07.841920+0000","last_unstale":"2026-02-24T07:44:50.688698+0000","last_undegraded":"2026-02-24T07:44:50.688698+0000","last_fullsized":"2026-02-24T07:44:50.688698+0000","mapping_epoch":30,"log_start":"40'40487","ondisk_log_start":"40'40487","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'28226","last_scrub_stamp":"2026-02-24T07:44:18.020379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_clean_scrub_stamp":"2026-02-24T07:44:18.020379+0000","objects_scrubbed":27705,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:45:27.051564+0000","scrub_duration":2601,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2007629824,"num_objects":30634,"num_object_clones":0,"num_object_copies":91902,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30634,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":40490,"num_write_kb":2275968,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.8","version":"40'40690","reported_seq":41048,"reported_epoch":40,"state":"active+clean","last_fresh":"2026-02-24T07:44:50.694063+0000","last_change":"2026-02-24T07:44:31.118780+0000","last_active":"2026-02-24T07:44:50.694063+0000","last_peered":"2026-02-24T07:44:50.694063+0000","last_clean":"2026-02-24T07:44:50.694063+0000","last_became_active":"2026-02-24T07:43:07.853758+0000","last_became_peered":"2026-02-24T07:43:07.853758+0000","last_unstale":"2026-02-24T07:44:50.694063+0000","last_undegraded":"2026-02-24T07:44:50.694063+0000","last_fullsized":"2026-02-24T07:44:50.694063+0000","mapping_epoch":30,"log_start":"40'40687","ondisk_log_start":"40'40687","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'32571","last_scrub_stamp":"2026-02-24T07:44:31.118720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_clean_scrub_stamp":"2026-02-24T07:44:31.118720+0000","objects_scrubbed":32077,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:45:44.647478+0000","scrub_duration":2775,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2009202688,"num_objects":30658,"num_object_clones":0,"num_object_copies":91974,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30658,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":40690,"num_write_kb":2283136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"40'41087","reported_seq":41480,"reported_epoch":40,"state":"active+clean","last_fresh":"2026-02-24T07:44:50.709571+0000","last_change":"2026-02-24T07:44:41.349757+0000","last_active":"2026-02-24T07:44:50.709571+0000","last_peered":"2026-02-24T07:44:50.709571+0000","last_clean":"2026-02-24T07:44:50.709571+0000","last_became_active":"2026-02-24T07:43:07.854377+0000","last_became_peered":"2026-02-24T07:43:07.854377+0000","last_unstale":"2026-02-24T07:44:50.709571+0000","last_undegraded":"2026-02-24T07:44:50.709571+0000","last_fullsized":"2026-02-24T07:44:50.709571+0000","mapping_epoch":30,"log_start":"40'41084","ondisk_log_start":"40'41084","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'36997","last_scrub_stamp":"2026-02-24T07:44:41.349228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_clean_scrub_stamp":"2026-02-24T07:44:41.349228+0000","objects_scrubbed":35574,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:45:50.611975+0000","scrub_duration":3205,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2026438656,"num_objects":30921,"num_object_clones":0,"num_object_copies":92763,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30921,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":41087,"num_write_kb":2304256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.a","version":"40'41033","reported_seq":41388,"reported_epoch":40,"state":"active+clean","last_fresh":"2026-02-24T07:44:50.709527+0000","last_change":"2026-02-24T07:44:27.902144+0000","last_active":"2026-02-24T07:44:50.709527+0000","last_peered":"2026-02-24T07:44:50.709527+0000","last_clean":"2026-02-24T07:44:50.709527+0000","last_became_active":"2026-02-24T07:43:07.841721+0000","last_became_peered":"2026-02-24T07:43:07.841721+0000","last_unstale":"2026-02-24T07:44:50.709527+0000","last_undegraded":"2026-02-24T07:44:50.709527+0000","last_fullsized":"2026-02-24T07:44:50.709527+0000","mapping_epoch":30,"log_start":"40'41030","ondisk_log_start":"40'41030","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'31794","last_scrub_stamp":"2026-02-24T07:44:27.902040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_clean_scrub_stamp":"2026-02-24T07:44:27.902040+0000","objects_scrubbed":31292,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:45:39.265782+0000","scrub_duration":3732,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2022113280,"num_objects":30855,"num_object_clones":0,"num_object_copies":92565,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30855,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":41033,"num_write_kb":2300416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.b","version":"40'41184","reported_seq":41579,"reported_epoch":40,"state":"active+clean","last_fresh":"2026-02-24T07:44:50.685040+0000","last_change":"2026-02-24T07:44:39.639056+0000","last_active":"2026-02-24T07:44:50.685040+0000","last_peered":"2026-02-24T07:44:50.685040+0000","last_clean":"2026-02-24T07:44:50.685040+0000","last_became_active":"2026-02-24T07:43:07.853069+0000","last_became_peered":"2026-02-24T07:43:07.853069+0000","last_unstale":"2026-02-24T07:44:50.685040+0000","last_undegraded":"2026-02-24T07:44:50.685040+0000","last_fullsized":"2026-02-24T07:44:50.685040+0000","mapping_epoch":30,"log_start":"40'41181","ondisk_log_start":"40'41181","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'36298","last_scrub_stamp":"2026-02-24T07:44:39.638986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_clean_scrub_stamp":"2026-02-24T07:44:39.638986+0000","objects_scrubbed":35656,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:46:09.128452+0000","scrub_duration":3228,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2021785600,"num_objects":30850,"num_object_clones":0,"num_object_copies":92550,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30850,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":41184,"num_write_kb":2305088,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":32308592664,"num_objects":492991,"num_object_clones":0,"num_object_copies":1478973,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":492991,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":655145,"num_write_kb":36740289,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":96947875840,"data_stored":96947863624,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":49,"ondisk_log_size":49,"up":48,"acting":48,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":240,"num_write":135,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":3,"up_from":25,"seq":107374182426,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":28681360,"kb_used_data":27767612,"kb_used_omap":4,"kb_used_meta":913659,"kb_avail":705321840,"statfs":{"total":751619276800,"available":722249564160,"internally_reserved":0,"allocated":28434034688,"data_stored":28433779976,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4448,"internal_metadata":935587488},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,5],"upper_bound":4},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":19,"seq":81604378652,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":23342604,"kb_used_data":21684152,"kb_used_omap":18,"kb_used_meta":1658413,"kb_avail":710660596,"statfs":{"total":751619276800,"available":727716450304,"internally_reserved":0,"allocated":22204571648,"data_stored":22204321008,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":18551,"internal_metadata":1698215817},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542173,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":25622988,"kb_used_data":23820152,"kb_used_omap":20,"kb_used_meta":1802731,"kb_avail":708380212,"statfs":{"total":751619276800,"available":725381337088,"internally_reserved":0,"allocated":24391835648,"data_stored":24391584488,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":20504,"internal_metadata":1845997544},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705696,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":23650000,"kb_used_data":21973628,"kb_used_omap":24,"kb_used_meta":1676327,"kb_avail":710353200,"statfs":{"total":751619276800,"available":727401676800,"internally_reserved":0,"allocated":22500995072,"data_stored":22500740360,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":25061,"internal_metadata":1716559387},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":22225096704,"data_stored":22225092632,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":24260055040,"data_stored":24260050968,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":22199468032,"data_stored":22199468032,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28263256064,"data_stored":28263251992,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-24T07:44:54.517 INFO:tasks.cephadm.ceph_manager.ceph:increase pool size by 10 2026-02-24T07:44:54.518 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph osd pool set .mgr pg_num 81 2026-02-24T07:44:54.653 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:44:54.769 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:44:54 trial035 ceph-mon[39284]: pgmap v71: 17 pgs: 17 active+clean; 30 GiB data, 97 GiB used, 2.6 TiB / 2.7 TiB avail; 0 B/s wr, 5.81k op/s 2026-02-24T07:44:54.769 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:44:54 trial035 ceph-mon[39284]: from='client.14501 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2026-02-24T07:44:55.098 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:44:55.070+0000 7f928f1d9640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:44:55.099 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:44:55.070+0000 7f928f1d9640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:44:55.205 INFO:teuthology.orchestra.run.trial035.stderr:Error EPERM: crimson_allow_pg_split is false; pg_num increase denied for crimson pool 2026-02-24T07:44:55.244 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-24T07:44:55.409 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2026-02-24T07:44:55.539 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:44:55.659 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:44:55.656+0000 7fcb7ad43640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:44:55.659 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:44:55.657+0000 7fcb7ad43640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:44:55.674 INFO:teuthology.orchestra.run.trial035.stdout:osd.0: {} 2026-02-24T07:44:55.674 INFO:teuthology.orchestra.run.trial035.stderr:osd.0: osd_enable_op_tracker = '' 2026-02-24T07:44:55.688 INFO:teuthology.orchestra.run.trial035.stdout:osd.1: {} 2026-02-24T07:44:55.689 INFO:teuthology.orchestra.run.trial035.stderr:osd.1: osd_enable_op_tracker = '' 2026-02-24T07:44:55.700 INFO:teuthology.orchestra.run.trial035.stdout:osd.2: {} 2026-02-24T07:44:55.700 INFO:teuthology.orchestra.run.trial035.stderr:osd.2: osd_enable_op_tracker = '' 2026-02-24T07:44:55.714 INFO:teuthology.orchestra.run.trial035.stdout:osd.3: {} 2026-02-24T07:44:55.715 INFO:teuthology.orchestra.run.trial035.stderr:osd.3: osd_enable_op_tracker = '' 2026-02-24T07:44:55.954 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:44:55 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/2491142956' entity='client.admin' cmd={"prefix": "osd pool set", "pool": ".mgr", "var": "pg_num", "val": "81"} : dispatch 2026-02-24T07:44:55.954 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:44:55 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/3621688073' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2026-02-24T07:44:57.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:44:56 trial035 ceph-mon[39284]: pgmap v72: 17 pgs: 17 active+clean; 30 GiB data, 94 GiB used, 2.6 TiB / 2.7 TiB avail; 0 B/s wr, 6.02k op/s 2026-02-24T07:44:57.205 INFO:journalctl@ceph.mgr.x.trial035.stdout:Feb 24 07:44:56 trial035 ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-mgr-x[39621]: ::ffff:10.20.193.91 - - [24/Feb/2026:07:44:56] "GET /metrics HTTP/1.1" 200 4223 "" "Prometheus/3.6.0" 2026-02-24T07:44:57.356 INFO:journalctl@ceph.prometheus.a.trial091.stdout:Feb 24 07:44:56 trial091 ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-prometheus-a[66007]: time=2026-02-24T07:44:56.973Z level=ERROR source=scrape.go:1630 msg="Failed to determine correct type of scrape target." component="scrape manager" scrape_pool=ceph target=http://trial035:9283/metrics content_type="text/html;charset=utf-8" fallback_media_type="" err="received unsupported Content-Type \"text/html;charset=utf-8\" and no fallback_scrape_protocol specified for target" 2026-02-24T07:44:57.753 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2026-02-24T07:44:57.895 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:44:58.015 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:44:58.013+0000 7f41f35fd640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:44:58.015 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:44:58.013+0000 7f41f35fd640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:44:58.029 INFO:teuthology.orchestra.run.trial035.stdout:osd.0: {} 2026-02-24T07:44:58.029 INFO:teuthology.orchestra.run.trial035.stderr:osd.0: osd_enable_op_tracker = '' 2026-02-24T07:44:58.041 INFO:teuthology.orchestra.run.trial035.stdout:osd.1: {} 2026-02-24T07:44:58.041 INFO:teuthology.orchestra.run.trial035.stderr:osd.1: osd_enable_op_tracker = '' 2026-02-24T07:44:58.054 INFO:teuthology.orchestra.run.trial035.stdout:osd.2: {} 2026-02-24T07:44:58.055 INFO:teuthology.orchestra.run.trial035.stderr:osd.2: osd_enable_op_tracker = '' 2026-02-24T07:44:58.067 INFO:teuthology.orchestra.run.trial035.stdout:osd.3: {} 2026-02-24T07:44:58.068 INFO:teuthology.orchestra.run.trial035.stderr:osd.3: osd_enable_op_tracker = '' 2026-02-24T07:44:59.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:44:58 trial035 ceph-mon[39284]: pgmap v73: 17 pgs: 17 active+clean; 28 GiB data, 90 GiB used, 2.6 TiB / 2.7 TiB avail; 0 B/s wr, 8.95k op/s 2026-02-24T07:44:59.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:44:58 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/4219081896' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2026-02-24T07:45:00.106 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2026-02-24T07:45:00.246 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2026-02-24T07:45:00.247 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2026-02-24T07:45:00.247 INFO:tasks.thrashosds.thrasher:Growing pool .mgr 2026-02-24T07:45:00.247 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph pg dump --format=json 2026-02-24T07:45:00.248 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:45:00.803 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:45:00.724+0000 7f535d115640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:45:00.803 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:45:00.724+0000 7f535d115640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:45:00.804 INFO:teuthology.orchestra.run.trial035.stderr:osd.0: osd_enable_op_tracker = '' 2026-02-24T07:45:00.804 INFO:teuthology.orchestra.run.trial035.stderr:osd.1: osd_enable_op_tracker = '' 2026-02-24T07:45:00.804 INFO:teuthology.orchestra.run.trial035.stderr:osd.2: osd_enable_op_tracker = '' 2026-02-24T07:45:00.804 INFO:teuthology.orchestra.run.trial035.stderr:osd.3: osd_enable_op_tracker = '' 2026-02-24T07:45:00.804 INFO:teuthology.orchestra.run.trial035.stdout:osd.0: {} 2026-02-24T07:45:00.805 INFO:teuthology.orchestra.run.trial035.stdout:osd.1: {} 2026-02-24T07:45:00.805 INFO:teuthology.orchestra.run.trial035.stdout:osd.2: {} 2026-02-24T07:45:00.805 INFO:teuthology.orchestra.run.trial035.stdout:osd.3: {} 2026-02-24T07:45:01.176 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:45:00 trial035 ceph-mon[39284]: pgmap v74: 17 pgs: 17 active+clean; 28 GiB data, 90 GiB used, 2.6 TiB / 2.7 TiB avail; 0 B/s wr, 6.15k op/s 2026-02-24T07:45:01.177 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:45:00 trial035 ceph-mon[39284]: from='mgr.14284 10.20.193.35:0/3681373222' entity='mgr.x' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2026-02-24T07:45:01.177 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:45:00 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/1824918857' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2026-02-24T07:45:01.438 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:45:02.128 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:45:02.101+0000 7f89fcd64640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:45:02.128 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:45:02.101+0000 7f89fcd64640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:45:02.235 INFO:teuthology.orchestra.run.trial035.stdout: 2026-02-24T07:45:02.235 INFO:teuthology.orchestra.run.trial035.stderr:dumped all 2026-02-24T07:45:02.275 INFO:teuthology.orchestra.run.trial035.stdout:{"pg_ready":true,"pg_map":{"version":75,"stamp":"2026-02-24T07:45:01.674378+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":27025343032,"num_objects":412368,"num_object_clones":0,"num_object_copies":1237104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":412368,"num_whiteouts":0,"num_read":137,"num_read_kb":241,"num_write":735905,"num_write_kb":36742439,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":53,"ondisk_log_size":53,"up":51,"acting":51,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":51,"num_osds":4,"num_per_pool_osds":4,"num_per_pool_omap_osds":4,"kb":2936012800,"kb_used":86320696,"kb_used_data":79817960,"kb_used_omap":66,"kb_used_meta":6502333,"kb_avail":2849692104,"statfs":{"total":3006477107200,"available":2918084714496,"internally_reserved":0,"allocated":81733591040,"data_stored":81732579816,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":68564,"internal_metadata":6658389036},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-7509704704,"num_objects":-114589,"num_object_clones":0,"num_object_copies":-343767,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-114589,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":114589,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001466"},"pg_stats":[{"pgid":"2.f","version":"40'45325","reported_seq":45659,"reported_epoch":40,"state":"active+clean","last_fresh":"2026-02-24T07:45:00.666939+0000","last_change":"2026-02-24T07:44:23.320237+0000","last_active":"2026-02-24T07:45:00.666939+0000","last_peered":"2026-02-24T07:45:00.666939+0000","last_clean":"2026-02-24T07:45:00.666939+0000","last_became_active":"2026-02-24T07:43:07.852899+0000","last_became_peered":"2026-02-24T07:43:07.852899+0000","last_unstale":"2026-02-24T07:45:00.666939+0000","last_undegraded":"2026-02-24T07:45:00.666939+0000","last_fullsized":"2026-02-24T07:45:00.666939+0000","mapping_epoch":30,"log_start":"40'45322","ondisk_log_start":"40'45322","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'30039","last_scrub_stamp":"2026-02-24T07:44:23.320202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_clean_scrub_stamp":"2026-02-24T07:44:23.320202+0000","objects_scrubbed":29558,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:45:24.615266+0000","scrub_duration":2598,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1670053888,"num_objects":25483,"num_object_clones":0,"num_object_copies":76449,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25483,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45325,"num_write_kb":2265856,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.e","version":"40'46132","reported_seq":46446,"reported_epoch":40,"state":"active+clean","last_fresh":"2026-02-24T07:45:00.707535+0000","last_change":"2026-02-24T07:44:16.083399+0000","last_active":"2026-02-24T07:45:00.707535+0000","last_peered":"2026-02-24T07:45:00.707535+0000","last_clean":"2026-02-24T07:45:00.707535+0000","last_became_active":"2026-02-24T07:43:07.852815+0000","last_became_peered":"2026-02-24T07:43:07.852815+0000","last_unstale":"2026-02-24T07:45:00.707535+0000","last_undegraded":"2026-02-24T07:45:00.707535+0000","last_fullsized":"2026-02-24T07:45:00.707535+0000","mapping_epoch":30,"log_start":"40'46129","ondisk_log_start":"40'46129","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'27767","last_scrub_stamp":"2026-02-24T07:44:16.083320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_clean_scrub_stamp":"2026-02-24T07:44:16.083320+0000","objects_scrubbed":27311,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:45:28.111749+0000","scrub_duration":2943,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1692008448,"num_objects":25818,"num_object_clones":0,"num_object_copies":77454,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25818,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":46132,"num_write_kb":2302400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.d","version":"40'46295","reported_seq":46645,"reported_epoch":40,"state":"active+clean","last_fresh":"2026-02-24T07:45:00.736265+0000","last_change":"2026-02-24T07:44:26.596366+0000","last_active":"2026-02-24T07:45:00.736265+0000","last_peered":"2026-02-24T07:45:00.736265+0000","last_clean":"2026-02-24T07:45:00.736265+0000","last_became_active":"2026-02-24T07:43:07.853975+0000","last_became_peered":"2026-02-24T07:43:07.853975+0000","last_unstale":"2026-02-24T07:45:00.736265+0000","last_undegraded":"2026-02-24T07:45:00.736265+0000","last_fullsized":"2026-02-24T07:45:00.736265+0000","mapping_epoch":30,"log_start":"40'46291","ondisk_log_start":"40'46291","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'31656","last_scrub_stamp":"2026-02-24T07:44:26.596292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_clean_scrub_stamp":"2026-02-24T07:44:26.596292+0000","objects_scrubbed":31097,"log_size":4,"log_dups_size":3000,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:45:55.416603+0000","scrub_duration":3380,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1696464920,"num_objects":25887,"num_object_clones":0,"num_object_copies":77661,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25887,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":46295,"num_write_kb":2309761,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"40'46200","reported_seq":46558,"reported_epoch":40,"state":"active+clean","last_fresh":"2026-02-24T07:45:00.707233+0000","last_change":"2026-02-24T07:44:29.667393+0000","last_active":"2026-02-24T07:45:00.707233+0000","last_peered":"2026-02-24T07:45:00.707233+0000","last_clean":"2026-02-24T07:45:00.707233+0000","last_became_active":"2026-02-24T07:43:07.854428+0000","last_became_peered":"2026-02-24T07:43:07.854428+0000","last_unstale":"2026-02-24T07:45:00.707233+0000","last_undegraded":"2026-02-24T07:45:00.707233+0000","last_fullsized":"2026-02-24T07:45:00.707233+0000","mapping_epoch":30,"log_start":"40'46197","ondisk_log_start":"40'46197","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'32466","last_scrub_stamp":"2026-02-24T07:44:29.667348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_clean_scrub_stamp":"2026-02-24T07:44:29.667348+0000","objects_scrubbed":31970,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:45:43.092916+0000","scrub_duration":3552,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1705902080,"num_objects":26030,"num_object_clones":0,"num_object_copies":78090,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26030,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":46200,"num_write_kb":2311360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"29'99","reported_seq":210,"reported_epoch":40,"state":"active+clean","last_fresh":"2026-02-24T07:43:59.386388+0000","last_change":"2026-02-24T07:43:59.386353+0000","last_active":"2026-02-24T07:43:59.386388+0000","last_peered":"2026-02-24T07:43:59.386388+0000","last_clean":"2026-02-24T07:43:59.386388+0000","last_became_active":"2026-02-24T07:42:54.186719+0000","last_became_peered":"2026-02-24T07:42:54.186719+0000","last_unstale":"2026-02-24T07:43:59.386388+0000","last_undegraded":"2026-02-24T07:43:59.386388+0000","last_fullsized":"2026-02-24T07:43:59.386388+0000","mapping_epoch":27,"log_start":"29'96","ondisk_log_start":"29'96","created":20,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'99","last_scrub_stamp":"2026-02-24T07:43:59.386324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:42:43.640776+0000","last_clean_scrub_stamp":"2026-02-24T07:43:59.386324+0000","objects_scrubbed":2,"log_size":3,"log_dups_size":96,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:45:24.339819+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":240,"num_write":135,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"40'46212","reported_seq":46548,"reported_epoch":40,"state":"active+clean","last_fresh":"2026-02-24T07:45:00.693203+0000","last_change":"2026-02-24T07:44:22.665897+0000","last_active":"2026-02-24T07:45:00.693203+0000","last_peered":"2026-02-24T07:45:00.693203+0000","last_clean":"2026-02-24T07:45:00.693203+0000","last_became_active":"2026-02-24T07:43:07.842530+0000","last_became_peered":"2026-02-24T07:43:07.842530+0000","last_unstale":"2026-02-24T07:45:00.693203+0000","last_undegraded":"2026-02-24T07:45:00.693203+0000","last_fullsized":"2026-02-24T07:45:00.693203+0000","mapping_epoch":30,"log_start":"40'46209","ondisk_log_start":"40'46209","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'30210","last_scrub_stamp":"2026-02-24T07:44:22.665762+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_clean_scrub_stamp":"2026-02-24T07:44:22.665762+0000","objects_scrubbed":29682,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:45:26.371453+0000","scrub_duration":3226,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1697120256,"num_objects":25896,"num_object_clones":0,"num_object_copies":77688,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25896,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":46212,"num_write_kb":2307456,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"40'45842","reported_seq":46194,"reported_epoch":40,"state":"active+clean","last_fresh":"2026-02-24T07:45:00.695010+0000","last_change":"2026-02-24T07:44:28.446165+0000","last_active":"2026-02-24T07:45:00.695010+0000","last_peered":"2026-02-24T07:45:00.695010+0000","last_clean":"2026-02-24T07:45:00.695010+0000","last_became_active":"2026-02-24T07:43:07.853885+0000","last_became_peered":"2026-02-24T07:43:07.853885+0000","last_unstale":"2026-02-24T07:45:00.695010+0000","last_undegraded":"2026-02-24T07:45:00.695010+0000","last_fullsized":"2026-02-24T07:45:00.695010+0000","mapping_epoch":30,"log_start":"40'45838","ondisk_log_start":"40'45838","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'31685","last_scrub_stamp":"2026-02-24T07:44:28.446064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_clean_scrub_stamp":"2026-02-24T07:44:28.446064+0000","objects_scrubbed":31280,"log_size":4,"log_dups_size":3000,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:45:52.594601+0000","scrub_duration":3117,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1663434752,"num_objects":25382,"num_object_clones":0,"num_object_copies":76146,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25382,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45842,"num_write_kb":2279168,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"40'46125","reported_seq":46496,"reported_epoch":40,"state":"active+clean","last_fresh":"2026-02-24T07:45:00.705980+0000","last_change":"2026-02-24T07:44:33.357960+0000","last_active":"2026-02-24T07:45:00.705980+0000","last_peered":"2026-02-24T07:45:00.705980+0000","last_clean":"2026-02-24T07:45:00.705980+0000","last_became_active":"2026-02-24T07:43:07.854212+0000","last_became_peered":"2026-02-24T07:43:07.854212+0000","last_unstale":"2026-02-24T07:45:00.705980+0000","last_undegraded":"2026-02-24T07:45:00.705980+0000","last_fullsized":"2026-02-24T07:45:00.705980+0000","mapping_epoch":30,"log_start":"40'46122","ondisk_log_start":"40'46122","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'33783","last_scrub_stamp":"2026-02-24T07:44:33.357894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_clean_scrub_stamp":"2026-02-24T07:44:33.357894+0000","objects_scrubbed":33200,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:45:38.914707+0000","scrub_duration":3247,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1704919040,"num_objects":26015,"num_object_clones":0,"num_object_copies":78045,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26015,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":46125,"num_write_kb":2308480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[1,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"40'46404","reported_seq":46795,"reported_epoch":40,"state":"active+clean","last_fresh":"2026-02-24T07:45:00.692131+0000","last_change":"2026-02-24T07:44:42.974303+0000","last_active":"2026-02-24T07:45:00.692131+0000","last_peered":"2026-02-24T07:45:00.692131+0000","last_clean":"2026-02-24T07:45:00.692131+0000","last_became_active":"2026-02-24T07:43:07.842864+0000","last_became_peered":"2026-02-24T07:43:07.842864+0000","last_unstale":"2026-02-24T07:45:00.692131+0000","last_undegraded":"2026-02-24T07:45:00.692131+0000","last_fullsized":"2026-02-24T07:45:00.692131+0000","mapping_epoch":30,"log_start":"40'46401","ondisk_log_start":"40'46401","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'37783","last_scrub_stamp":"2026-02-24T07:44:42.974258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_clean_scrub_stamp":"2026-02-24T07:44:42.974258+0000","objects_scrubbed":35124,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:45:49.565319+0000","scrub_duration":3593,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1681260544,"num_objects":25654,"num_object_clones":0,"num_object_copies":76962,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25654,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":46404,"num_write_kb":2305856,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.4","version":"40'45801","reported_seq":46116,"reported_epoch":40,"state":"active+clean","last_fresh":"2026-02-24T07:45:00.703432+0000","last_change":"2026-02-24T07:44:16.909673+0000","last_active":"2026-02-24T07:45:00.703432+0000","last_peered":"2026-02-24T07:45:00.703432+0000","last_clean":"2026-02-24T07:45:00.703432+0000","last_became_active":"2026-02-24T07:43:07.853966+0000","last_became_peered":"2026-02-24T07:43:07.853966+0000","last_unstale":"2026-02-24T07:45:00.703432+0000","last_undegraded":"2026-02-24T07:45:00.703432+0000","last_fullsized":"2026-02-24T07:45:00.703432+0000","mapping_epoch":30,"log_start":"40'45798","ondisk_log_start":"40'45798","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'27932","last_scrub_stamp":"2026-02-24T07:44:16.909624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_clean_scrub_stamp":"2026-02-24T07:44:16.909624+0000","objects_scrubbed":27480,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:45:43.624405+0000","scrub_duration":2774,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1689976832,"num_objects":25787,"num_object_clones":0,"num_object_copies":77361,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25787,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45801,"num_write_kb":2290816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"40'45927","reported_seq":46306,"reported_epoch":40,"state":"active+clean","last_fresh":"2026-02-24T07:45:00.685645+0000","last_change":"2026-02-24T07:44:35.717081+0000","last_active":"2026-02-24T07:45:00.685645+0000","last_peered":"2026-02-24T07:45:00.685645+0000","last_clean":"2026-02-24T07:45:00.685645+0000","last_became_active":"2026-02-24T07:43:07.853878+0000","last_became_peered":"2026-02-24T07:43:07.853878+0000","last_unstale":"2026-02-24T07:45:00.685645+0000","last_undegraded":"2026-02-24T07:45:00.685645+0000","last_fullsized":"2026-02-24T07:45:00.685645+0000","mapping_epoch":30,"log_start":"40'45924","ondisk_log_start":"40'45924","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'34578","last_scrub_stamp":"2026-02-24T07:44:35.717022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_clean_scrub_stamp":"2026-02-24T07:44:35.717022+0000","objects_scrubbed":33884,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:46:00.809440+0000","scrub_duration":3376,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1684865024,"num_objects":25709,"num_object_clones":0,"num_object_copies":77127,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25709,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45927,"num_write_kb":2292352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"40'46068","reported_seq":46428,"reported_epoch":40,"state":"active+clean","last_fresh":"2026-02-24T07:45:00.706135+0000","last_change":"2026-02-24T07:44:30.553008+0000","last_active":"2026-02-24T07:45:00.706135+0000","last_peered":"2026-02-24T07:45:00.706135+0000","last_clean":"2026-02-24T07:45:00.706135+0000","last_became_active":"2026-02-24T07:43:07.841993+0000","last_became_peered":"2026-02-24T07:43:07.841993+0000","last_unstale":"2026-02-24T07:45:00.706135+0000","last_undegraded":"2026-02-24T07:45:00.706135+0000","last_fullsized":"2026-02-24T07:45:00.706135+0000","mapping_epoch":30,"log_start":"40'46065","ondisk_log_start":"40'46065","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'32640","last_scrub_stamp":"2026-02-24T07:44:30.552965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_clean_scrub_stamp":"2026-02-24T07:44:30.552965+0000","objects_scrubbed":32121,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:45:56.537381+0000","scrub_duration":3414,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1687027712,"num_objects":25742,"num_object_clones":0,"num_object_copies":77226,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25742,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":46068,"num_write_kb":2297920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"40'45423","reported_seq":45739,"reported_epoch":40,"state":"active+clean","last_fresh":"2026-02-24T07:45:00.694560+0000","last_change":"2026-02-24T07:44:18.020443+0000","last_active":"2026-02-24T07:45:00.694560+0000","last_peered":"2026-02-24T07:45:00.694560+0000","last_clean":"2026-02-24T07:45:00.694560+0000","last_became_active":"2026-02-24T07:43:07.841920+0000","last_became_peered":"2026-02-24T07:43:07.841920+0000","last_unstale":"2026-02-24T07:45:00.694560+0000","last_undegraded":"2026-02-24T07:45:00.694560+0000","last_fullsized":"2026-02-24T07:45:00.694560+0000","mapping_epoch":30,"log_start":"40'45420","ondisk_log_start":"40'45420","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'28226","last_scrub_stamp":"2026-02-24T07:44:18.020379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_clean_scrub_stamp":"2026-02-24T07:44:18.020379+0000","objects_scrubbed":27705,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:45:27.051564+0000","scrub_duration":2601,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1684340736,"num_objects":25701,"num_object_clones":0,"num_object_copies":77103,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25701,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45423,"num_write_kb":2275968,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.8","version":"40'45677","reported_seq":46036,"reported_epoch":40,"state":"active+clean","last_fresh":"2026-02-24T07:45:00.694248+0000","last_change":"2026-02-24T07:44:31.118780+0000","last_active":"2026-02-24T07:45:00.694248+0000","last_peered":"2026-02-24T07:45:00.694248+0000","last_clean":"2026-02-24T07:45:00.694248+0000","last_became_active":"2026-02-24T07:43:07.853758+0000","last_became_peered":"2026-02-24T07:43:07.853758+0000","last_unstale":"2026-02-24T07:45:00.694248+0000","last_undegraded":"2026-02-24T07:45:00.694248+0000","last_fullsized":"2026-02-24T07:45:00.694248+0000","mapping_epoch":30,"log_start":"40'45674","ondisk_log_start":"40'45674","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'32571","last_scrub_stamp":"2026-02-24T07:44:31.118720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_clean_scrub_stamp":"2026-02-24T07:44:31.118720+0000","objects_scrubbed":32077,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:45:44.647478+0000","scrub_duration":2775,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1682374656,"num_objects":25671,"num_object_clones":0,"num_object_copies":77013,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25671,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45677,"num_write_kb":2283136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"40'46058","reported_seq":46453,"reported_epoch":40,"state":"active+clean","last_fresh":"2026-02-24T07:45:00.707677+0000","last_change":"2026-02-24T07:44:41.349757+0000","last_active":"2026-02-24T07:45:00.707677+0000","last_peered":"2026-02-24T07:45:00.707677+0000","last_clean":"2026-02-24T07:45:00.707677+0000","last_became_active":"2026-02-24T07:43:07.854377+0000","last_became_peered":"2026-02-24T07:43:07.854377+0000","last_unstale":"2026-02-24T07:45:00.707677+0000","last_undegraded":"2026-02-24T07:45:00.707677+0000","last_fullsized":"2026-02-24T07:45:00.707677+0000","mapping_epoch":30,"log_start":"40'46055","ondisk_log_start":"40'46055","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'36997","last_scrub_stamp":"2026-02-24T07:44:41.349228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_clean_scrub_stamp":"2026-02-24T07:44:41.349228+0000","objects_scrubbed":35574,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:45:50.611975+0000","scrub_duration":3205,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1700659200,"num_objects":25950,"num_object_clones":0,"num_object_copies":77850,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25950,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":46058,"num_write_kb":2304256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.a","version":"40'46121","reported_seq":46476,"reported_epoch":40,"state":"active+clean","last_fresh":"2026-02-24T07:45:00.736320+0000","last_change":"2026-02-24T07:44:27.902144+0000","last_active":"2026-02-24T07:45:00.736320+0000","last_peered":"2026-02-24T07:45:00.736320+0000","last_clean":"2026-02-24T07:45:00.736320+0000","last_became_active":"2026-02-24T07:43:07.841721+0000","last_became_peered":"2026-02-24T07:43:07.841721+0000","last_unstale":"2026-02-24T07:45:00.736320+0000","last_undegraded":"2026-02-24T07:45:00.736320+0000","last_fullsized":"2026-02-24T07:45:00.736320+0000","mapping_epoch":30,"log_start":"40'46118","ondisk_log_start":"40'46118","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'31794","last_scrub_stamp":"2026-02-24T07:44:27.902040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_clean_scrub_stamp":"2026-02-24T07:44:27.902040+0000","objects_scrubbed":31292,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:45:39.265782+0000","scrub_duration":3732,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1688666112,"num_objects":25767,"num_object_clones":0,"num_object_copies":77301,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25767,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":46121,"num_write_kb":2300416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.b","version":"40'46160","reported_seq":46555,"reported_epoch":40,"state":"active+clean","last_fresh":"2026-02-24T07:45:00.689999+0000","last_change":"2026-02-24T07:44:39.639056+0000","last_active":"2026-02-24T07:45:00.689999+0000","last_peered":"2026-02-24T07:45:00.689999+0000","last_clean":"2026-02-24T07:45:00.689999+0000","last_became_active":"2026-02-24T07:43:07.853069+0000","last_became_peered":"2026-02-24T07:43:07.853069+0000","last_unstale":"2026-02-24T07:45:00.689999+0000","last_undegraded":"2026-02-24T07:45:00.689999+0000","last_fullsized":"2026-02-24T07:45:00.689999+0000","mapping_epoch":30,"log_start":"40'46157","ondisk_log_start":"40'46157","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'36298","last_scrub_stamp":"2026-02-24T07:44:39.638986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_clean_scrub_stamp":"2026-02-24T07:44:39.638986+0000","objects_scrubbed":35656,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:46:09.128452+0000","scrub_duration":3228,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1695678464,"num_objects":25874,"num_object_clones":0,"num_object_copies":77622,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25874,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":46160,"num_write_kb":2305088,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":27024752664,"num_objects":412366,"num_object_clones":0,"num_object_copies":1237098,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":412366,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":735770,"num_write_kb":36740289,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":81099763712,"data_stored":81099751496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":50,"ondisk_log_size":50,"up":48,"acting":48,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":240,"num_write":135,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":3,"up_from":25,"seq":107374182428,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":24315536,"kb_used_data":23275388,"kb_used_omap":4,"kb_used_meta":1039995,"kb_avail":709687664,"statfs":{"total":751619276800,"available":726720167936,"internally_reserved":0,"allocated":23833997312,"data_stored":23833742600,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4448,"internal_metadata":1064955552},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":19,"seq":81604378654,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":19939916,"kb_used_data":18173176,"kb_used_omap":18,"kb_used_meta":1766701,"kb_avail":714063284,"statfs":{"total":751619276800,"available":731200802816,"internally_reserved":0,"allocated":18609332224,"data_stored":18609081584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":18551,"internal_metadata":1809102729},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542175,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":21909068,"kb_used_data":19997880,"kb_used_omap":20,"kb_used_meta":1911019,"kb_avail":712094132,"statfs":{"total":751619276800,"available":729184391168,"internally_reserved":0,"allocated":20477829120,"data_stored":20477577960,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":20504,"internal_metadata":1956884456},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705698,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":20156176,"kb_used_data":18371516,"kb_used_omap":24,"kb_used_meta":1784615,"kb_avail":713847024,"statfs":{"total":751619276800,"available":730979352576,"internally_reserved":0,"allocated":18812432384,"data_stored":18812177672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":25061,"internal_metadata":1827446299},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":18600169472,"data_stored":18600165400,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":20289097728,"data_stored":20289093656,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":18565169152,"data_stored":18565169152,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":23645327360,"data_stored":23645323288,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-24T07:45:02.276 INFO:tasks.cephadm.ceph_manager.ceph:increase pool size by 10 2026-02-24T07:45:02.276 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph osd pool set .mgr pg_num 91 2026-02-24T07:45:02.416 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:45:02.540 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:45:02.539+0000 7f908af5a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:45:02.541 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:45:02.539+0000 7f908af5a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:45:02.673 INFO:teuthology.orchestra.run.trial035.stderr:Error EPERM: crimson_allow_pg_split is false; pg_num increase denied for crimson pool 2026-02-24T07:45:02.710 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-24T07:45:03.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:45:02 trial035 ceph-mon[39284]: pgmap v75: 17 pgs: 17 active+clean; 25 GiB data, 82 GiB used, 2.7 TiB / 2.7 TiB avail; 0 B/s wr, 9.55k op/s 2026-02-24T07:45:03.205 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:45:02 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/3884471371' entity='client.admin' cmd={"prefix": "osd pool set", "pool": ".mgr", "var": "pg_num", "val": "91"} : dispatch 2026-02-24T07:45:03.563 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2026-02-24T07:45:03.696 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:45:03.820 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:45:03.818+0000 7f00a897c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:45:03.820 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:45:03.818+0000 7f00a897c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:45:03.834 INFO:teuthology.orchestra.run.trial035.stdout:osd.0: {} 2026-02-24T07:45:03.835 INFO:teuthology.orchestra.run.trial035.stderr:osd.0: osd_enable_op_tracker = '' 2026-02-24T07:45:03.848 INFO:teuthology.orchestra.run.trial035.stdout:osd.1: {} 2026-02-24T07:45:03.848 INFO:teuthology.orchestra.run.trial035.stderr:osd.1: osd_enable_op_tracker = '' 2026-02-24T07:45:03.860 INFO:teuthology.orchestra.run.trial035.stdout:osd.2: {} 2026-02-24T07:45:03.861 INFO:teuthology.orchestra.run.trial035.stderr:osd.2: osd_enable_op_tracker = '' 2026-02-24T07:45:03.873 INFO:teuthology.orchestra.run.trial035.stdout:osd.3: {} 2026-02-24T07:45:03.873 INFO:teuthology.orchestra.run.trial035.stderr:osd.3: osd_enable_op_tracker = '' 2026-02-24T07:45:04.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:45:03 trial035 ceph-mon[39284]: from='client.14511 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2026-02-24T07:45:04.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:45:03 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/4121089122' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2026-02-24T07:45:05.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:45:04 trial035 ceph-mon[39284]: pgmap v76: 17 pgs: 17 active+clean; 25 GiB data, 82 GiB used, 2.7 TiB / 2.7 TiB avail; 0 B/s wr, 6.72k op/s 2026-02-24T07:45:05.205 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:45:04 trial035 ceph-mon[39284]: purged_snaps scrub starts 2026-02-24T07:45:05.205 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:45:04 trial035 ceph-mon[39284]: purged_snaps scrub ok 2026-02-24T07:45:05.913 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2026-02-24T07:45:06.047 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:45:06.167 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:45:06.166+0000 7f46a1987640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:45:06.167 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:45:06.166+0000 7f46a1987640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:45:06.180 INFO:teuthology.orchestra.run.trial035.stdout:osd.0: {} 2026-02-24T07:45:06.180 INFO:teuthology.orchestra.run.trial035.stderr:osd.0: osd_enable_op_tracker = '' 2026-02-24T07:45:06.193 INFO:teuthology.orchestra.run.trial035.stdout:osd.1: {} 2026-02-24T07:45:06.193 INFO:teuthology.orchestra.run.trial035.stderr:osd.1: osd_enable_op_tracker = '' 2026-02-24T07:45:06.205 INFO:teuthology.orchestra.run.trial035.stdout:osd.2: {} 2026-02-24T07:45:06.205 INFO:teuthology.orchestra.run.trial035.stderr:osd.2: osd_enable_op_tracker = '' 2026-02-24T07:45:06.216 INFO:teuthology.orchestra.run.trial035.stdout:osd.3: {} 2026-02-24T07:45:06.216 INFO:teuthology.orchestra.run.trial035.stderr:osd.3: osd_enable_op_tracker = '' 2026-02-24T07:45:07.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:45:06 trial035 ceph-mon[39284]: pgmap v77: 17 pgs: 17 active+clean; 25 GiB data, 79 GiB used, 2.7 TiB / 2.7 TiB avail; 0 B/s wr, 6.90k op/s 2026-02-24T07:45:07.205 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:45:06 trial035 ceph-mon[39284]: from='mgr.14284 10.20.193.35:0/3681373222' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2026-02-24T07:45:07.205 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:45:06 trial035 ceph-mon[39284]: from='mgr.14284 10.20.193.35:0/3681373222' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-24T07:45:07.205 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:45:06 trial035 ceph-mon[39284]: from='mgr.14284 10.20.193.35:0/3681373222' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2026-02-24T07:45:07.205 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:45:06 trial035 ceph-mon[39284]: from='mgr.14284 10.20.193.35:0/3681373222' entity='mgr.x' 2026-02-24T07:45:07.205 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:45:06 trial035 ceph-mon[39284]: from='mgr.14284 10.20.193.35:0/3681373222' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2026-02-24T07:45:07.205 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:45:06 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/93769673' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2026-02-24T07:45:07.206 INFO:journalctl@ceph.mgr.x.trial035.stdout:Feb 24 07:45:06 trial035 ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-mgr-x[39621]: ::ffff:10.20.193.91 - - [24/Feb/2026:07:45:06] "GET /metrics HTTP/1.1" 200 4225 "" "Prometheus/3.6.0" 2026-02-24T07:45:07.356 INFO:journalctl@ceph.prometheus.a.trial091.stdout:Feb 24 07:45:06 trial091 ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-prometheus-a[66007]: time=2026-02-24T07:45:06.973Z level=ERROR source=scrape.go:1630 msg="Failed to determine correct type of scrape target." component="scrape manager" scrape_pool=ceph target=http://trial035:9283/metrics content_type="text/html;charset=utf-8" fallback_media_type="" err="received unsupported Content-Type \"text/html;charset=utf-8\" and no fallback_scrape_protocol specified for target" 2026-02-24T07:45:07.712 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2026-02-24T07:45:07.713 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2026-02-24T07:45:07.713 INFO:tasks.thrashosds.thrasher:Growing pool unique_pool_0 2026-02-24T07:45:07.713 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph pg dump --format=json 2026-02-24T07:45:07.848 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:45:08.254 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2026-02-24T07:45:08.366 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:45:08.346+0000 7f9c45e15640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:45:08.366 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:45:08.346+0000 7f9c45e15640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:45:08.383 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:45:08.485 INFO:teuthology.orchestra.run.trial035.stdout: 2026-02-24T07:45:08.485 INFO:teuthology.orchestra.run.trial035.stderr:dumped all 2026-02-24T07:45:08.954 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:45:08 trial035 ceph-mon[39284]: pgmap v78: 17 pgs: 17 active+clean; 23 GiB data, 74 GiB used, 2.7 TiB / 2.7 TiB avail; 0 B/s wr, 9.53k op/s 2026-02-24T07:45:09.272 INFO:teuthology.orchestra.run.trial035.stdout:{"pg_ready":true,"pg_map":{"version":78,"stamp":"2026-02-24T07:45:07.675126+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":24642912824,"num_objects":376015,"num_object_clones":0,"num_object_copies":1128045,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":376015,"num_whiteouts":0,"num_read":137,"num_read_kb":241,"num_write":772258,"num_write_kb":36742439,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":51,"ondisk_log_size":51,"up":51,"acting":51,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":51,"num_osds":4,"num_per_pool_osds":4,"num_per_pool_omap_osds":4,"kb":2936012800,"kb_used":77603384,"kb_used_data":72886440,"kb_used_omap":51,"kb_used_meta":4716748,"kb_avail":2858409416,"statfs":{"total":3006477107200,"available":2927011241984,"internally_reserved":0,"allocated":74635714560,"data_stored":74634703336,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":53157,"internal_metadata":4829950043},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[6,3],"upper_bound":4},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-7499284480,"num_objects":-114430,"num_object_clones":0,"num_object_copies":-343290,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-114430,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":114430,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001492"},"pg_stats":[{"pgid":"2.f","version":"40'47534","reported_seq":47868,"reported_epoch":40,"state":"active+clean","last_fresh":"2026-02-24T07:45:05.667653+0000","last_change":"2026-02-24T07:44:23.320237+0000","last_active":"2026-02-24T07:45:05.667653+0000","last_peered":"2026-02-24T07:45:05.667653+0000","last_clean":"2026-02-24T07:45:05.667653+0000","last_became_active":"2026-02-24T07:43:07.852899+0000","last_became_peered":"2026-02-24T07:43:07.852899+0000","last_unstale":"2026-02-24T07:45:05.667653+0000","last_undegraded":"2026-02-24T07:45:05.667653+0000","last_fullsized":"2026-02-24T07:45:05.667653+0000","mapping_epoch":30,"log_start":"40'47531","ondisk_log_start":"40'47531","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'30039","last_scrub_stamp":"2026-02-24T07:44:23.320202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_clean_scrub_stamp":"2026-02-24T07:44:23.320202+0000","objects_scrubbed":29558,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:45:24.615266+0000","scrub_duration":2598,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1525284864,"num_objects":23274,"num_object_clones":0,"num_object_copies":69822,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23274,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":47534,"num_write_kb":2265856,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.e","version":"40'48420","reported_seq":48734,"reported_epoch":40,"state":"active+clean","last_fresh":"2026-02-24T07:45:05.731469+0000","last_change":"2026-02-24T07:44:16.083399+0000","last_active":"2026-02-24T07:45:05.731469+0000","last_peered":"2026-02-24T07:45:05.731469+0000","last_clean":"2026-02-24T07:45:05.731469+0000","last_became_active":"2026-02-24T07:43:07.852815+0000","last_became_peered":"2026-02-24T07:43:07.852815+0000","last_unstale":"2026-02-24T07:45:05.731469+0000","last_undegraded":"2026-02-24T07:45:05.731469+0000","last_fullsized":"2026-02-24T07:45:05.731469+0000","mapping_epoch":30,"log_start":"40'48417","ondisk_log_start":"40'48417","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'27767","last_scrub_stamp":"2026-02-24T07:44:16.083320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_clean_scrub_stamp":"2026-02-24T07:44:16.083320+0000","objects_scrubbed":27311,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:45:28.111749+0000","scrub_duration":2943,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1542062080,"num_objects":23530,"num_object_clones":0,"num_object_copies":70590,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23530,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":48420,"num_write_kb":2302400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.d","version":"40'48624","reported_seq":48976,"reported_epoch":40,"state":"active+clean","last_fresh":"2026-02-24T07:45:05.735866+0000","last_change":"2026-02-24T07:44:26.596366+0000","last_active":"2026-02-24T07:45:05.735866+0000","last_peered":"2026-02-24T07:45:05.735866+0000","last_clean":"2026-02-24T07:45:05.735866+0000","last_became_active":"2026-02-24T07:43:07.853975+0000","last_became_peered":"2026-02-24T07:43:07.853975+0000","last_unstale":"2026-02-24T07:45:05.735866+0000","last_undegraded":"2026-02-24T07:45:05.735866+0000","last_fullsized":"2026-02-24T07:45:05.735866+0000","mapping_epoch":30,"log_start":"40'48621","ondisk_log_start":"40'48621","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'31656","last_scrub_stamp":"2026-02-24T07:44:26.596292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_clean_scrub_stamp":"2026-02-24T07:44:26.596292+0000","objects_scrubbed":31097,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:45:55.416603+0000","scrub_duration":3380,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1543831576,"num_objects":23558,"num_object_clones":0,"num_object_copies":70674,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23558,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":48624,"num_write_kb":2309761,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"40'48529","reported_seq":48887,"reported_epoch":40,"state":"active+clean","last_fresh":"2026-02-24T07:45:05.733749+0000","last_change":"2026-02-24T07:44:29.667393+0000","last_active":"2026-02-24T07:45:05.733749+0000","last_peered":"2026-02-24T07:45:05.733749+0000","last_clean":"2026-02-24T07:45:05.733749+0000","last_became_active":"2026-02-24T07:43:07.854428+0000","last_became_peered":"2026-02-24T07:43:07.854428+0000","last_unstale":"2026-02-24T07:45:05.733749+0000","last_undegraded":"2026-02-24T07:45:05.733749+0000","last_fullsized":"2026-02-24T07:45:05.733749+0000","mapping_epoch":30,"log_start":"40'48526","ondisk_log_start":"40'48526","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'32466","last_scrub_stamp":"2026-02-24T07:44:29.667348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_clean_scrub_stamp":"2026-02-24T07:44:29.667348+0000","objects_scrubbed":31970,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:45:43.092916+0000","scrub_duration":3552,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1553268736,"num_objects":23701,"num_object_clones":0,"num_object_copies":71103,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23701,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":48529,"num_write_kb":2311360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"29'99","reported_seq":210,"reported_epoch":40,"state":"active+clean","last_fresh":"2026-02-24T07:43:59.386388+0000","last_change":"2026-02-24T07:43:59.386353+0000","last_active":"2026-02-24T07:43:59.386388+0000","last_peered":"2026-02-24T07:43:59.386388+0000","last_clean":"2026-02-24T07:43:59.386388+0000","last_became_active":"2026-02-24T07:42:54.186719+0000","last_became_peered":"2026-02-24T07:42:54.186719+0000","last_unstale":"2026-02-24T07:43:59.386388+0000","last_undegraded":"2026-02-24T07:43:59.386388+0000","last_fullsized":"2026-02-24T07:43:59.386388+0000","mapping_epoch":27,"log_start":"29'96","ondisk_log_start":"29'96","created":20,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'99","last_scrub_stamp":"2026-02-24T07:43:59.386324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:42:43.640776+0000","last_clean_scrub_stamp":"2026-02-24T07:43:59.386324+0000","objects_scrubbed":2,"log_size":3,"log_dups_size":96,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:45:24.339819+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":240,"num_write":135,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"40'48486","reported_seq":48821,"reported_epoch":40,"state":"active+clean","last_fresh":"2026-02-24T07:45:05.691992+0000","last_change":"2026-02-24T07:44:22.665897+0000","last_active":"2026-02-24T07:45:05.691992+0000","last_peered":"2026-02-24T07:45:05.691992+0000","last_clean":"2026-02-24T07:45:05.691992+0000","last_became_active":"2026-02-24T07:43:07.842530+0000","last_became_peered":"2026-02-24T07:43:07.842530+0000","last_unstale":"2026-02-24T07:45:05.691992+0000","last_undegraded":"2026-02-24T07:45:05.691992+0000","last_fullsized":"2026-02-24T07:45:05.691992+0000","mapping_epoch":30,"log_start":"40'48483","ondisk_log_start":"40'48483","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'30210","last_scrub_stamp":"2026-02-24T07:44:22.665762+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_clean_scrub_stamp":"2026-02-24T07:44:22.665762+0000","objects_scrubbed":29682,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:45:26.371453+0000","scrub_duration":3226,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1548091392,"num_objects":23622,"num_object_clones":0,"num_object_copies":70866,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23622,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":48486,"num_write_kb":2307456,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"40'48043","reported_seq":48395,"reported_epoch":40,"state":"active+clean","last_fresh":"2026-02-24T07:45:05.694129+0000","last_change":"2026-02-24T07:44:28.446165+0000","last_active":"2026-02-24T07:45:05.694129+0000","last_peered":"2026-02-24T07:45:05.694129+0000","last_clean":"2026-02-24T07:45:05.694129+0000","last_became_active":"2026-02-24T07:43:07.853885+0000","last_became_peered":"2026-02-24T07:43:07.853885+0000","last_unstale":"2026-02-24T07:45:05.694129+0000","last_undegraded":"2026-02-24T07:45:05.694129+0000","last_fullsized":"2026-02-24T07:45:05.694129+0000","mapping_epoch":30,"log_start":"40'48040","ondisk_log_start":"40'48040","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'31685","last_scrub_stamp":"2026-02-24T07:44:28.446064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_clean_scrub_stamp":"2026-02-24T07:44:28.446064+0000","objects_scrubbed":31280,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:45:52.594601+0000","scrub_duration":3117,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1519190016,"num_objects":23181,"num_object_clones":0,"num_object_copies":69543,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23181,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":48043,"num_write_kb":2279168,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"40'48448","reported_seq":48819,"reported_epoch":40,"state":"active+clean","last_fresh":"2026-02-24T07:45:05.728300+0000","last_change":"2026-02-24T07:44:33.357960+0000","last_active":"2026-02-24T07:45:05.728300+0000","last_peered":"2026-02-24T07:45:05.728300+0000","last_clean":"2026-02-24T07:45:05.728300+0000","last_became_active":"2026-02-24T07:43:07.854212+0000","last_became_peered":"2026-02-24T07:43:07.854212+0000","last_unstale":"2026-02-24T07:45:05.728300+0000","last_undegraded":"2026-02-24T07:45:05.728300+0000","last_fullsized":"2026-02-24T07:45:05.728300+0000","mapping_epoch":30,"log_start":"40'48445","ondisk_log_start":"40'48445","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'33783","last_scrub_stamp":"2026-02-24T07:44:33.357894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_clean_scrub_stamp":"2026-02-24T07:44:33.357894+0000","objects_scrubbed":33200,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:45:38.914707+0000","scrub_duration":3247,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1552678912,"num_objects":23692,"num_object_clones":0,"num_object_copies":71076,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23692,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":48448,"num_write_kb":2308480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[1,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"40'48624","reported_seq":49015,"reported_epoch":40,"state":"active+clean","last_fresh":"2026-02-24T07:45:05.689409+0000","last_change":"2026-02-24T07:44:42.974303+0000","last_active":"2026-02-24T07:45:05.689409+0000","last_peered":"2026-02-24T07:45:05.689409+0000","last_clean":"2026-02-24T07:45:05.689409+0000","last_became_active":"2026-02-24T07:43:07.842864+0000","last_became_peered":"2026-02-24T07:43:07.842864+0000","last_unstale":"2026-02-24T07:45:05.689409+0000","last_undegraded":"2026-02-24T07:45:05.689409+0000","last_fullsized":"2026-02-24T07:45:05.689409+0000","mapping_epoch":30,"log_start":"40'48621","ondisk_log_start":"40'48621","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'37783","last_scrub_stamp":"2026-02-24T07:44:42.974258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_clean_scrub_stamp":"2026-02-24T07:44:42.974258+0000","objects_scrubbed":35124,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:45:49.565319+0000","scrub_duration":3593,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1535770624,"num_objects":23434,"num_object_clones":0,"num_object_copies":70302,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23434,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":48624,"num_write_kb":2305856,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.4","version":"40'48055","reported_seq":48370,"reported_epoch":40,"state":"active+clean","last_fresh":"2026-02-24T07:45:05.733057+0000","last_change":"2026-02-24T07:44:16.909673+0000","last_active":"2026-02-24T07:45:05.733057+0000","last_peered":"2026-02-24T07:45:05.733057+0000","last_clean":"2026-02-24T07:45:05.733057+0000","last_became_active":"2026-02-24T07:43:07.853966+0000","last_became_peered":"2026-02-24T07:43:07.853966+0000","last_unstale":"2026-02-24T07:45:05.733057+0000","last_undegraded":"2026-02-24T07:45:05.733057+0000","last_fullsized":"2026-02-24T07:45:05.733057+0000","mapping_epoch":30,"log_start":"40'48052","ondisk_log_start":"40'48052","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'27932","last_scrub_stamp":"2026-02-24T07:44:16.909624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_clean_scrub_stamp":"2026-02-24T07:44:16.909624+0000","objects_scrubbed":27480,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:45:43.624405+0000","scrub_duration":2774,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1542258688,"num_objects":23533,"num_object_clones":0,"num_object_copies":70599,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23533,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":48055,"num_write_kb":2290816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"40'48245","reported_seq":48624,"reported_epoch":40,"state":"active+clean","last_fresh":"2026-02-24T07:45:05.695157+0000","last_change":"2026-02-24T07:44:35.717081+0000","last_active":"2026-02-24T07:45:05.695157+0000","last_peered":"2026-02-24T07:45:05.695157+0000","last_clean":"2026-02-24T07:45:05.695157+0000","last_became_active":"2026-02-24T07:43:07.853878+0000","last_became_peered":"2026-02-24T07:43:07.853878+0000","last_unstale":"2026-02-24T07:45:05.695157+0000","last_undegraded":"2026-02-24T07:45:05.695157+0000","last_fullsized":"2026-02-24T07:45:05.695157+0000","mapping_epoch":30,"log_start":"40'48242","ondisk_log_start":"40'48242","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'34578","last_scrub_stamp":"2026-02-24T07:44:35.717022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_clean_scrub_stamp":"2026-02-24T07:44:35.717022+0000","objects_scrubbed":33884,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:46:00.809440+0000","scrub_duration":3376,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1532952576,"num_objects":23391,"num_object_clones":0,"num_object_copies":70173,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23391,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":48245,"num_write_kb":2292352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"40'48379","reported_seq":48739,"reported_epoch":40,"state":"active+clean","last_fresh":"2026-02-24T07:45:05.733247+0000","last_change":"2026-02-24T07:44:30.553008+0000","last_active":"2026-02-24T07:45:05.733247+0000","last_peered":"2026-02-24T07:45:05.733247+0000","last_clean":"2026-02-24T07:45:05.733247+0000","last_became_active":"2026-02-24T07:43:07.841993+0000","last_became_peered":"2026-02-24T07:43:07.841993+0000","last_unstale":"2026-02-24T07:45:05.733247+0000","last_undegraded":"2026-02-24T07:45:05.733247+0000","last_fullsized":"2026-02-24T07:45:05.733247+0000","mapping_epoch":30,"log_start":"40'48376","ondisk_log_start":"40'48376","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'32640","last_scrub_stamp":"2026-02-24T07:44:30.552965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_clean_scrub_stamp":"2026-02-24T07:44:30.552965+0000","objects_scrubbed":32121,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:45:56.537381+0000","scrub_duration":3414,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1535574016,"num_objects":23431,"num_object_clones":0,"num_object_copies":70293,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23431,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":48379,"num_write_kb":2297920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"40'47624","reported_seq":47941,"reported_epoch":40,"state":"active+clean","last_fresh":"2026-02-24T07:45:05.691659+0000","last_change":"2026-02-24T07:44:18.020443+0000","last_active":"2026-02-24T07:45:05.691659+0000","last_peered":"2026-02-24T07:45:05.691659+0000","last_clean":"2026-02-24T07:45:05.691659+0000","last_became_active":"2026-02-24T07:43:07.841920+0000","last_became_peered":"2026-02-24T07:43:07.841920+0000","last_unstale":"2026-02-24T07:45:05.691659+0000","last_undegraded":"2026-02-24T07:45:05.691659+0000","last_fullsized":"2026-02-24T07:45:05.691659+0000","mapping_epoch":30,"log_start":"40'47621","ondisk_log_start":"40'47621","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'28226","last_scrub_stamp":"2026-02-24T07:44:18.020379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_clean_scrub_stamp":"2026-02-24T07:44:18.020379+0000","objects_scrubbed":27705,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:45:27.051564+0000","scrub_duration":2601,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1540096000,"num_objects":23500,"num_object_clones":0,"num_object_copies":70500,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23500,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":47624,"num_write_kb":2275968,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.8","version":"40'47952","reported_seq":48311,"reported_epoch":40,"state":"active+clean","last_fresh":"2026-02-24T07:45:05.693640+0000","last_change":"2026-02-24T07:44:31.118780+0000","last_active":"2026-02-24T07:45:05.693640+0000","last_peered":"2026-02-24T07:45:05.693640+0000","last_clean":"2026-02-24T07:45:05.693640+0000","last_became_active":"2026-02-24T07:43:07.853758+0000","last_became_peered":"2026-02-24T07:43:07.853758+0000","last_unstale":"2026-02-24T07:45:05.693640+0000","last_undegraded":"2026-02-24T07:45:05.693640+0000","last_fullsized":"2026-02-24T07:45:05.693640+0000","mapping_epoch":30,"log_start":"40'47949","ondisk_log_start":"40'47949","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'32571","last_scrub_stamp":"2026-02-24T07:44:31.118720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_clean_scrub_stamp":"2026-02-24T07:44:31.118720+0000","objects_scrubbed":32077,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:45:44.647478+0000","scrub_duration":2775,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1533280256,"num_objects":23396,"num_object_clones":0,"num_object_copies":70188,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23396,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":47952,"num_write_kb":2283136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"40'48299","reported_seq":48694,"reported_epoch":40,"state":"active+clean","last_fresh":"2026-02-24T07:45:05.733607+0000","last_change":"2026-02-24T07:44:41.349757+0000","last_active":"2026-02-24T07:45:05.733607+0000","last_peered":"2026-02-24T07:45:05.733607+0000","last_clean":"2026-02-24T07:45:05.733607+0000","last_became_active":"2026-02-24T07:43:07.854377+0000","last_became_peered":"2026-02-24T07:43:07.854377+0000","last_unstale":"2026-02-24T07:45:05.733607+0000","last_undegraded":"2026-02-24T07:45:05.733607+0000","last_fullsized":"2026-02-24T07:45:05.733607+0000","mapping_epoch":30,"log_start":"40'48296","ondisk_log_start":"40'48296","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'36997","last_scrub_stamp":"2026-02-24T07:44:41.349228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_clean_scrub_stamp":"2026-02-24T07:44:41.349228+0000","objects_scrubbed":35574,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:45:50.611975+0000","scrub_duration":3205,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1553793024,"num_objects":23709,"num_object_clones":0,"num_object_copies":71127,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23709,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":48299,"num_write_kb":2304256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.a","version":"40'48409","reported_seq":48764,"reported_epoch":40,"state":"active+clean","last_fresh":"2026-02-24T07:45:05.731193+0000","last_change":"2026-02-24T07:44:27.902144+0000","last_active":"2026-02-24T07:45:05.731193+0000","last_peered":"2026-02-24T07:45:05.731193+0000","last_clean":"2026-02-24T07:45:05.731193+0000","last_became_active":"2026-02-24T07:43:07.841721+0000","last_became_peered":"2026-02-24T07:43:07.841721+0000","last_unstale":"2026-02-24T07:45:05.731193+0000","last_undegraded":"2026-02-24T07:45:05.731193+0000","last_fullsized":"2026-02-24T07:45:05.731193+0000","mapping_epoch":30,"log_start":"40'48406","ondisk_log_start":"40'48406","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'31794","last_scrub_stamp":"2026-02-24T07:44:27.902040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_clean_scrub_stamp":"2026-02-24T07:44:27.902040+0000","objects_scrubbed":31292,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:45:39.265782+0000","scrub_duration":3732,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1538719744,"num_objects":23479,"num_object_clones":0,"num_object_copies":70437,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23479,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":48409,"num_write_kb":2300416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.b","version":"40'48452","reported_seq":48847,"reported_epoch":40,"state":"active+clean","last_fresh":"2026-02-24T07:45:05.694107+0000","last_change":"2026-02-24T07:44:39.639056+0000","last_active":"2026-02-24T07:45:05.694107+0000","last_peered":"2026-02-24T07:45:05.694107+0000","last_clean":"2026-02-24T07:45:05.694107+0000","last_became_active":"2026-02-24T07:43:07.853069+0000","last_became_peered":"2026-02-24T07:43:07.853069+0000","last_unstale":"2026-02-24T07:45:05.694107+0000","last_undegraded":"2026-02-24T07:45:05.694107+0000","last_fullsized":"2026-02-24T07:45:05.694107+0000","mapping_epoch":30,"log_start":"40'48449","ondisk_log_start":"40'48449","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'36298","last_scrub_stamp":"2026-02-24T07:44:39.638986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_clean_scrub_stamp":"2026-02-24T07:44:39.638986+0000","objects_scrubbed":35656,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:46:09.128452+0000","scrub_duration":3228,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1545469952,"num_objects":23582,"num_object_clones":0,"num_object_copies":70746,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23582,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":48452,"num_write_kb":2305088,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":24642322456,"num_objects":376013,"num_object_clones":0,"num_object_copies":1128039,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":376013,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":772123,"num_write_kb":36740289,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":73955553280,"data_stored":73955541064,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":48,"ondisk_log_size":48,"up":48,"acting":48,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":240,"num_write":135,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":3,"up_from":25,"seq":107374182429,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":21623632,"kb_used_data":21278396,"kb_used_omap":4,"kb_used_meta":345211,"kb_avail":712379568,"statfs":{"total":751619276800,"available":729476677632,"internally_reserved":0,"allocated":21789077504,"data_stored":21788822792,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4448,"internal_metadata":353496736},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[4,2],"upper_bound":4},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":19,"seq":81604378655,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":18366924,"kb_used_data":16564088,"kb_used_omap":18,"kb_used_meta":1802797,"kb_avail":715636276,"statfs":{"total":751619276800,"available":732811546624,"internally_reserved":0,"allocated":16961626112,"data_stored":16961375472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19202,"internal_metadata":1846064382},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542176,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":19033676,"kb_used_data":18285624,"kb_used_omap":4,"kb_used_meta":748027,"kb_avail":714969524,"statfs":{"total":751619276800,"available":732128792576,"internally_reserved":0,"allocated":18724478976,"data_stored":18724227816,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4446,"internal_metadata":765980322},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1],"upper_bound":4},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705699,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":18579152,"kb_used_data":16758332,"kb_used_omap":24,"kb_used_meta":1820711,"kb_avail":715424048,"statfs":{"total":751619276800,"available":732594225152,"internally_reserved":0,"allocated":17160531968,"data_stored":17160277256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":25061,"internal_metadata":1864408603},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16960000000,"data_stored":16959995928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":18499309568,"data_stored":18499305496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16931618816,"data_stored":16931618816,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":21564624896,"data_stored":21564620824,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-24T07:45:09.273 INFO:tasks.cephadm.ceph_manager.ceph:increase pool size by 10 2026-02-24T07:45:09.274 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph osd pool set unique_pool_0 pg_num 46 2026-02-24T07:45:09.349 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:45:09.347+0000 7f9a22ef7640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:45:09.349 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:45:09.348+0000 7f9a22ef7640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:45:09.364 INFO:teuthology.orchestra.run.trial035.stdout:osd.0: {} 2026-02-24T07:45:09.364 INFO:teuthology.orchestra.run.trial035.stderr:osd.0: osd_enable_op_tracker = '' 2026-02-24T07:45:09.380 INFO:teuthology.orchestra.run.trial035.stdout:osd.1: {} 2026-02-24T07:45:09.380 INFO:teuthology.orchestra.run.trial035.stderr:osd.1: osd_enable_op_tracker = '' 2026-02-24T07:45:09.393 INFO:teuthology.orchestra.run.trial035.stdout:osd.2: {} 2026-02-24T07:45:09.394 INFO:teuthology.orchestra.run.trial035.stderr:osd.2: osd_enable_op_tracker = '' 2026-02-24T07:45:09.403 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:45:09.406 INFO:teuthology.orchestra.run.trial035.stdout:osd.3: {} 2026-02-24T07:45:09.406 INFO:teuthology.orchestra.run.trial035.stderr:osd.3: osd_enable_op_tracker = '' 2026-02-24T07:45:09.520 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:45:09.519+0000 7f5d53945640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:45:09.520 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:45:09.519+0000 7f5d53945640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:45:09.655 INFO:teuthology.orchestra.run.trial035.stderr:Error EPERM: crimson_allow_pg_split is false; pg_num increase denied for crimson pool 2026-02-24T07:45:09.691 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-24T07:45:10.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:45:09 trial035 ceph-mon[39284]: from='client.14519 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2026-02-24T07:45:10.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:45:09 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/4193700238' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2026-02-24T07:45:10.205 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:45:09 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/269687472' entity='client.admin' cmd={"prefix": "osd pool set", "pool": "unique_pool_0", "var": "pg_num", "val": "46"} : dispatch 2026-02-24T07:45:11.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:45:10 trial035 ceph-mon[39284]: pgmap v79: 17 pgs: 17 active+clean; 23 GiB data, 74 GiB used, 2.7 TiB / 2.7 TiB avail; 0 B/s wr, 6.42k op/s 2026-02-24T07:45:11.483 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2026-02-24T07:45:11.624 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:45:11.751 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:45:11.749+0000 7fe381e7b640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:45:11.752 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:45:11.750+0000 7fe381e7b640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:45:11.766 INFO:teuthology.orchestra.run.trial035.stdout:osd.0: {} 2026-02-24T07:45:11.767 INFO:teuthology.orchestra.run.trial035.stderr:osd.0: osd_enable_op_tracker = '' 2026-02-24T07:45:11.781 INFO:teuthology.orchestra.run.trial035.stdout:osd.1: {} 2026-02-24T07:45:11.781 INFO:teuthology.orchestra.run.trial035.stderr:osd.1: osd_enable_op_tracker = '' 2026-02-24T07:45:11.792 INFO:teuthology.orchestra.run.trial035.stdout:osd.2: {} 2026-02-24T07:45:11.793 INFO:teuthology.orchestra.run.trial035.stderr:osd.2: osd_enable_op_tracker = '' 2026-02-24T07:45:11.805 INFO:teuthology.orchestra.run.trial035.stdout:osd.3: {} 2026-02-24T07:45:11.806 INFO:teuthology.orchestra.run.trial035.stderr:osd.3: osd_enable_op_tracker = '' 2026-02-24T07:45:12.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:45:11 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/3648087834' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2026-02-24T07:45:13.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:45:12 trial035 ceph-mon[39284]: pgmap v80: 17 pgs: 17 active+clean; 21 GiB data, 68 GiB used, 2.7 TiB / 2.7 TiB avail; 0 B/s wr, 9.49k op/s 2026-02-24T07:45:13.860 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2026-02-24T07:45:13.995 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:45:14.118 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:45:14.116+0000 7fee13eb4640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:45:14.119 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:45:14.116+0000 7fee13eb4640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:45:14.133 INFO:teuthology.orchestra.run.trial035.stdout:osd.0: {} 2026-02-24T07:45:14.133 INFO:teuthology.orchestra.run.trial035.stderr:osd.0: osd_enable_op_tracker = '' 2026-02-24T07:45:14.145 INFO:teuthology.orchestra.run.trial035.stdout:osd.1: {} 2026-02-24T07:45:14.145 INFO:teuthology.orchestra.run.trial035.stderr:osd.1: osd_enable_op_tracker = '' 2026-02-24T07:45:14.157 INFO:teuthology.orchestra.run.trial035.stdout:osd.2: {} 2026-02-24T07:45:14.158 INFO:teuthology.orchestra.run.trial035.stderr:osd.2: osd_enable_op_tracker = '' 2026-02-24T07:45:14.170 INFO:teuthology.orchestra.run.trial035.stdout:osd.3: {} 2026-02-24T07:45:14.171 INFO:teuthology.orchestra.run.trial035.stderr:osd.3: osd_enable_op_tracker = '' 2026-02-24T07:45:14.692 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2026-02-24T07:45:14.693 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2026-02-24T07:45:14.693 INFO:tasks.thrashosds.thrasher:Growing pool unique_pool_0 2026-02-24T07:45:14.693 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph pg dump --format=json 2026-02-24T07:45:14.833 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:45:14.961 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:45:14.958+0000 7fcb2577d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:45:14.961 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:45:14.959+0000 7fcb2577d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:45:15.101 INFO:teuthology.orchestra.run.trial035.stdout: 2026-02-24T07:45:15.102 INFO:teuthology.orchestra.run.trial035.stderr:dumped all 2026-02-24T07:45:15.102 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:45:14 trial035 ceph-mon[39284]: pgmap v81: 17 pgs: 17 active+clean; 21 GiB data, 68 GiB used, 2.7 TiB / 2.7 TiB avail; 0 B/s wr, 6.10k op/s 2026-02-24T07:45:15.102 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:45:14 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/2955962849' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2026-02-24T07:45:15.145 INFO:teuthology.orchestra.run.trial035.stdout:{"pg_ready":true,"pg_map":{"version":81,"stamp":"2026-02-24T07:45:13.675919+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":22228369976,"num_objects":339172,"num_object_clones":0,"num_object_copies":1017516,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":339172,"num_whiteouts":0,"num_read":137,"num_read_kb":241,"num_write":809101,"num_write_kb":36742439,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":51,"ondisk_log_size":51,"up":51,"acting":51,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":51,"num_osds":4,"num_per_pool_osds":4,"num_per_pool_omap_osds":4,"kb":2936012800,"kb_used":70793592,"kb_used_data":65896040,"kb_used_omap":51,"kb_used_meta":4897228,"kb_avail":2865219208,"statfs":{"total":3006477107200,"available":2933984468992,"internally_reserved":0,"allocated":67477544960,"data_stored":67476533736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":53157,"internal_metadata":5014761563},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[6],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-4796973056,"num_objects":-73196,"num_object_clones":0,"num_object_copies":-219588,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-73196,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":73196,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001541"},"pg_stats":[{"pgid":"2.f","version":"40'49751","reported_seq":50083,"reported_epoch":40,"state":"active+clean","last_fresh":"2026-02-24T07:45:10.669814+0000","last_change":"2026-02-24T07:44:23.320237+0000","last_active":"2026-02-24T07:45:10.669814+0000","last_peered":"2026-02-24T07:45:10.669814+0000","last_clean":"2026-02-24T07:45:10.669814+0000","last_became_active":"2026-02-24T07:43:07.852899+0000","last_became_peered":"2026-02-24T07:43:07.852899+0000","last_unstale":"2026-02-24T07:45:10.669814+0000","last_undegraded":"2026-02-24T07:45:10.669814+0000","last_fullsized":"2026-02-24T07:45:10.669814+0000","mapping_epoch":30,"log_start":"40'49748","ondisk_log_start":"40'49748","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'30039","last_scrub_stamp":"2026-02-24T07:44:23.320202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_clean_scrub_stamp":"2026-02-24T07:44:23.320202+0000","objects_scrubbed":29558,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:45:24.615266+0000","scrub_duration":2598,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1379991552,"num_objects":21057,"num_object_clones":0,"num_object_copies":63171,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21057,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":49751,"num_write_kb":2265856,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.e","version":"40'50781","reported_seq":51095,"reported_epoch":40,"state":"active+clean","last_fresh":"2026-02-24T07:45:10.732986+0000","last_change":"2026-02-24T07:44:16.083399+0000","last_active":"2026-02-24T07:45:10.732986+0000","last_peered":"2026-02-24T07:45:10.732986+0000","last_clean":"2026-02-24T07:45:10.732986+0000","last_became_active":"2026-02-24T07:43:07.852815+0000","last_became_peered":"2026-02-24T07:43:07.852815+0000","last_unstale":"2026-02-24T07:45:10.732986+0000","last_undegraded":"2026-02-24T07:45:10.732986+0000","last_fullsized":"2026-02-24T07:45:10.732986+0000","mapping_epoch":30,"log_start":"40'50778","ondisk_log_start":"40'50778","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'27767","last_scrub_stamp":"2026-02-24T07:44:16.083320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_clean_scrub_stamp":"2026-02-24T07:44:16.083320+0000","objects_scrubbed":27311,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:45:28.111749+0000","scrub_duration":2943,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1387331584,"num_objects":21169,"num_object_clones":0,"num_object_copies":63507,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21169,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":50781,"num_write_kb":2302400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.d","version":"40'50938","reported_seq":51290,"reported_epoch":40,"state":"active+clean","last_fresh":"2026-02-24T07:45:10.728766+0000","last_change":"2026-02-24T07:44:26.596366+0000","last_active":"2026-02-24T07:45:10.728766+0000","last_peered":"2026-02-24T07:45:10.728766+0000","last_clean":"2026-02-24T07:45:10.728766+0000","last_became_active":"2026-02-24T07:43:07.853975+0000","last_became_peered":"2026-02-24T07:43:07.853975+0000","last_unstale":"2026-02-24T07:45:10.728766+0000","last_undegraded":"2026-02-24T07:45:10.728766+0000","last_fullsized":"2026-02-24T07:45:10.728766+0000","mapping_epoch":30,"log_start":"40'50935","ondisk_log_start":"40'50935","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'31656","last_scrub_stamp":"2026-02-24T07:44:26.596292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_clean_scrub_stamp":"2026-02-24T07:44:26.596292+0000","objects_scrubbed":31097,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:45:55.416603+0000","scrub_duration":3380,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1392181272,"num_objects":21244,"num_object_clones":0,"num_object_copies":63732,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21244,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":50938,"num_write_kb":2309761,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"40'50895","reported_seq":51253,"reported_epoch":40,"state":"active+clean","last_fresh":"2026-02-24T07:45:10.737767+0000","last_change":"2026-02-24T07:44:29.667393+0000","last_active":"2026-02-24T07:45:10.737767+0000","last_peered":"2026-02-24T07:45:10.737767+0000","last_clean":"2026-02-24T07:45:10.737767+0000","last_became_active":"2026-02-24T07:43:07.854428+0000","last_became_peered":"2026-02-24T07:43:07.854428+0000","last_unstale":"2026-02-24T07:45:10.737767+0000","last_undegraded":"2026-02-24T07:45:10.737767+0000","last_fullsized":"2026-02-24T07:45:10.737767+0000","mapping_epoch":30,"log_start":"40'50892","ondisk_log_start":"40'50892","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'32466","last_scrub_stamp":"2026-02-24T07:44:29.667348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_clean_scrub_stamp":"2026-02-24T07:44:29.667348+0000","objects_scrubbed":31970,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:45:43.092916+0000","scrub_duration":3552,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1398210560,"num_objects":21335,"num_object_clones":0,"num_object_copies":64005,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21335,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":50895,"num_write_kb":2311360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"29'99","reported_seq":210,"reported_epoch":40,"state":"active+clean","last_fresh":"2026-02-24T07:43:59.386388+0000","last_change":"2026-02-24T07:43:59.386353+0000","last_active":"2026-02-24T07:43:59.386388+0000","last_peered":"2026-02-24T07:43:59.386388+0000","last_clean":"2026-02-24T07:43:59.386388+0000","last_became_active":"2026-02-24T07:42:54.186719+0000","last_became_peered":"2026-02-24T07:42:54.186719+0000","last_unstale":"2026-02-24T07:43:59.386388+0000","last_undegraded":"2026-02-24T07:43:59.386388+0000","last_fullsized":"2026-02-24T07:43:59.386388+0000","mapping_epoch":27,"log_start":"29'96","ondisk_log_start":"29'96","created":20,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'99","last_scrub_stamp":"2026-02-24T07:43:59.386324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:42:43.640776+0000","last_clean_scrub_stamp":"2026-02-24T07:43:59.386324+0000","objects_scrubbed":2,"log_size":3,"log_dups_size":96,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:45:24.339819+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":240,"num_write":135,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"40'50783","reported_seq":51118,"reported_epoch":40,"state":"active+clean","last_fresh":"2026-02-24T07:45:10.696218+0000","last_change":"2026-02-24T07:44:22.665897+0000","last_active":"2026-02-24T07:45:10.696218+0000","last_peered":"2026-02-24T07:45:10.696218+0000","last_clean":"2026-02-24T07:45:10.696218+0000","last_became_active":"2026-02-24T07:43:07.842530+0000","last_became_peered":"2026-02-24T07:43:07.842530+0000","last_unstale":"2026-02-24T07:45:10.696218+0000","last_undegraded":"2026-02-24T07:45:10.696218+0000","last_fullsized":"2026-02-24T07:45:10.696218+0000","mapping_epoch":30,"log_start":"40'50780","ondisk_log_start":"40'50780","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'30210","last_scrub_stamp":"2026-02-24T07:44:22.665762+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_clean_scrub_stamp":"2026-02-24T07:44:22.665762+0000","objects_scrubbed":29682,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:45:26.371453+0000","scrub_duration":3226,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1397555200,"num_objects":21325,"num_object_clones":0,"num_object_copies":63975,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21325,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":50783,"num_write_kb":2307456,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"40'50275","reported_seq":50627,"reported_epoch":40,"state":"active+clean","last_fresh":"2026-02-24T07:45:10.689638+0000","last_change":"2026-02-24T07:44:28.446165+0000","last_active":"2026-02-24T07:45:10.689638+0000","last_peered":"2026-02-24T07:45:10.689638+0000","last_clean":"2026-02-24T07:45:10.689638+0000","last_became_active":"2026-02-24T07:43:07.853885+0000","last_became_peered":"2026-02-24T07:43:07.853885+0000","last_unstale":"2026-02-24T07:45:10.689638+0000","last_undegraded":"2026-02-24T07:45:10.689638+0000","last_fullsized":"2026-02-24T07:45:10.689638+0000","mapping_epoch":30,"log_start":"40'50272","ondisk_log_start":"40'50272","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'31685","last_scrub_stamp":"2026-02-24T07:44:28.446064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_clean_scrub_stamp":"2026-02-24T07:44:28.446064+0000","objects_scrubbed":31280,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:45:52.594601+0000","scrub_duration":3117,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1372913664,"num_objects":20949,"num_object_clones":0,"num_object_copies":62847,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20949,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":50275,"num_write_kb":2279168,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"40'50750","reported_seq":51121,"reported_epoch":40,"state":"active+clean","last_fresh":"2026-02-24T07:45:10.737952+0000","last_change":"2026-02-24T07:44:33.357960+0000","last_active":"2026-02-24T07:45:10.737952+0000","last_peered":"2026-02-24T07:45:10.737952+0000","last_clean":"2026-02-24T07:45:10.737952+0000","last_became_active":"2026-02-24T07:43:07.854212+0000","last_became_peered":"2026-02-24T07:43:07.854212+0000","last_unstale":"2026-02-24T07:45:10.737952+0000","last_undegraded":"2026-02-24T07:45:10.737952+0000","last_fullsized":"2026-02-24T07:45:10.737952+0000","mapping_epoch":30,"log_start":"40'50747","ondisk_log_start":"40'50747","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'33783","last_scrub_stamp":"2026-02-24T07:44:33.357894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_clean_scrub_stamp":"2026-02-24T07:44:33.357894+0000","objects_scrubbed":33200,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:45:38.914707+0000","scrub_duration":3247,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1401815040,"num_objects":21390,"num_object_clones":0,"num_object_copies":64170,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21390,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":50750,"num_write_kb":2308480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[1,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"40'50990","reported_seq":51381,"reported_epoch":40,"state":"active+clean","last_fresh":"2026-02-24T07:45:10.689605+0000","last_change":"2026-02-24T07:44:42.974303+0000","last_active":"2026-02-24T07:45:10.689605+0000","last_peered":"2026-02-24T07:45:10.689605+0000","last_clean":"2026-02-24T07:45:10.689605+0000","last_became_active":"2026-02-24T07:43:07.842864+0000","last_became_peered":"2026-02-24T07:43:07.842864+0000","last_unstale":"2026-02-24T07:45:10.689605+0000","last_undegraded":"2026-02-24T07:45:10.689605+0000","last_fullsized":"2026-02-24T07:45:10.689605+0000","mapping_epoch":30,"log_start":"40'50987","ondisk_log_start":"40'50987","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'37783","last_scrub_stamp":"2026-02-24T07:44:42.974258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_clean_scrub_stamp":"2026-02-24T07:44:42.974258+0000","objects_scrubbed":35124,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:45:49.565319+0000","scrub_duration":3593,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1380712448,"num_objects":21068,"num_object_clones":0,"num_object_copies":63204,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21068,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":50990,"num_write_kb":2305856,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.4","version":"40'50348","reported_seq":50663,"reported_epoch":40,"state":"active+clean","last_fresh":"2026-02-24T07:45:10.735707+0000","last_change":"2026-02-24T07:44:16.909673+0000","last_active":"2026-02-24T07:45:10.735707+0000","last_peered":"2026-02-24T07:45:10.735707+0000","last_clean":"2026-02-24T07:45:10.735707+0000","last_became_active":"2026-02-24T07:43:07.853966+0000","last_became_peered":"2026-02-24T07:43:07.853966+0000","last_unstale":"2026-02-24T07:45:10.735707+0000","last_undegraded":"2026-02-24T07:45:10.735707+0000","last_fullsized":"2026-02-24T07:45:10.735707+0000","mapping_epoch":30,"log_start":"40'50345","ondisk_log_start":"40'50345","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'27932","last_scrub_stamp":"2026-02-24T07:44:16.909624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_clean_scrub_stamp":"2026-02-24T07:44:16.909624+0000","objects_scrubbed":27480,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:45:43.624405+0000","scrub_duration":2774,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1391984640,"num_objects":21240,"num_object_clones":0,"num_object_copies":63720,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21240,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":50348,"num_write_kb":2290816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"40'50469","reported_seq":50848,"reported_epoch":40,"state":"active+clean","last_fresh":"2026-02-24T07:45:10.691038+0000","last_change":"2026-02-24T07:44:35.717081+0000","last_active":"2026-02-24T07:45:10.691038+0000","last_peered":"2026-02-24T07:45:10.691038+0000","last_clean":"2026-02-24T07:45:10.691038+0000","last_became_active":"2026-02-24T07:43:07.853878+0000","last_became_peered":"2026-02-24T07:43:07.853878+0000","last_unstale":"2026-02-24T07:45:10.691038+0000","last_undegraded":"2026-02-24T07:45:10.691038+0000","last_fullsized":"2026-02-24T07:45:10.691038+0000","mapping_epoch":30,"log_start":"40'50466","ondisk_log_start":"40'50466","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'34578","last_scrub_stamp":"2026-02-24T07:44:35.717022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_clean_scrub_stamp":"2026-02-24T07:44:35.717022+0000","objects_scrubbed":33884,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:46:00.809440+0000","scrub_duration":3376,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1387200512,"num_objects":21167,"num_object_clones":0,"num_object_copies":63501,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21167,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":50469,"num_write_kb":2292352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"40'50676","reported_seq":51035,"reported_epoch":40,"state":"active+clean","last_fresh":"2026-02-24T07:45:10.737881+0000","last_change":"2026-02-24T07:44:30.553008+0000","last_active":"2026-02-24T07:45:10.737881+0000","last_peered":"2026-02-24T07:45:10.737881+0000","last_clean":"2026-02-24T07:45:10.737881+0000","last_became_active":"2026-02-24T07:43:07.841993+0000","last_became_peered":"2026-02-24T07:43:07.841993+0000","last_unstale":"2026-02-24T07:45:10.737881+0000","last_undegraded":"2026-02-24T07:45:10.737881+0000","last_fullsized":"2026-02-24T07:45:10.737881+0000","mapping_epoch":30,"log_start":"40'50673","ondisk_log_start":"40'50673","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'32640","last_scrub_stamp":"2026-02-24T07:44:30.552965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_clean_scrub_stamp":"2026-02-24T07:44:30.552965+0000","objects_scrubbed":32121,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:45:56.537381+0000","scrub_duration":3414,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1385037824,"num_objects":21134,"num_object_clones":0,"num_object_copies":63402,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21134,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":50676,"num_write_kb":2297920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"40'49942","reported_seq":50259,"reported_epoch":40,"state":"active+clean","last_fresh":"2026-02-24T07:45:10.694468+0000","last_change":"2026-02-24T07:44:18.020443+0000","last_active":"2026-02-24T07:45:10.694468+0000","last_peered":"2026-02-24T07:45:10.694468+0000","last_clean":"2026-02-24T07:45:10.694468+0000","last_became_active":"2026-02-24T07:43:07.841920+0000","last_became_peered":"2026-02-24T07:43:07.841920+0000","last_unstale":"2026-02-24T07:45:10.694468+0000","last_undegraded":"2026-02-24T07:45:10.694468+0000","last_fullsized":"2026-02-24T07:45:10.694468+0000","mapping_epoch":30,"log_start":"40'49939","ondisk_log_start":"40'49939","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'28226","last_scrub_stamp":"2026-02-24T07:44:18.020379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_clean_scrub_stamp":"2026-02-24T07:44:18.020379+0000","objects_scrubbed":27705,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:45:27.051564+0000","scrub_duration":2601,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1388183552,"num_objects":21182,"num_object_clones":0,"num_object_copies":63546,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21182,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":49942,"num_write_kb":2275968,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.8","version":"40'50215","reported_seq":50573,"reported_epoch":40,"state":"active+clean","last_fresh":"2026-02-24T07:45:10.692397+0000","last_change":"2026-02-24T07:44:31.118780+0000","last_active":"2026-02-24T07:45:10.692397+0000","last_peered":"2026-02-24T07:45:10.692397+0000","last_clean":"2026-02-24T07:45:10.692397+0000","last_became_active":"2026-02-24T07:43:07.853758+0000","last_became_peered":"2026-02-24T07:43:07.853758+0000","last_unstale":"2026-02-24T07:45:10.692397+0000","last_undegraded":"2026-02-24T07:45:10.692397+0000","last_fullsized":"2026-02-24T07:45:10.692397+0000","mapping_epoch":30,"log_start":"40'50212","ondisk_log_start":"40'50212","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'32571","last_scrub_stamp":"2026-02-24T07:44:31.118720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_clean_scrub_stamp":"2026-02-24T07:44:31.118720+0000","objects_scrubbed":32077,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:45:44.647478+0000","scrub_duration":2775,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1384972288,"num_objects":21133,"num_object_clones":0,"num_object_copies":63399,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21133,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":50215,"num_write_kb":2283136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"40'50666","reported_seq":51061,"reported_epoch":40,"state":"active+clean","last_fresh":"2026-02-24T07:45:10.733423+0000","last_change":"2026-02-24T07:44:41.349757+0000","last_active":"2026-02-24T07:45:10.733423+0000","last_peered":"2026-02-24T07:45:10.733423+0000","last_clean":"2026-02-24T07:45:10.733423+0000","last_became_active":"2026-02-24T07:43:07.854377+0000","last_became_peered":"2026-02-24T07:43:07.854377+0000","last_unstale":"2026-02-24T07:45:10.733423+0000","last_undegraded":"2026-02-24T07:45:10.733423+0000","last_fullsized":"2026-02-24T07:45:10.733423+0000","mapping_epoch":30,"log_start":"40'50663","ondisk_log_start":"40'50663","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'36997","last_scrub_stamp":"2026-02-24T07:44:41.349228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_clean_scrub_stamp":"2026-02-24T07:44:41.349228+0000","objects_scrubbed":35574,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:45:50.611975+0000","scrub_duration":3205,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1398669312,"num_objects":21342,"num_object_clones":0,"num_object_copies":64026,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21342,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":50666,"num_write_kb":2304256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.a","version":"40'50759","reported_seq":51114,"reported_epoch":40,"state":"active+clean","last_fresh":"2026-02-24T07:45:10.737620+0000","last_change":"2026-02-24T07:44:27.902144+0000","last_active":"2026-02-24T07:45:10.737620+0000","last_peered":"2026-02-24T07:45:10.737620+0000","last_clean":"2026-02-24T07:45:10.737620+0000","last_became_active":"2026-02-24T07:43:07.841721+0000","last_became_peered":"2026-02-24T07:43:07.841721+0000","last_unstale":"2026-02-24T07:45:10.737620+0000","last_undegraded":"2026-02-24T07:45:10.737620+0000","last_fullsized":"2026-02-24T07:45:10.737620+0000","mapping_epoch":30,"log_start":"40'50756","ondisk_log_start":"40'50756","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'31794","last_scrub_stamp":"2026-02-24T07:44:27.902040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_clean_scrub_stamp":"2026-02-24T07:44:27.902040+0000","objects_scrubbed":31292,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:45:39.265782+0000","scrub_duration":3732,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1384710144,"num_objects":21129,"num_object_clones":0,"num_object_copies":63387,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21129,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":50759,"num_write_kb":2300416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.b","version":"40'50728","reported_seq":51123,"reported_epoch":40,"state":"active+clean","last_fresh":"2026-02-24T07:45:10.692257+0000","last_change":"2026-02-24T07:44:39.639056+0000","last_active":"2026-02-24T07:45:10.692257+0000","last_peered":"2026-02-24T07:45:10.692257+0000","last_clean":"2026-02-24T07:45:10.692257+0000","last_became_active":"2026-02-24T07:43:07.853069+0000","last_became_peered":"2026-02-24T07:43:07.853069+0000","last_unstale":"2026-02-24T07:45:10.692257+0000","last_undegraded":"2026-02-24T07:45:10.692257+0000","last_fullsized":"2026-02-24T07:45:10.692257+0000","mapping_epoch":30,"log_start":"40'50725","ondisk_log_start":"40'50725","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'36298","last_scrub_stamp":"2026-02-24T07:44:39.638986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_clean_scrub_stamp":"2026-02-24T07:44:39.638986+0000","objects_scrubbed":35656,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:46:09.128452+0000","scrub_duration":3228,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1396310016,"num_objects":21306,"num_object_clones":0,"num_object_copies":63918,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21306,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":50728,"num_write_kb":2305088,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":22227779608,"num_objects":339170,"num_object_clones":0,"num_object_copies":1017510,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":339170,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":808966,"num_write_kb":36740289,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":66716119040,"data_stored":66716106824,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":48,"ondisk_log_size":48,"up":48,"acting":48,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":240,"num_write":135,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":3,"up_from":25,"seq":107374182430,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":19640080,"kb_used_data":19240636,"kb_used_omap":4,"kb_used_meta":399355,"kb_avail":714363120,"statfs":{"total":751619276800,"available":731507834880,"internally_reserved":0,"allocated":19702411264,"data_stored":19702156552,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4448,"internal_metadata":408940192},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":19,"seq":81604378656,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":16830156,"kb_used_data":14973112,"kb_used_omap":18,"kb_used_meta":1856941,"kb_avail":717173044,"statfs":{"total":751619276800,"available":734385197056,"internally_reserved":0,"allocated":15332466688,"data_stored":15332216048,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19202,"internal_metadata":1901507838},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542177,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":17275020,"kb_used_data":16490808,"kb_used_omap":4,"kb_used_meta":784123,"kb_avail":716728180,"statfs":{"total":751619276800,"available":733929656320,"internally_reserved":0,"allocated":16886587392,"data_stored":16886336232,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4446,"internal_metadata":802942626},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[6],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705700,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":17048336,"kb_used_data":15191484,"kb_used_omap":24,"kb_used_meta":1856807,"kb_avail":716954864,"statfs":{"total":751619276800,"available":734161780736,"internally_reserved":0,"allocated":15556079616,"data_stored":15555824904,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":25061,"internal_metadata":1901370907},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":15308623872,"data_stored":15308619800,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16673738752,"data_stored":16673734680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":15278735360,"data_stored":15278735360,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":19455021056,"data_stored":19455016984,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-24T07:45:15.147 INFO:tasks.cephadm.ceph_manager.ceph:increase pool size by 10 2026-02-24T07:45:15.148 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph osd pool set unique_pool_0 pg_num 56 2026-02-24T07:45:15.285 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:45:15.411 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:45:15.408+0000 7f2d3a0ab640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:45:15.411 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:45:15.409+0000 7f2d3a0ab640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:45:15.554 INFO:teuthology.orchestra.run.trial035.stderr:Error EPERM: crimson_allow_pg_split is false; pg_num increase denied for crimson pool 2026-02-24T07:45:15.591 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-24T07:45:16.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:45:15 trial035 ceph-mon[39284]: from='mgr.14284 10.20.193.35:0/3681373222' entity='mgr.x' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2026-02-24T07:45:16.205 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:45:15 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/1029423060' entity='client.admin' cmd={"prefix": "osd pool set", "pool": "unique_pool_0", "var": "pg_num", "val": "56"} : dispatch 2026-02-24T07:45:16.209 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2026-02-24T07:45:16.347 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:45:16.466 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:45:16.463+0000 7f88da9df640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:45:16.466 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:45:16.464+0000 7f88da9df640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:45:16.482 INFO:teuthology.orchestra.run.trial035.stdout:osd.0: {} 2026-02-24T07:45:16.482 INFO:teuthology.orchestra.run.trial035.stderr:osd.0: osd_enable_op_tracker = '' 2026-02-24T07:45:16.497 INFO:teuthology.orchestra.run.trial035.stdout:osd.1: {} 2026-02-24T07:45:16.497 INFO:teuthology.orchestra.run.trial035.stderr:osd.1: osd_enable_op_tracker = '' 2026-02-24T07:45:16.510 INFO:teuthology.orchestra.run.trial035.stdout:osd.2: {} 2026-02-24T07:45:16.510 INFO:teuthology.orchestra.run.trial035.stderr:osd.2: osd_enable_op_tracker = '' 2026-02-24T07:45:16.523 INFO:teuthology.orchestra.run.trial035.stdout:osd.3: {} 2026-02-24T07:45:16.523 INFO:teuthology.orchestra.run.trial035.stderr:osd.3: osd_enable_op_tracker = '' 2026-02-24T07:45:17.204 INFO:journalctl@ceph.mgr.x.trial035.stdout:Feb 24 07:45:16 trial035 ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-mgr-x[39621]: ::ffff:10.20.193.91 - - [24/Feb/2026:07:45:16] "GET /metrics HTTP/1.1" 200 4231 "" "Prometheus/3.6.0" 2026-02-24T07:45:17.205 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:45:16 trial035 ceph-mon[39284]: from='client.14529 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2026-02-24T07:45:17.205 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:45:16 trial035 ceph-mon[39284]: pgmap v82: 17 pgs: 17 active+clean; 21 GiB data, 62 GiB used, 2.7 TiB / 2.7 TiB avail; 0 B/s wr, 6.33k op/s 2026-02-24T07:45:17.205 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:45:16 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/632006965' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2026-02-24T07:45:17.264 INFO:journalctl@ceph.prometheus.a.trial091.stdout:Feb 24 07:45:16 trial091 ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-prometheus-a[66007]: time=2026-02-24T07:45:16.972Z level=ERROR source=scrape.go:1630 msg="Failed to determine correct type of scrape target." component="scrape manager" scrape_pool=ceph target=http://trial035:9283/metrics content_type="text/html;charset=utf-8" fallback_media_type="" err="received unsupported Content-Type \"text/html;charset=utf-8\" and no fallback_scrape_protocol specified for target" 2026-02-24T07:45:18.566 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2026-02-24T07:45:18.705 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:45:18.818 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:45:18.815+0000 7fefd4176640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:45:18.818 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:45:18.815+0000 7fefd4176640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:45:18.833 INFO:teuthology.orchestra.run.trial035.stdout:osd.0: {} 2026-02-24T07:45:18.833 INFO:teuthology.orchestra.run.trial035.stderr:osd.0: osd_enable_op_tracker = '' 2026-02-24T07:45:18.850 INFO:teuthology.orchestra.run.trial035.stdout:osd.1: {} 2026-02-24T07:45:18.850 INFO:teuthology.orchestra.run.trial035.stderr:osd.1: osd_enable_op_tracker = '' 2026-02-24T07:45:18.861 INFO:teuthology.orchestra.run.trial035.stdout:osd.2: {} 2026-02-24T07:45:18.862 INFO:teuthology.orchestra.run.trial035.stderr:osd.2: osd_enable_op_tracker = '' 2026-02-24T07:45:18.873 INFO:teuthology.orchestra.run.trial035.stdout:osd.3: {} 2026-02-24T07:45:18.873 INFO:teuthology.orchestra.run.trial035.stderr:osd.3: osd_enable_op_tracker = '' 2026-02-24T07:45:19.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:45:18 trial035 ceph-mon[39284]: pgmap v83: 17 pgs: 17 active+clean; 18 GiB data, 58 GiB used, 2.7 TiB / 2.7 TiB avail; 0 B/s wr, 9.49k op/s 2026-02-24T07:45:19.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:45:18 trial035 ceph-mon[39284]: purged_snaps scrub starts 2026-02-24T07:45:19.205 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:45:18 trial035 ceph-mon[39284]: purged_snaps scrub ok 2026-02-24T07:45:19.205 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:45:18 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/4004806921' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2026-02-24T07:45:20.591 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2026-02-24T07:45:20.591 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2026-02-24T07:45:20.592 INFO:tasks.thrashosds.thrasher:Growing pool unique_pool_0 2026-02-24T07:45:20.592 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph pg dump --format=json 2026-02-24T07:45:20.728 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:45:20.848 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:45:20.846+0000 7f4758b49640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:45:20.848 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:45:20.846+0000 7f4758b49640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:45:20.936 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2026-02-24T07:45:20.954 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:45:20 trial035 ceph-mon[39284]: pgmap v84: 17 pgs: 17 active+clean; 18 GiB data, 58 GiB used, 2.7 TiB / 2.7 TiB avail; 0 B/s wr, 6.65k op/s 2026-02-24T07:45:20.973 INFO:teuthology.orchestra.run.trial035.stdout: 2026-02-24T07:45:20.974 INFO:teuthology.orchestra.run.trial035.stderr:dumped all 2026-02-24T07:45:21.015 INFO:teuthology.orchestra.run.trial035.stdout:{"pg_ready":true,"pg_map":{"version":84,"stamp":"2026-02-24T07:45:19.676670+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19411894840,"num_objects":296196,"num_object_clones":0,"num_object_copies":888588,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":296196,"num_whiteouts":0,"num_read":137,"num_read_kb":241,"num_write":852077,"num_write_kb":36742439,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":51,"ondisk_log_size":51,"up":51,"acting":51,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":51,"num_osds":4,"num_per_pool_osds":4,"num_per_pool_omap_osds":4,"kb":2936012800,"kb_used":60328312,"kb_used_data":57633000,"kb_used_omap":17,"kb_used_meta":2695150,"kb_avail":2875684488,"statfs":{"total":3006477107200,"available":2944700915712,"internally_reserved":0,"allocated":59016192000,"data_stored":59015180776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":17793,"internal_metadata":2759834239},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[6],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-5231017984,"num_objects":-79819,"num_object_clones":0,"num_object_copies":-239457,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-79819,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":79819,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001544"},"pg_stats":[{"pgid":"2.f","version":"40'52521","reported_seq":52855,"reported_epoch":40,"state":"active+clean","last_fresh":"2026-02-24T07:45:15.666942+0000","last_change":"2026-02-24T07:44:23.320237+0000","last_active":"2026-02-24T07:45:15.666942+0000","last_peered":"2026-02-24T07:45:15.666942+0000","last_clean":"2026-02-24T07:45:15.666942+0000","last_became_active":"2026-02-24T07:43:07.852899+0000","last_became_peered":"2026-02-24T07:43:07.852899+0000","last_unstale":"2026-02-24T07:45:15.666942+0000","last_undegraded":"2026-02-24T07:45:15.666942+0000","last_fullsized":"2026-02-24T07:45:15.666942+0000","mapping_epoch":30,"log_start":"40'52518","ondisk_log_start":"40'52518","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'30039","last_scrub_stamp":"2026-02-24T07:44:23.320202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_clean_scrub_stamp":"2026-02-24T07:44:23.320202+0000","objects_scrubbed":29558,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:45:24.615266+0000","scrub_duration":2598,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1198456832,"num_objects":18287,"num_object_clones":0,"num_object_copies":54861,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18287,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":52521,"num_write_kb":2265856,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.e","version":"40'53508","reported_seq":53821,"reported_epoch":40,"state":"active+clean","last_fresh":"2026-02-24T07:45:15.737931+0000","last_change":"2026-02-24T07:44:16.083399+0000","last_active":"2026-02-24T07:45:15.737931+0000","last_peered":"2026-02-24T07:45:15.737931+0000","last_clean":"2026-02-24T07:45:15.737931+0000","last_became_active":"2026-02-24T07:43:07.852815+0000","last_became_peered":"2026-02-24T07:43:07.852815+0000","last_unstale":"2026-02-24T07:45:15.737931+0000","last_undegraded":"2026-02-24T07:45:15.737931+0000","last_fullsized":"2026-02-24T07:45:15.737931+0000","mapping_epoch":30,"log_start":"40'53505","ondisk_log_start":"40'53505","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'27767","last_scrub_stamp":"2026-02-24T07:44:16.083320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_clean_scrub_stamp":"2026-02-24T07:44:16.083320+0000","objects_scrubbed":27311,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:45:28.111749+0000","scrub_duration":2943,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1208614912,"num_objects":18442,"num_object_clones":0,"num_object_copies":55326,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18442,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":53508,"num_write_kb":2302400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.d","version":"40'53608","reported_seq":53960,"reported_epoch":40,"state":"active+clean","last_fresh":"2026-02-24T07:45:15.738287+0000","last_change":"2026-02-24T07:44:26.596366+0000","last_active":"2026-02-24T07:45:15.738287+0000","last_peered":"2026-02-24T07:45:15.738287+0000","last_clean":"2026-02-24T07:45:15.738287+0000","last_became_active":"2026-02-24T07:43:07.853975+0000","last_became_peered":"2026-02-24T07:43:07.853975+0000","last_unstale":"2026-02-24T07:45:15.738287+0000","last_undegraded":"2026-02-24T07:45:15.738287+0000","last_fullsized":"2026-02-24T07:45:15.738287+0000","mapping_epoch":30,"log_start":"40'53605","ondisk_log_start":"40'53605","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'31656","last_scrub_stamp":"2026-02-24T07:44:26.596292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_clean_scrub_stamp":"2026-02-24T07:44:26.596292+0000","objects_scrubbed":31097,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:45:55.416603+0000","scrub_duration":3380,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1217200152,"num_objects":18574,"num_object_clones":0,"num_object_copies":55722,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18574,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":53608,"num_write_kb":2309761,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"40'53612","reported_seq":53969,"reported_epoch":40,"state":"active+clean","last_fresh":"2026-02-24T07:45:15.734564+0000","last_change":"2026-02-24T07:44:29.667393+0000","last_active":"2026-02-24T07:45:15.734564+0000","last_peered":"2026-02-24T07:45:15.734564+0000","last_clean":"2026-02-24T07:45:15.734564+0000","last_became_active":"2026-02-24T07:43:07.854428+0000","last_became_peered":"2026-02-24T07:43:07.854428+0000","last_unstale":"2026-02-24T07:45:15.734564+0000","last_undegraded":"2026-02-24T07:45:15.734564+0000","last_fullsized":"2026-02-24T07:45:15.734564+0000","mapping_epoch":30,"log_start":"40'53609","ondisk_log_start":"40'53609","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'32466","last_scrub_stamp":"2026-02-24T07:44:29.667348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_clean_scrub_stamp":"2026-02-24T07:44:29.667348+0000","objects_scrubbed":31970,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:45:43.092916+0000","scrub_duration":3552,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1220149248,"num_objects":18618,"num_object_clones":0,"num_object_copies":55854,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18618,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":53612,"num_write_kb":2311360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"29'99","reported_seq":210,"reported_epoch":40,"state":"active+clean","last_fresh":"2026-02-24T07:43:59.386388+0000","last_change":"2026-02-24T07:43:59.386353+0000","last_active":"2026-02-24T07:43:59.386388+0000","last_peered":"2026-02-24T07:43:59.386388+0000","last_clean":"2026-02-24T07:43:59.386388+0000","last_became_active":"2026-02-24T07:42:54.186719+0000","last_became_peered":"2026-02-24T07:42:54.186719+0000","last_unstale":"2026-02-24T07:43:59.386388+0000","last_undegraded":"2026-02-24T07:43:59.386388+0000","last_fullsized":"2026-02-24T07:43:59.386388+0000","mapping_epoch":27,"log_start":"29'96","ondisk_log_start":"29'96","created":20,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'99","last_scrub_stamp":"2026-02-24T07:43:59.386324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:42:43.640776+0000","last_clean_scrub_stamp":"2026-02-24T07:43:59.386324+0000","objects_scrubbed":2,"log_size":3,"log_dups_size":96,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:45:24.339819+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":240,"num_write":135,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"40'53467","reported_seq":53803,"reported_epoch":40,"state":"active+clean","last_fresh":"2026-02-24T07:45:15.696263+0000","last_change":"2026-02-24T07:44:22.665897+0000","last_active":"2026-02-24T07:45:15.696263+0000","last_peered":"2026-02-24T07:45:15.696263+0000","last_clean":"2026-02-24T07:45:15.696263+0000","last_became_active":"2026-02-24T07:43:07.842530+0000","last_became_peered":"2026-02-24T07:43:07.842530+0000","last_unstale":"2026-02-24T07:45:15.696263+0000","last_undegraded":"2026-02-24T07:45:15.696263+0000","last_fullsized":"2026-02-24T07:45:15.696263+0000","mapping_epoch":30,"log_start":"40'53464","ondisk_log_start":"40'53464","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'30210","last_scrub_stamp":"2026-02-24T07:44:22.665762+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_clean_scrub_stamp":"2026-02-24T07:44:22.665762+0000","objects_scrubbed":29682,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:45:26.371453+0000","scrub_duration":3226,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1221656576,"num_objects":18641,"num_object_clones":0,"num_object_copies":55923,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18641,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":53467,"num_write_kb":2307456,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"40'52922","reported_seq":53274,"reported_epoch":40,"state":"active+clean","last_fresh":"2026-02-24T07:45:15.689099+0000","last_change":"2026-02-24T07:44:28.446165+0000","last_active":"2026-02-24T07:45:15.689099+0000","last_peered":"2026-02-24T07:45:15.689099+0000","last_clean":"2026-02-24T07:45:15.689099+0000","last_became_active":"2026-02-24T07:43:07.853885+0000","last_became_peered":"2026-02-24T07:43:07.853885+0000","last_unstale":"2026-02-24T07:45:15.689099+0000","last_undegraded":"2026-02-24T07:45:15.689099+0000","last_fullsized":"2026-02-24T07:45:15.689099+0000","mapping_epoch":30,"log_start":"40'52919","ondisk_log_start":"40'52919","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'31685","last_scrub_stamp":"2026-02-24T07:44:28.446064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_clean_scrub_stamp":"2026-02-24T07:44:28.446064+0000","objects_scrubbed":31280,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:45:52.594601+0000","scrub_duration":3117,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1199439872,"num_objects":18302,"num_object_clones":0,"num_object_copies":54906,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18302,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":52922,"num_write_kb":2279168,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"40'53487","reported_seq":53857,"reported_epoch":40,"state":"active+clean","last_fresh":"2026-02-24T07:45:15.736222+0000","last_change":"2026-02-24T07:44:33.357960+0000","last_active":"2026-02-24T07:45:15.736222+0000","last_peered":"2026-02-24T07:45:15.736222+0000","last_clean":"2026-02-24T07:45:15.736222+0000","last_became_active":"2026-02-24T07:43:07.854212+0000","last_became_peered":"2026-02-24T07:43:07.854212+0000","last_unstale":"2026-02-24T07:45:15.736222+0000","last_undegraded":"2026-02-24T07:45:15.736222+0000","last_fullsized":"2026-02-24T07:45:15.736222+0000","mapping_epoch":30,"log_start":"40'53484","ondisk_log_start":"40'53484","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'33783","last_scrub_stamp":"2026-02-24T07:44:33.357894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_clean_scrub_stamp":"2026-02-24T07:44:33.357894+0000","objects_scrubbed":33200,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:45:38.914707+0000","scrub_duration":3247,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1222443008,"num_objects":18653,"num_object_clones":0,"num_object_copies":55959,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18653,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":53487,"num_write_kb":2308480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[1,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"40'53634","reported_seq":54025,"reported_epoch":40,"state":"active+clean","last_fresh":"2026-02-24T07:45:15.692450+0000","last_change":"2026-02-24T07:44:42.974303+0000","last_active":"2026-02-24T07:45:15.692450+0000","last_peered":"2026-02-24T07:45:15.692450+0000","last_clean":"2026-02-24T07:45:15.692450+0000","last_became_active":"2026-02-24T07:43:07.842864+0000","last_became_peered":"2026-02-24T07:43:07.842864+0000","last_unstale":"2026-02-24T07:45:15.692450+0000","last_undegraded":"2026-02-24T07:45:15.692450+0000","last_fullsized":"2026-02-24T07:45:15.692450+0000","mapping_epoch":30,"log_start":"40'53631","ondisk_log_start":"40'53631","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'37783","last_scrub_stamp":"2026-02-24T07:44:42.974258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_clean_scrub_stamp":"2026-02-24T07:44:42.974258+0000","objects_scrubbed":35124,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:45:49.565319+0000","scrub_duration":3593,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1207435264,"num_objects":18424,"num_object_clones":0,"num_object_copies":55272,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18424,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":53634,"num_write_kb":2305856,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.4","version":"40'53036","reported_seq":53351,"reported_epoch":40,"state":"active+clean","last_fresh":"2026-02-24T07:45:15.731968+0000","last_change":"2026-02-24T07:44:16.909673+0000","last_active":"2026-02-24T07:45:15.731968+0000","last_peered":"2026-02-24T07:45:15.731968+0000","last_clean":"2026-02-24T07:45:15.731968+0000","last_became_active":"2026-02-24T07:43:07.853966+0000","last_became_peered":"2026-02-24T07:43:07.853966+0000","last_unstale":"2026-02-24T07:45:15.731968+0000","last_undegraded":"2026-02-24T07:45:15.731968+0000","last_fullsized":"2026-02-24T07:45:15.731968+0000","mapping_epoch":30,"log_start":"40'53033","ondisk_log_start":"40'53033","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'27932","last_scrub_stamp":"2026-02-24T07:44:16.909624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_clean_scrub_stamp":"2026-02-24T07:44:16.909624+0000","objects_scrubbed":27480,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:45:43.624405+0000","scrub_duration":2774,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1215823872,"num_objects":18552,"num_object_clones":0,"num_object_copies":55656,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18552,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":53036,"num_write_kb":2290816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"40'53119","reported_seq":53497,"reported_epoch":40,"state":"active+clean","last_fresh":"2026-02-24T07:45:15.694456+0000","last_change":"2026-02-24T07:44:35.717081+0000","last_active":"2026-02-24T07:45:15.694456+0000","last_peered":"2026-02-24T07:45:15.694456+0000","last_clean":"2026-02-24T07:45:15.694456+0000","last_became_active":"2026-02-24T07:43:07.853878+0000","last_became_peered":"2026-02-24T07:43:07.853878+0000","last_unstale":"2026-02-24T07:45:15.694456+0000","last_undegraded":"2026-02-24T07:45:15.694456+0000","last_fullsized":"2026-02-24T07:45:15.694456+0000","mapping_epoch":30,"log_start":"40'53116","ondisk_log_start":"40'53116","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'34578","last_scrub_stamp":"2026-02-24T07:44:35.717022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_clean_scrub_stamp":"2026-02-24T07:44:35.717022+0000","objects_scrubbed":33884,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:46:00.809440+0000","scrub_duration":3376,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1213530112,"num_objects":18517,"num_object_clones":0,"num_object_copies":55551,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18517,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":53119,"num_write_kb":2292352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"40'53335","reported_seq":53695,"reported_epoch":40,"state":"active+clean","last_fresh":"2026-02-24T07:45:15.729521+0000","last_change":"2026-02-24T07:44:30.553008+0000","last_active":"2026-02-24T07:45:15.729521+0000","last_peered":"2026-02-24T07:45:15.729521+0000","last_clean":"2026-02-24T07:45:15.729521+0000","last_became_active":"2026-02-24T07:43:07.841993+0000","last_became_peered":"2026-02-24T07:43:07.841993+0000","last_unstale":"2026-02-24T07:45:15.729521+0000","last_undegraded":"2026-02-24T07:45:15.729521+0000","last_fullsized":"2026-02-24T07:45:15.729521+0000","mapping_epoch":30,"log_start":"40'53332","ondisk_log_start":"40'53332","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'32640","last_scrub_stamp":"2026-02-24T07:44:30.552965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_clean_scrub_stamp":"2026-02-24T07:44:30.552965+0000","objects_scrubbed":32121,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:45:56.537381+0000","scrub_duration":3414,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1210777600,"num_objects":18475,"num_object_clones":0,"num_object_copies":55425,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18475,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":53335,"num_write_kb":2297920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"40'52585","reported_seq":52902,"reported_epoch":40,"state":"active+clean","last_fresh":"2026-02-24T07:45:15.692602+0000","last_change":"2026-02-24T07:44:18.020443+0000","last_active":"2026-02-24T07:45:15.692602+0000","last_peered":"2026-02-24T07:45:15.692602+0000","last_clean":"2026-02-24T07:45:15.692602+0000","last_became_active":"2026-02-24T07:43:07.841920+0000","last_became_peered":"2026-02-24T07:43:07.841920+0000","last_unstale":"2026-02-24T07:45:15.692602+0000","last_undegraded":"2026-02-24T07:45:15.692602+0000","last_fullsized":"2026-02-24T07:45:15.692602+0000","mapping_epoch":30,"log_start":"40'52582","ondisk_log_start":"40'52582","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'28226","last_scrub_stamp":"2026-02-24T07:44:18.020379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_clean_scrub_stamp":"2026-02-24T07:44:18.020379+0000","objects_scrubbed":27705,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:45:27.051564+0000","scrub_duration":2601,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1214971904,"num_objects":18539,"num_object_clones":0,"num_object_copies":55617,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18539,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":52585,"num_write_kb":2275968,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.8","version":"40'52895","reported_seq":53254,"reported_epoch":40,"state":"active+clean","last_fresh":"2026-02-24T07:45:15.696349+0000","last_change":"2026-02-24T07:44:31.118780+0000","last_active":"2026-02-24T07:45:15.696349+0000","last_peered":"2026-02-24T07:45:15.696349+0000","last_clean":"2026-02-24T07:45:15.696349+0000","last_became_active":"2026-02-24T07:43:07.853758+0000","last_became_peered":"2026-02-24T07:43:07.853758+0000","last_unstale":"2026-02-24T07:45:15.696349+0000","last_undegraded":"2026-02-24T07:45:15.696349+0000","last_fullsized":"2026-02-24T07:45:15.696349+0000","mapping_epoch":30,"log_start":"40'52892","ondisk_log_start":"40'52892","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'32571","last_scrub_stamp":"2026-02-24T07:44:31.118720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_clean_scrub_stamp":"2026-02-24T07:44:31.118720+0000","objects_scrubbed":32077,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:45:44.647478+0000","scrub_duration":2775,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1209335808,"num_objects":18453,"num_object_clones":0,"num_object_copies":55359,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18453,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":52895,"num_write_kb":2283136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"40'53342","reported_seq":53737,"reported_epoch":40,"state":"active+clean","last_fresh":"2026-02-24T07:45:15.735887+0000","last_change":"2026-02-24T07:44:41.349757+0000","last_active":"2026-02-24T07:45:15.735887+0000","last_peered":"2026-02-24T07:45:15.735887+0000","last_clean":"2026-02-24T07:45:15.735887+0000","last_became_active":"2026-02-24T07:43:07.854377+0000","last_became_peered":"2026-02-24T07:43:07.854377+0000","last_unstale":"2026-02-24T07:45:15.735887+0000","last_undegraded":"2026-02-24T07:45:15.735887+0000","last_fullsized":"2026-02-24T07:45:15.735887+0000","mapping_epoch":30,"log_start":"40'53339","ondisk_log_start":"40'53339","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'36997","last_scrub_stamp":"2026-02-24T07:44:41.349228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_clean_scrub_stamp":"2026-02-24T07:44:41.349228+0000","objects_scrubbed":35574,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:45:50.611975+0000","scrub_duration":3205,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1223294976,"num_objects":18666,"num_object_clones":0,"num_object_copies":55998,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18666,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":53342,"num_write_kb":2304256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.a","version":"40'53414","reported_seq":53769,"reported_epoch":40,"state":"active+clean","last_fresh":"2026-02-24T07:45:15.733002+0000","last_change":"2026-02-24T07:44:27.902144+0000","last_active":"2026-02-24T07:45:15.733002+0000","last_peered":"2026-02-24T07:45:15.733002+0000","last_clean":"2026-02-24T07:45:15.733002+0000","last_became_active":"2026-02-24T07:43:07.841721+0000","last_became_peered":"2026-02-24T07:43:07.841721+0000","last_unstale":"2026-02-24T07:45:15.733002+0000","last_undegraded":"2026-02-24T07:45:15.733002+0000","last_fullsized":"2026-02-24T07:45:15.733002+0000","mapping_epoch":30,"log_start":"40'53411","ondisk_log_start":"40'53411","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'31794","last_scrub_stamp":"2026-02-24T07:44:27.902040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_clean_scrub_stamp":"2026-02-24T07:44:27.902040+0000","objects_scrubbed":31292,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:45:39.265782+0000","scrub_duration":3732,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1210712064,"num_objects":18474,"num_object_clones":0,"num_object_copies":55422,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18474,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":53414,"num_write_kb":2300416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.b","version":"40'53457","reported_seq":53851,"reported_epoch":40,"state":"active+clean","last_fresh":"2026-02-24T07:45:15.694824+0000","last_change":"2026-02-24T07:44:39.639056+0000","last_active":"2026-02-24T07:45:15.694824+0000","last_peered":"2026-02-24T07:45:15.694824+0000","last_clean":"2026-02-24T07:45:15.694824+0000","last_became_active":"2026-02-24T07:43:07.853069+0000","last_became_peered":"2026-02-24T07:43:07.853069+0000","last_unstale":"2026-02-24T07:45:15.694824+0000","last_undegraded":"2026-02-24T07:45:15.694824+0000","last_fullsized":"2026-02-24T07:45:15.694824+0000","mapping_epoch":30,"log_start":"40'53454","ondisk_log_start":"40'53454","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'36298","last_scrub_stamp":"2026-02-24T07:44:39.638986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_clean_scrub_stamp":"2026-02-24T07:44:39.638986+0000","objects_scrubbed":35656,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:46:09.128452+0000","scrub_duration":3228,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1217462272,"num_objects":18577,"num_object_clones":0,"num_object_copies":55731,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18577,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":53457,"num_write_kb":2305088,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":19411304472,"num_objects":296194,"num_object_clones":0,"num_object_copies":888582,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":296194,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":851942,"num_write_kb":36740289,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":58270101504,"data_stored":58270089288,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":48,"ondisk_log_size":48,"up":48,"acting":48,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":240,"num_write":135,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":3,"up_from":25,"seq":107374182431,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":17299408,"kb_used_data":16827836,"kb_used_omap":4,"kb_used_meta":471547,"kb_avail":716703792,"statfs":{"total":751619276800,"available":733904683008,"internally_reserved":0,"allocated":17231704064,"data_stored":17231449352,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4448,"internal_metadata":482864800},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":19,"seq":81604378657,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":13738700,"kb_used_data":13054968,"kb_used_omap":4,"kb_used_meta":683643,"kb_avail":720264500,"statfs":{"total":751619276800,"available":737550848000,"internally_reserved":0,"allocated":13368287232,"data_stored":13368036592,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4449,"internal_metadata":700051103},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542178,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":15313228,"kb_used_data":14456888,"kb_used_omap":4,"kb_used_meta":856315,"kb_avail":718689972,"statfs":{"total":751619276800,"available":735938531328,"internally_reserved":0,"allocated":14803853312,"data_stored":14803602152,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4446,"internal_metadata":876867234},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[6],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705701,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":13976976,"kb_used_data":13293308,"kb_used_omap":4,"kb_used_meta":683643,"kb_avail":720026224,"statfs":{"total":751619276800,"available":737306853376,"internally_reserved":0,"allocated":13612347392,"data_stored":13612092680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4450,"internal_metadata":700051102},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":13365415936,"data_stored":13365411864,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":14566100992,"data_stored":14566096920,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":13342801920,"data_stored":13342801920,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16995782656,"data_stored":16995778584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-24T07:45:21.018 INFO:tasks.cephadm.ceph_manager.ceph:increase pool size by 10 2026-02-24T07:45:21.018 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph osd pool set unique_pool_0 pg_num 66 2026-02-24T07:45:21.070 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:45:21.191 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:45:21.188+0000 7f3729b49640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:45:21.191 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:45:21.189+0000 7f3729b49640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:45:21.192 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:45:21.205 INFO:teuthology.orchestra.run.trial035.stdout:osd.0: {} 2026-02-24T07:45:21.205 INFO:teuthology.orchestra.run.trial035.stderr:osd.0: osd_enable_op_tracker = '' 2026-02-24T07:45:21.220 INFO:teuthology.orchestra.run.trial035.stdout:osd.1: {} 2026-02-24T07:45:21.221 INFO:teuthology.orchestra.run.trial035.stderr:osd.1: osd_enable_op_tracker = '' 2026-02-24T07:45:21.232 INFO:teuthology.orchestra.run.trial035.stdout:osd.2: {} 2026-02-24T07:45:21.233 INFO:teuthology.orchestra.run.trial035.stderr:osd.2: osd_enable_op_tracker = '' 2026-02-24T07:45:21.245 INFO:teuthology.orchestra.run.trial035.stdout:osd.3: {} 2026-02-24T07:45:21.245 INFO:teuthology.orchestra.run.trial035.stderr:osd.3: osd_enable_op_tracker = '' 2026-02-24T07:45:21.308 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:45:21.306+0000 7f96196f1640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:45:21.309 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:45:21.307+0000 7f96196f1640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:45:21.464 INFO:teuthology.orchestra.run.trial035.stderr:Error EPERM: crimson_allow_pg_split is false; pg_num increase denied for crimson pool 2026-02-24T07:45:21.505 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-24T07:45:22.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:45:21 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/342719017' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2026-02-24T07:45:22.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:45:21 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/3632705806' entity='client.admin' cmd={"prefix": "osd pool set", "pool": "unique_pool_0", "var": "pg_num", "val": "66"} : dispatch 2026-02-24T07:45:23.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:45:22 trial035 ceph-mon[39284]: from='client.14537 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2026-02-24T07:45:23.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:45:22 trial035 ceph-mon[39284]: pgmap v85: 17 pgs: 17 active+clean; 16 GiB data, 50 GiB used, 2.7 TiB / 2.7 TiB avail; 0 B/s wr, 10.07k op/s 2026-02-24T07:45:23.284 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2026-02-24T07:45:23.417 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:45:23.535 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:45:23.532+0000 7f9def5c4640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:45:23.535 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:45:23.533+0000 7f9def5c4640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:45:23.550 INFO:teuthology.orchestra.run.trial035.stdout:osd.0: {} 2026-02-24T07:45:23.550 INFO:teuthology.orchestra.run.trial035.stderr:osd.0: osd_enable_op_tracker = '' 2026-02-24T07:45:23.563 INFO:teuthology.orchestra.run.trial035.stdout:osd.1: {} 2026-02-24T07:45:23.564 INFO:teuthology.orchestra.run.trial035.stderr:osd.1: osd_enable_op_tracker = '' 2026-02-24T07:45:23.576 INFO:teuthology.orchestra.run.trial035.stdout:osd.2: {} 2026-02-24T07:45:23.576 INFO:teuthology.orchestra.run.trial035.stderr:osd.2: osd_enable_op_tracker = '' 2026-02-24T07:45:23.589 INFO:teuthology.orchestra.run.trial035.stdout:osd.3: {} 2026-02-24T07:45:23.589 INFO:teuthology.orchestra.run.trial035.stderr:osd.3: osd_enable_op_tracker = '' 2026-02-24T07:45:24.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:45:23 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/4077597033' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2026-02-24T07:45:25.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:45:24 trial035 ceph-mon[39284]: pgmap v86: 17 pgs: 17 active+clean; 16 GiB data, 50 GiB used, 2.7 TiB / 2.7 TiB avail; 0 B/s wr, 7.00k op/s 2026-02-24T07:45:25.628 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2026-02-24T07:45:25.761 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:45:25.918 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:45:25 trial035 ceph-mon[39284]: purged_snaps scrub starts 2026-02-24T07:45:25.918 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:45:25 trial035 ceph-mon[39284]: purged_snaps scrub ok 2026-02-24T07:45:25.918 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:45:25 trial035 ceph-mon[39284]: 2.f scrub starts 2026-02-24T07:45:26.269 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:45:26.248+0000 7f81bae24640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:45:26.269 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:45:26.249+0000 7f81bae24640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:45:26.281 INFO:teuthology.orchestra.run.trial035.stdout:osd.0: {} 2026-02-24T07:45:26.281 INFO:teuthology.orchestra.run.trial035.stderr:osd.0: osd_enable_op_tracker = '' 2026-02-24T07:45:26.296 INFO:teuthology.orchestra.run.trial035.stdout:osd.1: {} 2026-02-24T07:45:26.296 INFO:teuthology.orchestra.run.trial035.stderr:osd.1: osd_enable_op_tracker = '' 2026-02-24T07:45:26.308 INFO:teuthology.orchestra.run.trial035.stdout:osd.2: {} 2026-02-24T07:45:26.308 INFO:teuthology.orchestra.run.trial035.stderr:osd.2: osd_enable_op_tracker = '' 2026-02-24T07:45:26.320 INFO:teuthology.orchestra.run.trial035.stdout:osd.3: {} 2026-02-24T07:45:26.321 INFO:teuthology.orchestra.run.trial035.stderr:osd.3: osd_enable_op_tracker = '' 2026-02-24T07:45:26.506 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2026-02-24T07:45:26.506 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2026-02-24T07:45:26.506 INFO:tasks.thrashosds.thrasher:Growing pool .mgr 2026-02-24T07:45:26.507 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph pg dump --format=json 2026-02-24T07:45:26.812 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:45:27.204 INFO:journalctl@ceph.mgr.x.trial035.stdout:Feb 24 07:45:26 trial035 ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-mgr-x[39621]: ::ffff:10.20.193.91 - - [24/Feb/2026:07:45:26] "GET /metrics HTTP/1.1" 200 4231 "" "Prometheus/3.6.0" 2026-02-24T07:45:27.205 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:45:26 trial035 ceph-mon[39284]: pgmap v87: 17 pgs: 1 active+clean+scrubbing, 16 active+clean; 15 GiB data, 46 GiB used, 2.7 TiB / 2.7 TiB avail; 0 B/s wr, 7.20k op/s 2026-02-24T07:45:27.205 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:45:26 trial035 ceph-mon[39284]: 1.0 scrub starts 2026-02-24T07:45:27.205 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:45:26 trial035 ceph-mon[39284]: 1.0 scrub ok 2026-02-24T07:45:27.205 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:45:26 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/2175638708' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2026-02-24T07:45:27.205 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:45:26 trial035 ceph-mon[39284]: 2.f scrub ok 2026-02-24T07:45:27.356 INFO:journalctl@ceph.prometheus.a.trial091.stdout:Feb 24 07:45:26 trial091 ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-prometheus-a[66007]: time=2026-02-24T07:45:26.973Z level=ERROR source=scrape.go:1630 msg="Failed to determine correct type of scrape target." component="scrape manager" scrape_pool=ceph target=http://trial035:9283/metrics content_type="text/html;charset=utf-8" fallback_media_type="" err="received unsupported Content-Type \"text/html;charset=utf-8\" and no fallback_scrape_protocol specified for target" 2026-02-24T07:45:27.421 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:45:27.409+0000 7f25fed1e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:45:27.421 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:45:27.410+0000 7f25fed1e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:45:27.541 INFO:teuthology.orchestra.run.trial035.stdout: 2026-02-24T07:45:27.541 INFO:teuthology.orchestra.run.trial035.stderr:dumped all 2026-02-24T07:45:28.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:45:27 trial035 ceph-mon[39284]: 2.3 scrub starts 2026-02-24T07:45:28.251 INFO:teuthology.orchestra.run.trial035.stdout:{"pg_ready":true,"pg_map":{"version":87,"stamp":"2026-02-24T07:45:25.677412+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":16568484408,"num_objects":252809,"num_object_clones":0,"num_object_copies":758427,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":252809,"num_whiteouts":0,"num_read":137,"num_read_kb":241,"num_write":895464,"num_write_kb":36742439,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":51,"ondisk_log_size":51,"up":51,"acting":51,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":51,"num_osds":4,"num_per_pool_osds":4,"num_per_pool_omap_osds":4,"kb":2936012800,"kb_used":48671800,"kb_used_data":46292520,"kb_used_omap":18,"kb_used_meta":2379117,"kb_avail":2887341000,"statfs":{"total":3006477107200,"available":2956637184000,"internally_reserved":0,"allocated":47403540480,"data_stored":47402529256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":18445,"internal_metadata":2436216819},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[6],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-5659885568,"num_objects":-86363,"num_object_clones":0,"num_object_copies":-259089,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-86363,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":86363,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001493"},"pg_stats":[{"pgid":"2.f","version":"40'57491","reported_seq":57836,"reported_epoch":40,"state":"active+clean+scrubbing","last_fresh":"2026-02-24T07:45:25.670875+0000","last_change":"2026-02-24T07:45:25.607632+0000","last_active":"2026-02-24T07:45:25.670875+0000","last_peered":"2026-02-24T07:45:25.670875+0000","last_clean":"2026-02-24T07:45:25.670875+0000","last_became_active":"2026-02-24T07:43:07.852899+0000","last_became_peered":"2026-02-24T07:43:07.852899+0000","last_unstale":"2026-02-24T07:45:25.670875+0000","last_undegraded":"2026-02-24T07:45:25.670875+0000","last_fullsized":"2026-02-24T07:45:25.670875+0000","mapping_epoch":30,"log_start":"40'57488","ondisk_log_start":"40'57488","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'30039","last_scrub_stamp":"2026-02-24T07:44:23.320202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_clean_scrub_stamp":"2026-02-24T07:44:23.320202+0000","objects_scrubbed":899,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"scrubbing for 1s","scrub_duration":2598,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":872742912,"num_objects":13317,"num_object_clones":0,"num_object_copies":39951,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13317,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":57491,"num_write_kb":2265856,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.e","version":"40'56100","reported_seq":56414,"reported_epoch":40,"state":"active+clean","last_fresh":"2026-02-24T07:45:20.734757+0000","last_change":"2026-02-24T07:44:16.083399+0000","last_active":"2026-02-24T07:45:20.734757+0000","last_peered":"2026-02-24T07:45:20.734757+0000","last_clean":"2026-02-24T07:45:20.734757+0000","last_became_active":"2026-02-24T07:43:07.852815+0000","last_became_peered":"2026-02-24T07:43:07.852815+0000","last_unstale":"2026-02-24T07:45:20.734757+0000","last_undegraded":"2026-02-24T07:45:20.734757+0000","last_fullsized":"2026-02-24T07:45:20.734757+0000","mapping_epoch":30,"log_start":"40'56097","ondisk_log_start":"40'56097","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'27767","last_scrub_stamp":"2026-02-24T07:44:16.083320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_clean_scrub_stamp":"2026-02-24T07:44:16.083320+0000","objects_scrubbed":27311,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:45:28.111749+0000","scrub_duration":2943,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1038745600,"num_objects":15850,"num_object_clones":0,"num_object_copies":47550,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15850,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":56100,"num_write_kb":2302400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.d","version":"40'56204","reported_seq":56556,"reported_epoch":40,"state":"active+clean","last_fresh":"2026-02-24T07:45:20.738657+0000","last_change":"2026-02-24T07:44:26.596366+0000","last_active":"2026-02-24T07:45:20.738657+0000","last_peered":"2026-02-24T07:45:20.738657+0000","last_clean":"2026-02-24T07:45:20.738657+0000","last_became_active":"2026-02-24T07:43:07.853975+0000","last_became_peered":"2026-02-24T07:43:07.853975+0000","last_unstale":"2026-02-24T07:45:20.738657+0000","last_undegraded":"2026-02-24T07:45:20.738657+0000","last_fullsized":"2026-02-24T07:45:20.738657+0000","mapping_epoch":30,"log_start":"40'56201","ondisk_log_start":"40'56201","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'31656","last_scrub_stamp":"2026-02-24T07:44:26.596292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_clean_scrub_stamp":"2026-02-24T07:44:26.596292+0000","objects_scrubbed":31097,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:45:55.416603+0000","scrub_duration":3380,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1047068696,"num_objects":15978,"num_object_clones":0,"num_object_copies":47934,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15978,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":56204,"num_write_kb":2309761,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"40'56167","reported_seq":56525,"reported_epoch":40,"state":"active+clean","last_fresh":"2026-02-24T07:45:20.738548+0000","last_change":"2026-02-24T07:44:29.667393+0000","last_active":"2026-02-24T07:45:20.738548+0000","last_peered":"2026-02-24T07:45:20.738548+0000","last_clean":"2026-02-24T07:45:20.738548+0000","last_became_active":"2026-02-24T07:43:07.854428+0000","last_became_peered":"2026-02-24T07:43:07.854428+0000","last_unstale":"2026-02-24T07:45:20.738548+0000","last_undegraded":"2026-02-24T07:45:20.738548+0000","last_fullsized":"2026-02-24T07:45:20.738548+0000","mapping_epoch":30,"log_start":"40'56164","ondisk_log_start":"40'56164","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'32466","last_scrub_stamp":"2026-02-24T07:44:29.667348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_clean_scrub_stamp":"2026-02-24T07:44:29.667348+0000","objects_scrubbed":31970,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:45:43.092916+0000","scrub_duration":3552,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1052704768,"num_objects":16063,"num_object_clones":0,"num_object_copies":48189,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16063,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":56167,"num_write_kb":2311360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"29'99","reported_seq":210,"reported_epoch":40,"state":"active+clean","last_fresh":"2026-02-24T07:43:59.386388+0000","last_change":"2026-02-24T07:43:59.386353+0000","last_active":"2026-02-24T07:43:59.386388+0000","last_peered":"2026-02-24T07:43:59.386388+0000","last_clean":"2026-02-24T07:43:59.386388+0000","last_became_active":"2026-02-24T07:42:54.186719+0000","last_became_peered":"2026-02-24T07:42:54.186719+0000","last_unstale":"2026-02-24T07:43:59.386388+0000","last_undegraded":"2026-02-24T07:43:59.386388+0000","last_fullsized":"2026-02-24T07:43:59.386388+0000","mapping_epoch":27,"log_start":"29'96","ondisk_log_start":"29'96","created":20,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'99","last_scrub_stamp":"2026-02-24T07:43:59.386324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:42:43.640776+0000","last_clean_scrub_stamp":"2026-02-24T07:43:59.386324+0000","objects_scrubbed":2,"log_size":3,"log_dups_size":96,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:45:24.339819+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":240,"num_write":135,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"40'55978","reported_seq":56314,"reported_epoch":40,"state":"active+clean","last_fresh":"2026-02-24T07:45:20.696208+0000","last_change":"2026-02-24T07:44:22.665897+0000","last_active":"2026-02-24T07:45:20.696208+0000","last_peered":"2026-02-24T07:45:20.696208+0000","last_clean":"2026-02-24T07:45:20.696208+0000","last_became_active":"2026-02-24T07:43:07.842530+0000","last_became_peered":"2026-02-24T07:43:07.842530+0000","last_unstale":"2026-02-24T07:45:20.696208+0000","last_undegraded":"2026-02-24T07:45:20.696208+0000","last_fullsized":"2026-02-24T07:45:20.696208+0000","mapping_epoch":30,"log_start":"40'55975","ondisk_log_start":"40'55975","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'30210","last_scrub_stamp":"2026-02-24T07:44:22.665762+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_clean_scrub_stamp":"2026-02-24T07:44:22.665762+0000","objects_scrubbed":29682,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:45:26.371453+0000","scrub_duration":3226,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1057095680,"num_objects":16130,"num_object_clones":0,"num_object_copies":48390,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16130,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":55978,"num_write_kb":2307456,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"40'55412","reported_seq":55763,"reported_epoch":40,"state":"active+clean","last_fresh":"2026-02-24T07:45:20.697163+0000","last_change":"2026-02-24T07:44:28.446165+0000","last_active":"2026-02-24T07:45:20.697163+0000","last_peered":"2026-02-24T07:45:20.697163+0000","last_clean":"2026-02-24T07:45:20.697163+0000","last_became_active":"2026-02-24T07:43:07.853885+0000","last_became_peered":"2026-02-24T07:43:07.853885+0000","last_unstale":"2026-02-24T07:45:20.697163+0000","last_undegraded":"2026-02-24T07:45:20.697163+0000","last_fullsized":"2026-02-24T07:45:20.697163+0000","mapping_epoch":30,"log_start":"40'55409","ondisk_log_start":"40'55409","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'31685","last_scrub_stamp":"2026-02-24T07:44:28.446064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_clean_scrub_stamp":"2026-02-24T07:44:28.446064+0000","objects_scrubbed":31280,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:45:52.594601+0000","scrub_duration":3117,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1036255232,"num_objects":15812,"num_object_clones":0,"num_object_copies":47436,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15812,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":55412,"num_write_kb":2279168,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"40'56074","reported_seq":56445,"reported_epoch":40,"state":"active+clean","last_fresh":"2026-02-24T07:45:20.733062+0000","last_change":"2026-02-24T07:44:33.357960+0000","last_active":"2026-02-24T07:45:20.733062+0000","last_peered":"2026-02-24T07:45:20.733062+0000","last_clean":"2026-02-24T07:45:20.733062+0000","last_became_active":"2026-02-24T07:43:07.854212+0000","last_became_peered":"2026-02-24T07:43:07.854212+0000","last_unstale":"2026-02-24T07:45:20.733062+0000","last_undegraded":"2026-02-24T07:45:20.733062+0000","last_fullsized":"2026-02-24T07:45:20.733062+0000","mapping_epoch":30,"log_start":"40'56071","ondisk_log_start":"40'56071","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'33783","last_scrub_stamp":"2026-02-24T07:44:33.357894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_clean_scrub_stamp":"2026-02-24T07:44:33.357894+0000","objects_scrubbed":33200,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:45:38.914707+0000","scrub_duration":3247,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1052901376,"num_objects":16066,"num_object_clones":0,"num_object_copies":48198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16066,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":56074,"num_write_kb":2308480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[1,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"40'56120","reported_seq":56511,"reported_epoch":40,"state":"active+clean","last_fresh":"2026-02-24T07:45:20.694834+0000","last_change":"2026-02-24T07:44:42.974303+0000","last_active":"2026-02-24T07:45:20.694834+0000","last_peered":"2026-02-24T07:45:20.694834+0000","last_clean":"2026-02-24T07:45:20.694834+0000","last_became_active":"2026-02-24T07:43:07.842864+0000","last_became_peered":"2026-02-24T07:43:07.842864+0000","last_unstale":"2026-02-24T07:45:20.694834+0000","last_undegraded":"2026-02-24T07:45:20.694834+0000","last_fullsized":"2026-02-24T07:45:20.694834+0000","mapping_epoch":30,"log_start":"40'56117","ondisk_log_start":"40'56117","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'37783","last_scrub_stamp":"2026-02-24T07:44:42.974258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_clean_scrub_stamp":"2026-02-24T07:44:42.974258+0000","objects_scrubbed":35124,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:45:49.565319+0000","scrub_duration":3593,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1044512768,"num_objects":15938,"num_object_clones":0,"num_object_copies":47814,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15938,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":56120,"num_write_kb":2305856,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.4","version":"40'55645","reported_seq":55960,"reported_epoch":40,"state":"active+clean","last_fresh":"2026-02-24T07:45:20.737213+0000","last_change":"2026-02-24T07:44:16.909673+0000","last_active":"2026-02-24T07:45:20.737213+0000","last_peered":"2026-02-24T07:45:20.737213+0000","last_clean":"2026-02-24T07:45:20.737213+0000","last_became_active":"2026-02-24T07:43:07.853966+0000","last_became_peered":"2026-02-24T07:43:07.853966+0000","last_unstale":"2026-02-24T07:45:20.737213+0000","last_undegraded":"2026-02-24T07:45:20.737213+0000","last_fullsized":"2026-02-24T07:45:20.737213+0000","mapping_epoch":30,"log_start":"40'55642","ondisk_log_start":"40'55642","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'27932","last_scrub_stamp":"2026-02-24T07:44:16.909624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_clean_scrub_stamp":"2026-02-24T07:44:16.909624+0000","objects_scrubbed":27480,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:45:43.624405+0000","scrub_duration":2774,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1044840448,"num_objects":15943,"num_object_clones":0,"num_object_copies":47829,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15943,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":55645,"num_write_kb":2290816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"40'55723","reported_seq":56100,"reported_epoch":40,"state":"active+clean","last_fresh":"2026-02-24T07:45:20.697026+0000","last_change":"2026-02-24T07:44:35.717081+0000","last_active":"2026-02-24T07:45:20.697026+0000","last_peered":"2026-02-24T07:45:20.697026+0000","last_clean":"2026-02-24T07:45:20.697026+0000","last_became_active":"2026-02-24T07:43:07.853878+0000","last_became_peered":"2026-02-24T07:43:07.853878+0000","last_unstale":"2026-02-24T07:45:20.697026+0000","last_undegraded":"2026-02-24T07:45:20.697026+0000","last_fullsized":"2026-02-24T07:45:20.697026+0000","mapping_epoch":30,"log_start":"40'55720","ondisk_log_start":"40'55720","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'34578","last_scrub_stamp":"2026-02-24T07:44:35.717022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_clean_scrub_stamp":"2026-02-24T07:44:35.717022+0000","objects_scrubbed":33884,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:46:00.809440+0000","scrub_duration":3376,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1042874368,"num_objects":15913,"num_object_clones":0,"num_object_copies":47739,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15913,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":55723,"num_write_kb":2292352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"40'55934","reported_seq":56294,"reported_epoch":40,"state":"active+clean","last_fresh":"2026-02-24T07:45:20.739061+0000","last_change":"2026-02-24T07:44:30.553008+0000","last_active":"2026-02-24T07:45:20.739061+0000","last_peered":"2026-02-24T07:45:20.739061+0000","last_clean":"2026-02-24T07:45:20.739061+0000","last_became_active":"2026-02-24T07:43:07.841993+0000","last_became_peered":"2026-02-24T07:43:07.841993+0000","last_unstale":"2026-02-24T07:45:20.739061+0000","last_undegraded":"2026-02-24T07:45:20.739061+0000","last_fullsized":"2026-02-24T07:45:20.739061+0000","mapping_epoch":30,"log_start":"40'55931","ondisk_log_start":"40'55931","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'32640","last_scrub_stamp":"2026-02-24T07:44:30.552965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_clean_scrub_stamp":"2026-02-24T07:44:30.552965+0000","objects_scrubbed":32121,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:45:56.537381+0000","scrub_duration":3414,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1040449536,"num_objects":15876,"num_object_clones":0,"num_object_copies":47628,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15876,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":55934,"num_write_kb":2297920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"40'55087","reported_seq":55404,"reported_epoch":40,"state":"active+clean","last_fresh":"2026-02-24T07:45:20.689241+0000","last_change":"2026-02-24T07:44:18.020443+0000","last_active":"2026-02-24T07:45:20.689241+0000","last_peered":"2026-02-24T07:45:20.689241+0000","last_clean":"2026-02-24T07:45:20.689241+0000","last_became_active":"2026-02-24T07:43:07.841920+0000","last_became_peered":"2026-02-24T07:43:07.841920+0000","last_unstale":"2026-02-24T07:45:20.689241+0000","last_undegraded":"2026-02-24T07:45:20.689241+0000","last_fullsized":"2026-02-24T07:45:20.689241+0000","mapping_epoch":30,"log_start":"40'55084","ondisk_log_start":"40'55084","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'28226","last_scrub_stamp":"2026-02-24T07:44:18.020379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_clean_scrub_stamp":"2026-02-24T07:44:18.020379+0000","objects_scrubbed":27705,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:45:27.051564+0000","scrub_duration":2601,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1051000832,"num_objects":16037,"num_object_clones":0,"num_object_copies":48111,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16037,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":55087,"num_write_kb":2275968,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.8","version":"40'55401","reported_seq":55760,"reported_epoch":40,"state":"active+clean","last_fresh":"2026-02-24T07:45:20.691806+0000","last_change":"2026-02-24T07:44:31.118780+0000","last_active":"2026-02-24T07:45:20.691806+0000","last_peered":"2026-02-24T07:45:20.691806+0000","last_clean":"2026-02-24T07:45:20.691806+0000","last_became_active":"2026-02-24T07:43:07.853758+0000","last_became_peered":"2026-02-24T07:43:07.853758+0000","last_unstale":"2026-02-24T07:45:20.691806+0000","last_undegraded":"2026-02-24T07:45:20.691806+0000","last_fullsized":"2026-02-24T07:45:20.691806+0000","mapping_epoch":30,"log_start":"40'55398","ondisk_log_start":"40'55398","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'32571","last_scrub_stamp":"2026-02-24T07:44:31.118720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_clean_scrub_stamp":"2026-02-24T07:44:31.118720+0000","objects_scrubbed":32077,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:45:44.647478+0000","scrub_duration":2775,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1045102592,"num_objects":15947,"num_object_clones":0,"num_object_copies":47841,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15947,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":55401,"num_write_kb":2283136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"40'55944","reported_seq":56339,"reported_epoch":40,"state":"active+clean","last_fresh":"2026-02-24T07:45:20.729556+0000","last_change":"2026-02-24T07:44:41.349757+0000","last_active":"2026-02-24T07:45:20.729556+0000","last_peered":"2026-02-24T07:45:20.729556+0000","last_clean":"2026-02-24T07:45:20.729556+0000","last_became_active":"2026-02-24T07:43:07.854377+0000","last_became_peered":"2026-02-24T07:43:07.854377+0000","last_unstale":"2026-02-24T07:45:20.729556+0000","last_undegraded":"2026-02-24T07:45:20.729556+0000","last_fullsized":"2026-02-24T07:45:20.729556+0000","mapping_epoch":30,"log_start":"40'55941","ondisk_log_start":"40'55941","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'36997","last_scrub_stamp":"2026-02-24T07:44:41.349228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_clean_scrub_stamp":"2026-02-24T07:44:41.349228+0000","objects_scrubbed":35574,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:45:50.611975+0000","scrub_duration":3205,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1052770304,"num_objects":16064,"num_object_clones":0,"num_object_copies":48192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16064,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":55944,"num_write_kb":2304256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.a","version":"40'56046","reported_seq":56401,"reported_epoch":40,"state":"active+clean","last_fresh":"2026-02-24T07:45:20.735099+0000","last_change":"2026-02-24T07:44:27.902144+0000","last_active":"2026-02-24T07:45:20.735099+0000","last_peered":"2026-02-24T07:45:20.735099+0000","last_clean":"2026-02-24T07:45:20.735099+0000","last_became_active":"2026-02-24T07:43:07.841721+0000","last_became_peered":"2026-02-24T07:43:07.841721+0000","last_unstale":"2026-02-24T07:45:20.735099+0000","last_undegraded":"2026-02-24T07:45:20.735099+0000","last_fullsized":"2026-02-24T07:45:20.735099+0000","mapping_epoch":30,"log_start":"40'56043","ondisk_log_start":"40'56043","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'31794","last_scrub_stamp":"2026-02-24T07:44:27.902040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_clean_scrub_stamp":"2026-02-24T07:44:27.902040+0000","objects_scrubbed":31292,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:45:39.265782+0000","scrub_duration":3732,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1038221312,"num_objects":15842,"num_object_clones":0,"num_object_copies":47526,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15842,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":56046,"num_write_kb":2300416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.b","version":"40'56003","reported_seq":56398,"reported_epoch":40,"state":"active+clean","last_fresh":"2026-02-24T07:45:20.697080+0000","last_change":"2026-02-24T07:44:39.639056+0000","last_active":"2026-02-24T07:45:20.697080+0000","last_peered":"2026-02-24T07:45:20.697080+0000","last_clean":"2026-02-24T07:45:20.697080+0000","last_became_active":"2026-02-24T07:43:07.853069+0000","last_became_peered":"2026-02-24T07:43:07.853069+0000","last_unstale":"2026-02-24T07:45:20.697080+0000","last_undegraded":"2026-02-24T07:45:20.697080+0000","last_fullsized":"2026-02-24T07:45:20.697080+0000","mapping_epoch":30,"log_start":"40'56000","ondisk_log_start":"40'56000","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'36298","last_scrub_stamp":"2026-02-24T07:44:39.638986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_clean_scrub_stamp":"2026-02-24T07:44:39.638986+0000","objects_scrubbed":35656,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:46:09.128452+0000","scrub_duration":3228,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1050607616,"num_objects":16031,"num_object_clones":0,"num_object_copies":48093,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16031,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":56003,"num_write_kb":2305088,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":16567894040,"num_objects":252807,"num_object_clones":0,"num_object_copies":758421,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":252807,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":895329,"num_write_kb":36740289,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":46738321408,"data_stored":46738309192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":93,"internal_metadata":0},"log_size":48,"ondisk_log_size":48,"up":48,"acting":48,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":240,"num_write":135,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":3,"up_from":25,"seq":107374182432,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":15016336,"kb_used_data":14490620,"kb_used_omap":4,"kb_used_meta":525691,"kb_avail":718986864,"statfs":{"total":751619276800,"available":736242548736,"internally_reserved":0,"allocated":14838394880,"data_stored":14838140168,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4448,"internal_metadata":538308256},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":19,"seq":81604378659,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":10326284,"kb_used_data":9552312,"kb_used_omap":4,"kb_used_meta":773883,"kb_avail":723676916,"statfs":{"total":751619276800,"available":741045161984,"internally_reserved":0,"allocated":9781567488,"data_stored":9781316848,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4449,"internal_metadata":792456863},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542179,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":12766604,"kb_used_data":12478968,"kb_used_omap":4,"kb_used_meta":287611,"kb_avail":721236596,"statfs":{"total":751619276800,"available":738546274304,"internally_reserved":0,"allocated":12778463232,"data_stored":12778212072,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":5098,"internal_metadata":294513686},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705703,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":10562576,"kb_used_data":9770620,"kb_used_omap":4,"kb_used_meta":791931,"kb_avail":723440624,"statfs":{"total":751619276800,"available":740803198976,"internally_reserved":0,"allocated":10005114880,"data_stored":10004860168,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4450,"internal_metadata":810938014},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[6],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":9784201216,"data_stored":9784197144,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12551655424,"data_stored":12551651352,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":9759358976,"data_stored":9759358976,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":93,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":14643105792,"data_stored":14643101720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-24T07:45:28.253 INFO:tasks.cephadm.ceph_manager.ceph:increase pool size by 10 2026-02-24T07:45:28.253 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph osd pool set .mgr pg_num 101 2026-02-24T07:45:28.387 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:45:28.927 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2026-02-24T07:45:28.956 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:45:28.890+0000 7fd8b6fb1640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:45:28.956 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:45:28.890+0000 7fd8b6fb1640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:45:29.021 INFO:teuthology.orchestra.run.trial035.stderr:Error EPERM: crimson_allow_pg_split is false; pg_num increase denied for crimson pool 2026-02-24T07:45:29.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:45:28 trial035 ceph-mon[39284]: from='client.14547 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2026-02-24T07:45:29.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:45:28 trial035 ceph-mon[39284]: pgmap v88: 17 pgs: 1 active+clean+scrubbing, 16 active+clean; 13 GiB data, 43 GiB used, 2.7 TiB / 2.7 TiB avail; 0 B/s wr, 9.96k op/s 2026-02-24T07:45:29.241 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:45:29.521 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-24T07:45:29.697 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:45:29.695+0000 7ffb623b1640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:45:29.697 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:45:29.695+0000 7ffb623b1640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:45:29.715 INFO:teuthology.orchestra.run.trial035.stdout:osd.0: {} 2026-02-24T07:45:29.715 INFO:teuthology.orchestra.run.trial035.stderr:osd.0: osd_enable_op_tracker = '' 2026-02-24T07:45:29.729 INFO:teuthology.orchestra.run.trial035.stdout:osd.1: {} 2026-02-24T07:45:29.729 INFO:teuthology.orchestra.run.trial035.stderr:osd.1: osd_enable_op_tracker = '' 2026-02-24T07:45:29.740 INFO:teuthology.orchestra.run.trial035.stdout:osd.2: {} 2026-02-24T07:45:29.740 INFO:teuthology.orchestra.run.trial035.stderr:osd.2: osd_enable_op_tracker = '' 2026-02-24T07:45:29.752 INFO:teuthology.orchestra.run.trial035.stdout:osd.3: {} 2026-02-24T07:45:29.752 INFO:teuthology.orchestra.run.trial035.stderr:osd.3: osd_enable_op_tracker = '' 2026-02-24T07:45:30.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:45:29 trial035 ceph-mon[39284]: 2.3 scrub ok 2026-02-24T07:45:30.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:45:29 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/1589740397' entity='client.admin' cmd={"prefix": "osd pool set", "pool": ".mgr", "var": "pg_num", "val": "101"} : dispatch 2026-02-24T07:45:30.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:45:29 trial035 ceph-mon[39284]: 2.e scrub starts 2026-02-24T07:45:30.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:45:29 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/1256496562' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2026-02-24T07:45:30.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:45:29 trial035 ceph-mon[39284]: from='mgr.14284 10.20.193.35:0/3681373222' entity='mgr.x' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2026-02-24T07:45:31.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:45:30 trial035 ceph-mon[39284]: pgmap v89: 17 pgs: 1 active+clean+scrubbing, 16 active+clean; 13 GiB data, 43 GiB used, 2.7 TiB / 2.7 TiB avail; 0 B/s wr, 6.61k op/s 2026-02-24T07:45:31.205 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:45:30 trial035 ceph-mon[39284]: 2.e scrub ok 2026-02-24T07:45:31.793 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2026-02-24T07:45:31.925 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:45:32.043 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:45:32.040+0000 7fe5d4c2b640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:45:32.043 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:45:32.041+0000 7fe5d4c2b640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:45:32.057 INFO:teuthology.orchestra.run.trial035.stdout:osd.0: {} 2026-02-24T07:45:32.057 INFO:teuthology.orchestra.run.trial035.stderr:osd.0: osd_enable_op_tracker = '' 2026-02-24T07:45:32.068 INFO:teuthology.orchestra.run.trial035.stdout:osd.1: {} 2026-02-24T07:45:32.069 INFO:teuthology.orchestra.run.trial035.stderr:osd.1: osd_enable_op_tracker = '' 2026-02-24T07:45:32.069 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:45:31 trial035 ceph-mon[39284]: pgmap v90: 17 pgs: 17 active+clean; 11 GiB data, 37 GiB used, 2.7 TiB / 2.7 TiB avail; 0 B/s wr, 9.13k op/s 2026-02-24T07:45:32.082 INFO:teuthology.orchestra.run.trial035.stdout:osd.2: {} 2026-02-24T07:45:32.082 INFO:teuthology.orchestra.run.trial035.stderr:osd.2: osd_enable_op_tracker = '' 2026-02-24T07:45:32.095 INFO:teuthology.orchestra.run.trial035.stdout:osd.3: {} 2026-02-24T07:45:32.095 INFO:teuthology.orchestra.run.trial035.stderr:osd.3: osd_enable_op_tracker = '' 2026-02-24T07:45:33.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:45:32 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/927139569' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2026-02-24T07:45:33.205 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:45:32 trial035 ceph-mon[39284]: 2.7 scrub starts 2026-02-24T07:45:34.135 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2026-02-24T07:45:34.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:45:33 trial035 ceph-mon[39284]: 2.7 scrub ok 2026-02-24T07:45:34.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:45:33 trial035 ceph-mon[39284]: pgmap v91: 17 pgs: 17 active+clean; 11 GiB data, 37 GiB used, 2.7 TiB / 2.7 TiB avail; 0 B/s wr, 5.71k op/s 2026-02-24T07:45:34.269 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:45:34.388 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:45:34.386+0000 7fa83a09b640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:45:34.388 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:45:34.387+0000 7fa83a09b640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:45:34.403 INFO:teuthology.orchestra.run.trial035.stdout:osd.0: {} 2026-02-24T07:45:34.404 INFO:teuthology.orchestra.run.trial035.stderr:osd.0: osd_enable_op_tracker = '' 2026-02-24T07:45:34.416 INFO:teuthology.orchestra.run.trial035.stdout:osd.1: {} 2026-02-24T07:45:34.416 INFO:teuthology.orchestra.run.trial035.stderr:osd.1: osd_enable_op_tracker = '' 2026-02-24T07:45:34.430 INFO:teuthology.orchestra.run.trial035.stdout:osd.2: {} 2026-02-24T07:45:34.430 INFO:teuthology.orchestra.run.trial035.stderr:osd.2: osd_enable_op_tracker = '' 2026-02-24T07:45:34.441 INFO:teuthology.orchestra.run.trial035.stdout:osd.3: {} 2026-02-24T07:45:34.441 INFO:teuthology.orchestra.run.trial035.stderr:osd.3: osd_enable_op_tracker = '' 2026-02-24T07:45:34.522 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2026-02-24T07:45:34.522 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2026-02-24T07:45:34.522 INFO:tasks.thrashosds.thrasher:Growing pool unique_pool_0 2026-02-24T07:45:34.523 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph pg dump --format=json 2026-02-24T07:45:34.651 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:45:34.779 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:45:34.777+0000 7ff588939640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:45:34.779 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:45:34.778+0000 7ff588939640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:45:34.912 INFO:teuthology.orchestra.run.trial035.stdout: 2026-02-24T07:45:34.912 INFO:teuthology.orchestra.run.trial035.stderr:dumped all 2026-02-24T07:45:35.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:45:34 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/4003031906' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2026-02-24T07:45:35.440 INFO:teuthology.orchestra.run.trial035.stdout:{"pg_ready":true,"pg_map":{"version":91,"stamp":"2026-02-24T07:45:33.678403+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":12233540152,"num_objects":186663,"num_object_clones":0,"num_object_copies":559989,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":186663,"num_whiteouts":0,"num_read":137,"num_read_kb":241,"num_write":961610,"num_write_kb":36742439,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":51,"ondisk_log_size":51,"up":51,"acting":51,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":51,"num_osds":4,"num_per_pool_osds":4,"num_per_pool_omap_osds":4,"kb":2936012800,"kb_used":38876152,"kb_used_data":36811880,"kb_used_omap":19,"kb_used_meta":2064044,"kb_avail":2897136648,"statfs":{"total":3006477107200,"available":2966667927552,"internally_reserved":0,"allocated":37695365120,"data_stored":37694353896,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19468,"internal_metadata":2113582068},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[4,1,10,4,0,19],"upper_bound":64},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-4487315456,"num_objects":-68471,"num_object_clones":0,"num_object_copies":-205413,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-68471,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":68471,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001492"},"pg_stats":[{"pgid":"2.f","version":"40'59341","reported_seq":59815,"reported_epoch":40,"state":"active+clean","last_fresh":"2026-02-24T07:45:30.673952+0000","last_change":"2026-02-24T07:45:26.689907+0000","last_active":"2026-02-24T07:45:30.673952+0000","last_peered":"2026-02-24T07:45:30.673952+0000","last_clean":"2026-02-24T07:45:30.673952+0000","last_became_active":"2026-02-24T07:43:07.852899+0000","last_became_peered":"2026-02-24T07:43:07.852899+0000","last_unstale":"2026-02-24T07:45:30.673952+0000","last_undegraded":"2026-02-24T07:45:30.673952+0000","last_fullsized":"2026-02-24T07:45:30.673952+0000","mapping_epoch":30,"log_start":"40'59338","ondisk_log_start":"40'59338","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'57836","last_scrub_stamp":"2026-02-24T07:45:26.689827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_clean_scrub_stamp":"2026-02-24T07:45:26.689827+0000","objects_scrubbed":13173,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:46:28.773425+0000","scrub_duration":1082,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":751501312,"num_objects":11467,"num_object_clones":0,"num_object_copies":34401,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11467,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":59341,"num_write_kb":2265856,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.e","version":"40'60267","reported_seq":60710,"reported_epoch":40,"state":"active+clean","last_fresh":"2026-02-24T07:45:30.736815+0000","last_change":"2026-02-24T07:45:30.620522+0000","last_active":"2026-02-24T07:45:30.736815+0000","last_peered":"2026-02-24T07:45:30.736815+0000","last_clean":"2026-02-24T07:45:30.736815+0000","last_became_active":"2026-02-24T07:43:07.852815+0000","last_became_peered":"2026-02-24T07:43:07.852815+0000","last_unstale":"2026-02-24T07:45:30.736815+0000","last_undegraded":"2026-02-24T07:45:30.736815+0000","last_fullsized":"2026-02-24T07:45:30.736815+0000","mapping_epoch":30,"log_start":"40'60264","ondisk_log_start":"40'60264","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'60215","last_scrub_stamp":"2026-02-24T07:45:30.620432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_clean_scrub_stamp":"2026-02-24T07:45:30.620432+0000","objects_scrubbed":12027,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:46:34.106668+0000","scrub_duration":1600,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":765657088,"num_objects":11683,"num_object_clones":0,"num_object_copies":35049,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11683,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":60267,"num_write_kb":2302400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.d","version":"40'60457","reported_seq":60809,"reported_epoch":40,"state":"active+clean","last_fresh":"2026-02-24T07:45:30.736093+0000","last_change":"2026-02-24T07:44:26.596366+0000","last_active":"2026-02-24T07:45:30.736093+0000","last_peered":"2026-02-24T07:45:30.736093+0000","last_clean":"2026-02-24T07:45:30.736093+0000","last_became_active":"2026-02-24T07:43:07.853975+0000","last_became_peered":"2026-02-24T07:43:07.853975+0000","last_unstale":"2026-02-24T07:45:30.736093+0000","last_undegraded":"2026-02-24T07:45:30.736093+0000","last_fullsized":"2026-02-24T07:45:30.736093+0000","mapping_epoch":30,"log_start":"40'60454","ondisk_log_start":"40'60454","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'31656","last_scrub_stamp":"2026-02-24T07:44:26.596292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_clean_scrub_stamp":"2026-02-24T07:44:26.596292+0000","objects_scrubbed":31097,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:45:55.416603+0000","scrub_duration":3380,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":768344088,"num_objects":11725,"num_object_clones":0,"num_object_copies":35175,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11725,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":60457,"num_write_kb":2309761,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"40'60412","reported_seq":60770,"reported_epoch":40,"state":"active+clean","last_fresh":"2026-02-24T07:45:30.734024+0000","last_change":"2026-02-24T07:44:29.667393+0000","last_active":"2026-02-24T07:45:30.734024+0000","last_peered":"2026-02-24T07:45:30.734024+0000","last_clean":"2026-02-24T07:45:30.734024+0000","last_became_active":"2026-02-24T07:43:07.854428+0000","last_became_peered":"2026-02-24T07:43:07.854428+0000","last_unstale":"2026-02-24T07:45:30.734024+0000","last_undegraded":"2026-02-24T07:45:30.734024+0000","last_fullsized":"2026-02-24T07:45:30.734024+0000","mapping_epoch":30,"log_start":"40'60409","ondisk_log_start":"40'60409","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'32466","last_scrub_stamp":"2026-02-24T07:44:29.667348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_clean_scrub_stamp":"2026-02-24T07:44:29.667348+0000","objects_scrubbed":31970,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:45:43.092916+0000","scrub_duration":3552,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":774504448,"num_objects":11818,"num_object_clones":0,"num_object_copies":35454,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11818,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":60412,"num_write_kb":2311360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"29'99","reported_seq":218,"reported_epoch":40,"state":"active+clean","last_fresh":"2026-02-24T07:45:26.125455+0000","last_change":"2026-02-24T07:45:26.125455+0000","last_active":"2026-02-24T07:45:26.125455+0000","last_peered":"2026-02-24T07:45:26.125455+0000","last_clean":"2026-02-24T07:45:26.125455+0000","last_became_active":"2026-02-24T07:42:54.186719+0000","last_became_peered":"2026-02-24T07:42:54.186719+0000","last_unstale":"2026-02-24T07:45:26.125455+0000","last_undegraded":"2026-02-24T07:45:26.125455+0000","last_fullsized":"2026-02-24T07:45:26.125455+0000","mapping_epoch":27,"log_start":"29'96","ondisk_log_start":"29'96","created":20,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'99","last_scrub_stamp":"2026-02-24T07:45:26.125426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:42:43.640776+0000","last_clean_scrub_stamp":"2026-02-24T07:45:26.125426+0000","objects_scrubbed":2,"log_size":3,"log_dups_size":96,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:46:34.126365+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":240,"num_write":135,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"40'60302","reported_seq":60778,"reported_epoch":40,"state":"active+clean","last_fresh":"2026-02-24T07:45:30.697833+0000","last_change":"2026-02-24T07:45:28.799643+0000","last_active":"2026-02-24T07:45:30.697833+0000","last_peered":"2026-02-24T07:45:30.697833+0000","last_clean":"2026-02-24T07:45:30.697833+0000","last_became_active":"2026-02-24T07:43:07.842530+0000","last_became_peered":"2026-02-24T07:43:07.842530+0000","last_unstale":"2026-02-24T07:45:30.697833+0000","last_undegraded":"2026-02-24T07:45:30.697833+0000","last_fullsized":"2026-02-24T07:45:30.697833+0000","mapping_epoch":30,"log_start":"40'60299","ondisk_log_start":"40'60299","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'59525","last_scrub_stamp":"2026-02-24T07:45:28.799605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_clean_scrub_stamp":"2026-02-24T07:45:28.799605+0000","objects_scrubbed":12919,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:46:45.174239+0000","scrub_duration":1680,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":773718016,"num_objects":11806,"num_object_clones":0,"num_object_copies":35418,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11806,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":60302,"num_write_kb":2307456,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"40'59631","reported_seq":59983,"reported_epoch":40,"state":"active+clean","last_fresh":"2026-02-24T07:45:30.696761+0000","last_change":"2026-02-24T07:44:28.446165+0000","last_active":"2026-02-24T07:45:30.696761+0000","last_peered":"2026-02-24T07:45:30.696761+0000","last_clean":"2026-02-24T07:45:30.696761+0000","last_became_active":"2026-02-24T07:43:07.853885+0000","last_became_peered":"2026-02-24T07:43:07.853885+0000","last_unstale":"2026-02-24T07:45:30.696761+0000","last_undegraded":"2026-02-24T07:45:30.696761+0000","last_fullsized":"2026-02-24T07:45:30.696761+0000","mapping_epoch":30,"log_start":"40'59628","ondisk_log_start":"40'59628","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'31685","last_scrub_stamp":"2026-02-24T07:44:28.446064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_clean_scrub_stamp":"2026-02-24T07:44:28.446064+0000","objects_scrubbed":31280,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:45:52.594601+0000","scrub_duration":3117,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":759758848,"num_objects":11593,"num_object_clones":0,"num_object_copies":34779,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11593,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":59631,"num_write_kb":2279168,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"40'60371","reported_seq":60742,"reported_epoch":40,"state":"active+clean","last_fresh":"2026-02-24T07:45:30.738278+0000","last_change":"2026-02-24T07:44:33.357960+0000","last_active":"2026-02-24T07:45:30.738278+0000","last_peered":"2026-02-24T07:45:30.738278+0000","last_clean":"2026-02-24T07:45:30.738278+0000","last_became_active":"2026-02-24T07:43:07.854212+0000","last_became_peered":"2026-02-24T07:43:07.854212+0000","last_unstale":"2026-02-24T07:45:30.738278+0000","last_undegraded":"2026-02-24T07:45:30.738278+0000","last_fullsized":"2026-02-24T07:45:30.738278+0000","mapping_epoch":30,"log_start":"40'60368","ondisk_log_start":"40'60368","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'33783","last_scrub_stamp":"2026-02-24T07:44:33.357894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_clean_scrub_stamp":"2026-02-24T07:44:33.357894+0000","objects_scrubbed":33200,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:45:38.914707+0000","scrub_duration":3247,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":771293184,"num_objects":11769,"num_object_clones":0,"num_object_copies":35307,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11769,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":60371,"num_write_kb":2308480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[1,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"40'60457","reported_seq":60848,"reported_epoch":40,"state":"active+clean","last_fresh":"2026-02-24T07:45:30.696711+0000","last_change":"2026-02-24T07:44:42.974303+0000","last_active":"2026-02-24T07:45:30.696711+0000","last_peered":"2026-02-24T07:45:30.696711+0000","last_clean":"2026-02-24T07:45:30.696711+0000","last_became_active":"2026-02-24T07:43:07.842864+0000","last_became_peered":"2026-02-24T07:43:07.842864+0000","last_unstale":"2026-02-24T07:45:30.696711+0000","last_undegraded":"2026-02-24T07:45:30.696711+0000","last_fullsized":"2026-02-24T07:45:30.696711+0000","mapping_epoch":30,"log_start":"40'60454","ondisk_log_start":"40'60454","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'37783","last_scrub_stamp":"2026-02-24T07:44:42.974258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_clean_scrub_stamp":"2026-02-24T07:44:42.974258+0000","objects_scrubbed":35124,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:45:49.565319+0000","scrub_duration":3593,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":760283136,"num_objects":11601,"num_object_clones":0,"num_object_copies":34803,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11601,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":60457,"num_write_kb":2305856,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.4","version":"40'59917","reported_seq":60232,"reported_epoch":40,"state":"active+clean","last_fresh":"2026-02-24T07:45:30.739909+0000","last_change":"2026-02-24T07:44:16.909673+0000","last_active":"2026-02-24T07:45:30.739909+0000","last_peered":"2026-02-24T07:45:30.739909+0000","last_clean":"2026-02-24T07:45:30.739909+0000","last_became_active":"2026-02-24T07:43:07.853966+0000","last_became_peered":"2026-02-24T07:43:07.853966+0000","last_unstale":"2026-02-24T07:45:30.739909+0000","last_undegraded":"2026-02-24T07:45:30.739909+0000","last_fullsized":"2026-02-24T07:45:30.739909+0000","mapping_epoch":30,"log_start":"40'59914","ondisk_log_start":"40'59914","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'27932","last_scrub_stamp":"2026-02-24T07:44:16.909624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_clean_scrub_stamp":"2026-02-24T07:44:16.909624+0000","objects_scrubbed":27480,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:45:43.624405+0000","scrub_duration":2774,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":764870656,"num_objects":11671,"num_object_clones":0,"num_object_copies":35013,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11671,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":59917,"num_write_kb":2290816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"40'60090","reported_seq":60469,"reported_epoch":40,"state":"active+clean","last_fresh":"2026-02-24T07:45:30.698162+0000","last_change":"2026-02-24T07:44:35.717081+0000","last_active":"2026-02-24T07:45:30.698162+0000","last_peered":"2026-02-24T07:45:30.698162+0000","last_clean":"2026-02-24T07:45:30.698162+0000","last_became_active":"2026-02-24T07:43:07.853878+0000","last_became_peered":"2026-02-24T07:43:07.853878+0000","last_unstale":"2026-02-24T07:45:30.698162+0000","last_undegraded":"2026-02-24T07:45:30.698162+0000","last_fullsized":"2026-02-24T07:45:30.698162+0000","mapping_epoch":30,"log_start":"40'60087","ondisk_log_start":"40'60087","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'34578","last_scrub_stamp":"2026-02-24T07:44:35.717022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_clean_scrub_stamp":"2026-02-24T07:44:35.717022+0000","objects_scrubbed":33884,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:46:00.809440+0000","scrub_duration":3376,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":756678656,"num_objects":11546,"num_object_clones":0,"num_object_copies":34638,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11546,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":60090,"num_write_kb":2292352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"40'60275","reported_seq":60636,"reported_epoch":40,"state":"active+clean","last_fresh":"2026-02-24T07:45:30.727960+0000","last_change":"2026-02-24T07:44:30.553008+0000","last_active":"2026-02-24T07:45:30.727960+0000","last_peered":"2026-02-24T07:45:30.727960+0000","last_clean":"2026-02-24T07:45:30.727960+0000","last_became_active":"2026-02-24T07:43:07.841993+0000","last_became_peered":"2026-02-24T07:43:07.841993+0000","last_unstale":"2026-02-24T07:45:30.727960+0000","last_undegraded":"2026-02-24T07:45:30.727960+0000","last_fullsized":"2026-02-24T07:45:30.727960+0000","mapping_epoch":30,"log_start":"40'60272","ondisk_log_start":"40'60272","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'32640","last_scrub_stamp":"2026-02-24T07:44:30.552965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_clean_scrub_stamp":"2026-02-24T07:44:30.552965+0000","objects_scrubbed":32121,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:45:56.537381+0000","scrub_duration":3414,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":755957760,"num_objects":11535,"num_object_clones":0,"num_object_copies":34605,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11535,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":60275,"num_write_kb":2297920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"40'59407","reported_seq":59724,"reported_epoch":40,"state":"active+clean","last_fresh":"2026-02-24T07:45:30.698110+0000","last_change":"2026-02-24T07:44:18.020443+0000","last_active":"2026-02-24T07:45:30.698110+0000","last_peered":"2026-02-24T07:45:30.698110+0000","last_clean":"2026-02-24T07:45:30.698110+0000","last_became_active":"2026-02-24T07:43:07.841920+0000","last_became_peered":"2026-02-24T07:43:07.841920+0000","last_unstale":"2026-02-24T07:45:30.698110+0000","last_undegraded":"2026-02-24T07:45:30.698110+0000","last_fullsized":"2026-02-24T07:45:30.698110+0000","mapping_epoch":30,"log_start":"40'59404","ondisk_log_start":"40'59404","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'28226","last_scrub_stamp":"2026-02-24T07:44:18.020379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_clean_scrub_stamp":"2026-02-24T07:44:18.020379+0000","objects_scrubbed":27705,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"queued for scrub","scrub_duration":2601,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":767885312,"num_objects":11717,"num_object_clones":0,"num_object_copies":35151,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11717,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":59407,"num_write_kb":2275968,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.8","version":"40'59578","reported_seq":59937,"reported_epoch":40,"state":"active+clean","last_fresh":"2026-02-24T07:45:30.693905+0000","last_change":"2026-02-24T07:44:31.118780+0000","last_active":"2026-02-24T07:45:30.693905+0000","last_peered":"2026-02-24T07:45:30.693905+0000","last_clean":"2026-02-24T07:45:30.693905+0000","last_became_active":"2026-02-24T07:43:07.853758+0000","last_became_peered":"2026-02-24T07:43:07.853758+0000","last_unstale":"2026-02-24T07:45:30.693905+0000","last_undegraded":"2026-02-24T07:45:30.693905+0000","last_fullsized":"2026-02-24T07:45:30.693905+0000","mapping_epoch":30,"log_start":"40'59575","ondisk_log_start":"40'59575","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'32571","last_scrub_stamp":"2026-02-24T07:44:31.118720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_clean_scrub_stamp":"2026-02-24T07:44:31.118720+0000","objects_scrubbed":32077,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:45:44.647478+0000","scrub_duration":2775,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":771358720,"num_objects":11770,"num_object_clones":0,"num_object_copies":35310,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11770,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":59578,"num_write_kb":2283136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"40'60315","reported_seq":60710,"reported_epoch":40,"state":"active+clean","last_fresh":"2026-02-24T07:45:30.728915+0000","last_change":"2026-02-24T07:44:41.349757+0000","last_active":"2026-02-24T07:45:30.728915+0000","last_peered":"2026-02-24T07:45:30.728915+0000","last_clean":"2026-02-24T07:45:30.728915+0000","last_became_active":"2026-02-24T07:43:07.854377+0000","last_became_peered":"2026-02-24T07:43:07.854377+0000","last_unstale":"2026-02-24T07:45:30.728915+0000","last_undegraded":"2026-02-24T07:45:30.728915+0000","last_fullsized":"2026-02-24T07:45:30.728915+0000","mapping_epoch":30,"log_start":"40'60312","ondisk_log_start":"40'60312","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'36997","last_scrub_stamp":"2026-02-24T07:44:41.349228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_clean_scrub_stamp":"2026-02-24T07:44:41.349228+0000","objects_scrubbed":35574,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:45:50.611975+0000","scrub_duration":3205,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":766312448,"num_objects":11693,"num_object_clones":0,"num_object_copies":35079,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11693,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":60315,"num_write_kb":2304256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.a","version":"40'60267","reported_seq":60622,"reported_epoch":40,"state":"active+clean","last_fresh":"2026-02-24T07:45:30.737311+0000","last_change":"2026-02-24T07:44:27.902144+0000","last_active":"2026-02-24T07:45:30.737311+0000","last_peered":"2026-02-24T07:45:30.737311+0000","last_clean":"2026-02-24T07:45:30.737311+0000","last_became_active":"2026-02-24T07:43:07.841721+0000","last_became_peered":"2026-02-24T07:43:07.841721+0000","last_unstale":"2026-02-24T07:45:30.737311+0000","last_undegraded":"2026-02-24T07:45:30.737311+0000","last_fullsized":"2026-02-24T07:45:30.737311+0000","mapping_epoch":30,"log_start":"40'60264","ondisk_log_start":"40'60264","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'31794","last_scrub_stamp":"2026-02-24T07:44:27.902040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_clean_scrub_stamp":"2026-02-24T07:44:27.902040+0000","objects_scrubbed":31292,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:45:39.265782+0000","scrub_duration":3732,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":761593856,"num_objects":11621,"num_object_clones":0,"num_object_copies":34863,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11621,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":60267,"num_write_kb":2300416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.b","version":"40'60388","reported_seq":60784,"reported_epoch":40,"state":"active+clean","last_fresh":"2026-02-24T07:45:30.698785+0000","last_change":"2026-02-24T07:44:39.639056+0000","last_active":"2026-02-24T07:45:30.698785+0000","last_peered":"2026-02-24T07:45:30.698785+0000","last_clean":"2026-02-24T07:45:30.698785+0000","last_became_active":"2026-02-24T07:43:07.853069+0000","last_became_peered":"2026-02-24T07:43:07.853069+0000","last_unstale":"2026-02-24T07:45:30.698785+0000","last_undegraded":"2026-02-24T07:45:30.698785+0000","last_fullsized":"2026-02-24T07:45:30.698785+0000","mapping_epoch":30,"log_start":"40'60385","ondisk_log_start":"40'60385","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'36298","last_scrub_stamp":"2026-02-24T07:44:39.638986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_clean_scrub_stamp":"2026-02-24T07:44:39.638986+0000","objects_scrubbed":35656,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:46:09.128452+0000","scrub_duration":3228,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":763232256,"num_objects":11646,"num_object_clones":0,"num_object_copies":34938,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11646,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":60388,"num_write_kb":2305088,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":12232949784,"num_objects":186661,"num_object_clones":0,"num_object_copies":559983,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":186661,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":961475,"num_write_kb":36740289,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":36731105280,"data_stored":36731093064,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":279,"internal_metadata":0},"log_size":48,"ondisk_log_size":48,"up":48,"acting":48,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":240,"num_write":135,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":744,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":3,"up_from":25,"seq":107374182434,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":11308624,"kb_used_data":10674556,"kb_used_omap":5,"kb_used_meta":633978,"kb_avail":722694576,"statfs":{"total":751619276800,"available":740039245824,"internally_reserved":0,"allocated":10930745344,"data_stored":10930490632,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":5285,"internal_metadata":649194331},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[4,1,10,4],"upper_bound":16},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":19,"seq":81604378660,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":9311244,"kb_used_data":8500152,"kb_used_omap":4,"kb_used_meta":811067,"kb_avail":724691956,"statfs":{"total":751619276800,"available":742084562944,"internally_reserved":0,"allocated":8704155648,"data_stored":8703905008,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4542,"internal_metadata":830533186},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542181,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":9598668,"kb_used_data":9220792,"kb_used_omap":5,"kb_used_meta":377850,"kb_avail":724404532,"statfs":{"total":751619276800,"available":741790240768,"internally_reserved":0,"allocated":9442091008,"data_stored":9441839848,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":5191,"internal_metadata":386919353},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,19],"upper_bound":64},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705704,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":8657616,"kb_used_data":8416380,"kb_used_omap":4,"kb_used_meta":241147,"kb_avail":725345584,"statfs":{"total":751619276800,"available":742753878016,"internally_reserved":0,"allocated":8618373120,"data_stored":8618118408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4450,"internal_metadata":246935198},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":744,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8429965312,"data_stored":8429961240,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":9184940032,"data_stored":9184935960,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":93,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8407154688,"data_stored":8407154688,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":93,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":10709045248,"data_stored":10709041176,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":93,"internal_metadata":0}]}} 2026-02-24T07:45:35.441 INFO:tasks.cephadm.ceph_manager.ceph:increase pool size by 10 2026-02-24T07:45:35.442 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph osd pool set unique_pool_0 pg_num 76 2026-02-24T07:45:35.569 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:45:35.955 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:45:35 trial035 ceph-mon[39284]: from='client.14557 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2026-02-24T07:45:36.128 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:45:36.076+0000 7f912a607640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:45:36.128 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:45:36.076+0000 7f912a607640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:45:36.225 INFO:teuthology.orchestra.run.trial035.stderr:Error EPERM: crimson_allow_pg_split is false; pg_num increase denied for crimson pool 2026-02-24T07:45:36.225 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:45:35 trial035 ceph-mon[39284]: pgmap v92: 17 pgs: 17 active+clean; 11 GiB data, 35 GiB used, 2.7 TiB / 2.7 TiB avail; 0 B/s wr, 5.90k op/s 2026-02-24T07:45:36.382 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-24T07:45:36.482 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2026-02-24T07:45:36.612 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:45:36.733 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:45:36.732+0000 7f1b2f212640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:45:36.733 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:45:36.732+0000 7f1b2f212640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:45:36.749 INFO:teuthology.orchestra.run.trial035.stdout:osd.0: {} 2026-02-24T07:45:36.749 INFO:teuthology.orchestra.run.trial035.stderr:osd.0: osd_enable_op_tracker = '' 2026-02-24T07:45:36.763 INFO:teuthology.orchestra.run.trial035.stdout:osd.1: {} 2026-02-24T07:45:36.764 INFO:teuthology.orchestra.run.trial035.stderr:osd.1: osd_enable_op_tracker = '' 2026-02-24T07:45:36.774 INFO:teuthology.orchestra.run.trial035.stdout:osd.2: {} 2026-02-24T07:45:36.775 INFO:teuthology.orchestra.run.trial035.stderr:osd.2: osd_enable_op_tracker = '' 2026-02-24T07:45:36.788 INFO:teuthology.orchestra.run.trial035.stdout:osd.3: {} 2026-02-24T07:45:36.788 INFO:teuthology.orchestra.run.trial035.stderr:osd.3: osd_enable_op_tracker = '' 2026-02-24T07:45:37.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:45:36 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/339773727' entity='client.admin' cmd={"prefix": "osd pool set", "pool": "unique_pool_0", "var": "pg_num", "val": "76"} : dispatch 2026-02-24T07:45:37.205 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:45:36 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/1183723161' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2026-02-24T07:45:37.205 INFO:journalctl@ceph.mgr.x.trial035.stdout:Feb 24 07:45:36 trial035 ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-mgr-x[39621]: ::ffff:10.20.193.91 - - [24/Feb/2026:07:45:36] "GET /metrics HTTP/1.1" 200 4231 "" "Prometheus/3.6.0" 2026-02-24T07:45:37.356 INFO:journalctl@ceph.prometheus.a.trial091.stdout:Feb 24 07:45:36 trial091 ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-prometheus-a[66007]: time=2026-02-24T07:45:36.973Z level=ERROR source=scrape.go:1630 msg="Failed to determine correct type of scrape target." component="scrape manager" scrape_pool=ceph target=http://trial035:9283/metrics content_type="text/html;charset=utf-8" fallback_media_type="" err="received unsupported Content-Type \"text/html;charset=utf-8\" and no fallback_scrape_protocol specified for target" 2026-02-24T07:45:38.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:45:37 trial035 ceph-mon[39284]: pgmap v93: 17 pgs: 17 active+clean; 9.1 GiB data, 30 GiB used, 2.7 TiB / 2.7 TiB avail; 0 B/s wr, 8.63k op/s 2026-02-24T07:45:38.828 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2026-02-24T07:45:38.960 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:45:39.084 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:45:39.082+0000 7f8a4dae5640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:45:39.084 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:45:39.082+0000 7f8a4dae5640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:45:39.098 INFO:teuthology.orchestra.run.trial035.stdout:osd.0: {} 2026-02-24T07:45:39.098 INFO:teuthology.orchestra.run.trial035.stderr:osd.0: osd_enable_op_tracker = '' 2026-02-24T07:45:39.111 INFO:teuthology.orchestra.run.trial035.stdout:osd.1: {} 2026-02-24T07:45:39.112 INFO:teuthology.orchestra.run.trial035.stderr:osd.1: osd_enable_op_tracker = '' 2026-02-24T07:45:39.124 INFO:teuthology.orchestra.run.trial035.stdout:osd.2: {} 2026-02-24T07:45:39.125 INFO:teuthology.orchestra.run.trial035.stderr:osd.2: osd_enable_op_tracker = '' 2026-02-24T07:45:39.136 INFO:teuthology.orchestra.run.trial035.stdout:osd.3: {} 2026-02-24T07:45:39.136 INFO:teuthology.orchestra.run.trial035.stderr:osd.3: osd_enable_op_tracker = '' 2026-02-24T07:45:40.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:45:39 trial035 ceph-mon[39284]: 2.1 scrub starts 2026-02-24T07:45:40.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:45:39 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/2010096221' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2026-02-24T07:45:41.178 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2026-02-24T07:45:41.203 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:45:40 trial035 ceph-mon[39284]: pgmap v94: 17 pgs: 17 active+clean; 9.1 GiB data, 30 GiB used, 2.7 TiB / 2.7 TiB avail; 0 B/s wr, 5.63k op/s 2026-02-24T07:45:41.203 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:45:40 trial035 ceph-mon[39284]: 2.1 scrub ok 2026-02-24T07:45:41.310 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:45:41.384 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2026-02-24T07:45:41.384 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2026-02-24T07:45:41.384 INFO:tasks.thrashosds.thrasher:Growing pool unique_pool_0 2026-02-24T07:45:41.385 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph pg dump --format=json 2026-02-24T07:45:41.594 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:45:41.794 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:45:41.717+0000 7f1cc1825640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:45:41.794 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:45:41.717+0000 7f1cc1825640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:45:41.794 INFO:teuthology.orchestra.run.trial035.stderr:osd.0: osd_enable_op_tracker = '' 2026-02-24T07:45:41.794 INFO:teuthology.orchestra.run.trial035.stderr:osd.1: osd_enable_op_tracker = '' 2026-02-24T07:45:41.794 INFO:teuthology.orchestra.run.trial035.stderr:osd.2: osd_enable_op_tracker = '' 2026-02-24T07:45:41.794 INFO:teuthology.orchestra.run.trial035.stderr:osd.3: osd_enable_op_tracker = '' 2026-02-24T07:45:41.794 INFO:teuthology.orchestra.run.trial035.stdout:osd.0: {} 2026-02-24T07:45:41.794 INFO:teuthology.orchestra.run.trial035.stdout:osd.1: {} 2026-02-24T07:45:41.794 INFO:teuthology.orchestra.run.trial035.stdout:osd.2: {} 2026-02-24T07:45:41.794 INFO:teuthology.orchestra.run.trial035.stdout:osd.3: {} 2026-02-24T07:45:42.134 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:45:41 trial035 ceph-mon[39284]: 2.a scrub starts 2026-02-24T07:45:42.135 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:45:41 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/2861334525' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2026-02-24T07:45:42.881 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:45:42.879+0000 7f6c901e5640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:45:42.882 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:45:42.880+0000 7f6c901e5640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:45:43.018 INFO:teuthology.orchestra.run.trial035.stdout: 2026-02-24T07:45:43.018 INFO:teuthology.orchestra.run.trial035.stderr:dumped all 2026-02-24T07:45:43.019 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:45:42 trial035 ceph-mon[39284]: pgmap v95: 17 pgs: 17 active+clean; 6.9 GiB data, 23 GiB used, 2.7 TiB / 2.7 TiB avail; 0 B/s wr, 8.71k op/s 2026-02-24T07:45:43.019 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:45:42 trial035 ceph-mon[39284]: 2.a scrub ok 2026-02-24T07:45:43.057 INFO:teuthology.orchestra.run.trial035.stdout:{"pg_ready":true,"pg_map":{"version":95,"stamp":"2026-02-24T07:45:41.679385+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":7362118200,"num_objects":112331,"num_object_clones":0,"num_object_copies":336993,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":112331,"num_whiteouts":0,"num_read":137,"num_read_kb":241,"num_write":1035942,"num_write_kb":36742439,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":53,"ondisk_log_size":53,"up":51,"acting":51,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":51,"num_osds":4,"num_per_pool_osds":4,"num_per_pool_omap_osds":4,"kb":2936012800,"kb_used":24199288,"kb_used_data":22308008,"kb_used_omap":19,"kb_used_meta":1891052,"kb_avail":2911813512,"statfs":{"total":3006477107200,"available":2981697036288,"internally_reserved":0,"allocated":22843400192,"data_stored":22842388968,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19654,"internal_metadata":1936438074},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-6850478080,"num_objects":-104530,"num_object_clones":0,"num_object_copies":-313590,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-104530,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":104530,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001492"},"pg_stats":[{"pgid":"2.f","version":"40'63933","reported_seq":64408,"reported_epoch":40,"state":"active+clean","last_fresh":"2026-02-24T07:45:40.674108+0000","last_change":"2026-02-24T07:45:26.689907+0000","last_active":"2026-02-24T07:45:40.674108+0000","last_peered":"2026-02-24T07:45:40.674108+0000","last_clean":"2026-02-24T07:45:40.674108+0000","last_became_active":"2026-02-24T07:43:07.852899+0000","last_became_peered":"2026-02-24T07:43:07.852899+0000","last_unstale":"2026-02-24T07:45:40.674108+0000","last_undegraded":"2026-02-24T07:45:40.674108+0000","last_fullsized":"2026-02-24T07:45:40.674108+0000","mapping_epoch":30,"log_start":"40'63930","ondisk_log_start":"40'63930","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'57836","last_scrub_stamp":"2026-02-24T07:45:26.689827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_clean_scrub_stamp":"2026-02-24T07:45:26.689827+0000","objects_scrubbed":13173,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:46:28.773425+0000","scrub_duration":1082,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":450560000,"num_objects":6875,"num_object_clones":0,"num_object_copies":20625,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6875,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":63933,"num_write_kb":2265856,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.e","version":"40'64864","reported_seq":65306,"reported_epoch":40,"state":"active+clean","last_fresh":"2026-02-24T07:45:40.738499+0000","last_change":"2026-02-24T07:45:30.620522+0000","last_active":"2026-02-24T07:45:40.738499+0000","last_peered":"2026-02-24T07:45:40.738499+0000","last_clean":"2026-02-24T07:45:40.738499+0000","last_became_active":"2026-02-24T07:43:07.852815+0000","last_became_peered":"2026-02-24T07:43:07.852815+0000","last_unstale":"2026-02-24T07:45:40.738499+0000","last_undegraded":"2026-02-24T07:45:40.738499+0000","last_fullsized":"2026-02-24T07:45:40.738499+0000","mapping_epoch":30,"log_start":"40'64861","ondisk_log_start":"40'64861","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'60215","last_scrub_stamp":"2026-02-24T07:45:30.620432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_clean_scrub_stamp":"2026-02-24T07:45:30.620432+0000","objects_scrubbed":12027,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:46:34.106668+0000","scrub_duration":1600,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":464388096,"num_objects":7086,"num_object_clones":0,"num_object_copies":21258,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7086,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":64864,"num_write_kb":2302400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.d","version":"40'65196","reported_seq":65548,"reported_epoch":40,"state":"active+clean","last_fresh":"2026-02-24T07:45:40.741477+0000","last_change":"2026-02-24T07:44:26.596366+0000","last_active":"2026-02-24T07:45:40.741477+0000","last_peered":"2026-02-24T07:45:40.741477+0000","last_clean":"2026-02-24T07:45:40.741477+0000","last_became_active":"2026-02-24T07:43:07.853975+0000","last_became_peered":"2026-02-24T07:43:07.853975+0000","last_unstale":"2026-02-24T07:45:40.741477+0000","last_undegraded":"2026-02-24T07:45:40.741477+0000","last_fullsized":"2026-02-24T07:45:40.741477+0000","mapping_epoch":30,"log_start":"40'65193","ondisk_log_start":"40'65193","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'31656","last_scrub_stamp":"2026-02-24T07:44:26.596292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_clean_scrub_stamp":"2026-02-24T07:44:26.596292+0000","objects_scrubbed":31097,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:45:55.416603+0000","scrub_duration":3380,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":457768984,"num_objects":6986,"num_object_clones":0,"num_object_copies":20958,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6986,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":65196,"num_write_kb":2309761,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"40'65155","reported_seq":65514,"reported_epoch":40,"state":"active+clean","last_fresh":"2026-02-24T07:45:40.741042+0000","last_change":"2026-02-24T07:44:29.667393+0000","last_active":"2026-02-24T07:45:40.741042+0000","last_peered":"2026-02-24T07:45:40.741042+0000","last_clean":"2026-02-24T07:45:40.741042+0000","last_became_active":"2026-02-24T07:43:07.854428+0000","last_became_peered":"2026-02-24T07:43:07.854428+0000","last_unstale":"2026-02-24T07:45:40.741042+0000","last_undegraded":"2026-02-24T07:45:40.741042+0000","last_fullsized":"2026-02-24T07:45:40.741042+0000","mapping_epoch":30,"log_start":"40'65152","ondisk_log_start":"40'65152","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'32466","last_scrub_stamp":"2026-02-24T07:44:29.667348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_clean_scrub_stamp":"2026-02-24T07:44:29.667348+0000","objects_scrubbed":31970,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:45:43.092916+0000","scrub_duration":3552,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":463667200,"num_objects":7075,"num_object_clones":0,"num_object_copies":21225,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7075,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":65155,"num_write_kb":2311360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"29'99","reported_seq":218,"reported_epoch":40,"state":"active+clean","last_fresh":"2026-02-24T07:45:26.125455+0000","last_change":"2026-02-24T07:45:26.125455+0000","last_active":"2026-02-24T07:45:26.125455+0000","last_peered":"2026-02-24T07:45:26.125455+0000","last_clean":"2026-02-24T07:45:26.125455+0000","last_became_active":"2026-02-24T07:42:54.186719+0000","last_became_peered":"2026-02-24T07:42:54.186719+0000","last_unstale":"2026-02-24T07:45:26.125455+0000","last_undegraded":"2026-02-24T07:45:26.125455+0000","last_fullsized":"2026-02-24T07:45:26.125455+0000","mapping_epoch":27,"log_start":"29'96","ondisk_log_start":"29'96","created":20,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'99","last_scrub_stamp":"2026-02-24T07:45:26.125426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:42:43.640776+0000","last_clean_scrub_stamp":"2026-02-24T07:45:26.125426+0000","objects_scrubbed":2,"log_size":3,"log_dups_size":96,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:46:34.126365+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":240,"num_write":135,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"40'65083","reported_seq":65559,"reported_epoch":40,"state":"active+clean","last_fresh":"2026-02-24T07:45:40.699489+0000","last_change":"2026-02-24T07:45:28.799643+0000","last_active":"2026-02-24T07:45:40.699489+0000","last_peered":"2026-02-24T07:45:40.699489+0000","last_clean":"2026-02-24T07:45:40.699489+0000","last_became_active":"2026-02-24T07:43:07.842530+0000","last_became_peered":"2026-02-24T07:43:07.842530+0000","last_unstale":"2026-02-24T07:45:40.699489+0000","last_undegraded":"2026-02-24T07:45:40.699489+0000","last_fullsized":"2026-02-24T07:45:40.699489+0000","mapping_epoch":30,"log_start":"40'65080","ondisk_log_start":"40'65080","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'59525","last_scrub_stamp":"2026-02-24T07:45:28.799605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_clean_scrub_stamp":"2026-02-24T07:45:28.799605+0000","objects_scrubbed":12919,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:46:45.174239+0000","scrub_duration":1680,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":460390400,"num_objects":7025,"num_object_clones":0,"num_object_copies":21075,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7025,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":65083,"num_write_kb":2307456,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"40'64245","reported_seq":64597,"reported_epoch":40,"state":"active+clean","last_fresh":"2026-02-24T07:45:40.692047+0000","last_change":"2026-02-24T07:44:28.446165+0000","last_active":"2026-02-24T07:45:40.692047+0000","last_peered":"2026-02-24T07:45:40.692047+0000","last_clean":"2026-02-24T07:45:40.692047+0000","last_became_active":"2026-02-24T07:43:07.853885+0000","last_became_peered":"2026-02-24T07:43:07.853885+0000","last_unstale":"2026-02-24T07:45:40.692047+0000","last_undegraded":"2026-02-24T07:45:40.692047+0000","last_fullsized":"2026-02-24T07:45:40.692047+0000","mapping_epoch":30,"log_start":"40'64242","ondisk_log_start":"40'64242","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'31685","last_scrub_stamp":"2026-02-24T07:44:28.446064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_clean_scrub_stamp":"2026-02-24T07:44:28.446064+0000","objects_scrubbed":31280,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:45:52.594601+0000","scrub_duration":3117,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":457375744,"num_objects":6979,"num_object_clones":0,"num_object_copies":20937,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6979,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":64245,"num_write_kb":2279168,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"40'65090","reported_seq":65547,"reported_epoch":40,"state":"active+clean","last_fresh":"2026-02-24T07:45:40.741172+0000","last_change":"2026-02-24T07:45:39.807832+0000","last_active":"2026-02-24T07:45:40.741172+0000","last_peered":"2026-02-24T07:45:40.741172+0000","last_clean":"2026-02-24T07:45:40.741172+0000","last_became_active":"2026-02-24T07:43:07.854212+0000","last_became_peered":"2026-02-24T07:43:07.854212+0000","last_unstale":"2026-02-24T07:45:40.741172+0000","last_undegraded":"2026-02-24T07:45:40.741172+0000","last_fullsized":"2026-02-24T07:45:40.741172+0000","mapping_epoch":30,"log_start":"40'65087","ondisk_log_start":"40'65087","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'64631","last_scrub_stamp":"2026-02-24T07:45:39.807738+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_clean_scrub_stamp":"2026-02-24T07:45:39.807738+0000","objects_scrubbed":7725,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:47:08.262241+0000","scrub_duration":866,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":462028800,"num_objects":7050,"num_object_clones":0,"num_object_copies":21150,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7050,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":65090,"num_write_kb":2308480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[1,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"40'65082","reported_seq":65473,"reported_epoch":40,"state":"active+clean","last_fresh":"2026-02-24T07:45:40.699915+0000","last_change":"2026-02-24T07:44:42.974303+0000","last_active":"2026-02-24T07:45:40.699915+0000","last_peered":"2026-02-24T07:45:40.699915+0000","last_clean":"2026-02-24T07:45:40.699915+0000","last_became_active":"2026-02-24T07:43:07.842864+0000","last_became_peered":"2026-02-24T07:43:07.842864+0000","last_unstale":"2026-02-24T07:45:40.699915+0000","last_undegraded":"2026-02-24T07:45:40.699915+0000","last_fullsized":"2026-02-24T07:45:40.699915+0000","mapping_epoch":30,"log_start":"40'65077","ondisk_log_start":"40'65077","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'37783","last_scrub_stamp":"2026-02-24T07:44:42.974258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_clean_scrub_stamp":"2026-02-24T07:44:42.974258+0000","objects_scrubbed":35124,"log_size":5,"log_dups_size":3000,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:45:49.565319+0000","scrub_duration":3593,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":457179136,"num_objects":6976,"num_object_clones":0,"num_object_copies":20928,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6976,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":65082,"num_write_kb":2305856,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.4","version":"40'64562","reported_seq":64877,"reported_epoch":40,"state":"active+clean","last_fresh":"2026-02-24T07:45:40.742170+0000","last_change":"2026-02-24T07:44:16.909673+0000","last_active":"2026-02-24T07:45:40.742170+0000","last_peered":"2026-02-24T07:45:40.742170+0000","last_clean":"2026-02-24T07:45:40.742170+0000","last_became_active":"2026-02-24T07:43:07.853966+0000","last_became_peered":"2026-02-24T07:43:07.853966+0000","last_unstale":"2026-02-24T07:45:40.742170+0000","last_undegraded":"2026-02-24T07:45:40.742170+0000","last_fullsized":"2026-02-24T07:45:40.742170+0000","mapping_epoch":30,"log_start":"40'64559","ondisk_log_start":"40'64559","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'27932","last_scrub_stamp":"2026-02-24T07:44:16.909624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_clean_scrub_stamp":"2026-02-24T07:44:16.909624+0000","objects_scrubbed":27480,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:45:43.624405+0000","scrub_duration":2774,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":460455936,"num_objects":7026,"num_object_clones":0,"num_object_copies":21078,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7026,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":64562,"num_write_kb":2290816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"40'64595","reported_seq":64974,"reported_epoch":40,"state":"active+clean","last_fresh":"2026-02-24T07:45:40.692861+0000","last_change":"2026-02-24T07:44:35.717081+0000","last_active":"2026-02-24T07:45:40.692861+0000","last_peered":"2026-02-24T07:45:40.692861+0000","last_clean":"2026-02-24T07:45:40.692861+0000","last_became_active":"2026-02-24T07:43:07.853878+0000","last_became_peered":"2026-02-24T07:43:07.853878+0000","last_unstale":"2026-02-24T07:45:40.692861+0000","last_undegraded":"2026-02-24T07:45:40.692861+0000","last_fullsized":"2026-02-24T07:45:40.692861+0000","mapping_epoch":30,"log_start":"40'64592","ondisk_log_start":"40'64592","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'34578","last_scrub_stamp":"2026-02-24T07:44:35.717022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_clean_scrub_stamp":"2026-02-24T07:44:35.717022+0000","objects_scrubbed":33884,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:46:00.809440+0000","scrub_duration":3376,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":461438976,"num_objects":7041,"num_object_clones":0,"num_object_copies":21123,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7041,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":64595,"num_write_kb":2292352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"40'64887","reported_seq":65248,"reported_epoch":40,"state":"active+clean","last_fresh":"2026-02-24T07:45:40.738478+0000","last_change":"2026-02-24T07:44:30.553008+0000","last_active":"2026-02-24T07:45:40.738478+0000","last_peered":"2026-02-24T07:45:40.738478+0000","last_clean":"2026-02-24T07:45:40.738478+0000","last_became_active":"2026-02-24T07:43:07.841993+0000","last_became_peered":"2026-02-24T07:43:07.841993+0000","last_unstale":"2026-02-24T07:45:40.738478+0000","last_undegraded":"2026-02-24T07:45:40.738478+0000","last_fullsized":"2026-02-24T07:45:40.738478+0000","mapping_epoch":30,"log_start":"40'64884","ondisk_log_start":"40'64884","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'32640","last_scrub_stamp":"2026-02-24T07:44:30.552965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_clean_scrub_stamp":"2026-02-24T07:44:30.552965+0000","objects_scrubbed":32121,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:45:56.537381+0000","scrub_duration":3414,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":453705728,"num_objects":6923,"num_object_clones":0,"num_object_copies":20769,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6923,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":64887,"num_write_kb":2297920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"40'64121","reported_seq":64555,"reported_epoch":40,"state":"active+clean","last_fresh":"2026-02-24T07:45:40.699993+0000","last_change":"2026-02-24T07:45:33.195132+0000","last_active":"2026-02-24T07:45:40.699993+0000","last_peered":"2026-02-24T07:45:40.699993+0000","last_clean":"2026-02-24T07:45:40.699993+0000","last_became_active":"2026-02-24T07:43:07.841920+0000","last_became_peered":"2026-02-24T07:43:07.841920+0000","last_unstale":"2026-02-24T07:45:40.699993+0000","last_undegraded":"2026-02-24T07:45:40.699993+0000","last_fullsized":"2026-02-24T07:45:40.699993+0000","mapping_epoch":30,"log_start":"40'64118","ondisk_log_start":"40'64118","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'60653","last_scrub_stamp":"2026-02-24T07:45:33.195015+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_clean_scrub_stamp":"2026-02-24T07:45:33.195015+0000","objects_scrubbed":10713,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:46:42.852418+0000","scrub_duration":1030,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":458948608,"num_objects":7003,"num_object_clones":0,"num_object_copies":21009,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7003,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":64121,"num_write_kb":2275968,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.8","version":"40'64226","reported_seq":64585,"reported_epoch":40,"state":"active+clean","last_fresh":"2026-02-24T07:45:40.699409+0000","last_change":"2026-02-24T07:44:31.118780+0000","last_active":"2026-02-24T07:45:40.699409+0000","last_peered":"2026-02-24T07:45:40.699409+0000","last_clean":"2026-02-24T07:45:40.699409+0000","last_became_active":"2026-02-24T07:43:07.853758+0000","last_became_peered":"2026-02-24T07:43:07.853758+0000","last_unstale":"2026-02-24T07:45:40.699409+0000","last_undegraded":"2026-02-24T07:45:40.699409+0000","last_fullsized":"2026-02-24T07:45:40.699409+0000","mapping_epoch":30,"log_start":"40'64223","ondisk_log_start":"40'64223","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'32571","last_scrub_stamp":"2026-02-24T07:44:31.118720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_clean_scrub_stamp":"2026-02-24T07:44:31.118720+0000","objects_scrubbed":32077,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:45:44.647478+0000","scrub_duration":2775,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":466747392,"num_objects":7122,"num_object_clones":0,"num_object_copies":21366,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7122,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":64226,"num_write_kb":2283136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"40'65020","reported_seq":65415,"reported_epoch":40,"state":"active+clean","last_fresh":"2026-02-24T07:45:40.736595+0000","last_change":"2026-02-24T07:44:41.349757+0000","last_active":"2026-02-24T07:45:40.736595+0000","last_peered":"2026-02-24T07:45:40.736595+0000","last_clean":"2026-02-24T07:45:40.736595+0000","last_became_active":"2026-02-24T07:43:07.854377+0000","last_became_peered":"2026-02-24T07:43:07.854377+0000","last_unstale":"2026-02-24T07:45:40.736595+0000","last_undegraded":"2026-02-24T07:45:40.736595+0000","last_fullsized":"2026-02-24T07:45:40.736595+0000","mapping_epoch":30,"log_start":"40'65017","ondisk_log_start":"40'65017","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'36997","last_scrub_stamp":"2026-02-24T07:44:41.349228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_clean_scrub_stamp":"2026-02-24T07:44:41.349228+0000","objects_scrubbed":35574,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:45:50.611975+0000","scrub_duration":3205,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":457965568,"num_objects":6988,"num_object_clones":0,"num_object_copies":20964,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6988,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":65020,"num_write_kb":2304256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.a","version":"40'64784","reported_seq":65137,"reported_epoch":40,"state":"active+clean","last_fresh":"2026-02-24T07:45:40.739524+0000","last_change":"2026-02-24T07:44:27.902144+0000","last_active":"2026-02-24T07:45:40.739524+0000","last_peered":"2026-02-24T07:45:40.739524+0000","last_clean":"2026-02-24T07:45:40.739524+0000","last_became_active":"2026-02-24T07:43:07.841721+0000","last_became_peered":"2026-02-24T07:43:07.841721+0000","last_unstale":"2026-02-24T07:45:40.739524+0000","last_undegraded":"2026-02-24T07:45:40.739524+0000","last_fullsized":"2026-02-24T07:45:40.739524+0000","mapping_epoch":30,"log_start":"40'64781","ondisk_log_start":"40'64781","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'31794","last_scrub_stamp":"2026-02-24T07:44:27.902040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_clean_scrub_stamp":"2026-02-24T07:44:27.902040+0000","objects_scrubbed":31292,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"queued for scrub","scrub_duration":3732,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":465567744,"num_objects":7104,"num_object_clones":0,"num_object_copies":21312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7104,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":64784,"num_write_kb":2300416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.b","version":"40'64964","reported_seq":65360,"reported_epoch":40,"state":"active+clean","last_fresh":"2026-02-24T07:45:40.699651+0000","last_change":"2026-02-24T07:44:39.639056+0000","last_active":"2026-02-24T07:45:40.699651+0000","last_peered":"2026-02-24T07:45:40.699651+0000","last_clean":"2026-02-24T07:45:40.699651+0000","last_became_active":"2026-02-24T07:43:07.853069+0000","last_became_peered":"2026-02-24T07:43:07.853069+0000","last_unstale":"2026-02-24T07:45:40.699651+0000","last_undegraded":"2026-02-24T07:45:40.699651+0000","last_fullsized":"2026-02-24T07:45:40.699651+0000","mapping_epoch":30,"log_start":"40'64961","ondisk_log_start":"40'64961","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'36298","last_scrub_stamp":"2026-02-24T07:44:39.638986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_clean_scrub_stamp":"2026-02-24T07:44:39.638986+0000","objects_scrubbed":35656,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:46:09.128452+0000","scrub_duration":3228,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":463339520,"num_objects":7070,"num_object_clones":0,"num_object_copies":21210,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7070,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":64964,"num_write_kb":2305088,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":7361527832,"num_objects":112329,"num_object_clones":0,"num_object_copies":336987,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":112329,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1035807,"num_write_kb":36740289,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":22108778496,"data_stored":22108766280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":465,"internal_metadata":0},"log_size":50,"ondisk_log_size":50,"up":48,"acting":48,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":240,"num_write":135,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":744,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":3,"up_from":25,"seq":107374182436,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":7284624,"kb_used_data":6524220,"kb_used_omap":5,"kb_used_meta":760314,"kb_avail":726718576,"statfs":{"total":751619276800,"available":744159821824,"internally_reserved":0,"allocated":6680801280,"data_stored":6680546568,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":5378,"internal_metadata":778562302},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":19,"seq":81604378662,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":5298700,"kb_used_data":4967352,"kb_used_omap":4,"kb_used_meta":331259,"kb_avail":728704500,"statfs":{"total":751619276800,"available":746193408000,"internally_reserved":0,"allocated":5086568448,"data_stored":5086317808,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4542,"internal_metadata":339209794},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542183,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":6164940,"kb_used_data":5696824,"kb_used_omap":5,"kb_used_meta":468090,"kb_avail":727838260,"statfs":{"total":751619276800,"available":745306378240,"internally_reserved":0,"allocated":5833547776,"data_stored":5833296616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":5284,"internal_metadata":479325020},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705706,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":5451024,"kb_used_data":5119612,"kb_used_omap":4,"kb_used_meta":331387,"kb_avail":728552176,"statfs":{"total":751619276800,"available":746037428224,"internally_reserved":0,"allocated":5242482688,"data_stored":5242227976,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4450,"internal_metadata":339340958},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":744,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":5074980864,"data_stored":5074976792,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":5513285632,"data_stored":5513281560,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":186,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":5076418560,"data_stored":5076418560,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":93,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":6444093440,"data_stored":6444089368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":186,"internal_metadata":0}]}} 2026-02-24T07:45:43.059 INFO:tasks.cephadm.ceph_manager.ceph:increase pool size by 10 2026-02-24T07:45:43.060 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph osd pool set unique_pool_0 pg_num 86 2026-02-24T07:45:43.196 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:45:43.316 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:45:43.315+0000 7fdd81d54640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:45:43.316 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:45:43.315+0000 7fdd81d54640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:45:43.450 INFO:teuthology.orchestra.run.trial035.stderr:Error EPERM: crimson_allow_pg_split is false; pg_num increase denied for crimson pool 2026-02-24T07:45:43.485 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-24T07:45:44.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:45:43 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/2126015221' entity='client.admin' cmd={"prefix": "osd pool set", "pool": "unique_pool_0", "var": "pg_num", "val": "86"} : dispatch 2026-02-24T07:45:44.389 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2026-02-24T07:45:44.527 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:45:44.648 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:45:44.646+0000 7fe95ebff640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:45:44.648 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:45:44.647+0000 7fe95ebff640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:45:44.661 INFO:teuthology.orchestra.run.trial035.stdout:osd.0: {} 2026-02-24T07:45:44.662 INFO:teuthology.orchestra.run.trial035.stderr:osd.0: osd_enable_op_tracker = '' 2026-02-24T07:45:44.677 INFO:teuthology.orchestra.run.trial035.stdout:osd.1: {} 2026-02-24T07:45:44.677 INFO:teuthology.orchestra.run.trial035.stderr:osd.1: osd_enable_op_tracker = '' 2026-02-24T07:45:44.689 INFO:teuthology.orchestra.run.trial035.stdout:osd.2: {} 2026-02-24T07:45:44.689 INFO:teuthology.orchestra.run.trial035.stderr:osd.2: osd_enable_op_tracker = '' 2026-02-24T07:45:44.702 INFO:teuthology.orchestra.run.trial035.stdout:osd.3: {} 2026-02-24T07:45:44.702 INFO:teuthology.orchestra.run.trial035.stderr:osd.3: osd_enable_op_tracker = '' 2026-02-24T07:45:44.954 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:45:44 trial035 ceph-mon[39284]: from='client.14567 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2026-02-24T07:45:44.954 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:45:44 trial035 ceph-mon[39284]: pgmap v96: 17 pgs: 17 active+clean; 6.9 GiB data, 23 GiB used, 2.7 TiB / 2.7 TiB avail; 0 B/s wr, 6.19k op/s 2026-02-24T07:45:44.954 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:45:44 trial035 ceph-mon[39284]: 2.c scrub starts 2026-02-24T07:45:44.954 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:45:44 trial035 ceph-mon[39284]: 2.c scrub ok 2026-02-24T07:45:44.954 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:45:44 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/1457055528' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2026-02-24T07:45:46.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:45:45 trial035 ceph-mon[39284]: from='mgr.14284 10.20.193.35:0/3681373222' entity='mgr.x' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2026-02-24T07:45:46.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:45:45 trial035 ceph-mon[39284]: 2.8 scrub starts 2026-02-24T07:45:46.742 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2026-02-24T07:45:46.871 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:45:47.100 INFO:journalctl@ceph.mgr.x.trial035.stdout:Feb 24 07:45:46 trial035 ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-mgr-x[39621]: ::ffff:10.20.193.91 - - [24/Feb/2026:07:45:46] "GET /metrics HTTP/1.1" 200 4231 "" "Prometheus/3.6.0" 2026-02-24T07:45:47.100 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:45:46 trial035 ceph-mon[39284]: 2.8 scrub ok 2026-02-24T07:45:47.100 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:45:46 trial035 ceph-mon[39284]: pgmap v97: 17 pgs: 17 active+clean; 6.7 GiB data, 22 GiB used, 2.7 TiB / 2.7 TiB avail; 0 B/s wr, 6.37k op/s 2026-02-24T07:45:47.257 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:45:47.220+0000 7f41bbd3e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:45:47.258 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:45:47.220+0000 7f41bbd3e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:45:47.258 INFO:teuthology.orchestra.run.trial035.stderr:osd.0: osd_enable_op_tracker = '' 2026-02-24T07:45:47.258 INFO:teuthology.orchestra.run.trial035.stderr:osd.1: osd_enable_op_tracker = '' 2026-02-24T07:45:47.258 INFO:teuthology.orchestra.run.trial035.stdout:osd.0: {} 2026-02-24T07:45:47.258 INFO:teuthology.orchestra.run.trial035.stdout:osd.1: {} 2026-02-24T07:45:47.261 INFO:teuthology.orchestra.run.trial035.stdout:osd.2: {} 2026-02-24T07:45:47.262 INFO:teuthology.orchestra.run.trial035.stderr:osd.2: osd_enable_op_tracker = '' 2026-02-24T07:45:47.265 INFO:journalctl@ceph.prometheus.a.trial091.stdout:Feb 24 07:45:46 trial091 ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-prometheus-a[66007]: time=2026-02-24T07:45:46.973Z level=ERROR source=scrape.go:1630 msg="Failed to determine correct type of scrape target." component="scrape manager" scrape_pool=ceph target=http://trial035:9283/metrics content_type="text/html;charset=utf-8" fallback_media_type="" err="received unsupported Content-Type \"text/html;charset=utf-8\" and no fallback_scrape_protocol specified for target" 2026-02-24T07:45:47.273 INFO:teuthology.orchestra.run.trial035.stdout:osd.3: {} 2026-02-24T07:45:47.273 INFO:teuthology.orchestra.run.trial035.stderr:osd.3: osd_enable_op_tracker = '' 2026-02-24T07:45:48.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:45:47 trial035 ceph-mon[39284]: 2.4 scrub starts 2026-02-24T07:45:48.205 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:45:47 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/2086029325' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2026-02-24T07:45:48.205 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:45:47 trial035 ceph-mon[39284]: 2.4 scrub ok 2026-02-24T07:45:48.486 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2026-02-24T07:45:48.487 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2026-02-24T07:45:48.487 INFO:tasks.thrashosds.thrasher:trim_stale_osdmaps 2026-02-24T07:45:48.487 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph tell osd.0 'trim stale osdmaps' 2026-02-24T07:45:48.621 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:45:48.734 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:45:48.732+0000 7fc122d1a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:45:48.734 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:45:48.732+0000 7fc122d1a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:45:48.755 INFO:teuthology.orchestra.run.trial035.stderr:no valid command found; 10 closest matches: 2026-02-24T07:45:48.755 INFO:teuthology.orchestra.run.trial035.stderr:0 2026-02-24T07:45:48.755 INFO:teuthology.orchestra.run.trial035.stderr:1 2026-02-24T07:45:48.755 INFO:teuthology.orchestra.run.trial035.stderr:2 2026-02-24T07:45:48.755 INFO:teuthology.orchestra.run.trial035.stderr:abort 2026-02-24T07:45:48.755 INFO:teuthology.orchestra.run.trial035.stderr:assert 2026-02-24T07:45:48.756 INFO:teuthology.orchestra.run.trial035.stderr:bench [] [] [] [] 2026-02-24T07:45:48.756 INFO:teuthology.orchestra.run.trial035.stderr:bluefs debug_inject_read_zeros 2026-02-24T07:45:48.756 INFO:teuthology.orchestra.run.trial035.stderr:bluefs files list 2026-02-24T07:45:48.756 INFO:teuthology.orchestra.run.trial035.stderr:bluefs stats 2026-02-24T07:45:48.756 INFO:teuthology.orchestra.run.trial035.stderr:bluestore allocator dump block 2026-02-24T07:45:48.756 INFO:teuthology.orchestra.run.trial035.stderr:Error EINVAL: invalid command 2026-02-24T07:45:48.787 DEBUG:teuthology.orchestra.run:got remote process result: 22 2026-02-24T07:45:48.787 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2026-02-24T07:45:48.788 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph tell osd.1 'trim stale osdmaps' 2026-02-24T07:45:48.922 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:45:49.035 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:45:49.032+0000 7f6639427640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:45:49.035 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:45:49.033+0000 7f6639427640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:45:49.036 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:45:48 trial035 ceph-mon[39284]: pgmap v98: 17 pgs: 17 active+clean; 4.8 GiB data, 17 GiB used, 2.7 TiB / 2.7 TiB avail; 0 B/s wr, 8.85k op/s 2026-02-24T07:45:49.055 INFO:teuthology.orchestra.run.trial035.stderr:no valid command found; 10 closest matches: 2026-02-24T07:45:49.055 INFO:teuthology.orchestra.run.trial035.stderr:0 2026-02-24T07:45:49.055 INFO:teuthology.orchestra.run.trial035.stderr:1 2026-02-24T07:45:49.055 INFO:teuthology.orchestra.run.trial035.stderr:2 2026-02-24T07:45:49.055 INFO:teuthology.orchestra.run.trial035.stderr:abort 2026-02-24T07:45:49.055 INFO:teuthology.orchestra.run.trial035.stderr:assert 2026-02-24T07:45:49.055 INFO:teuthology.orchestra.run.trial035.stderr:bench [] [] [] [] 2026-02-24T07:45:49.055 INFO:teuthology.orchestra.run.trial035.stderr:bluefs debug_inject_read_zeros 2026-02-24T07:45:49.056 INFO:teuthology.orchestra.run.trial035.stderr:bluefs files list 2026-02-24T07:45:49.056 INFO:teuthology.orchestra.run.trial035.stderr:bluefs stats 2026-02-24T07:45:49.056 INFO:teuthology.orchestra.run.trial035.stderr:bluestore allocator dump block 2026-02-24T07:45:49.056 INFO:teuthology.orchestra.run.trial035.stderr:Error EINVAL: invalid command 2026-02-24T07:45:49.085 DEBUG:teuthology.orchestra.run:got remote process result: 22 2026-02-24T07:45:49.086 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2026-02-24T07:45:49.086 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph tell osd.2 'trim stale osdmaps' 2026-02-24T07:45:49.220 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:45:49.333 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:45:49.331+0000 7f60b8710640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:45:49.334 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:45:49.331+0000 7f60b8710640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:45:49.353 INFO:teuthology.orchestra.run.trial035.stderr:no valid command found; 10 closest matches: 2026-02-24T07:45:49.353 INFO:teuthology.orchestra.run.trial035.stderr:0 2026-02-24T07:45:49.353 INFO:teuthology.orchestra.run.trial035.stderr:1 2026-02-24T07:45:49.353 INFO:teuthology.orchestra.run.trial035.stderr:2 2026-02-24T07:45:49.353 INFO:teuthology.orchestra.run.trial035.stderr:abort 2026-02-24T07:45:49.353 INFO:teuthology.orchestra.run.trial035.stderr:assert 2026-02-24T07:45:49.353 INFO:teuthology.orchestra.run.trial035.stderr:bench [] [] [] [] 2026-02-24T07:45:49.354 INFO:teuthology.orchestra.run.trial035.stderr:bluefs debug_inject_read_zeros 2026-02-24T07:45:49.354 INFO:teuthology.orchestra.run.trial035.stderr:bluefs files list 2026-02-24T07:45:49.354 INFO:teuthology.orchestra.run.trial035.stderr:bluefs stats 2026-02-24T07:45:49.354 INFO:teuthology.orchestra.run.trial035.stderr:bluestore allocator dump block 2026-02-24T07:45:49.354 INFO:teuthology.orchestra.run.trial035.stderr:Error EINVAL: invalid command 2026-02-24T07:45:49.383 DEBUG:teuthology.orchestra.run:got remote process result: 22 2026-02-24T07:45:49.384 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2026-02-24T07:45:49.384 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph tell osd.3 'trim stale osdmaps' 2026-02-24T07:45:49.510 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2026-02-24T07:45:49.515 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:45:49.639 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:45:49.637+0000 7f23b73c6640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:45:49.639 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:45:49.637+0000 7f23b73c6640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:45:49.646 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:45:49.662 INFO:teuthology.orchestra.run.trial035.stderr:no valid command found; 10 closest matches: 2026-02-24T07:45:49.662 INFO:teuthology.orchestra.run.trial035.stderr:0 2026-02-24T07:45:49.662 INFO:teuthology.orchestra.run.trial035.stderr:1 2026-02-24T07:45:49.662 INFO:teuthology.orchestra.run.trial035.stderr:2 2026-02-24T07:45:49.662 INFO:teuthology.orchestra.run.trial035.stderr:abort 2026-02-24T07:45:49.662 INFO:teuthology.orchestra.run.trial035.stderr:assert 2026-02-24T07:45:49.662 INFO:teuthology.orchestra.run.trial035.stderr:bench [] [] [] [] 2026-02-24T07:45:49.662 INFO:teuthology.orchestra.run.trial035.stderr:bluefs debug_inject_read_zeros 2026-02-24T07:45:49.662 INFO:teuthology.orchestra.run.trial035.stderr:bluefs files list 2026-02-24T07:45:49.662 INFO:teuthology.orchestra.run.trial035.stderr:bluefs stats 2026-02-24T07:45:49.662 INFO:teuthology.orchestra.run.trial035.stderr:bluestore allocator dump block 2026-02-24T07:45:49.663 INFO:teuthology.orchestra.run.trial035.stderr:Error EINVAL: invalid command 2026-02-24T07:45:49.695 DEBUG:teuthology.orchestra.run:got remote process result: 22 2026-02-24T07:45:49.696 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2026-02-24T07:45:49.778 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:45:49.776+0000 7f282df8b640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:45:49.778 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:45:49.776+0000 7f282df8b640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:45:49.793 INFO:teuthology.orchestra.run.trial035.stdout:osd.0: {} 2026-02-24T07:45:49.794 INFO:teuthology.orchestra.run.trial035.stderr:osd.0: osd_enable_op_tracker = '' 2026-02-24T07:45:49.809 INFO:teuthology.orchestra.run.trial035.stdout:osd.1: {} 2026-02-24T07:45:49.809 INFO:teuthology.orchestra.run.trial035.stderr:osd.1: osd_enable_op_tracker = '' 2026-02-24T07:45:49.820 INFO:teuthology.orchestra.run.trial035.stdout:osd.2: {} 2026-02-24T07:45:49.821 INFO:teuthology.orchestra.run.trial035.stderr:osd.2: osd_enable_op_tracker = '' 2026-02-24T07:45:49.833 INFO:teuthology.orchestra.run.trial035.stdout:osd.3: {} 2026-02-24T07:45:49.833 INFO:teuthology.orchestra.run.trial035.stderr:osd.3: osd_enable_op_tracker = '' 2026-02-24T07:45:50.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:45:49 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/1310517174' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2026-02-24T07:45:51.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:45:50 trial035 ceph-mon[39284]: pgmap v99: 17 pgs: 17 active+clean; 4.8 GiB data, 17 GiB used, 2.7 TiB / 2.7 TiB avail; 0 B/s wr, 5.93k op/s 2026-02-24T07:45:51.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:45:50 trial035 ceph-mon[39284]: 2.2 scrub starts 2026-02-24T07:45:51.872 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2026-02-24T07:45:52.003 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:45:52.115 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:45:52.113+0000 7f9aaf6ac640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:45:52.116 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:45:52.113+0000 7f9aaf6ac640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:45:52.116 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:45:51 trial035 ceph-mon[39284]: 2.2 scrub ok 2026-02-24T07:45:52.116 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:45:51 trial035 ceph-mon[39284]: 2.9 scrub starts 2026-02-24T07:45:52.116 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:45:51 trial035 ceph-mon[39284]: 2.9 scrub ok 2026-02-24T07:45:52.131 INFO:teuthology.orchestra.run.trial035.stdout:osd.0: {} 2026-02-24T07:45:52.131 INFO:teuthology.orchestra.run.trial035.stderr:osd.0: osd_enable_op_tracker = '' 2026-02-24T07:45:52.145 INFO:teuthology.orchestra.run.trial035.stdout:osd.1: {} 2026-02-24T07:45:52.145 INFO:teuthology.orchestra.run.trial035.stderr:osd.1: osd_enable_op_tracker = '' 2026-02-24T07:45:52.158 INFO:teuthology.orchestra.run.trial035.stdout:osd.2: {} 2026-02-24T07:45:52.158 INFO:teuthology.orchestra.run.trial035.stderr:osd.2: osd_enable_op_tracker = '' 2026-02-24T07:45:52.171 INFO:teuthology.orchestra.run.trial035.stdout:osd.3: {} 2026-02-24T07:45:52.171 INFO:teuthology.orchestra.run.trial035.stderr:osd.3: osd_enable_op_tracker = '' 2026-02-24T07:45:53.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:45:52 trial035 ceph-mon[39284]: pgmap v100: 17 pgs: 17 active+clean; 2.5 GiB data, 10 GiB used, 2.7 TiB / 2.7 TiB avail; 0 B/s wr, 9.05k op/s 2026-02-24T07:45:53.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:45:52 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/2725413746' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2026-02-24T07:45:54.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:45:53 trial035 ceph-mon[39284]: 2.0 scrub starts 2026-02-24T07:45:54.208 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2026-02-24T07:45:54.345 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:45:54.459 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:45:54.457+0000 7f401d53d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:45:54.459 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:45:54.457+0000 7f401d53d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:45:54.474 INFO:teuthology.orchestra.run.trial035.stdout:osd.0: {} 2026-02-24T07:45:54.474 INFO:teuthology.orchestra.run.trial035.stderr:osd.0: osd_enable_op_tracker = '' 2026-02-24T07:45:54.487 INFO:teuthology.orchestra.run.trial035.stdout:osd.1: {} 2026-02-24T07:45:54.487 INFO:teuthology.orchestra.run.trial035.stderr:osd.1: osd_enable_op_tracker = '' 2026-02-24T07:45:54.499 INFO:teuthology.orchestra.run.trial035.stdout:osd.2: {} 2026-02-24T07:45:54.499 INFO:teuthology.orchestra.run.trial035.stderr:osd.2: osd_enable_op_tracker = '' 2026-02-24T07:45:54.512 INFO:teuthology.orchestra.run.trial035.stdout:osd.3: {} 2026-02-24T07:45:54.512 INFO:teuthology.orchestra.run.trial035.stderr:osd.3: osd_enable_op_tracker = '' 2026-02-24T07:45:54.697 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2026-02-24T07:45:54.697 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2026-02-24T07:45:54.697 INFO:tasks.thrashosds.thrasher:Growing pool unique_pool_0 2026-02-24T07:45:54.698 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph pg dump --format=json 2026-02-24T07:45:54.825 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:45:54.949 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:45:54.947+0000 7fcf56798640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:45:54.950 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:45:54.947+0000 7fcf56798640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:45:55.088 INFO:teuthology.orchestra.run.trial035.stdout: 2026-02-24T07:45:55.089 INFO:teuthology.orchestra.run.trial035.stderr:dumped all 2026-02-24T07:45:55.098 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:45:54 trial035 ceph-mon[39284]: 2.0 scrub ok 2026-02-24T07:45:55.098 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:45:54 trial035 ceph-mon[39284]: pgmap v101: 17 pgs: 17 active+clean; 2.5 GiB data, 10 GiB used, 2.7 TiB / 2.7 TiB avail; 0 B/s wr, 5.98k op/s 2026-02-24T07:45:55.098 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:45:54 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/2071636625' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2026-02-24T07:45:55.129 INFO:teuthology.orchestra.run.trial035.stdout:{"pg_ready":true,"pg_map":{"version":101,"stamp":"2026-02-24T07:45:53.680931+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":2661286456,"num_objects":40602,"num_object_clones":0,"num_object_copies":121806,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40602,"num_whiteouts":0,"num_read":137,"num_read_kb":241,"num_write":1107671,"num_write_kb":36742439,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":53,"ondisk_log_size":53,"up":51,"acting":51,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":51,"num_osds":4,"num_per_pool_osds":4,"num_per_pool_omap_osds":4,"kb":2936012800,"kb_used":10853816,"kb_used_data":8583464,"kb_used_omap":19,"kb_used_meta":2270060,"kb_avail":2925158984,"statfs":{"total":3006477107200,"available":2995362799616,"internally_reserved":0,"allocated":8789467136,"data_stored":8788455912,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19959,"internal_metadata":2324541961},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-4700831744,"num_objects":-71729,"num_object_clones":0,"num_object_copies":-215187,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-71729,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":71729,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001545"},"pg_stats":[{"pgid":"2.f","version":"40'68383","reported_seq":68858,"reported_epoch":40,"state":"active+clean","last_fresh":"2026-02-24T07:45:50.672235+0000","last_change":"2026-02-24T07:45:26.689907+0000","last_active":"2026-02-24T07:45:50.672235+0000","last_peered":"2026-02-24T07:45:50.672235+0000","last_clean":"2026-02-24T07:45:50.672235+0000","last_became_active":"2026-02-24T07:43:07.852899+0000","last_became_peered":"2026-02-24T07:43:07.852899+0000","last_unstale":"2026-02-24T07:45:50.672235+0000","last_undegraded":"2026-02-24T07:45:50.672235+0000","last_fullsized":"2026-02-24T07:45:50.672235+0000","mapping_epoch":30,"log_start":"40'68380","ondisk_log_start":"40'68380","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'57836","last_scrub_stamp":"2026-02-24T07:45:26.689827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_clean_scrub_stamp":"2026-02-24T07:45:26.689827+0000","objects_scrubbed":13173,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:46:28.773425+0000","scrub_duration":1082,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":158924800,"num_objects":2425,"num_object_clones":0,"num_object_copies":7275,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2425,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":68383,"num_write_kb":2265856,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.e","version":"40'69393","reported_seq":69836,"reported_epoch":40,"state":"active+clean","last_fresh":"2026-02-24T07:45:50.743152+0000","last_change":"2026-02-24T07:45:30.620522+0000","last_active":"2026-02-24T07:45:50.743152+0000","last_peered":"2026-02-24T07:45:50.743152+0000","last_clean":"2026-02-24T07:45:50.743152+0000","last_became_active":"2026-02-24T07:43:07.852815+0000","last_became_peered":"2026-02-24T07:43:07.852815+0000","last_unstale":"2026-02-24T07:45:50.743152+0000","last_undegraded":"2026-02-24T07:45:50.743152+0000","last_fullsized":"2026-02-24T07:45:50.743152+0000","mapping_epoch":30,"log_start":"40'69390","ondisk_log_start":"40'69390","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'60215","last_scrub_stamp":"2026-02-24T07:45:30.620432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_clean_scrub_stamp":"2026-02-24T07:45:30.620432+0000","objects_scrubbed":12027,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:46:34.106668+0000","scrub_duration":1600,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":167575552,"num_objects":2557,"num_object_clones":0,"num_object_copies":7671,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2557,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":69393,"num_write_kb":2302400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.d","version":"40'69661","reported_seq":70013,"reported_epoch":40,"state":"active+clean","last_fresh":"2026-02-24T07:45:50.739712+0000","last_change":"2026-02-24T07:44:26.596366+0000","last_active":"2026-02-24T07:45:50.739712+0000","last_peered":"2026-02-24T07:45:50.739712+0000","last_clean":"2026-02-24T07:45:50.739712+0000","last_became_active":"2026-02-24T07:43:07.853975+0000","last_became_peered":"2026-02-24T07:43:07.853975+0000","last_unstale":"2026-02-24T07:45:50.739712+0000","last_undegraded":"2026-02-24T07:45:50.739712+0000","last_fullsized":"2026-02-24T07:45:50.739712+0000","mapping_epoch":30,"log_start":"40'69658","ondisk_log_start":"40'69658","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'31656","last_scrub_stamp":"2026-02-24T07:44:26.596292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_clean_scrub_stamp":"2026-02-24T07:44:26.596292+0000","objects_scrubbed":31097,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:45:55.416603+0000","scrub_duration":3380,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":165150744,"num_objects":2521,"num_object_clones":0,"num_object_copies":7563,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2521,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":69661,"num_write_kb":2309761,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"40'69611","reported_seq":70035,"reported_epoch":40,"state":"active+clean","last_fresh":"2026-02-24T07:45:50.741666+0000","last_change":"2026-02-24T07:45:44.471623+0000","last_active":"2026-02-24T07:45:50.741666+0000","last_peered":"2026-02-24T07:45:50.741666+0000","last_clean":"2026-02-24T07:45:50.741666+0000","last_became_active":"2026-02-24T07:43:07.854428+0000","last_became_peered":"2026-02-24T07:43:07.854428+0000","last_unstale":"2026-02-24T07:45:50.741666+0000","last_undegraded":"2026-02-24T07:45:50.741666+0000","last_fullsized":"2026-02-24T07:45:50.741666+0000","mapping_epoch":30,"log_start":"40'69608","ondisk_log_start":"40'69608","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'66689","last_scrub_stamp":"2026-02-24T07:45:44.471562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_clean_scrub_stamp":"2026-02-24T07:45:44.471562+0000","objects_scrubbed":5672,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:47:01.720994+0000","scrub_duration":614,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":171638784,"num_objects":2619,"num_object_clones":0,"num_object_copies":7857,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2619,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":69611,"num_write_kb":2311360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"29'99","reported_seq":218,"reported_epoch":40,"state":"active+clean","last_fresh":"2026-02-24T07:45:26.125455+0000","last_change":"2026-02-24T07:45:26.125455+0000","last_active":"2026-02-24T07:45:26.125455+0000","last_peered":"2026-02-24T07:45:26.125455+0000","last_clean":"2026-02-24T07:45:26.125455+0000","last_became_active":"2026-02-24T07:42:54.186719+0000","last_became_peered":"2026-02-24T07:42:54.186719+0000","last_unstale":"2026-02-24T07:45:26.125455+0000","last_undegraded":"2026-02-24T07:45:26.125455+0000","last_fullsized":"2026-02-24T07:45:26.125455+0000","mapping_epoch":27,"log_start":"29'96","ondisk_log_start":"29'96","created":20,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'99","last_scrub_stamp":"2026-02-24T07:45:26.125426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:42:43.640776+0000","last_clean_scrub_stamp":"2026-02-24T07:45:26.125426+0000","objects_scrubbed":2,"log_size":3,"log_dups_size":96,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:46:34.126365+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":240,"num_write":135,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"40'69557","reported_seq":70033,"reported_epoch":40,"state":"active+clean","last_fresh":"2026-02-24T07:45:50.696629+0000","last_change":"2026-02-24T07:45:28.799643+0000","last_active":"2026-02-24T07:45:50.696629+0000","last_peered":"2026-02-24T07:45:50.696629+0000","last_clean":"2026-02-24T07:45:50.696629+0000","last_became_active":"2026-02-24T07:43:07.842530+0000","last_became_peered":"2026-02-24T07:43:07.842530+0000","last_unstale":"2026-02-24T07:45:50.696629+0000","last_undegraded":"2026-02-24T07:45:50.696629+0000","last_fullsized":"2026-02-24T07:45:50.696629+0000","mapping_epoch":30,"log_start":"40'69554","ondisk_log_start":"40'69554","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'59525","last_scrub_stamp":"2026-02-24T07:45:28.799605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_clean_scrub_stamp":"2026-02-24T07:45:28.799605+0000","objects_scrubbed":12919,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:46:45.174239+0000","scrub_duration":1680,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":167182336,"num_objects":2551,"num_object_clones":0,"num_object_copies":7653,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2551,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":69557,"num_write_kb":2307456,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"40'68707","reported_seq":69059,"reported_epoch":40,"state":"active+clean","last_fresh":"2026-02-24T07:45:50.694522+0000","last_change":"2026-02-24T07:44:28.446165+0000","last_active":"2026-02-24T07:45:50.694522+0000","last_peered":"2026-02-24T07:45:50.694522+0000","last_clean":"2026-02-24T07:45:50.694522+0000","last_became_active":"2026-02-24T07:43:07.853885+0000","last_became_peered":"2026-02-24T07:43:07.853885+0000","last_unstale":"2026-02-24T07:45:50.694522+0000","last_undegraded":"2026-02-24T07:45:50.694522+0000","last_fullsized":"2026-02-24T07:45:50.694522+0000","mapping_epoch":30,"log_start":"40'68704","ondisk_log_start":"40'68704","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'31685","last_scrub_stamp":"2026-02-24T07:44:28.446064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_clean_scrub_stamp":"2026-02-24T07:44:28.446064+0000","objects_scrubbed":31280,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:45:52.594601+0000","scrub_duration":3117,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":164954112,"num_objects":2517,"num_object_clones":0,"num_object_copies":7551,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2517,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":68707,"num_write_kb":2279168,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"40'69553","reported_seq":70010,"reported_epoch":40,"state":"active+clean","last_fresh":"2026-02-24T07:45:50.741697+0000","last_change":"2026-02-24T07:45:39.807832+0000","last_active":"2026-02-24T07:45:50.741697+0000","last_peered":"2026-02-24T07:45:50.741697+0000","last_clean":"2026-02-24T07:45:50.741697+0000","last_became_active":"2026-02-24T07:43:07.854212+0000","last_became_peered":"2026-02-24T07:43:07.854212+0000","last_unstale":"2026-02-24T07:45:50.741697+0000","last_undegraded":"2026-02-24T07:45:50.741697+0000","last_fullsized":"2026-02-24T07:45:50.741697+0000","mapping_epoch":30,"log_start":"40'69550","ondisk_log_start":"40'69550","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'64631","last_scrub_stamp":"2026-02-24T07:45:39.807738+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_clean_scrub_stamp":"2026-02-24T07:45:39.807738+0000","objects_scrubbed":7725,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:47:08.262241+0000","scrub_duration":866,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":169541632,"num_objects":2587,"num_object_clones":0,"num_object_copies":7761,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2587,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":69553,"num_write_kb":2308480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[1,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"40'69510","reported_seq":69937,"reported_epoch":40,"state":"active+clean","last_fresh":"2026-02-24T07:45:50.698200+0000","last_change":"2026-02-24T07:45:50.513620+0000","last_active":"2026-02-24T07:45:50.698200+0000","last_peered":"2026-02-24T07:45:50.698200+0000","last_clean":"2026-02-24T07:45:50.698200+0000","last_became_active":"2026-02-24T07:43:07.842864+0000","last_became_peered":"2026-02-24T07:43:07.842864+0000","last_unstale":"2026-02-24T07:45:50.698200+0000","last_undegraded":"2026-02-24T07:45:50.698200+0000","last_fullsized":"2026-02-24T07:45:50.698200+0000","mapping_epoch":30,"log_start":"40'69507","ondisk_log_start":"40'69507","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'69415","last_scrub_stamp":"2026-02-24T07:45:50.513497+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_clean_scrub_stamp":"2026-02-24T07:45:50.513497+0000","objects_scrubbed":2718,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:46:58.443563+0000","scrub_duration":326,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":166985728,"num_objects":2548,"num_object_clones":0,"num_object_copies":7644,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2548,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":69510,"num_write_kb":2305856,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.4","version":"40'69092","reported_seq":69458,"reported_epoch":40,"state":"active+clean","last_fresh":"2026-02-24T07:45:50.737891+0000","last_change":"2026-02-24T07:45:47.509218+0000","last_active":"2026-02-24T07:45:50.737891+0000","last_peered":"2026-02-24T07:45:50.737891+0000","last_clean":"2026-02-24T07:45:50.737891+0000","last_became_active":"2026-02-24T07:43:07.853966+0000","last_became_peered":"2026-02-24T07:43:07.853966+0000","last_unstale":"2026-02-24T07:45:50.737891+0000","last_undegraded":"2026-02-24T07:45:50.737891+0000","last_fullsized":"2026-02-24T07:45:50.737891+0000","mapping_epoch":30,"log_start":"40'69089","ondisk_log_start":"40'69089","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'67502","last_scrub_stamp":"2026-02-24T07:45:47.509181+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_clean_scrub_stamp":"2026-02-24T07:45:47.509181+0000","objects_scrubbed":4178,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:46:54.664422+0000","scrub_duration":653,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":163577856,"num_objects":2496,"num_object_clones":0,"num_object_copies":7488,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2496,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":69092,"num_write_kb":2290816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"40'69124","reported_seq":69503,"reported_epoch":40,"state":"active+clean","last_fresh":"2026-02-24T07:45:50.700072+0000","last_change":"2026-02-24T07:44:35.717081+0000","last_active":"2026-02-24T07:45:50.700072+0000","last_peered":"2026-02-24T07:45:50.700072+0000","last_clean":"2026-02-24T07:45:50.700072+0000","last_became_active":"2026-02-24T07:43:07.853878+0000","last_became_peered":"2026-02-24T07:43:07.853878+0000","last_unstale":"2026-02-24T07:45:50.700072+0000","last_undegraded":"2026-02-24T07:45:50.700072+0000","last_fullsized":"2026-02-24T07:45:50.700072+0000","mapping_epoch":30,"log_start":"40'69121","ondisk_log_start":"40'69121","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'34578","last_scrub_stamp":"2026-02-24T07:44:35.717022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_clean_scrub_stamp":"2026-02-24T07:44:35.717022+0000","objects_scrubbed":33884,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:46:00.809440+0000","scrub_duration":3376,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":164626432,"num_objects":2512,"num_object_clones":0,"num_object_copies":7536,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2512,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":69124,"num_write_kb":2292352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"40'69224","reported_seq":69585,"reported_epoch":40,"state":"active+clean","last_fresh":"2026-02-24T07:45:50.739877+0000","last_change":"2026-02-24T07:44:30.553008+0000","last_active":"2026-02-24T07:45:50.739877+0000","last_peered":"2026-02-24T07:45:50.739877+0000","last_clean":"2026-02-24T07:45:50.739877+0000","last_became_active":"2026-02-24T07:43:07.841993+0000","last_became_peered":"2026-02-24T07:43:07.841993+0000","last_unstale":"2026-02-24T07:45:50.739877+0000","last_undegraded":"2026-02-24T07:45:50.739877+0000","last_fullsized":"2026-02-24T07:45:50.739877+0000","mapping_epoch":30,"log_start":"40'69221","ondisk_log_start":"40'69221","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'32640","last_scrub_stamp":"2026-02-24T07:44:30.552965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_clean_scrub_stamp":"2026-02-24T07:44:30.552965+0000","objects_scrubbed":32121,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:45:56.537381+0000","scrub_duration":3414,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":169476096,"num_objects":2586,"num_object_clones":0,"num_object_copies":7758,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2586,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":69224,"num_write_kb":2297920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"40'68606","reported_seq":69040,"reported_epoch":40,"state":"active+clean","last_fresh":"2026-02-24T07:45:50.701148+0000","last_change":"2026-02-24T07:45:33.195132+0000","last_active":"2026-02-24T07:45:50.701148+0000","last_peered":"2026-02-24T07:45:50.701148+0000","last_clean":"2026-02-24T07:45:50.701148+0000","last_became_active":"2026-02-24T07:43:07.841920+0000","last_became_peered":"2026-02-24T07:43:07.841920+0000","last_unstale":"2026-02-24T07:45:50.701148+0000","last_undegraded":"2026-02-24T07:45:50.701148+0000","last_fullsized":"2026-02-24T07:45:50.701148+0000","mapping_epoch":30,"log_start":"40'68601","ondisk_log_start":"40'68601","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'60653","last_scrub_stamp":"2026-02-24T07:45:33.195015+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_clean_scrub_stamp":"2026-02-24T07:45:33.195015+0000","objects_scrubbed":10713,"log_size":5,"log_dups_size":3000,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:46:42.852418+0000","scrub_duration":1030,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":165019648,"num_objects":2518,"num_object_clones":0,"num_object_copies":7554,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2518,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":68606,"num_write_kb":2275968,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.8","version":"40'68769","reported_seq":69187,"reported_epoch":40,"state":"active+clean","last_fresh":"2026-02-24T07:45:50.700869+0000","last_change":"2026-02-24T07:45:45.648796+0000","last_active":"2026-02-24T07:45:50.700869+0000","last_peered":"2026-02-24T07:45:50.700869+0000","last_clean":"2026-02-24T07:45:50.700869+0000","last_became_active":"2026-02-24T07:43:07.853758+0000","last_became_peered":"2026-02-24T07:43:07.853758+0000","last_unstale":"2026-02-24T07:45:50.700869+0000","last_undegraded":"2026-02-24T07:45:50.700869+0000","last_fullsized":"2026-02-24T07:45:50.700869+0000","mapping_epoch":30,"log_start":"40'68766","ondisk_log_start":"40'68766","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'66354","last_scrub_stamp":"2026-02-24T07:45:45.648748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_clean_scrub_stamp":"2026-02-24T07:45:45.648748+0000","objects_scrubbed":5086,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:46:48.741191+0000","scrub_duration":493,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":169017344,"num_objects":2579,"num_object_clones":0,"num_object_copies":7737,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2579,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":68769,"num_write_kb":2283136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"40'69479","reported_seq":69874,"reported_epoch":40,"state":"active+clean","last_fresh":"2026-02-24T07:45:50.743780+0000","last_change":"2026-02-24T07:44:41.349757+0000","last_active":"2026-02-24T07:45:50.743780+0000","last_peered":"2026-02-24T07:45:50.743780+0000","last_clean":"2026-02-24T07:45:50.743780+0000","last_became_active":"2026-02-24T07:43:07.854377+0000","last_became_peered":"2026-02-24T07:43:07.854377+0000","last_unstale":"2026-02-24T07:45:50.743780+0000","last_undegraded":"2026-02-24T07:45:50.743780+0000","last_fullsized":"2026-02-24T07:45:50.743780+0000","mapping_epoch":30,"log_start":"40'69476","ondisk_log_start":"40'69476","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'36997","last_scrub_stamp":"2026-02-24T07:44:41.349228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_clean_scrub_stamp":"2026-02-24T07:44:41.349228+0000","objects_scrubbed":35574,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"queued for scrub","scrub_duration":3205,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":165740544,"num_objects":2529,"num_object_clones":0,"num_object_copies":7587,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2529,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":69479,"num_write_kb":2304256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.a","version":"40'69331","reported_seq":69763,"reported_epoch":40,"state":"active+clean","last_fresh":"2026-02-24T07:45:50.742891+0000","last_change":"2026-02-24T07:45:41.891852+0000","last_active":"2026-02-24T07:45:50.742891+0000","last_peered":"2026-02-24T07:45:50.742891+0000","last_clean":"2026-02-24T07:45:50.742891+0000","last_became_active":"2026-02-24T07:43:07.841721+0000","last_became_peered":"2026-02-24T07:43:07.841721+0000","last_unstale":"2026-02-24T07:45:50.742891+0000","last_undegraded":"2026-02-24T07:45:50.742891+0000","last_fullsized":"2026-02-24T07:45:50.742891+0000","mapping_epoch":30,"log_start":"40'69328","ondisk_log_start":"40'69328","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'65231","last_scrub_stamp":"2026-02-24T07:45:41.891811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_clean_scrub_stamp":"2026-02-24T07:45:41.891811+0000","objects_scrubbed":6826,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:47:07.090556+0000","scrub_duration":972,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":167575552,"num_objects":2557,"num_object_clones":0,"num_object_copies":7671,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2557,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":69331,"num_write_kb":2300416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.b","version":"40'69536","reported_seq":69932,"reported_epoch":40,"state":"active+clean","last_fresh":"2026-02-24T07:45:50.696393+0000","last_change":"2026-02-24T07:44:39.639056+0000","last_active":"2026-02-24T07:45:50.696393+0000","last_peered":"2026-02-24T07:45:50.696393+0000","last_clean":"2026-02-24T07:45:50.696393+0000","last_became_active":"2026-02-24T07:43:07.853069+0000","last_became_peered":"2026-02-24T07:43:07.853069+0000","last_unstale":"2026-02-24T07:45:50.696393+0000","last_undegraded":"2026-02-24T07:45:50.696393+0000","last_fullsized":"2026-02-24T07:45:50.696393+0000","mapping_epoch":30,"log_start":"40'69533","ondisk_log_start":"40'69533","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'36298","last_scrub_stamp":"2026-02-24T07:44:39.638986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:43:06.832892+0000","last_clean_scrub_stamp":"2026-02-24T07:44:39.638986+0000","objects_scrubbed":35656,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:46:09.128452+0000","scrub_duration":3228,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":163708928,"num_objects":2498,"num_object_clones":0,"num_object_copies":7494,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2498,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":69536,"num_write_kb":2305088,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":2660696088,"num_objects":40600,"num_object_clones":0,"num_object_copies":121800,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40600,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1107536,"num_write_kb":36740289,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8009428992,"data_stored":8009416776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":2077,"internal_metadata":0},"log_size":50,"ondisk_log_size":50,"up":48,"acting":48,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":240,"num_write":135,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":930,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":3,"up_from":25,"seq":107374182438,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":3376208,"kb_used_data":2507580,"kb_used_omap":5,"kb_used_meta":868602,"kb_avail":730626992,"statfs":{"total":751619276800,"available":748162039808,"internally_reserved":0,"allocated":2567761920,"data_stored":2567507208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":5467,"internal_metadata":889449125},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":19,"seq":81604378664,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":2245388,"kb_used_data":1823736,"kb_used_omap":4,"kb_used_meta":421499,"kb_avail":731757812,"statfs":{"total":751619276800,"available":749319999488,"internally_reserved":0,"allocated":1867505664,"data_stored":1867255024,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4542,"internal_metadata":431615554},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542185,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":2835404,"kb_used_data":2276984,"kb_used_omap":5,"kb_used_meta":558330,"kb_avail":731167796,"statfs":{"total":751619276800,"available":748715823104,"internally_reserved":0,"allocated":2331631616,"data_stored":2331380456,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":5500,"internal_metadata":571730564},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705708,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":2396816,"kb_used_data":1975164,"kb_used_omap":4,"kb_used_meta":421627,"kb_avail":731606384,"statfs":{"total":751619276800,"available":749164937216,"internally_reserved":0,"allocated":2022567936,"data_stored":2022313224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4450,"internal_metadata":431746718},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":930,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":1834160128,"data_stored":1834156056,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":1998065664,"data_stored":1998061592,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":868,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":1839464448,"data_stored":1839464448,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":93,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":2337738752,"data_stored":2337734680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1116,"internal_metadata":0}]}} 2026-02-24T07:45:55.131 INFO:tasks.cephadm.ceph_manager.ceph:increase pool size by 10 2026-02-24T07:45:55.132 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph osd pool set unique_pool_0 pg_num 96 2026-02-24T07:45:55.262 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:45:55.384 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:45:55.382+0000 7f4277252640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:45:55.384 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:45:55.382+0000 7f4277252640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:45:55.506 INFO:teuthology.orchestra.run.trial035.stderr:Error EPERM: crimson_allow_pg_split is false; pg_num increase denied for crimson pool 2026-02-24T07:45:55.545 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-24T07:45:55.961 INFO:tasks.radosbench.radosbench.0.trial035.stdout:Removed 574067 objects 2026-02-24T07:45:55.961 INFO:tasks.radosbench.radosbench.0.trial035.stdout:Clean up completed and total clean up time :77.0221 2026-02-24T07:45:56.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:45:55 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/4157176001' entity='client.admin' cmd={"prefix": "osd pool set", "pool": "unique_pool_0", "var": "pg_num", "val": "96"} : dispatch 2026-02-24T07:45:56.551 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2026-02-24T07:45:56.667 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:45:56.767 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:45:56.765+0000 7fb4ef4a3640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:45:56.768 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:45:56.766+0000 7fb4ef4a3640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:45:56.780 INFO:teuthology.orchestra.run.trial035.stdout:osd.0: {} 2026-02-24T07:45:56.780 INFO:teuthology.orchestra.run.trial035.stderr:osd.0: osd_enable_op_tracker = '' 2026-02-24T07:45:56.790 INFO:teuthology.orchestra.run.trial035.stdout:osd.1: {} 2026-02-24T07:45:56.790 INFO:teuthology.orchestra.run.trial035.stderr:osd.1: osd_enable_op_tracker = '' 2026-02-24T07:45:56.801 INFO:teuthology.orchestra.run.trial035.stdout:osd.2: {} 2026-02-24T07:45:56.801 INFO:teuthology.orchestra.run.trial035.stderr:osd.2: osd_enable_op_tracker = '' 2026-02-24T07:45:56.812 INFO:teuthology.orchestra.run.trial035.stdout:osd.3: {} 2026-02-24T07:45:56.812 INFO:teuthology.orchestra.run.trial035.stderr:osd.3: osd_enable_op_tracker = '' 2026-02-24T07:45:57.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:45:56 trial035 ceph-mon[39284]: from='client.14589 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2026-02-24T07:45:57.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:45:56 trial035 ceph-mon[39284]: pgmap v102: 17 pgs: 17 active+clean; 2.3 GiB data, 8.8 GiB used, 2.7 TiB / 2.7 TiB avail; 0 B/s wr, 6.17k op/s 2026-02-24T07:45:57.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:45:56 trial035 ceph-mon[39284]: 2.d scrub starts 2026-02-24T07:45:57.205 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:45:56 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/755252032' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2026-02-24T07:45:57.205 INFO:journalctl@ceph.mgr.x.trial035.stdout:Feb 24 07:45:56 trial035 ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-mgr-x[39621]: ::ffff:10.20.193.91 - - [24/Feb/2026:07:45:56] "GET /metrics HTTP/1.1" 200 4231 "" "Prometheus/3.6.0" 2026-02-24T07:45:57.357 INFO:journalctl@ceph.prometheus.a.trial091.stdout:Feb 24 07:45:56 trial091 ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-prometheus-a[66007]: time=2026-02-24T07:45:56.973Z level=ERROR source=scrape.go:1630 msg="Failed to determine correct type of scrape target." component="scrape manager" scrape_pool=ceph target=http://trial035:9283/metrics content_type="text/html;charset=utf-8" fallback_media_type="" err="received unsupported Content-Type \"text/html;charset=utf-8\" and no fallback_scrape_protocol specified for target" 2026-02-24T07:45:58.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:45:57 trial035 ceph-mon[39284]: purged_snaps scrub starts 2026-02-24T07:45:58.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:45:57 trial035 ceph-mon[39284]: purged_snaps scrub ok 2026-02-24T07:45:58.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:45:57 trial035 ceph-mon[39284]: 2.d scrub ok 2026-02-24T07:45:58.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:45:57 trial035 ceph-mon[39284]: 2.6 scrub starts 2026-02-24T07:45:58.850 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2026-02-24T07:45:58.966 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:45:59.139 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:45:58 trial035 ceph-mon[39284]: pgmap v103: 17 pgs: 17 active+clean; 118 MiB data, 3.4 GiB used, 2.7 TiB / 2.7 TiB avail; 0 B/s wr, 9.03k op/s 2026-02-24T07:45:59.140 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:45:58 trial035 ceph-mon[39284]: 2.6 scrub ok 2026-02-24T07:45:59.501 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:45:59.431+0000 7fa309a86640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:45:59.501 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:45:59.431+0000 7fa309a86640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:45:59.501 INFO:teuthology.orchestra.run.trial035.stderr:osd.0: osd_enable_op_tracker = '' 2026-02-24T07:45:59.501 INFO:teuthology.orchestra.run.trial035.stderr:osd.1: osd_enable_op_tracker = '' 2026-02-24T07:45:59.501 INFO:teuthology.orchestra.run.trial035.stderr:osd.2: osd_enable_op_tracker = '' 2026-02-24T07:45:59.501 INFO:teuthology.orchestra.run.trial035.stderr:osd.3: osd_enable_op_tracker = '' 2026-02-24T07:45:59.501 INFO:teuthology.orchestra.run.trial035.stdout:osd.0: {} 2026-02-24T07:45:59.501 INFO:teuthology.orchestra.run.trial035.stdout:osd.1: {} 2026-02-24T07:45:59.501 INFO:teuthology.orchestra.run.trial035.stdout:osd.2: {} 2026-02-24T07:45:59.501 INFO:teuthology.orchestra.run.trial035.stdout:osd.3: {} 2026-02-24T07:45:59.721 INFO:tasks.cephadm.ceph_manager.ceph:removing pool_name unique_pool_0 2026-02-24T07:45:59.721 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph osd pool rm unique_pool_0 unique_pool_0 --yes-i-really-really-mean-it 2026-02-24T07:46:00.059 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:45:59 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/771027836' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2026-02-24T07:46:00.059 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:45:59 trial035 ceph-mon[39284]: from='mgr.14284 10.20.193.35:0/3681373222' entity='mgr.x' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2026-02-24T07:46:00.070 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:46:00.547 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2026-02-24T07:46:00.547 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2026-02-24T07:46:00.550 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:46:00.507+0000 7f5e9e2b9640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:46:00.550 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:46:00.507+0000 7f5e9e2b9640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:46:00.868 INFO:teuthology.orchestra.run.trial035.stderr:pool 'unique_pool_0' removed 2026-02-24T07:46:01.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:46:00 trial035 ceph-mon[39284]: pgmap v104: 17 pgs: 17 active+clean; 118 MiB data, 3.4 GiB used, 2.7 TiB / 2.7 TiB avail; 0 B/s wr, 6.35k op/s 2026-02-24T07:46:01.205 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:46:00 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/3701598152' entity='client.admin' cmd={"prefix": "osd pool rm", "pool": "unique_pool_0", "pool2": "unique_pool_0", "yes_i_really_really_mean_it": true} : dispatch 2026-02-24T07:46:01.415 INFO:teuthology.task.full_sequential:In full_sequential, running task radosbench... 2026-02-24T07:46:01.415 INFO:tasks.radosbench:Beginning radosbench... 2026-02-24T07:46:01.415 INFO:tasks.cephadm.ceph_manager.ceph:creating pool_name unique_pool_1 2026-02-24T07:46:01.415 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph osd pool create unique_pool_1 16 2026-02-24T07:46:01.534 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:46:01.663 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:46:01.661+0000 7f66e05fa640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:46:01.663 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:46:01.661+0000 7f66e05fa640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:46:01.872 INFO:teuthology.orchestra.run.trial035.stderr:pool 'unique_pool_1' created 2026-02-24T07:46:01.903 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:46:01 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/3701598152' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "unique_pool_0", "pool2": "unique_pool_0", "yes_i_really_really_mean_it": true}]': finished 2026-02-24T07:46:01.903 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:46:01 trial035 ceph-mon[39284]: osdmap e41: 4 total, 4 up, 4 in 2026-02-24T07:46:01.903 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:46:01 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/2035108899' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "unique_pool_1", "pg_num": 16} : dispatch 2026-02-24T07:46:01.913 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph osd pool application enable unique_pool_1 rados --yes-i-really-mean-it || true 2026-02-24T07:46:02.029 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:46:02.078 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2026-02-24T07:46:02.128 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:46:02.126+0000 7fbdf711f640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:46:02.129 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:46:02.126+0000 7fbdf711f640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:46:02.194 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:46:02.296 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:46:02.293+0000 7fb98612e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:46:02.296 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:46:02.294+0000 7fb98612e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:46:02.308 INFO:teuthology.orchestra.run.trial035.stdout:osd.0: {} 2026-02-24T07:46:02.308 INFO:teuthology.orchestra.run.trial035.stderr:osd.0: osd_enable_op_tracker = '' 2026-02-24T07:46:02.318 INFO:teuthology.orchestra.run.trial035.stdout:osd.1: {} 2026-02-24T07:46:02.318 INFO:teuthology.orchestra.run.trial035.stderr:osd.1: osd_enable_op_tracker = '' 2026-02-24T07:46:02.329 INFO:teuthology.orchestra.run.trial035.stdout:osd.2: {} 2026-02-24T07:46:02.330 INFO:teuthology.orchestra.run.trial035.stderr:osd.2: osd_enable_op_tracker = '' 2026-02-24T07:46:02.340 INFO:teuthology.orchestra.run.trial035.stdout:osd.3: {} 2026-02-24T07:46:02.340 INFO:teuthology.orchestra.run.trial035.stderr:osd.3: osd_enable_op_tracker = '' 2026-02-24T07:46:02.877 INFO:teuthology.orchestra.run.trial035.stderr:enabled application 'rados' on pool 'unique_pool_1' 2026-02-24T07:46:03.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:46:02 trial035 ceph-mon[39284]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 2.4 GiB used, 2.7 TiB / 2.7 TiB avail 2026-02-24T07:46:03.205 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:46:02 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/2035108899' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "unique_pool_1", "pg_num": 16}]': finished 2026-02-24T07:46:03.205 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:46:02 trial035 ceph-mon[39284]: osdmap e42: 4 total, 4 up, 4 in 2026-02-24T07:46:03.205 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:46:02 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/43700027' entity='client.admin' cmd={"prefix": "osd pool application enable", "pool": "unique_pool_1", "app": "rados", "yes_i_really_mean_it": true} : dispatch 2026-02-24T07:46:03.205 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:46:02 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/1433694186' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2026-02-24T07:46:03.919 DEBUG:teuthology.orchestra.run.trial035:> /bin/sh -c 'adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --no-log-to-stderr --name client.0 -t 16 -b 65536 --object-size 65536 -p unique_pool_1 bench 90 write' 2026-02-24T07:46:03.920 INFO:tasks.thrashosds.thrasher:Growing pool .mgr 2026-02-24T07:46:03.920 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph pg dump --format=json 2026-02-24T07:46:03.923 INFO:tasks.radosbench:joining radosbench (timing out after 3000s) 2026-02-24T07:46:03.924 INFO:teuthology.orchestra.run:waiting for 3000 2026-02-24T07:46:03.972 INFO:tasks.radosbench.radosbench.0.trial035.stdout:hints = 1 2026-02-24T07:46:03.972 INFO:tasks.radosbench.radosbench.0.trial035.stdout:Maintaining 16 concurrent writes of 65536 bytes to objects of size 65536 for up to 90 seconds or 0 objects 2026-02-24T07:46:03.972 INFO:tasks.radosbench.radosbench.0.trial035.stdout:Object prefix: benchmark_data_trial035_92015 2026-02-24T07:46:03.972 INFO:tasks.radosbench.radosbench.0.trial035.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2026-02-24T07:46:03.973 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 0 0 0 0 0 0 - 0 2026-02-24T07:46:04.056 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:46:04.173 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:46:03 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/43700027' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "unique_pool_1", "app": "rados", "yes_i_really_mean_it": true}]': finished 2026-02-24T07:46:04.174 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:46:03 trial035 ceph-mon[39284]: osdmap e43: 4 total, 4 up, 4 in 2026-02-24T07:46:04.174 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:46:03 trial035 ceph-mon[39284]: pgmap v109: 17 pgs: 16 unknown, 1 active+clean; 577 KiB data, 2.4 GiB used, 2.7 TiB / 2.7 TiB avail 2026-02-24T07:46:04.181 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:46:04.178+0000 7fd49a7e2640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:46:04.181 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:46:04.179+0000 7fd49a7e2640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:46:04.329 INFO:teuthology.orchestra.run.trial035.stdout: 2026-02-24T07:46:04.329 INFO:teuthology.orchestra.run.trial035.stderr:dumped all 2026-02-24T07:46:04.374 INFO:teuthology.orchestra.run.trial035.stdout:{"pg_ready":true,"pg_map":{"version":110,"stamp":"2026-02-24T07:46:03.879472+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":240,"num_write":135,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":3,"acting":3,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":51,"num_osds":4,"num_per_pool_osds":4,"num_per_pool_omap_osds":4,"kb":2936012800,"kb_used":2501164,"kb_used_data":3228,"kb_used_omap":20,"kb_used_meta":2497835,"kb_avail":2933511636,"statfs":{"total":3006477107200,"available":3003915915264,"internally_reserved":0,"allocated":3305472,"data_stored":2306464,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":21071,"internal_metadata":2557783473},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-122617880,"num_objects":-1872,"num_object_clones":0,"num_object_copies":-5616,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-1872,"num_whiteouts":0,"num_read":-1,"num_read_kb":-1,"num_write":-1146264,"num_write_kb":-36740289,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.197712"},"pg_stats":[{"pgid":"3.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2026-02-24T07:46:01.867949+0000","last_change":"2026-02-24T07:46:01.867949+0000","last_active":"2026-02-24T07:46:01.867949+0000","last_peered":"2026-02-24T07:46:01.867949+0000","last_clean":"2026-02-24T07:46:01.867949+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2026-02-24T07:46:01.867949+0000","last_undegraded":"2026-02-24T07:46:01.867949+0000","last_fullsized":"2026-02-24T07:46:01.867949+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_clean_scrub_stamp":"2026-02-24T07:46:01.867949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2026-02-24T07:46:01.867949+0000","last_change":"2026-02-24T07:46:01.867949+0000","last_active":"2026-02-24T07:46:01.867949+0000","last_peered":"2026-02-24T07:46:01.867949+0000","last_clean":"2026-02-24T07:46:01.867949+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2026-02-24T07:46:01.867949+0000","last_undegraded":"2026-02-24T07:46:01.867949+0000","last_fullsized":"2026-02-24T07:46:01.867949+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_clean_scrub_stamp":"2026-02-24T07:46:01.867949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2026-02-24T07:46:01.867949+0000","last_change":"2026-02-24T07:46:01.867949+0000","last_active":"2026-02-24T07:46:01.867949+0000","last_peered":"2026-02-24T07:46:01.867949+0000","last_clean":"2026-02-24T07:46:01.867949+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2026-02-24T07:46:01.867949+0000","last_undegraded":"2026-02-24T07:46:01.867949+0000","last_fullsized":"2026-02-24T07:46:01.867949+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_clean_scrub_stamp":"2026-02-24T07:46:01.867949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2026-02-24T07:46:01.867949+0000","last_change":"2026-02-24T07:46:01.867949+0000","last_active":"2026-02-24T07:46:01.867949+0000","last_peered":"2026-02-24T07:46:01.867949+0000","last_clean":"2026-02-24T07:46:01.867949+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2026-02-24T07:46:01.867949+0000","last_undegraded":"2026-02-24T07:46:01.867949+0000","last_fullsized":"2026-02-24T07:46:01.867949+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_clean_scrub_stamp":"2026-02-24T07:46:01.867949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2026-02-24T07:46:01.867949+0000","last_change":"2026-02-24T07:46:01.867949+0000","last_active":"2026-02-24T07:46:01.867949+0000","last_peered":"2026-02-24T07:46:01.867949+0000","last_clean":"2026-02-24T07:46:01.867949+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2026-02-24T07:46:01.867949+0000","last_undegraded":"2026-02-24T07:46:01.867949+0000","last_fullsized":"2026-02-24T07:46:01.867949+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_clean_scrub_stamp":"2026-02-24T07:46:01.867949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2026-02-24T07:46:01.867949+0000","last_change":"2026-02-24T07:46:01.867949+0000","last_active":"2026-02-24T07:46:01.867949+0000","last_peered":"2026-02-24T07:46:01.867949+0000","last_clean":"2026-02-24T07:46:01.867949+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2026-02-24T07:46:01.867949+0000","last_undegraded":"2026-02-24T07:46:01.867949+0000","last_fullsized":"2026-02-24T07:46:01.867949+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_clean_scrub_stamp":"2026-02-24T07:46:01.867949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2026-02-24T07:46:01.867949+0000","last_change":"2026-02-24T07:46:01.867949+0000","last_active":"2026-02-24T07:46:01.867949+0000","last_peered":"2026-02-24T07:46:01.867949+0000","last_clean":"2026-02-24T07:46:01.867949+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2026-02-24T07:46:01.867949+0000","last_undegraded":"2026-02-24T07:46:01.867949+0000","last_fullsized":"2026-02-24T07:46:01.867949+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_clean_scrub_stamp":"2026-02-24T07:46:01.867949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2026-02-24T07:46:01.867949+0000","last_change":"2026-02-24T07:46:01.867949+0000","last_active":"2026-02-24T07:46:01.867949+0000","last_peered":"2026-02-24T07:46:01.867949+0000","last_clean":"2026-02-24T07:46:01.867949+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2026-02-24T07:46:01.867949+0000","last_undegraded":"2026-02-24T07:46:01.867949+0000","last_fullsized":"2026-02-24T07:46:01.867949+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_clean_scrub_stamp":"2026-02-24T07:46:01.867949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2026-02-24T07:46:01.867949+0000","last_change":"2026-02-24T07:46:01.867949+0000","last_active":"2026-02-24T07:46:01.867949+0000","last_peered":"2026-02-24T07:46:01.867949+0000","last_clean":"2026-02-24T07:46:01.867949+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2026-02-24T07:46:01.867949+0000","last_undegraded":"2026-02-24T07:46:01.867949+0000","last_fullsized":"2026-02-24T07:46:01.867949+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_clean_scrub_stamp":"2026-02-24T07:46:01.867949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2026-02-24T07:46:01.867949+0000","last_change":"2026-02-24T07:46:01.867949+0000","last_active":"2026-02-24T07:46:01.867949+0000","last_peered":"2026-02-24T07:46:01.867949+0000","last_clean":"2026-02-24T07:46:01.867949+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2026-02-24T07:46:01.867949+0000","last_undegraded":"2026-02-24T07:46:01.867949+0000","last_fullsized":"2026-02-24T07:46:01.867949+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_clean_scrub_stamp":"2026-02-24T07:46:01.867949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2026-02-24T07:46:01.867949+0000","last_change":"2026-02-24T07:46:01.867949+0000","last_active":"2026-02-24T07:46:01.867949+0000","last_peered":"2026-02-24T07:46:01.867949+0000","last_clean":"2026-02-24T07:46:01.867949+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2026-02-24T07:46:01.867949+0000","last_undegraded":"2026-02-24T07:46:01.867949+0000","last_fullsized":"2026-02-24T07:46:01.867949+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_clean_scrub_stamp":"2026-02-24T07:46:01.867949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2026-02-24T07:46:01.867949+0000","last_change":"2026-02-24T07:46:01.867949+0000","last_active":"2026-02-24T07:46:01.867949+0000","last_peered":"2026-02-24T07:46:01.867949+0000","last_clean":"2026-02-24T07:46:01.867949+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2026-02-24T07:46:01.867949+0000","last_undegraded":"2026-02-24T07:46:01.867949+0000","last_fullsized":"2026-02-24T07:46:01.867949+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_clean_scrub_stamp":"2026-02-24T07:46:01.867949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2026-02-24T07:46:01.867949+0000","last_change":"2026-02-24T07:46:01.867949+0000","last_active":"2026-02-24T07:46:01.867949+0000","last_peered":"2026-02-24T07:46:01.867949+0000","last_clean":"2026-02-24T07:46:01.867949+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2026-02-24T07:46:01.867949+0000","last_undegraded":"2026-02-24T07:46:01.867949+0000","last_fullsized":"2026-02-24T07:46:01.867949+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_clean_scrub_stamp":"2026-02-24T07:46:01.867949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2026-02-24T07:46:01.867949+0000","last_change":"2026-02-24T07:46:01.867949+0000","last_active":"2026-02-24T07:46:01.867949+0000","last_peered":"2026-02-24T07:46:01.867949+0000","last_clean":"2026-02-24T07:46:01.867949+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2026-02-24T07:46:01.867949+0000","last_undegraded":"2026-02-24T07:46:01.867949+0000","last_fullsized":"2026-02-24T07:46:01.867949+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_clean_scrub_stamp":"2026-02-24T07:46:01.867949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2026-02-24T07:46:01.867949+0000","last_change":"2026-02-24T07:46:01.867949+0000","last_active":"2026-02-24T07:46:01.867949+0000","last_peered":"2026-02-24T07:46:01.867949+0000","last_clean":"2026-02-24T07:46:01.867949+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2026-02-24T07:46:01.867949+0000","last_undegraded":"2026-02-24T07:46:01.867949+0000","last_fullsized":"2026-02-24T07:46:01.867949+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_clean_scrub_stamp":"2026-02-24T07:46:01.867949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2026-02-24T07:46:01.867949+0000","last_change":"2026-02-24T07:46:01.867949+0000","last_active":"2026-02-24T07:46:01.867949+0000","last_peered":"2026-02-24T07:46:01.867949+0000","last_clean":"2026-02-24T07:46:01.867949+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2026-02-24T07:46:01.867949+0000","last_undegraded":"2026-02-24T07:46:01.867949+0000","last_fullsized":"2026-02-24T07:46:01.867949+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_clean_scrub_stamp":"2026-02-24T07:46:01.867949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"29'99","reported_seq":218,"reported_epoch":40,"state":"active+clean","last_fresh":"2026-02-24T07:45:26.125455+0000","last_change":"2026-02-24T07:45:26.125455+0000","last_active":"2026-02-24T07:45:26.125455+0000","last_peered":"2026-02-24T07:45:26.125455+0000","last_clean":"2026-02-24T07:45:26.125455+0000","last_became_active":"2026-02-24T07:42:54.186719+0000","last_became_peered":"2026-02-24T07:42:54.186719+0000","last_unstale":"2026-02-24T07:45:26.125455+0000","last_undegraded":"2026-02-24T07:45:26.125455+0000","last_fullsized":"2026-02-24T07:45:26.125455+0000","mapping_epoch":27,"log_start":"29'96","ondisk_log_start":"29'96","created":20,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'99","last_scrub_stamp":"2026-02-24T07:45:26.125426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:42:43.640776+0000","last_clean_scrub_stamp":"2026-02-24T07:45:26.125426+0000","objects_scrubbed":2,"log_size":3,"log_dups_size":96,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:46:34.126365+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":240,"num_write":135,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":16,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":240,"num_write":135,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1014,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":3,"up_from":25,"seq":107374182440,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":916876,"kb_used_data":952,"kb_used_omap":5,"kb_used_meta":915898,"kb_avail":733086324,"statfs":{"total":751619276800,"available":750680395776,"internally_reserved":0,"allocated":974848,"data_stored":724208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":5648,"internal_metadata":937880048},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":19,"seq":81604378666,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":476620,"kb_used_data":952,"kb_used_omap":4,"kb_used_meta":475643,"kb_avail":733526580,"statfs":{"total":751619276800,"available":751131217920,"internally_reserved":0,"allocated":974848,"data_stored":724208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4542,"internal_metadata":487059010},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542187,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":630920,"kb_used_data":372,"kb_used_omap":5,"kb_used_meta":630522,"kb_avail":733372280,"statfs":{"total":751619276800,"available":750973214720,"internally_reserved":0,"allocated":380928,"data_stored":133840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":5779,"internal_metadata":645654893},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705710,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":476748,"kb_used_data":952,"kb_used_omap":4,"kb_used_meta":475771,"kb_avail":733526452,"statfs":{"total":751619276800,"available":751131086848,"internally_reserved":0,"allocated":974848,"data_stored":724208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":5102,"internal_metadata":487189522},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1014,"internal_metadata":0}]}} 2026-02-24T07:46:04.376 INFO:tasks.cephadm.ceph_manager.ceph:increase pool size by 10 2026-02-24T07:46:04.376 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph osd pool set .mgr pg_num 111 2026-02-24T07:46:04.377 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2026-02-24T07:46:04.525 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:46:04.534 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:46:04.662 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:46:04.660+0000 7fdc15a90640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:46:04.663 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:46:04.660+0000 7fdc15a90640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:46:04.670 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:46:04.668+0000 7fe827470640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:46:04.671 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:46:04.668+0000 7fe827470640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:46:04.677 INFO:teuthology.orchestra.run.trial035.stdout:osd.0: {} 2026-02-24T07:46:04.677 INFO:teuthology.orchestra.run.trial035.stderr:osd.0: osd_enable_op_tracker = '' 2026-02-24T07:46:04.691 INFO:teuthology.orchestra.run.trial035.stdout:osd.1: {} 2026-02-24T07:46:04.692 INFO:teuthology.orchestra.run.trial035.stderr:osd.1: osd_enable_op_tracker = '' 2026-02-24T07:46:04.705 INFO:teuthology.orchestra.run.trial035.stdout:osd.2: {} 2026-02-24T07:46:04.706 INFO:teuthology.orchestra.run.trial035.stderr:osd.2: osd_enable_op_tracker = '' 2026-02-24T07:46:04.718 INFO:teuthology.orchestra.run.trial035.stdout:osd.3: {} 2026-02-24T07:46:04.719 INFO:teuthology.orchestra.run.trial035.stderr:osd.3: osd_enable_op_tracker = '' 2026-02-24T07:46:04.832 INFO:teuthology.orchestra.run.trial035.stderr:Error EPERM: crimson_allow_pg_split is false; pg_num increase denied for crimson pool 2026-02-24T07:46:04.872 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-24T07:46:04.972 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 1 16 7957 7941 496.27 496.312 0.00226202 0.00201052 2026-02-24T07:46:05.084 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:46:04 trial035 ceph-mon[39284]: osdmap e44: 4 total, 4 up, 4 in 2026-02-24T07:46:05.084 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:46:04 trial035 ceph-mon[39284]: from='client.14608 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2026-02-24T07:46:05.084 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:46:04 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/636625554' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2026-02-24T07:46:05.084 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:46:04 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/4220513342' entity='client.admin' cmd={"prefix": "osd pool set", "pool": ".mgr", "var": "pg_num", "val": "111"} : dispatch 2026-02-24T07:46:05.972 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 2 15 16169 16154 504.772 513.312 0.00251883 0.00197887 2026-02-24T07:46:06.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:46:05 trial035 ceph-mon[39284]: purged_snaps scrub starts 2026-02-24T07:46:06.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:46:05 trial035 ceph-mon[39284]: purged_snaps scrub ok 2026-02-24T07:46:06.205 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:46:05 trial035 ceph-mon[39284]: pgmap v111: 17 pgs: 12 unknown, 5 active+clean; 220 MiB data, 3.1 GiB used, 2.7 TiB / 2.7 TiB avail; 45 MiB/s wr, 727 op/s 2026-02-24T07:46:06.764 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2026-02-24T07:46:06.915 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:46:06.972 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 3 16 24291 24275 505.689 507.562 0.00230384 0.00197559 2026-02-24T07:46:07.046 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:46:07.044+0000 7fbca2956640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:46:07.046 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:46:07.044+0000 7fbca2956640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:46:07.047 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:46:06 trial035 ceph-mon[39284]: from='mgr.14284 10.20.193.35:0/3681373222' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2026-02-24T07:46:07.047 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:46:06 trial035 ceph-mon[39284]: from='mgr.14284 10.20.193.35:0/3681373222' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-24T07:46:07.047 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:46:06 trial035 ceph-mon[39284]: from='mgr.14284 10.20.193.35:0/3681373222' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2026-02-24T07:46:07.047 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:46:06 trial035 ceph-mon[39284]: from='mgr.14284 10.20.193.35:0/3681373222' entity='mgr.x' 2026-02-24T07:46:07.047 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:46:06 trial035 ceph-mon[39284]: from='mgr.14284 10.20.193.35:0/3681373222' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2026-02-24T07:46:07.047 INFO:journalctl@ceph.mgr.x.trial035.stdout:Feb 24 07:46:06 trial035 ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-mgr-x[39621]: ::ffff:10.20.193.91 - - [24/Feb/2026:07:46:06] "GET /metrics HTTP/1.1" 200 4217 "" "Prometheus/3.6.0" 2026-02-24T07:46:07.061 INFO:teuthology.orchestra.run.trial035.stdout:osd.0: {} 2026-02-24T07:46:07.062 INFO:teuthology.orchestra.run.trial035.stderr:osd.0: osd_enable_op_tracker = '' 2026-02-24T07:46:07.075 INFO:teuthology.orchestra.run.trial035.stdout:osd.1: {} 2026-02-24T07:46:07.075 INFO:teuthology.orchestra.run.trial035.stderr:osd.1: osd_enable_op_tracker = '' 2026-02-24T07:46:07.087 INFO:teuthology.orchestra.run.trial035.stdout:osd.2: {} 2026-02-24T07:46:07.087 INFO:teuthology.orchestra.run.trial035.stderr:osd.2: osd_enable_op_tracker = '' 2026-02-24T07:46:07.099 INFO:teuthology.orchestra.run.trial035.stdout:osd.3: {} 2026-02-24T07:46:07.099 INFO:teuthology.orchestra.run.trial035.stderr:osd.3: osd_enable_op_tracker = '' 2026-02-24T07:46:07.356 INFO:journalctl@ceph.prometheus.a.trial091.stdout:Feb 24 07:46:06 trial091 ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-prometheus-a[66007]: time=2026-02-24T07:46:06.973Z level=ERROR source=scrape.go:1630 msg="Failed to determine correct type of scrape target." component="scrape manager" scrape_pool=ceph target=http://trial035:9283/metrics content_type="text/html;charset=utf-8" fallback_media_type="" err="received unsupported Content-Type \"text/html;charset=utf-8\" and no fallback_scrape_protocol specified for target" 2026-02-24T07:46:07.972 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 4 16 32314 32298 504.616 501.438 0.00174078 0.00198003 2026-02-24T07:46:08.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:46:07 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/3636443529' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2026-02-24T07:46:08.205 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:46:07 trial035 ceph-mon[39284]: pgmap v112: 17 pgs: 17 active+clean; 883 MiB data, 4.3 GiB used, 2.7 TiB / 2.7 TiB avail; 147 MiB/s wr, 2.35k op/s 2026-02-24T07:46:08.972 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 5 16 40757 40741 509.222 527.688 0.00295525 0.00196199 2026-02-24T07:46:09.143 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2026-02-24T07:46:09.292 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:46:09.422 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:46:09.420+0000 7f8480d47640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:46:09.422 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:46:09.420+0000 7f8480d47640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:46:09.437 INFO:teuthology.orchestra.run.trial035.stdout:osd.0: {} 2026-02-24T07:46:09.438 INFO:teuthology.orchestra.run.trial035.stderr:osd.0: osd_enable_op_tracker = '' 2026-02-24T07:46:09.452 INFO:teuthology.orchestra.run.trial035.stdout:osd.1: {} 2026-02-24T07:46:09.452 INFO:teuthology.orchestra.run.trial035.stderr:osd.1: osd_enable_op_tracker = '' 2026-02-24T07:46:09.465 INFO:teuthology.orchestra.run.trial035.stdout:osd.2: {} 2026-02-24T07:46:09.465 INFO:teuthology.orchestra.run.trial035.stderr:osd.2: osd_enable_op_tracker = '' 2026-02-24T07:46:09.482 INFO:teuthology.orchestra.run.trial035.stdout:osd.3: {} 2026-02-24T07:46:09.482 INFO:teuthology.orchestra.run.trial035.stderr:osd.3: osd_enable_op_tracker = '' 2026-02-24T07:46:09.704 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:46:09 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/1026717570' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2026-02-24T07:46:09.874 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2026-02-24T07:46:09.874 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2026-02-24T07:46:09.875 INFO:tasks.thrashosds.thrasher:fixing pg num pool unique_pool_1 2026-02-24T07:46:09.875 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph pg dump --format=json 2026-02-24T07:46:09.973 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 6 16 48950 48934 509.69 512.062 0.00175353 0.0019604 2026-02-24T07:46:10.032 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:46:10.159 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:46:10.157+0000 7fc96cb58640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:46:10.159 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:46:10.158+0000 7fc96cb58640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:46:10.306 INFO:teuthology.orchestra.run.trial035.stdout: 2026-02-24T07:46:10.306 INFO:teuthology.orchestra.run.trial035.stderr:dumped all 2026-02-24T07:46:10.350 INFO:teuthology.orchestra.run.trial035.stdout:{"pg_ready":true,"pg_map":{"version":113,"stamp":"2026-02-24T07:46:09.683099+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":925565472,"num_objects":14116,"num_object_clones":0,"num_object_copies":42348,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14116,"num_whiteouts":0,"num_read":136,"num_read_kb":240,"num_write":14249,"num_write_kb":905446,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":52,"ondisk_log_size":52,"up":51,"acting":51,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":51,"num_osds":4,"num_per_pool_osds":4,"num_per_pool_omap_osds":4,"kb":2936012800,"kb_used":4482092,"kb_used_data":1893916,"kb_used_omap":31,"kb_used_meta":2588064,"kb_avail":2931530708,"statfs":{"total":3006477107200,"available":3001887444992,"internally_reserved":0,"allocated":1939369984,"data_stored":1938249136,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":31994,"internal_metadata":2650178310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[9,3],"upper_bound":4},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":924975104,"num_objects":14114,"num_object_clones":0,"num_object_copies":42342,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14114,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14114,"num_write_kb":903296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.811124"},"pg_stats":[{"pgid":"3.f","version":"44'874","reported_seq":888,"reported_epoch":44,"state":"active+clean","last_fresh":"2026-02-24T07:46:05.676884+0000","last_change":"2026-02-24T07:46:02.882095+0000","last_active":"2026-02-24T07:46:05.676884+0000","last_peered":"2026-02-24T07:46:05.676884+0000","last_clean":"2026-02-24T07:46:05.676884+0000","last_became_active":"2026-02-24T07:46:02.881625+0000","last_became_peered":"2026-02-24T07:46:02.881625+0000","last_unstale":"2026-02-24T07:46:05.676884+0000","last_undegraded":"2026-02-24T07:46:05.676884+0000","last_fullsized":"2026-02-24T07:46:05.676884+0000","mapping_epoch":42,"log_start":"44'871","ondisk_log_start":"44'871","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_clean_scrub_stamp":"2026-02-24T07:46:01.867949+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":871,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:47:12.812774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":57278464,"num_objects":874,"num_object_clones":0,"num_object_copies":2622,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":874,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":874,"num_write_kb":55936,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.e","version":"44'876","reported_seq":889,"reported_epoch":44,"state":"active+clean","last_fresh":"2026-02-24T07:46:05.673743+0000","last_change":"2026-02-24T07:46:02.881462+0000","last_active":"2026-02-24T07:46:05.673743+0000","last_peered":"2026-02-24T07:46:05.673743+0000","last_clean":"2026-02-24T07:46:05.673743+0000","last_became_active":"2026-02-24T07:46:02.881083+0000","last_became_peered":"2026-02-24T07:46:02.881083+0000","last_unstale":"2026-02-24T07:46:05.673743+0000","last_undegraded":"2026-02-24T07:46:05.673743+0000","last_fullsized":"2026-02-24T07:46:05.673743+0000","mapping_epoch":42,"log_start":"44'873","ondisk_log_start":"44'873","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_clean_scrub_stamp":"2026-02-24T07:46:01.867949+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":873,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:47:10.312296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":57409536,"num_objects":876,"num_object_clones":0,"num_object_copies":2628,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":876,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":876,"num_write_kb":56064,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"44'874","reported_seq":886,"reported_epoch":44,"state":"active+clean","last_fresh":"2026-02-24T07:46:05.704469+0000","last_change":"2026-02-24T07:46:02.883952+0000","last_active":"2026-02-24T07:46:05.704469+0000","last_peered":"2026-02-24T07:46:05.704469+0000","last_clean":"2026-02-24T07:46:05.704469+0000","last_became_active":"2026-02-24T07:46:02.883799+0000","last_became_peered":"2026-02-24T07:46:02.883799+0000","last_unstale":"2026-02-24T07:46:05.704469+0000","last_undegraded":"2026-02-24T07:46:05.704469+0000","last_fullsized":"2026-02-24T07:46:05.704469+0000","mapping_epoch":42,"log_start":"44'871","ondisk_log_start":"44'871","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_clean_scrub_stamp":"2026-02-24T07:46:01.867949+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":871,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:47:14.432862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":57278464,"num_objects":874,"num_object_clones":0,"num_object_copies":2622,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":874,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":874,"num_write_kb":55936,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"44'828","reported_seq":840,"reported_epoch":44,"state":"active+clean","last_fresh":"2026-02-24T07:46:05.702029+0000","last_change":"2026-02-24T07:46:02.880375+0000","last_active":"2026-02-24T07:46:05.702029+0000","last_peered":"2026-02-24T07:46:05.702029+0000","last_clean":"2026-02-24T07:46:05.702029+0000","last_became_active":"2026-02-24T07:46:02.880060+0000","last_became_peered":"2026-02-24T07:46:02.880060+0000","last_unstale":"2026-02-24T07:46:05.702029+0000","last_undegraded":"2026-02-24T07:46:05.702029+0000","last_fullsized":"2026-02-24T07:46:05.702029+0000","mapping_epoch":42,"log_start":"44'825","ondisk_log_start":"44'825","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_clean_scrub_stamp":"2026-02-24T07:46:01.867949+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":825,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:47:28.898648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":54263808,"num_objects":828,"num_object_clones":0,"num_object_copies":2484,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":828,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":828,"num_write_kb":52992,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"44'907","reported_seq":921,"reported_epoch":44,"state":"active+clean","last_fresh":"2026-02-24T07:46:05.745919+0000","last_change":"2026-02-24T07:46:02.880509+0000","last_active":"2026-02-24T07:46:05.745919+0000","last_peered":"2026-02-24T07:46:05.745919+0000","last_clean":"2026-02-24T07:46:05.745919+0000","last_became_active":"2026-02-24T07:46:02.880171+0000","last_became_peered":"2026-02-24T07:46:02.880171+0000","last_unstale":"2026-02-24T07:46:05.745919+0000","last_undegraded":"2026-02-24T07:46:05.745919+0000","last_fullsized":"2026-02-24T07:46:05.745919+0000","mapping_epoch":42,"log_start":"44'904","ondisk_log_start":"44'904","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_clean_scrub_stamp":"2026-02-24T07:46:01.867949+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":904,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:47:06.294580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":59441152,"num_objects":907,"num_object_clones":0,"num_object_copies":2721,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":907,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":907,"num_write_kb":58048,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.a","version":"44'912","reported_seq":926,"reported_epoch":44,"state":"active+clean","last_fresh":"2026-02-24T07:46:05.703369+0000","last_change":"2026-02-24T07:46:02.882761+0000","last_active":"2026-02-24T07:46:05.703369+0000","last_peered":"2026-02-24T07:46:05.703369+0000","last_clean":"2026-02-24T07:46:05.703369+0000","last_became_active":"2026-02-24T07:46:02.882274+0000","last_became_peered":"2026-02-24T07:46:02.882274+0000","last_unstale":"2026-02-24T07:46:05.703369+0000","last_undegraded":"2026-02-24T07:46:05.703369+0000","last_fullsized":"2026-02-24T07:46:05.703369+0000","mapping_epoch":42,"log_start":"44'909","ondisk_log_start":"44'909","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_clean_scrub_stamp":"2026-02-24T07:46:01.867949+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":909,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:47:11.014010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":59768832,"num_objects":912,"num_object_clones":0,"num_object_copies":2736,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":912,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":912,"num_write_kb":58368,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"44'857","reported_seq":871,"reported_epoch":44,"state":"active+clean","last_fresh":"2026-02-24T07:46:05.704586+0000","last_change":"2026-02-24T07:46:02.882303+0000","last_active":"2026-02-24T07:46:05.704586+0000","last_peered":"2026-02-24T07:46:05.704586+0000","last_clean":"2026-02-24T07:46:05.704586+0000","last_became_active":"2026-02-24T07:46:02.881951+0000","last_became_peered":"2026-02-24T07:46:02.881951+0000","last_unstale":"2026-02-24T07:46:05.704586+0000","last_undegraded":"2026-02-24T07:46:05.704586+0000","last_fullsized":"2026-02-24T07:46:05.704586+0000","mapping_epoch":42,"log_start":"44'854","ondisk_log_start":"44'854","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_clean_scrub_stamp":"2026-02-24T07:46:01.867949+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":854,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:47:19.768765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":56164352,"num_objects":857,"num_object_clones":0,"num_object_copies":2571,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":857,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":857,"num_write_kb":54848,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"44'890","reported_seq":904,"reported_epoch":44,"state":"active+clean","last_fresh":"2026-02-24T07:46:05.744148+0000","last_change":"2026-02-24T07:46:02.881463+0000","last_active":"2026-02-24T07:46:05.744148+0000","last_peered":"2026-02-24T07:46:05.744148+0000","last_clean":"2026-02-24T07:46:05.744148+0000","last_became_active":"2026-02-24T07:46:02.881345+0000","last_became_peered":"2026-02-24T07:46:02.881345+0000","last_unstale":"2026-02-24T07:46:05.744148+0000","last_undegraded":"2026-02-24T07:46:05.744148+0000","last_fullsized":"2026-02-24T07:46:05.744148+0000","mapping_epoch":42,"log_start":"44'887","ondisk_log_start":"44'887","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_clean_scrub_stamp":"2026-02-24T07:46:01.867949+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":887,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:47:11.310752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":58327040,"num_objects":890,"num_object_clones":0,"num_object_copies":2670,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":890,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":890,"num_write_kb":56960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"44'915","reported_seq":928,"reported_epoch":44,"state":"active+clean","last_fresh":"2026-02-24T07:46:05.745457+0000","last_change":"2026-02-24T07:46:02.880582+0000","last_active":"2026-02-24T07:46:05.745457+0000","last_peered":"2026-02-24T07:46:05.745457+0000","last_clean":"2026-02-24T07:46:05.745457+0000","last_became_active":"2026-02-24T07:46:02.880323+0000","last_became_peered":"2026-02-24T07:46:02.880323+0000","last_unstale":"2026-02-24T07:46:05.745457+0000","last_undegraded":"2026-02-24T07:46:05.745457+0000","last_fullsized":"2026-02-24T07:46:05.745457+0000","mapping_epoch":42,"log_start":"44'912","ondisk_log_start":"44'912","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_clean_scrub_stamp":"2026-02-24T07:46:01.867949+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":912,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:47:20.538625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":59965440,"num_objects":915,"num_object_clones":0,"num_object_copies":2745,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":915,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":915,"num_write_kb":58560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.6","version":"44'863","reported_seq":875,"reported_epoch":44,"state":"active+clean","last_fresh":"2026-02-24T07:46:05.698499+0000","last_change":"2026-02-24T07:46:02.880170+0000","last_active":"2026-02-24T07:46:05.698499+0000","last_peered":"2026-02-24T07:46:05.698499+0000","last_clean":"2026-02-24T07:46:05.698499+0000","last_became_active":"2026-02-24T07:46:02.880005+0000","last_became_peered":"2026-02-24T07:46:02.880005+0000","last_unstale":"2026-02-24T07:46:05.698499+0000","last_undegraded":"2026-02-24T07:46:05.698499+0000","last_fullsized":"2026-02-24T07:46:05.698499+0000","mapping_epoch":42,"log_start":"44'859","ondisk_log_start":"44'859","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_clean_scrub_stamp":"2026-02-24T07:46:01.867949+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":859,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:47:29.015277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":56557568,"num_objects":863,"num_object_clones":0,"num_object_copies":2589,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":863,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":863,"num_write_kb":55232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"44'870","reported_seq":883,"reported_epoch":44,"state":"active+clean","last_fresh":"2026-02-24T07:46:05.676756+0000","last_change":"2026-02-24T07:46:02.881931+0000","last_active":"2026-02-24T07:46:05.676756+0000","last_peered":"2026-02-24T07:46:05.676756+0000","last_clean":"2026-02-24T07:46:05.676756+0000","last_became_active":"2026-02-24T07:46:02.881420+0000","last_became_peered":"2026-02-24T07:46:02.881420+0000","last_unstale":"2026-02-24T07:46:05.676756+0000","last_undegraded":"2026-02-24T07:46:05.676756+0000","last_fullsized":"2026-02-24T07:46:05.676756+0000","mapping_epoch":42,"log_start":"44'867","ondisk_log_start":"44'867","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_clean_scrub_stamp":"2026-02-24T07:46:01.867949+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":867,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:47:08.126609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":57016320,"num_objects":870,"num_object_clones":0,"num_object_copies":2610,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":870,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":870,"num_write_kb":55680,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.4","version":"44'912","reported_seq":925,"reported_epoch":44,"state":"active+clean","last_fresh":"2026-02-24T07:46:05.745419+0000","last_change":"2026-02-24T07:46:02.880661+0000","last_active":"2026-02-24T07:46:05.745419+0000","last_peered":"2026-02-24T07:46:05.745419+0000","last_clean":"2026-02-24T07:46:05.745419+0000","last_became_active":"2026-02-24T07:46:02.880345+0000","last_became_peered":"2026-02-24T07:46:02.880345+0000","last_unstale":"2026-02-24T07:46:05.745419+0000","last_undegraded":"2026-02-24T07:46:05.745419+0000","last_fullsized":"2026-02-24T07:46:05.745419+0000","mapping_epoch":42,"log_start":"44'909","ondisk_log_start":"44'909","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_clean_scrub_stamp":"2026-02-24T07:46:01.867949+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":909,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:47:05.136529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":59768832,"num_objects":912,"num_object_clones":0,"num_object_copies":2736,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":912,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":912,"num_write_kb":58368,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"44'883","reported_seq":897,"reported_epoch":44,"state":"active+clean","last_fresh":"2026-02-24T07:46:05.676191+0000","last_change":"2026-02-24T07:46:02.881591+0000","last_active":"2026-02-24T07:46:05.676191+0000","last_peered":"2026-02-24T07:46:05.676191+0000","last_clean":"2026-02-24T07:46:05.676191+0000","last_became_active":"2026-02-24T07:46:02.881035+0000","last_became_peered":"2026-02-24T07:46:02.881035+0000","last_unstale":"2026-02-24T07:46:05.676191+0000","last_undegraded":"2026-02-24T07:46:05.676191+0000","last_fullsized":"2026-02-24T07:46:05.676191+0000","mapping_epoch":42,"log_start":"44'880","ondisk_log_start":"44'880","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_clean_scrub_stamp":"2026-02-24T07:46:01.867949+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":880,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:47:04.588786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":57868288,"num_objects":883,"num_object_clones":0,"num_object_copies":2649,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":883,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":883,"num_write_kb":56512,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"44'865","reported_seq":877,"reported_epoch":44,"state":"active+clean","last_fresh":"2026-02-24T07:46:05.696188+0000","last_change":"2026-02-24T07:46:02.879955+0000","last_active":"2026-02-24T07:46:05.696188+0000","last_peered":"2026-02-24T07:46:05.696188+0000","last_clean":"2026-02-24T07:46:05.696188+0000","last_became_active":"2026-02-24T07:46:02.879831+0000","last_became_peered":"2026-02-24T07:46:02.879831+0000","last_unstale":"2026-02-24T07:46:05.696188+0000","last_undegraded":"2026-02-24T07:46:05.696188+0000","last_fullsized":"2026-02-24T07:46:05.696188+0000","mapping_epoch":42,"log_start":"44'862","ondisk_log_start":"44'862","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_clean_scrub_stamp":"2026-02-24T07:46:01.867949+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":862,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:47:15.015015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":56688640,"num_objects":865,"num_object_clones":0,"num_object_copies":2595,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":865,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":865,"num_write_kb":55360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.0","version":"44'858","reported_seq":872,"reported_epoch":44,"state":"active+clean","last_fresh":"2026-02-24T07:46:05.744257+0000","last_change":"2026-02-24T07:46:02.880585+0000","last_active":"2026-02-24T07:46:05.744257+0000","last_peered":"2026-02-24T07:46:05.744257+0000","last_clean":"2026-02-24T07:46:05.744257+0000","last_became_active":"2026-02-24T07:46:02.880333+0000","last_became_peered":"2026-02-24T07:46:02.880333+0000","last_unstale":"2026-02-24T07:46:05.744257+0000","last_undegraded":"2026-02-24T07:46:05.744257+0000","last_fullsized":"2026-02-24T07:46:05.744257+0000","mapping_epoch":42,"log_start":"44'855","ondisk_log_start":"44'855","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_clean_scrub_stamp":"2026-02-24T07:46:01.867949+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":855,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:47:22.086512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":56229888,"num_objects":858,"num_object_clones":0,"num_object_copies":2574,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":858,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":858,"num_write_kb":54912,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2","version":"44'930","reported_seq":943,"reported_epoch":44,"state":"active+clean","last_fresh":"2026-02-24T07:46:05.745687+0000","last_change":"2026-02-24T07:46:02.880566+0000","last_active":"2026-02-24T07:46:05.745687+0000","last_peered":"2026-02-24T07:46:05.745687+0000","last_clean":"2026-02-24T07:46:05.745687+0000","last_became_active":"2026-02-24T07:46:02.880185+0000","last_became_peered":"2026-02-24T07:46:02.880185+0000","last_unstale":"2026-02-24T07:46:05.745687+0000","last_undegraded":"2026-02-24T07:46:05.745687+0000","last_fullsized":"2026-02-24T07:46:05.745687+0000","mapping_epoch":42,"log_start":"44'927","ondisk_log_start":"44'927","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_clean_scrub_stamp":"2026-02-24T07:46:01.867949+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":927,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:47:06.727531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":60948480,"num_objects":930,"num_object_clones":0,"num_object_copies":2790,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":930,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":930,"num_write_kb":59520,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"29'99","reported_seq":226,"reported_epoch":44,"state":"active+clean","last_fresh":"2026-02-24T07:46:03.880095+0000","last_change":"2026-02-24T07:45:26.125455+0000","last_active":"2026-02-24T07:46:03.880095+0000","last_peered":"2026-02-24T07:46:03.880095+0000","last_clean":"2026-02-24T07:46:03.880095+0000","last_became_active":"2026-02-24T07:42:54.186719+0000","last_became_peered":"2026-02-24T07:42:54.186719+0000","last_unstale":"2026-02-24T07:46:03.880095+0000","last_undegraded":"2026-02-24T07:46:03.880095+0000","last_fullsized":"2026-02-24T07:46:03.880095+0000","mapping_epoch":27,"log_start":"29'96","ondisk_log_start":"29'96","created":20,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'99","last_scrub_stamp":"2026-02-24T07:45:26.125426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:42:43.640776+0000","last_clean_scrub_stamp":"2026-02-24T07:45:26.125426+0000","objects_scrubbed":2,"log_size":3,"log_dups_size":96,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:46:34.126365+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":240,"num_write":135,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":16,"stat_sum":{"num_bytes":924975104,"num_objects":14114,"num_object_clones":0,"num_object_copies":42342,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14114,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14114,"num_write_kb":903296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2759917568,"data_stored":2759917568,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":49,"ondisk_log_size":49,"up":48,"acting":48,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":240,"num_write":135,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":2232,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":3,"up_from":25,"seq":107374182441,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":1338620,"kb_used_data":386600,"kb_used_omap":8,"kb_used_meta":951991,"kb_avail":732664580,"statfs":{"total":751619276800,"available":750248529920,"internally_reserved":0,"allocated":395878400,"data_stored":395597300,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":8416,"internal_metadata":974839584},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":19,"seq":81604378667,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":1200572,"kb_used_data":688808,"kb_used_omap":7,"kb_used_meta":511736,"kb_avail":732802628,"statfs":{"total":751619276800,"available":750389891072,"internally_reserved":0,"allocated":705339392,"data_stored":705058292,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":7181,"internal_metadata":524018675},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,1],"upper_bound":4},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542188,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":955512,"kb_used_data":324964,"kb_used_omap":8,"kb_used_meta":630519,"kb_avail":733047688,"statfs":{"total":751619276800,"available":750640832512,"internally_reserved":0,"allocated":332763136,"data_stored":332485588,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":8687,"internal_metadata":645651985},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,1],"upper_bound":4},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705711,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":987388,"kb_used_data":493544,"kb_used_omap":7,"kb_used_meta":493816,"kb_avail":733015812,"statfs":{"total":751619276800,"available":750608191488,"internally_reserved":0,"allocated":505389056,"data_stored":505107956,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":7710,"internal_metadata":505668066},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[4,1],"upper_bound":4},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":2232,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":739442688,"data_stored":739442688,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":647757824,"data_stored":647757824,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":794558464,"data_stored":794558464,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":578158592,"data_stored":578158592,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-24T07:46:10.351 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph osd pool set unique_pool_1 pgp_num 16 2026-02-24T07:46:10.491 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:46:10.618 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:46:10.616+0000 7f5d894a0640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:46:10.618 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:46:10.617+0000 7f5d894a0640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:46:10.954 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:46:10 trial035 ceph-mon[39284]: pgmap v113: 17 pgs: 17 active+clean; 883 MiB data, 4.3 GiB used, 2.7 TiB / 2.7 TiB avail; 113 MiB/s wr, 1.81k op/s 2026-02-24T07:46:10.972 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 7 15 57130 57115 509.916 511.312 0.00114437 0.00195965 2026-02-24T07:46:11.480 INFO:teuthology.orchestra.run.trial035.stderr:set pool 3 pgp_num to 16 2026-02-24T07:46:11.526 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2026-02-24T07:46:11.674 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:46:11.734 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:46:11 trial035 ceph-mon[39284]: from='client.14618 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2026-02-24T07:46:11.734 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:46:11 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/3511386341' entity='client.admin' cmd={"prefix": "osd pool set", "pool": "unique_pool_1", "var": "pgp_num", "val": "16"} : dispatch 2026-02-24T07:46:11.804 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:46:11.802+0000 7f3f22744640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:46:11.804 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:46:11.803+0000 7f3f22744640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:46:11.819 INFO:teuthology.orchestra.run.trial035.stdout:osd.0: {} 2026-02-24T07:46:11.819 INFO:teuthology.orchestra.run.trial035.stderr:osd.0: osd_enable_op_tracker = '' 2026-02-24T07:46:11.834 INFO:teuthology.orchestra.run.trial035.stdout:osd.1: {} 2026-02-24T07:46:11.835 INFO:teuthology.orchestra.run.trial035.stderr:osd.1: osd_enable_op_tracker = '' 2026-02-24T07:46:11.850 INFO:teuthology.orchestra.run.trial035.stdout:osd.2: {} 2026-02-24T07:46:11.851 INFO:teuthology.orchestra.run.trial035.stderr:osd.2: osd_enable_op_tracker = '' 2026-02-24T07:46:11.863 INFO:teuthology.orchestra.run.trial035.stdout:osd.3: {} 2026-02-24T07:46:11.863 INFO:teuthology.orchestra.run.trial035.stderr:osd.3: osd_enable_op_tracker = '' 2026-02-24T07:46:11.973 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 8 15 65289 65274 509.913 509.938 0.00230584 0.00195971 2026-02-24T07:46:12.954 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:46:12 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/3511386341' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "unique_pool_1", "var": "pgp_num", "val": "16"}]': finished 2026-02-24T07:46:12.954 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:46:12 trial035 ceph-mon[39284]: osdmap e45: 4 total, 4 up, 4 in 2026-02-24T07:46:12.954 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:46:12 trial035 ceph-mon[39284]: pgmap v115: 17 pgs: 17 active+clean; 3.4 GiB data, 12 GiB used, 2.7 TiB / 2.7 TiB avail; 429 MiB/s wr, 6.87k op/s 2026-02-24T07:46:12.954 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:46:12 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/3519326859' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2026-02-24T07:46:12.973 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 9 16 73518 73502 510.391 514.25 0.00311267 0.00195786 2026-02-24T07:46:13.904 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2026-02-24T07:46:13.973 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 10 15 81551 81536 509.561 502.125 0.00212017 0.00196105 2026-02-24T07:46:14.055 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:46:14.185 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:46:14.183+0000 7fce6ba61640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:46:14.185 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:46:14.183+0000 7fce6ba61640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:46:14.199 INFO:teuthology.orchestra.run.trial035.stdout:osd.0: {} 2026-02-24T07:46:14.199 INFO:teuthology.orchestra.run.trial035.stderr:osd.0: osd_enable_op_tracker = '' 2026-02-24T07:46:14.215 INFO:teuthology.orchestra.run.trial035.stdout:osd.1: {} 2026-02-24T07:46:14.215 INFO:teuthology.orchestra.run.trial035.stderr:osd.1: osd_enable_op_tracker = '' 2026-02-24T07:46:14.231 INFO:teuthology.orchestra.run.trial035.stdout:osd.2: {} 2026-02-24T07:46:14.232 INFO:teuthology.orchestra.run.trial035.stderr:osd.2: osd_enable_op_tracker = '' 2026-02-24T07:46:14.247 INFO:teuthology.orchestra.run.trial035.stdout:osd.3: {} 2026-02-24T07:46:14.247 INFO:teuthology.orchestra.run.trial035.stderr:osd.3: osd_enable_op_tracker = '' 2026-02-24T07:46:14.821 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:46:14 trial035 ceph-mon[39284]: pgmap v116: 17 pgs: 17 active+clean; 3.4 GiB data, 12 GiB used, 2.7 TiB / 2.7 TiB avail; 350 MiB/s wr, 5.61k op/s 2026-02-24T07:46:14.821 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:46:14 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/791933560' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2026-02-24T07:46:14.973 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 11 15 88259 88244 501.348 419.25 0.00163688 0.00199319 2026-02-24T07:46:15.973 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 12 16 96725 96709 503.654 529.062 0.00204472 0.00198405 2026-02-24T07:46:16.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:46:15 trial035 ceph-mon[39284]: from='mgr.14284 10.20.193.35:0/3681373222' entity='mgr.x' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2026-02-24T07:46:16.527 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2026-02-24T07:46:16.528 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2026-02-24T07:46:16.528 INFO:tasks.thrashosds.thrasher:Growing pool .mgr 2026-02-24T07:46:16.528 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph pg dump --format=json 2026-02-24T07:46:16.681 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:46:16.809 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:46:16.807+0000 7f44161e1640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:46:16.810 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:46:16.807+0000 7f44161e1640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:46:16.810 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:46:16 trial035 ceph-mon[39284]: pgmap v117: 17 pgs: 17 active+clean; 4.0 GiB data, 14 GiB used, 2.7 TiB / 2.7 TiB avail; 384 MiB/s wr, 6.14k op/s 2026-02-24T07:46:16.847 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2026-02-24T07:46:16.945 INFO:teuthology.orchestra.run.trial035.stdout: 2026-02-24T07:46:16.946 INFO:teuthology.orchestra.run.trial035.stderr:dumped all 2026-02-24T07:46:16.973 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 13 16 105223 105207 505.764 531.125 0.00158082 0.00197581 2026-02-24T07:46:16.991 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:46:17.017 INFO:teuthology.orchestra.run.trial035.stdout:{"pg_ready":true,"pg_map":{"version":117,"stamp":"2026-02-24T07:46:15.683814+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4255973920,"num_objects":64934,"num_object_clones":0,"num_object_copies":194802,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64934,"num_whiteouts":0,"num_read":136,"num_read_kb":240,"num_write":65067,"num_write_kb":4157798,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":52,"ondisk_log_size":52,"up":51,"acting":51,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":51,"num_osds":4,"num_per_pool_osds":4,"num_per_pool_omap_osds":4,"kb":2936012800,"kb_used":15186488,"kb_used_data":11966568,"kb_used_omap":32,"kb_used_meta":3219743,"kb_avail":2920826312,"statfs":{"total":3006477107200,"available":2990926143488,"internally_reserved":0,"allocated":12253765632,"data_stored":12252637240,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":33298,"internal_metadata":3297017326},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[9,5,1],"upper_bound":8},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":4025810944,"num_objects":61429,"num_object_clones":0,"num_object_copies":184287,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":61429,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":61429,"num_write_kb":3931456,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.001287"},"pg_stats":[{"pgid":"3.f","version":"45'5878","reported_seq":5895,"reported_epoch":45,"state":"active+clean","last_fresh":"2026-02-24T07:46:15.677153+0000","last_change":"2026-02-24T07:46:02.882095+0000","last_active":"2026-02-24T07:46:15.677153+0000","last_peered":"2026-02-24T07:46:15.677153+0000","last_clean":"2026-02-24T07:46:15.677153+0000","last_became_active":"2026-02-24T07:46:02.881625+0000","last_became_peered":"2026-02-24T07:46:02.881625+0000","last_unstale":"2026-02-24T07:46:15.677153+0000","last_undegraded":"2026-02-24T07:46:15.677153+0000","last_fullsized":"2026-02-24T07:46:15.677153+0000","mapping_epoch":42,"log_start":"45'5875","ondisk_log_start":"45'5875","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_clean_scrub_stamp":"2026-02-24T07:46:01.867949+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:47:09.267374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":385220608,"num_objects":5878,"num_object_clones":0,"num_object_copies":17634,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5878,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5878,"num_write_kb":376192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.e","version":"45'6010","reported_seq":6027,"reported_epoch":45,"state":"active+clean","last_fresh":"2026-02-24T07:46:15.676356+0000","last_change":"2026-02-24T07:46:02.881462+0000","last_active":"2026-02-24T07:46:15.676356+0000","last_peered":"2026-02-24T07:46:15.676356+0000","last_clean":"2026-02-24T07:46:15.676356+0000","last_became_active":"2026-02-24T07:46:02.881083+0000","last_became_peered":"2026-02-24T07:46:02.881083+0000","last_unstale":"2026-02-24T07:46:15.676356+0000","last_undegraded":"2026-02-24T07:46:15.676356+0000","last_fullsized":"2026-02-24T07:46:15.676356+0000","mapping_epoch":42,"log_start":"45'6007","ondisk_log_start":"45'6007","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_clean_scrub_stamp":"2026-02-24T07:46:01.867949+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:47:29.030602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393871360,"num_objects":6010,"num_object_clones":0,"num_object_copies":18030,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6010,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6010,"num_write_kb":384640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"44'3376","reported_seq":3390,"reported_epoch":44,"state":"active+clean","last_fresh":"2026-02-24T07:46:10.701161+0000","last_change":"2026-02-24T07:46:02.883952+0000","last_active":"2026-02-24T07:46:10.701161+0000","last_peered":"2026-02-24T07:46:10.701161+0000","last_clean":"2026-02-24T07:46:10.701161+0000","last_became_active":"2026-02-24T07:46:02.883799+0000","last_became_peered":"2026-02-24T07:46:02.883799+0000","last_unstale":"2026-02-24T07:46:10.701161+0000","last_undegraded":"2026-02-24T07:46:10.701161+0000","last_fullsized":"2026-02-24T07:46:10.701161+0000","mapping_epoch":42,"log_start":"44'3373","ondisk_log_start":"44'3373","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_clean_scrub_stamp":"2026-02-24T07:46:01.867949+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:47:14.432862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":221249536,"num_objects":3376,"num_object_clones":0,"num_object_copies":10128,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3376,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3376,"num_write_kb":216064,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"44'3351","reported_seq":3365,"reported_epoch":44,"state":"active+clean","last_fresh":"2026-02-24T07:46:10.699117+0000","last_change":"2026-02-24T07:46:02.880375+0000","last_active":"2026-02-24T07:46:10.699117+0000","last_peered":"2026-02-24T07:46:10.699117+0000","last_clean":"2026-02-24T07:46:10.699117+0000","last_became_active":"2026-02-24T07:46:02.880060+0000","last_became_peered":"2026-02-24T07:46:02.880060+0000","last_unstale":"2026-02-24T07:46:10.699117+0000","last_undegraded":"2026-02-24T07:46:10.699117+0000","last_fullsized":"2026-02-24T07:46:10.699117+0000","mapping_epoch":42,"log_start":"44'3348","ondisk_log_start":"44'3348","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_clean_scrub_stamp":"2026-02-24T07:46:01.867949+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:47:28.898648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":219611136,"num_objects":3351,"num_object_clones":0,"num_object_copies":10053,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3351,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3351,"num_write_kb":214464,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"44'3406","reported_seq":3420,"reported_epoch":44,"state":"active+clean","last_fresh":"2026-02-24T07:46:10.744755+0000","last_change":"2026-02-24T07:46:02.880509+0000","last_active":"2026-02-24T07:46:10.744755+0000","last_peered":"2026-02-24T07:46:10.744755+0000","last_clean":"2026-02-24T07:46:10.744755+0000","last_became_active":"2026-02-24T07:46:02.880171+0000","last_became_peered":"2026-02-24T07:46:02.880171+0000","last_unstale":"2026-02-24T07:46:10.744755+0000","last_undegraded":"2026-02-24T07:46:10.744755+0000","last_fullsized":"2026-02-24T07:46:10.744755+0000","mapping_epoch":42,"log_start":"44'3403","ondisk_log_start":"44'3403","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_clean_scrub_stamp":"2026-02-24T07:46:01.867949+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:47:06.294580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":223215616,"num_objects":3406,"num_object_clones":0,"num_object_copies":10218,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3406,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3406,"num_write_kb":217984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.a","version":"44'3478","reported_seq":3492,"reported_epoch":44,"state":"active+clean","last_fresh":"2026-02-24T07:46:10.701099+0000","last_change":"2026-02-24T07:46:02.882761+0000","last_active":"2026-02-24T07:46:10.701099+0000","last_peered":"2026-02-24T07:46:10.701099+0000","last_clean":"2026-02-24T07:46:10.701099+0000","last_became_active":"2026-02-24T07:46:02.882274+0000","last_became_peered":"2026-02-24T07:46:02.882274+0000","last_unstale":"2026-02-24T07:46:10.701099+0000","last_undegraded":"2026-02-24T07:46:10.701099+0000","last_fullsized":"2026-02-24T07:46:10.701099+0000","mapping_epoch":42,"log_start":"44'3475","ondisk_log_start":"44'3475","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_clean_scrub_stamp":"2026-02-24T07:46:01.867949+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:47:11.014010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":227934208,"num_objects":3478,"num_object_clones":0,"num_object_copies":10434,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3478,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3478,"num_write_kb":222592,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"44'3443","reported_seq":3457,"reported_epoch":44,"state":"active+clean","last_fresh":"2026-02-24T07:46:10.702021+0000","last_change":"2026-02-24T07:46:02.882303+0000","last_active":"2026-02-24T07:46:10.702021+0000","last_peered":"2026-02-24T07:46:10.702021+0000","last_clean":"2026-02-24T07:46:10.702021+0000","last_became_active":"2026-02-24T07:46:02.881951+0000","last_became_peered":"2026-02-24T07:46:02.881951+0000","last_unstale":"2026-02-24T07:46:10.702021+0000","last_undegraded":"2026-02-24T07:46:10.702021+0000","last_fullsized":"2026-02-24T07:46:10.702021+0000","mapping_epoch":42,"log_start":"44'3440","ondisk_log_start":"44'3440","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_clean_scrub_stamp":"2026-02-24T07:46:01.867949+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:47:19.768765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":225640448,"num_objects":3443,"num_object_clones":0,"num_object_copies":10329,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3443,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3443,"num_write_kb":220352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"44'3396","reported_seq":3410,"reported_epoch":44,"state":"active+clean","last_fresh":"2026-02-24T07:46:10.732723+0000","last_change":"2026-02-24T07:46:02.881463+0000","last_active":"2026-02-24T07:46:10.732723+0000","last_peered":"2026-02-24T07:46:10.732723+0000","last_clean":"2026-02-24T07:46:10.732723+0000","last_became_active":"2026-02-24T07:46:02.881345+0000","last_became_peered":"2026-02-24T07:46:02.881345+0000","last_unstale":"2026-02-24T07:46:10.732723+0000","last_undegraded":"2026-02-24T07:46:10.732723+0000","last_fullsized":"2026-02-24T07:46:10.732723+0000","mapping_epoch":42,"log_start":"44'3393","ondisk_log_start":"44'3393","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_clean_scrub_stamp":"2026-02-24T07:46:01.867949+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:47:11.310752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":222560256,"num_objects":3396,"num_object_clones":0,"num_object_copies":10188,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3396,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3396,"num_write_kb":217344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"44'3413","reported_seq":3427,"reported_epoch":44,"state":"active+clean","last_fresh":"2026-02-24T07:46:10.745236+0000","last_change":"2026-02-24T07:46:02.880582+0000","last_active":"2026-02-24T07:46:10.745236+0000","last_peered":"2026-02-24T07:46:10.745236+0000","last_clean":"2026-02-24T07:46:10.745236+0000","last_became_active":"2026-02-24T07:46:02.880323+0000","last_became_peered":"2026-02-24T07:46:02.880323+0000","last_unstale":"2026-02-24T07:46:10.745236+0000","last_undegraded":"2026-02-24T07:46:10.745236+0000","last_fullsized":"2026-02-24T07:46:10.745236+0000","mapping_epoch":42,"log_start":"44'3410","ondisk_log_start":"44'3410","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_clean_scrub_stamp":"2026-02-24T07:46:01.867949+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:47:20.538625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":223674368,"num_objects":3413,"num_object_clones":0,"num_object_copies":10239,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3413,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3413,"num_write_kb":218432,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.6","version":"44'3522","reported_seq":3536,"reported_epoch":44,"state":"active+clean","last_fresh":"2026-02-24T07:46:10.695836+0000","last_change":"2026-02-24T07:46:02.880170+0000","last_active":"2026-02-24T07:46:10.695836+0000","last_peered":"2026-02-24T07:46:10.695836+0000","last_clean":"2026-02-24T07:46:10.695836+0000","last_became_active":"2026-02-24T07:46:02.880005+0000","last_became_peered":"2026-02-24T07:46:02.880005+0000","last_unstale":"2026-02-24T07:46:10.695836+0000","last_undegraded":"2026-02-24T07:46:10.695836+0000","last_fullsized":"2026-02-24T07:46:10.695836+0000","mapping_epoch":42,"log_start":"44'3519","ondisk_log_start":"44'3519","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_clean_scrub_stamp":"2026-02-24T07:46:01.867949+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:47:29.015277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":230817792,"num_objects":3522,"num_object_clones":0,"num_object_copies":10566,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3522,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3522,"num_write_kb":225408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"45'5877","reported_seq":5894,"reported_epoch":45,"state":"active+clean","last_fresh":"2026-02-24T07:46:15.675006+0000","last_change":"2026-02-24T07:46:02.881931+0000","last_active":"2026-02-24T07:46:15.675006+0000","last_peered":"2026-02-24T07:46:15.675006+0000","last_clean":"2026-02-24T07:46:15.675006+0000","last_became_active":"2026-02-24T07:46:02.881420+0000","last_became_peered":"2026-02-24T07:46:02.881420+0000","last_unstale":"2026-02-24T07:46:15.675006+0000","last_undegraded":"2026-02-24T07:46:15.675006+0000","last_fullsized":"2026-02-24T07:46:15.675006+0000","mapping_epoch":42,"log_start":"45'5874","ondisk_log_start":"45'5874","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_clean_scrub_stamp":"2026-02-24T07:46:01.867949+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:47:09.535393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":385155072,"num_objects":5877,"num_object_clones":0,"num_object_copies":17631,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5877,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5877,"num_write_kb":376128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.4","version":"44'3430","reported_seq":3444,"reported_epoch":44,"state":"active+clean","last_fresh":"2026-02-24T07:46:10.743208+0000","last_change":"2026-02-24T07:46:02.880661+0000","last_active":"2026-02-24T07:46:10.743208+0000","last_peered":"2026-02-24T07:46:10.743208+0000","last_clean":"2026-02-24T07:46:10.743208+0000","last_became_active":"2026-02-24T07:46:02.880345+0000","last_became_peered":"2026-02-24T07:46:02.880345+0000","last_unstale":"2026-02-24T07:46:10.743208+0000","last_undegraded":"2026-02-24T07:46:10.743208+0000","last_fullsized":"2026-02-24T07:46:10.743208+0000","mapping_epoch":42,"log_start":"44'3427","ondisk_log_start":"44'3427","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_clean_scrub_stamp":"2026-02-24T07:46:01.867949+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:47:05.136529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":224788480,"num_objects":3430,"num_object_clones":0,"num_object_copies":10290,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3430,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3430,"num_write_kb":219520,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"45'5921","reported_seq":5938,"reported_epoch":45,"state":"active+clean","last_fresh":"2026-02-24T07:46:15.678652+0000","last_change":"2026-02-24T07:46:02.881591+0000","last_active":"2026-02-24T07:46:15.678652+0000","last_peered":"2026-02-24T07:46:15.678652+0000","last_clean":"2026-02-24T07:46:15.678652+0000","last_became_active":"2026-02-24T07:46:02.881035+0000","last_became_peered":"2026-02-24T07:46:02.881035+0000","last_unstale":"2026-02-24T07:46:15.678652+0000","last_undegraded":"2026-02-24T07:46:15.678652+0000","last_fullsized":"2026-02-24T07:46:15.678652+0000","mapping_epoch":42,"log_start":"45'5918","ondisk_log_start":"45'5918","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_clean_scrub_stamp":"2026-02-24T07:46:01.867949+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:47:05.565071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":388038656,"num_objects":5921,"num_object_clones":0,"num_object_copies":17763,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5921,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5921,"num_write_kb":378944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"44'3459","reported_seq":3471,"reported_epoch":44,"state":"active+clean","last_fresh":"2026-02-24T07:46:10.700112+0000","last_change":"2026-02-24T07:46:02.879955+0000","last_active":"2026-02-24T07:46:10.700112+0000","last_peered":"2026-02-24T07:46:10.700112+0000","last_clean":"2026-02-24T07:46:10.700112+0000","last_became_active":"2026-02-24T07:46:02.879831+0000","last_became_peered":"2026-02-24T07:46:02.879831+0000","last_unstale":"2026-02-24T07:46:10.700112+0000","last_undegraded":"2026-02-24T07:46:10.700112+0000","last_fullsized":"2026-02-24T07:46:10.700112+0000","mapping_epoch":42,"log_start":"44'3455","ondisk_log_start":"44'3455","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_clean_scrub_stamp":"2026-02-24T07:46:01.867949+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":3000,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:47:15.015015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":226689024,"num_objects":3459,"num_object_clones":0,"num_object_copies":10377,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3459,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3459,"num_write_kb":221376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.0","version":"44'3429","reported_seq":3442,"reported_epoch":44,"state":"active+clean","last_fresh":"2026-02-24T07:46:10.744559+0000","last_change":"2026-02-24T07:46:02.880585+0000","last_active":"2026-02-24T07:46:10.744559+0000","last_peered":"2026-02-24T07:46:10.744559+0000","last_clean":"2026-02-24T07:46:10.744559+0000","last_became_active":"2026-02-24T07:46:02.880333+0000","last_became_peered":"2026-02-24T07:46:02.880333+0000","last_unstale":"2026-02-24T07:46:10.744559+0000","last_undegraded":"2026-02-24T07:46:10.744559+0000","last_fullsized":"2026-02-24T07:46:10.744559+0000","mapping_epoch":42,"log_start":"44'3426","ondisk_log_start":"44'3426","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_clean_scrub_stamp":"2026-02-24T07:46:01.867949+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:47:22.086512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":224722944,"num_objects":3429,"num_object_clones":0,"num_object_copies":10287,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3429,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3429,"num_write_kb":219456,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2","version":"44'3543","reported_seq":3557,"reported_epoch":44,"state":"active+clean","last_fresh":"2026-02-24T07:46:10.745981+0000","last_change":"2026-02-24T07:46:02.880566+0000","last_active":"2026-02-24T07:46:10.745981+0000","last_peered":"2026-02-24T07:46:10.745981+0000","last_clean":"2026-02-24T07:46:10.745981+0000","last_became_active":"2026-02-24T07:46:02.880185+0000","last_became_peered":"2026-02-24T07:46:02.880185+0000","last_unstale":"2026-02-24T07:46:10.745981+0000","last_undegraded":"2026-02-24T07:46:10.745981+0000","last_fullsized":"2026-02-24T07:46:10.745981+0000","mapping_epoch":42,"log_start":"44'3540","ondisk_log_start":"44'3540","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_clean_scrub_stamp":"2026-02-24T07:46:01.867949+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:47:06.727531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":232194048,"num_objects":3543,"num_object_clones":0,"num_object_copies":10629,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3543,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3543,"num_write_kb":226752,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"29'99","reported_seq":226,"reported_epoch":44,"state":"active+clean","last_fresh":"2026-02-24T07:46:03.880095+0000","last_change":"2026-02-24T07:45:26.125455+0000","last_active":"2026-02-24T07:46:03.880095+0000","last_peered":"2026-02-24T07:46:03.880095+0000","last_clean":"2026-02-24T07:46:03.880095+0000","last_became_active":"2026-02-24T07:42:54.186719+0000","last_became_peered":"2026-02-24T07:42:54.186719+0000","last_unstale":"2026-02-24T07:46:03.880095+0000","last_undegraded":"2026-02-24T07:46:03.880095+0000","last_fullsized":"2026-02-24T07:46:03.880095+0000","mapping_epoch":27,"log_start":"29'96","ondisk_log_start":"29'96","created":20,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'99","last_scrub_stamp":"2026-02-24T07:45:26.125426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:42:43.640776+0000","last_clean_scrub_stamp":"2026-02-24T07:45:26.125426+0000","objects_scrubbed":2,"log_size":3,"log_dups_size":96,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:46:34.126365+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":240,"num_write":135,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":16,"stat_sum":{"num_bytes":4255383552,"num_objects":64932,"num_object_clones":0,"num_object_copies":194796,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64932,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":64932,"num_write_kb":4155648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":13070237696,"data_stored":13070237696,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":49,"ondisk_log_size":49,"up":48,"acting":48,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":240,"num_write":135,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":2232,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":3,"up_from":25,"seq":107374182442,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":3101820,"kb_used_data":2023464,"kb_used_omap":8,"kb_used_meta":1078327,"kb_avail":730901380,"statfs":{"total":751619276800,"available":748443013120,"internally_reserved":0,"allocated":2072027136,"data_stored":2071746036,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":8416,"internal_metadata":1104207648},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[4,2],"upper_bound":4},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":19,"seq":81604378669,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":5991624,"kb_used_data":5209076,"kb_used_omap":8,"kb_used_meta":782455,"kb_avail":728011576,"statfs":{"total":751619276800,"available":745483853824,"internally_reserved":0,"allocated":5334093824,"data_stored":5333805180,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":8485,"internal_metadata":801234651},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1],"upper_bound":4},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542189,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":2867768,"kb_used_data":2128932,"kb_used_omap":8,"kb_used_meta":738807,"kb_avail":731135432,"statfs":{"total":751619276800,"available":748682682368,"internally_reserved":0,"allocated":2180026368,"data_stored":2179748820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":8687,"internal_metadata":756538897},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[4,2,1],"upper_bound":8},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705712,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":3225276,"kb_used_data":2605096,"kb_used_omap":7,"kb_used_meta":620152,"kb_avail":730777924,"statfs":{"total":751619276800,"available":748316594176,"internally_reserved":0,"allocated":2667618304,"data_stored":2667337204,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":7710,"internal_metadata":635036130},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":2232,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":2906783744,"data_stored":2906783744,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":2495414272,"data_stored":2495414272,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":5417795584,"data_stored":5417795584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":2250244096,"data_stored":2250244096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-24T07:46:17.019 INFO:tasks.cephadm.ceph_manager.ceph:increase pool size by 10 2026-02-24T07:46:17.019 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph osd pool set .mgr pg_num 121 2026-02-24T07:46:17.061 INFO:journalctl@ceph.mgr.x.trial035.stdout:Feb 24 07:46:16 trial035 ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-mgr-x[39621]: ::ffff:10.20.193.91 - - [24/Feb/2026:07:46:16] "GET /metrics HTTP/1.1" 200 4217 "" "Prometheus/3.6.0" 2026-02-24T07:46:17.117 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:46:17.114+0000 7fd2fd2a5640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:46:17.117 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:46:17.114+0000 7fd2fd2a5640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:46:17.132 INFO:teuthology.orchestra.run.trial035.stdout:osd.0: {} 2026-02-24T07:46:17.132 INFO:teuthology.orchestra.run.trial035.stderr:osd.0: osd_enable_op_tracker = '' 2026-02-24T07:46:17.149 INFO:teuthology.orchestra.run.trial035.stdout:osd.1: {} 2026-02-24T07:46:17.149 INFO:teuthology.orchestra.run.trial035.stderr:osd.1: osd_enable_op_tracker = '' 2026-02-24T07:46:17.162 INFO:teuthology.orchestra.run.trial035.stdout:osd.2: {} 2026-02-24T07:46:17.162 INFO:teuthology.orchestra.run.trial035.stderr:osd.2: osd_enable_op_tracker = '' 2026-02-24T07:46:17.177 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:46:17.178 INFO:teuthology.orchestra.run.trial035.stdout:osd.3: {} 2026-02-24T07:46:17.178 INFO:teuthology.orchestra.run.trial035.stderr:osd.3: osd_enable_op_tracker = '' 2026-02-24T07:46:17.265 INFO:journalctl@ceph.prometheus.a.trial091.stdout:Feb 24 07:46:16 trial091 ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-prometheus-a[66007]: time=2026-02-24T07:46:16.973Z level=ERROR source=scrape.go:1630 msg="Failed to determine correct type of scrape target." component="scrape manager" scrape_pool=ceph target=http://trial035:9283/metrics content_type="text/html;charset=utf-8" fallback_media_type="" err="received unsupported Content-Type \"text/html;charset=utf-8\" and no fallback_scrape_protocol specified for target" 2026-02-24T07:46:17.374 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:46:17.362+0000 7fa56c59e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:46:17.374 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:46:17.362+0000 7fa56c59e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:46:17.522 INFO:teuthology.orchestra.run.trial035.stderr:Error EPERM: crimson_allow_pg_split is false; pg_num increase denied for crimson pool 2026-02-24T07:46:17.813 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-24T07:46:17.973 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 14 16 112013 111997 499.948 424.375 0.00190925 0.00199885 2026-02-24T07:46:18.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:46:17 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/536504828' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2026-02-24T07:46:18.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:46:17 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/557830432' entity='client.admin' cmd={"prefix": "osd pool set", "pool": ".mgr", "var": "pg_num", "val": "121"} : dispatch 2026-02-24T07:46:18.973 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 15 15 120343 120328 501.328 520.688 0.002085 0.00199335 2026-02-24T07:46:19.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:46:18 trial035 ceph-mon[39284]: from='client.14626 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2026-02-24T07:46:19.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:46:18 trial035 ceph-mon[39284]: pgmap v118: 17 pgs: 17 active+clean; 5.8 GiB data, 20 GiB used, 2.7 TiB / 2.7 TiB avail; 502 MiB/s wr, 8.03k op/s 2026-02-24T07:46:19.225 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2026-02-24T07:46:19.374 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:46:19.505 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:46:19.502+0000 7f0011782640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:46:19.505 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:46:19.502+0000 7f0011782640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:46:19.522 INFO:teuthology.orchestra.run.trial035.stdout:osd.0: {} 2026-02-24T07:46:19.522 INFO:teuthology.orchestra.run.trial035.stderr:osd.0: osd_enable_op_tracker = '' 2026-02-24T07:46:19.537 INFO:teuthology.orchestra.run.trial035.stdout:osd.1: {} 2026-02-24T07:46:19.538 INFO:teuthology.orchestra.run.trial035.stderr:osd.1: osd_enable_op_tracker = '' 2026-02-24T07:46:19.551 INFO:teuthology.orchestra.run.trial035.stdout:osd.2: {} 2026-02-24T07:46:19.551 INFO:teuthology.orchestra.run.trial035.stderr:osd.2: osd_enable_op_tracker = '' 2026-02-24T07:46:19.565 INFO:teuthology.orchestra.run.trial035.stdout:osd.3: {} 2026-02-24T07:46:19.565 INFO:teuthology.orchestra.run.trial035.stderr:osd.3: osd_enable_op_tracker = '' 2026-02-24T07:46:19.973 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 16 16 128371 128355 501.348 501.688 0.00227541 0.00199325 2026-02-24T07:46:20.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:46:19 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/2381448336' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2026-02-24T07:46:20.973 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 17 16 136657 136641 502.318 517.875 0.00142293 0.00198949 2026-02-24T07:46:21.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:46:20 trial035 ceph-mon[39284]: pgmap v119: 17 pgs: 17 active+clean; 5.8 GiB data, 20 GiB used, 2.7 TiB / 2.7 TiB avail; 502 MiB/s wr, 8.03k op/s 2026-02-24T07:46:21.609 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2026-02-24T07:46:21.758 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:46:21.885 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:46:21.883+0000 7f550be5c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:46:21.885 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:46:21.883+0000 7f550be5c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:46:21.901 INFO:teuthology.orchestra.run.trial035.stdout:osd.0: {} 2026-02-24T07:46:21.901 INFO:teuthology.orchestra.run.trial035.stderr:osd.0: osd_enable_op_tracker = '' 2026-02-24T07:46:21.914 INFO:teuthology.orchestra.run.trial035.stdout:osd.1: {} 2026-02-24T07:46:21.914 INFO:teuthology.orchestra.run.trial035.stderr:osd.1: osd_enable_op_tracker = '' 2026-02-24T07:46:21.927 INFO:teuthology.orchestra.run.trial035.stdout:osd.2: {} 2026-02-24T07:46:21.927 INFO:teuthology.orchestra.run.trial035.stderr:osd.2: osd_enable_op_tracker = '' 2026-02-24T07:46:21.943 INFO:teuthology.orchestra.run.trial035.stdout:osd.3: {} 2026-02-24T07:46:21.943 INFO:teuthology.orchestra.run.trial035.stderr:osd.3: osd_enable_op_tracker = '' 2026-02-24T07:46:21.973 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 18 16 144863 144847 502.902 512.875 0.00122152 0.00198715 2026-02-24T07:46:22.815 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2026-02-24T07:46:22.815 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2026-02-24T07:46:22.816 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph pg dump --format=json 2026-02-24T07:46:22.964 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:46:22.973 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 19 16 153076 153060 503.448 513.312 0.00127156 0.00198502 2026-02-24T07:46:23.138 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:46:22 trial035 ceph-mon[39284]: pgmap v120: 17 pgs: 17 active+clean; 8.2 GiB data, 28 GiB used, 2.7 TiB / 2.7 TiB avail; 486 MiB/s wr, 7.78k op/s 2026-02-24T07:46:23.139 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:46:22 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/3049444951' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2026-02-24T07:46:23.427 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:46:23.411+0000 7f4023979640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:46:23.428 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:46:23.411+0000 7f4023979640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:46:23.552 INFO:teuthology.orchestra.run.trial035.stdout: 2026-02-24T07:46:23.552 INFO:teuthology.orchestra.run.trial035.stderr:dumped all 2026-02-24T07:46:23.974 INFO:tasks.radosbench.radosbench.0.trial035.stdout:2026-02-24T07:46:23.973316+0000 min lat: 0.000742459 max lat: 0.0383889 avg lat: 0.0020124 2026-02-24T07:46:23.974 INFO:tasks.radosbench.radosbench.0.trial035.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2026-02-24T07:46:23.974 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 20 15 158937 158922 496.593 366.375 0.00174492 0.0020124 2026-02-24T07:46:23.986 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2026-02-24T07:46:24.160 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:46:24.177 INFO:teuthology.orchestra.run.trial035.stdout:{"pg_ready":true,"pg_map":{"version":120,"stamp":"2026-02-24T07:46:21.684647+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":8807973408,"num_objects":134392,"num_object_clones":0,"num_object_copies":403176,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":134392,"num_whiteouts":0,"num_read":136,"num_read_kb":240,"num_write":134525,"num_write_kb":8603110,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":53,"ondisk_log_size":53,"up":51,"acting":51,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":51,"num_osds":4,"num_per_pool_osds":4,"num_per_pool_omap_osds":4,"kb":2936012800,"kb_used":28937180,"kb_used_data":24959308,"kb_used_omap":34,"kb_used_meta":3977757,"kb_avail":2907075620,"statfs":{"total":3006477107200,"available":2976845434880,"internally_reserved":0,"allocated":25558331392,"data_stored":25557180368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":35261,"internal_metadata":4073223747},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":5205393408,"num_objects":79428,"num_object_clones":0,"num_object_copies":238284,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":79428,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":79428,"num_write_kb":5083392,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.205272"},"pg_stats":[{"pgid":"3.f","version":"45'8438","reported_seq":8455,"reported_epoch":45,"state":"active+clean","last_fresh":"2026-02-24T07:46:20.674372+0000","last_change":"2026-02-24T07:46:02.882095+0000","last_active":"2026-02-24T07:46:20.674372+0000","last_peered":"2026-02-24T07:46:20.674372+0000","last_clean":"2026-02-24T07:46:20.674372+0000","last_became_active":"2026-02-24T07:46:02.881625+0000","last_became_peered":"2026-02-24T07:46:02.881625+0000","last_unstale":"2026-02-24T07:46:20.674372+0000","last_undegraded":"2026-02-24T07:46:20.674372+0000","last_fullsized":"2026-02-24T07:46:20.674372+0000","mapping_epoch":42,"log_start":"45'8435","ondisk_log_start":"45'8435","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_clean_scrub_stamp":"2026-02-24T07:46:01.867949+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:47:09.267374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":552992768,"num_objects":8438,"num_object_clones":0,"num_object_copies":25314,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8438,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8438,"num_write_kb":540032,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.e","version":"45'8588","reported_seq":8604,"reported_epoch":45,"state":"active+clean","last_fresh":"2026-02-24T07:46:20.680265+0000","last_change":"2026-02-24T07:46:02.881462+0000","last_active":"2026-02-24T07:46:20.680265+0000","last_peered":"2026-02-24T07:46:20.680265+0000","last_clean":"2026-02-24T07:46:20.680265+0000","last_became_active":"2026-02-24T07:46:02.881083+0000","last_became_peered":"2026-02-24T07:46:02.881083+0000","last_unstale":"2026-02-24T07:46:20.680265+0000","last_undegraded":"2026-02-24T07:46:20.680265+0000","last_fullsized":"2026-02-24T07:46:20.680265+0000","mapping_epoch":42,"log_start":"45'8585","ondisk_log_start":"45'8585","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_clean_scrub_stamp":"2026-02-24T07:46:01.867949+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:47:29.030602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":562823168,"num_objects":8588,"num_object_clones":0,"num_object_copies":25764,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8588,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8588,"num_write_kb":549632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"45'8353","reported_seq":8370,"reported_epoch":45,"state":"active+clean","last_fresh":"2026-02-24T07:46:20.703037+0000","last_change":"2026-02-24T07:46:02.883952+0000","last_active":"2026-02-24T07:46:20.703037+0000","last_peered":"2026-02-24T07:46:20.703037+0000","last_clean":"2026-02-24T07:46:20.703037+0000","last_became_active":"2026-02-24T07:46:02.883799+0000","last_became_peered":"2026-02-24T07:46:02.883799+0000","last_unstale":"2026-02-24T07:46:20.703037+0000","last_undegraded":"2026-02-24T07:46:20.703037+0000","last_fullsized":"2026-02-24T07:46:20.703037+0000","mapping_epoch":42,"log_start":"45'8350","ondisk_log_start":"45'8350","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_clean_scrub_stamp":"2026-02-24T07:46:01.867949+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:47:02.402624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":547422208,"num_objects":8353,"num_object_clones":0,"num_object_copies":25059,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8353,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8353,"num_write_kb":534592,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"45'8189","reported_seq":8206,"reported_epoch":45,"state":"active+clean","last_fresh":"2026-02-24T07:46:20.706645+0000","last_change":"2026-02-24T07:46:02.880375+0000","last_active":"2026-02-24T07:46:20.706645+0000","last_peered":"2026-02-24T07:46:20.706645+0000","last_clean":"2026-02-24T07:46:20.706645+0000","last_became_active":"2026-02-24T07:46:02.880060+0000","last_became_peered":"2026-02-24T07:46:02.880060+0000","last_unstale":"2026-02-24T07:46:20.706645+0000","last_undegraded":"2026-02-24T07:46:20.706645+0000","last_fullsized":"2026-02-24T07:46:20.706645+0000","mapping_epoch":42,"log_start":"45'8186","ondisk_log_start":"45'8186","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_clean_scrub_stamp":"2026-02-24T07:46:01.867949+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:47:04.123135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":536674304,"num_objects":8189,"num_object_clones":0,"num_object_copies":24567,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8189,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8189,"num_write_kb":524096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"45'8404","reported_seq":8421,"reported_epoch":45,"state":"active+clean","last_fresh":"2026-02-24T07:46:20.743948+0000","last_change":"2026-02-24T07:46:02.880509+0000","last_active":"2026-02-24T07:46:20.743948+0000","last_peered":"2026-02-24T07:46:20.743948+0000","last_clean":"2026-02-24T07:46:20.743948+0000","last_became_active":"2026-02-24T07:46:02.880171+0000","last_became_peered":"2026-02-24T07:46:02.880171+0000","last_unstale":"2026-02-24T07:46:20.743948+0000","last_undegraded":"2026-02-24T07:46:20.743948+0000","last_fullsized":"2026-02-24T07:46:20.743948+0000","mapping_epoch":42,"log_start":"45'8401","ondisk_log_start":"45'8401","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_clean_scrub_stamp":"2026-02-24T07:46:01.867949+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:47:06.105346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":550764544,"num_objects":8404,"num_object_clones":0,"num_object_copies":25212,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8404,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8404,"num_write_kb":537856,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.a","version":"45'8475","reported_seq":8492,"reported_epoch":45,"state":"active+clean","last_fresh":"2026-02-24T07:46:20.700528+0000","last_change":"2026-02-24T07:46:02.882761+0000","last_active":"2026-02-24T07:46:20.700528+0000","last_peered":"2026-02-24T07:46:20.700528+0000","last_clean":"2026-02-24T07:46:20.700528+0000","last_became_active":"2026-02-24T07:46:02.882274+0000","last_became_peered":"2026-02-24T07:46:02.882274+0000","last_unstale":"2026-02-24T07:46:20.700528+0000","last_undegraded":"2026-02-24T07:46:20.700528+0000","last_fullsized":"2026-02-24T07:46:20.700528+0000","mapping_epoch":42,"log_start":"45'8472","ondisk_log_start":"45'8472","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_clean_scrub_stamp":"2026-02-24T07:46:01.867949+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:47:31.776406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":555417600,"num_objects":8475,"num_object_clones":0,"num_object_copies":25425,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8475,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8475,"num_write_kb":542400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"45'8447","reported_seq":8464,"reported_epoch":45,"state":"active+clean","last_fresh":"2026-02-24T07:46:20.705002+0000","last_change":"2026-02-24T07:46:02.882303+0000","last_active":"2026-02-24T07:46:20.705002+0000","last_peered":"2026-02-24T07:46:20.705002+0000","last_clean":"2026-02-24T07:46:20.705002+0000","last_became_active":"2026-02-24T07:46:02.881951+0000","last_became_peered":"2026-02-24T07:46:02.881951+0000","last_unstale":"2026-02-24T07:46:20.705002+0000","last_undegraded":"2026-02-24T07:46:20.705002+0000","last_fullsized":"2026-02-24T07:46:20.705002+0000","mapping_epoch":42,"log_start":"45'8444","ondisk_log_start":"45'8444","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_clean_scrub_stamp":"2026-02-24T07:46:01.867949+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:47:04.345724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":553582592,"num_objects":8447,"num_object_clones":0,"num_object_copies":25341,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8447,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8447,"num_write_kb":540608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"45'8342","reported_seq":8359,"reported_epoch":45,"state":"active+clean","last_fresh":"2026-02-24T07:46:20.743870+0000","last_change":"2026-02-24T07:46:02.881463+0000","last_active":"2026-02-24T07:46:20.743870+0000","last_peered":"2026-02-24T07:46:20.743870+0000","last_clean":"2026-02-24T07:46:20.743870+0000","last_became_active":"2026-02-24T07:46:02.881345+0000","last_became_peered":"2026-02-24T07:46:02.881345+0000","last_unstale":"2026-02-24T07:46:20.743870+0000","last_undegraded":"2026-02-24T07:46:20.743870+0000","last_fullsized":"2026-02-24T07:46:20.743870+0000","mapping_epoch":42,"log_start":"45'8339","ondisk_log_start":"45'8339","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_clean_scrub_stamp":"2026-02-24T07:46:01.867949+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:47:17.135287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":546701312,"num_objects":8342,"num_object_clones":0,"num_object_copies":25026,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8342,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8342,"num_write_kb":533888,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"45'8331","reported_seq":8348,"reported_epoch":45,"state":"active+clean","last_fresh":"2026-02-24T07:46:20.745271+0000","last_change":"2026-02-24T07:46:02.880582+0000","last_active":"2026-02-24T07:46:20.745271+0000","last_peered":"2026-02-24T07:46:20.745271+0000","last_clean":"2026-02-24T07:46:20.745271+0000","last_became_active":"2026-02-24T07:46:02.880323+0000","last_became_peered":"2026-02-24T07:46:02.880323+0000","last_unstale":"2026-02-24T07:46:20.745271+0000","last_undegraded":"2026-02-24T07:46:20.745271+0000","last_fullsized":"2026-02-24T07:46:20.745271+0000","mapping_epoch":42,"log_start":"45'8328","ondisk_log_start":"45'8328","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_clean_scrub_stamp":"2026-02-24T07:46:01.867949+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:47:31.420697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":545980416,"num_objects":8331,"num_object_clones":0,"num_object_copies":24993,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8331,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8331,"num_write_kb":533184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.6","version":"45'8468","reported_seq":8483,"reported_epoch":45,"state":"active+clean","last_fresh":"2026-02-24T07:46:20.701234+0000","last_change":"2026-02-24T07:46:02.880170+0000","last_active":"2026-02-24T07:46:20.701234+0000","last_peered":"2026-02-24T07:46:20.701234+0000","last_clean":"2026-02-24T07:46:20.701234+0000","last_became_active":"2026-02-24T07:46:02.880005+0000","last_became_peered":"2026-02-24T07:46:02.880005+0000","last_unstale":"2026-02-24T07:46:20.701234+0000","last_undegraded":"2026-02-24T07:46:20.701234+0000","last_fullsized":"2026-02-24T07:46:20.701234+0000","mapping_epoch":42,"log_start":"45'8464","ondisk_log_start":"45'8464","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_clean_scrub_stamp":"2026-02-24T07:46:01.867949+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":3000,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:47:14.923924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554958848,"num_objects":8468,"num_object_clones":0,"num_object_copies":25404,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8468,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8468,"num_write_kb":541952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"45'8446","reported_seq":8463,"reported_epoch":45,"state":"active+clean","last_fresh":"2026-02-24T07:46:20.663217+0000","last_change":"2026-02-24T07:46:02.881931+0000","last_active":"2026-02-24T07:46:20.663217+0000","last_peered":"2026-02-24T07:46:20.663217+0000","last_clean":"2026-02-24T07:46:20.663217+0000","last_became_active":"2026-02-24T07:46:02.881420+0000","last_became_peered":"2026-02-24T07:46:02.881420+0000","last_unstale":"2026-02-24T07:46:20.663217+0000","last_undegraded":"2026-02-24T07:46:20.663217+0000","last_fullsized":"2026-02-24T07:46:20.663217+0000","mapping_epoch":42,"log_start":"45'8443","ondisk_log_start":"45'8443","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_clean_scrub_stamp":"2026-02-24T07:46:01.867949+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:47:09.535393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":553517056,"num_objects":8446,"num_object_clones":0,"num_object_copies":25338,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8446,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8446,"num_write_kb":540544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.4","version":"45'8219","reported_seq":8236,"reported_epoch":45,"state":"active+clean","last_fresh":"2026-02-24T07:46:20.746170+0000","last_change":"2026-02-24T07:46:02.880661+0000","last_active":"2026-02-24T07:46:20.746170+0000","last_peered":"2026-02-24T07:46:20.746170+0000","last_clean":"2026-02-24T07:46:20.746170+0000","last_became_active":"2026-02-24T07:46:02.880345+0000","last_became_peered":"2026-02-24T07:46:02.880345+0000","last_unstale":"2026-02-24T07:46:20.746170+0000","last_undegraded":"2026-02-24T07:46:20.746170+0000","last_fullsized":"2026-02-24T07:46:20.746170+0000","mapping_epoch":42,"log_start":"45'8215","ondisk_log_start":"45'8215","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_clean_scrub_stamp":"2026-02-24T07:46:01.867949+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":3000,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:47:27.577904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":538640384,"num_objects":8219,"num_object_clones":0,"num_object_copies":24657,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8219,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8219,"num_write_kb":526016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"45'8366","reported_seq":8383,"reported_epoch":45,"state":"active+clean","last_fresh":"2026-02-24T07:46:20.674730+0000","last_change":"2026-02-24T07:46:02.881591+0000","last_active":"2026-02-24T07:46:20.674730+0000","last_peered":"2026-02-24T07:46:20.674730+0000","last_clean":"2026-02-24T07:46:20.674730+0000","last_became_active":"2026-02-24T07:46:02.881035+0000","last_became_peered":"2026-02-24T07:46:02.881035+0000","last_unstale":"2026-02-24T07:46:20.674730+0000","last_undegraded":"2026-02-24T07:46:20.674730+0000","last_fullsized":"2026-02-24T07:46:20.674730+0000","mapping_epoch":42,"log_start":"45'8363","ondisk_log_start":"45'8363","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_clean_scrub_stamp":"2026-02-24T07:46:01.867949+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:47:05.565071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":548274176,"num_objects":8366,"num_object_clones":0,"num_object_copies":25098,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8366,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8366,"num_write_kb":535424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"45'8554","reported_seq":8571,"reported_epoch":45,"state":"active+clean","last_fresh":"2026-02-24T07:46:20.699408+0000","last_change":"2026-02-24T07:46:02.879955+0000","last_active":"2026-02-24T07:46:20.699408+0000","last_peered":"2026-02-24T07:46:20.699408+0000","last_clean":"2026-02-24T07:46:20.699408+0000","last_became_active":"2026-02-24T07:46:02.879831+0000","last_became_peered":"2026-02-24T07:46:02.879831+0000","last_unstale":"2026-02-24T07:46:20.699408+0000","last_undegraded":"2026-02-24T07:46:20.699408+0000","last_fullsized":"2026-02-24T07:46:20.699408+0000","mapping_epoch":42,"log_start":"45'8551","ondisk_log_start":"45'8551","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_clean_scrub_stamp":"2026-02-24T07:46:01.867949+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:47:19.157247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":560594944,"num_objects":8554,"num_object_clones":0,"num_object_copies":25662,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8554,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8554,"num_write_kb":547456,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.0","version":"45'8326","reported_seq":8343,"reported_epoch":45,"state":"active+clean","last_fresh":"2026-02-24T07:46:20.746228+0000","last_change":"2026-02-24T07:46:02.880585+0000","last_active":"2026-02-24T07:46:20.746228+0000","last_peered":"2026-02-24T07:46:20.746228+0000","last_clean":"2026-02-24T07:46:20.746228+0000","last_became_active":"2026-02-24T07:46:02.880333+0000","last_became_peered":"2026-02-24T07:46:02.880333+0000","last_unstale":"2026-02-24T07:46:20.746228+0000","last_undegraded":"2026-02-24T07:46:20.746228+0000","last_fullsized":"2026-02-24T07:46:20.746228+0000","mapping_epoch":42,"log_start":"45'8323","ondisk_log_start":"45'8323","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_clean_scrub_stamp":"2026-02-24T07:46:01.867949+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:47:04.574448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":545652736,"num_objects":8326,"num_object_clones":0,"num_object_copies":24978,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8326,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8326,"num_write_kb":532864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2","version":"45'8444","reported_seq":8461,"reported_epoch":45,"state":"active+clean","last_fresh":"2026-02-24T07:46:20.737271+0000","last_change":"2026-02-24T07:46:02.880566+0000","last_active":"2026-02-24T07:46:20.737271+0000","last_peered":"2026-02-24T07:46:20.737271+0000","last_clean":"2026-02-24T07:46:20.737271+0000","last_became_active":"2026-02-24T07:46:02.880185+0000","last_became_peered":"2026-02-24T07:46:02.880185+0000","last_unstale":"2026-02-24T07:46:20.737271+0000","last_undegraded":"2026-02-24T07:46:20.737271+0000","last_fullsized":"2026-02-24T07:46:20.737271+0000","mapping_epoch":42,"log_start":"45'8441","ondisk_log_start":"45'8441","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_clean_scrub_stamp":"2026-02-24T07:46:01.867949+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:47:23.244686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":553385984,"num_objects":8444,"num_object_clones":0,"num_object_copies":25332,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8444,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8444,"num_write_kb":540416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"29'99","reported_seq":228,"reported_epoch":45,"state":"active+clean","last_fresh":"2026-02-24T07:46:11.479829+0000","last_change":"2026-02-24T07:45:26.125455+0000","last_active":"2026-02-24T07:46:11.479829+0000","last_peered":"2026-02-24T07:46:11.479829+0000","last_clean":"2026-02-24T07:46:11.479829+0000","last_became_active":"2026-02-24T07:42:54.186719+0000","last_became_peered":"2026-02-24T07:42:54.186719+0000","last_unstale":"2026-02-24T07:46:11.479829+0000","last_undegraded":"2026-02-24T07:46:11.479829+0000","last_fullsized":"2026-02-24T07:46:11.479829+0000","mapping_epoch":27,"log_start":"29'96","ondisk_log_start":"29'96","created":20,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'99","last_scrub_stamp":"2026-02-24T07:45:26.125426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:42:43.640776+0000","last_clean_scrub_stamp":"2026-02-24T07:45:26.125426+0000","objects_scrubbed":2,"log_size":3,"log_dups_size":96,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:46:34.126365+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":240,"num_write":135,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":16,"stat_sum":{"num_bytes":8807383040,"num_objects":134390,"num_object_clones":0,"num_object_copies":403170,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":134390,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":134390,"num_write_kb":8600960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":26411335680,"data_stored":26411335680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":50,"ondisk_log_size":50,"up":48,"acting":48,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":240,"num_write":135,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":2288,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":3,"up_from":25,"seq":107374182444,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":6430856,"kb_used_data":5190068,"kb_used_omap":8,"kb_used_meta":1240759,"kb_avail":727572344,"statfs":{"total":751619276800,"available":745034080256,"internally_reserved":0,"allocated":5314629632,"data_stored":5314340988,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9064,"internal_metadata":1270537368},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":19,"seq":81604378670,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":8350536,"kb_used_data":7441716,"kb_used_omap":8,"kb_used_meta":908791,"kb_avail":725652664,"statfs":{"total":751619276800,"available":743068327936,"internally_reserved":0,"allocated":7620317184,"data_stored":7620028540,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":8485,"internal_metadata":930602715},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542191,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":6561988,"kb_used_data":5606576,"kb_used_omap":9,"kb_used_meta":955382,"kb_avail":727441212,"statfs":{"total":751619276800,"available":744899801088,"internally_reserved":0,"allocated":5741133824,"data_stored":5740848732,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9350,"internal_metadata":978312058},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705714,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":7593800,"kb_used_data":6720948,"kb_used_omap":8,"kb_used_meta":872823,"kb_avail":726409400,"statfs":{"total":751619276800,"available":743843225600,"internally_reserved":0,"allocated":6882250752,"data_stored":6881962108,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":8362,"internal_metadata":893771606},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":2288,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":7129792512,"data_stored":7129792512,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":6060638208,"data_stored":6060638208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":7709589504,"data_stored":7709589504,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":5511315456,"data_stored":5511315456,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-24T07:46:24.696 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:46:24.679+0000 7f63b2865640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:46:24.696 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:46:24.679+0000 7f63b2865640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:46:24.696 INFO:teuthology.orchestra.run.trial035.stderr:osd.0: osd_enable_op_tracker = '' 2026-02-24T07:46:24.697 INFO:teuthology.orchestra.run.trial035.stdout:osd.0: {} 2026-02-24T07:46:24.715 INFO:teuthology.orchestra.run.trial035.stdout:osd.1: {} 2026-02-24T07:46:24.716 INFO:teuthology.orchestra.run.trial035.stderr:osd.1: osd_enable_op_tracker = '' 2026-02-24T07:46:24.728 INFO:teuthology.orchestra.run.trial035.stdout:osd.2: {} 2026-02-24T07:46:24.728 INFO:teuthology.orchestra.run.trial035.stderr:osd.2: osd_enable_op_tracker = '' 2026-02-24T07:46:24.742 INFO:teuthology.orchestra.run.trial035.stdout:osd.3: {} 2026-02-24T07:46:24.742 INFO:teuthology.orchestra.run.trial035.stderr:osd.3: osd_enable_op_tracker = '' 2026-02-24T07:46:24.954 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:46:24 trial035 ceph-mon[39284]: from='client.14636 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2026-02-24T07:46:24.954 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:46:24 trial035 ceph-mon[39284]: pgmap v121: 17 pgs: 17 active+clean; 8.2 GiB data, 28 GiB used, 2.7 TiB / 2.7 TiB avail; 414 MiB/s wr, 6.62k op/s 2026-02-24T07:46:24.955 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:46:24 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/1350623044' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2026-02-24T07:46:24.974 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 21 16 165258 165242 491.745 395 0.0026026 0.00203222 2026-02-24T07:46:25.974 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 22 16 172250 172234 489.255 437 0.00181114 0.00204258 2026-02-24T07:46:26.974 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 23 16 179795 179779 488.485 471.562 0.00165461 0.00204585 2026-02-24T07:46:27.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:46:26 trial035 ceph-mon[39284]: pgmap v122: 17 pgs: 17 active+clean; 8.7 GiB data, 30 GiB used, 2.7 TiB / 2.7 TiB avail; 460 MiB/s wr, 7.36k op/s 2026-02-24T07:46:27.205 INFO:journalctl@ceph.mgr.x.trial035.stdout:Feb 24 07:46:26 trial035 ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-mgr-x[39621]: ::ffff:10.20.193.91 - - [24/Feb/2026:07:46:26] "GET /metrics HTTP/1.1" 200 4217 "" "Prometheus/3.6.0" 2026-02-24T07:46:27.357 INFO:journalctl@ceph.prometheus.a.trial091.stdout:Feb 24 07:46:26 trial091 ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-prometheus-a[66007]: time=2026-02-24T07:46:26.973Z level=ERROR source=scrape.go:1630 msg="Failed to determine correct type of scrape target." component="scrape manager" scrape_pool=ceph target=http://trial035:9283/metrics content_type="text/html;charset=utf-8" fallback_media_type="" err="received unsupported Content-Type \"text/html;charset=utf-8\" and no fallback_scrape_protocol specified for target" 2026-02-24T07:46:27.426 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2026-02-24T07:46:27.572 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:46:27.705 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:46:27.702+0000 7f0a25970640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:46:27.705 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:46:27.702+0000 7f0a25970640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:46:27.720 INFO:teuthology.orchestra.run.trial035.stdout:osd.0: {} 2026-02-24T07:46:27.720 INFO:teuthology.orchestra.run.trial035.stderr:osd.0: osd_enable_op_tracker = '' 2026-02-24T07:46:27.735 INFO:teuthology.orchestra.run.trial035.stdout:osd.1: {} 2026-02-24T07:46:27.736 INFO:teuthology.orchestra.run.trial035.stderr:osd.1: osd_enable_op_tracker = '' 2026-02-24T07:46:27.750 INFO:teuthology.orchestra.run.trial035.stdout:osd.2: {} 2026-02-24T07:46:27.751 INFO:teuthology.orchestra.run.trial035.stderr:osd.2: osd_enable_op_tracker = '' 2026-02-24T07:46:27.765 INFO:teuthology.orchestra.run.trial035.stdout:osd.3: {} 2026-02-24T07:46:27.765 INFO:teuthology.orchestra.run.trial035.stderr:osd.3: osd_enable_op_tracker = '' 2026-02-24T07:46:27.974 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 24 16 187840 187824 489.08 502.812 0.00137731 0.0020434 2026-02-24T07:46:28.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:46:27 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/15207832' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2026-02-24T07:46:28.974 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 25 16 196136 196120 490.255 518.5 0.00246681 0.00203842 2026-02-24T07:46:29.181 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2026-02-24T07:46:29.181 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2026-02-24T07:46:29.181 INFO:tasks.thrashosds.thrasher:fixing pg num pool .mgr 2026-02-24T07:46:29.182 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph pg dump --format=json 2026-02-24T07:46:29.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:46:28 trial035 ceph-mon[39284]: pgmap v123: 17 pgs: 17 active+clean; 10 GiB data, 35 GiB used, 2.7 TiB / 2.7 TiB avail; 548 MiB/s wr, 8.76k op/s 2026-02-24T07:46:29.340 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:46:29.466 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:46:29.463+0000 7ff0bd6e4640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:46:29.466 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:46:29.463+0000 7ff0bd6e4640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:46:29.625 INFO:teuthology.orchestra.run.trial035.stdout: 2026-02-24T07:46:29.626 INFO:teuthology.orchestra.run.trial035.stderr:dumped all 2026-02-24T07:46:29.671 INFO:teuthology.orchestra.run.trial035.stdout:{"pg_ready":true,"pg_map":{"version":123,"stamp":"2026-02-24T07:46:27.685493+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":11148591648,"num_objects":170107,"num_object_clones":0,"num_object_copies":510321,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":170107,"num_whiteouts":0,"num_read":136,"num_read_kb":240,"num_write":170240,"num_write_kb":10888870,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":51,"ondisk_log_size":51,"up":51,"acting":51,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":51,"num_osds":4,"num_per_pool_osds":4,"num_per_pool_omap_osds":4,"kb":2936012800,"kb_used":36355868,"kb_used_data":31908748,"kb_used_omap":34,"kb_used_meta":4447005,"kb_avail":2899656932,"statfs":{"total":3006477107200,"available":2969248698368,"internally_reserved":0,"allocated":32674557952,"data_stored":32673406928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":35261,"internal_metadata":4553733699},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[7,3],"upper_bound":4},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":6892617728,"num_objects":105173,"num_object_clones":0,"num_object_copies":315519,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":105173,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":105173,"num_write_kb":6731072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001679"},"pg_stats":[{"pgid":"3.f","version":"45'10618","reported_seq":10635,"reported_epoch":45,"state":"active+clean","last_fresh":"2026-02-24T07:46:25.675023+0000","last_change":"2026-02-24T07:46:02.882095+0000","last_active":"2026-02-24T07:46:25.675023+0000","last_peered":"2026-02-24T07:46:25.675023+0000","last_clean":"2026-02-24T07:46:25.675023+0000","last_became_active":"2026-02-24T07:46:02.881625+0000","last_became_peered":"2026-02-24T07:46:02.881625+0000","last_unstale":"2026-02-24T07:46:25.675023+0000","last_undegraded":"2026-02-24T07:46:25.675023+0000","last_fullsized":"2026-02-24T07:46:25.675023+0000","mapping_epoch":42,"log_start":"45'10615","ondisk_log_start":"45'10615","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_clean_scrub_stamp":"2026-02-24T07:46:01.867949+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:47:09.267374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":695861248,"num_objects":10618,"num_object_clones":0,"num_object_copies":31854,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10618,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10618,"num_write_kb":679552,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.e","version":"45'10782","reported_seq":10799,"reported_epoch":45,"state":"active+clean","last_fresh":"2026-02-24T07:46:25.675708+0000","last_change":"2026-02-24T07:46:02.881462+0000","last_active":"2026-02-24T07:46:25.675708+0000","last_peered":"2026-02-24T07:46:25.675708+0000","last_clean":"2026-02-24T07:46:25.675708+0000","last_became_active":"2026-02-24T07:46:02.881083+0000","last_became_peered":"2026-02-24T07:46:02.881083+0000","last_unstale":"2026-02-24T07:46:25.675708+0000","last_undegraded":"2026-02-24T07:46:25.675708+0000","last_fullsized":"2026-02-24T07:46:25.675708+0000","mapping_epoch":42,"log_start":"45'10779","ondisk_log_start":"45'10779","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_clean_scrub_stamp":"2026-02-24T07:46:01.867949+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:47:29.030602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":706609152,"num_objects":10782,"num_object_clones":0,"num_object_copies":32346,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10782,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10782,"num_write_kb":690048,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"45'10658","reported_seq":10675,"reported_epoch":45,"state":"active+clean","last_fresh":"2026-02-24T07:46:25.704782+0000","last_change":"2026-02-24T07:46:02.883952+0000","last_active":"2026-02-24T07:46:25.704782+0000","last_peered":"2026-02-24T07:46:25.704782+0000","last_clean":"2026-02-24T07:46:25.704782+0000","last_became_active":"2026-02-24T07:46:02.883799+0000","last_became_peered":"2026-02-24T07:46:02.883799+0000","last_unstale":"2026-02-24T07:46:25.704782+0000","last_undegraded":"2026-02-24T07:46:25.704782+0000","last_fullsized":"2026-02-24T07:46:25.704782+0000","mapping_epoch":42,"log_start":"45'10655","ondisk_log_start":"45'10655","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_clean_scrub_stamp":"2026-02-24T07:46:01.867949+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:47:02.402624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":698482688,"num_objects":10658,"num_object_clones":0,"num_object_copies":31974,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10658,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10658,"num_write_kb":682112,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"45'10404","reported_seq":10421,"reported_epoch":45,"state":"active+clean","last_fresh":"2026-02-24T07:46:25.699729+0000","last_change":"2026-02-24T07:46:02.880375+0000","last_active":"2026-02-24T07:46:25.699729+0000","last_peered":"2026-02-24T07:46:25.699729+0000","last_clean":"2026-02-24T07:46:25.699729+0000","last_became_active":"2026-02-24T07:46:02.880060+0000","last_became_peered":"2026-02-24T07:46:02.880060+0000","last_unstale":"2026-02-24T07:46:25.699729+0000","last_undegraded":"2026-02-24T07:46:25.699729+0000","last_fullsized":"2026-02-24T07:46:25.699729+0000","mapping_epoch":42,"log_start":"45'10401","ondisk_log_start":"45'10401","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_clean_scrub_stamp":"2026-02-24T07:46:01.867949+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:47:04.123135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":681836544,"num_objects":10404,"num_object_clones":0,"num_object_copies":31212,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10404,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10404,"num_write_kb":665856,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"45'10641","reported_seq":10658,"reported_epoch":45,"state":"active+clean","last_fresh":"2026-02-24T07:46:25.745621+0000","last_change":"2026-02-24T07:46:02.880509+0000","last_active":"2026-02-24T07:46:25.745621+0000","last_peered":"2026-02-24T07:46:25.745621+0000","last_clean":"2026-02-24T07:46:25.745621+0000","last_became_active":"2026-02-24T07:46:02.880171+0000","last_became_peered":"2026-02-24T07:46:02.880171+0000","last_unstale":"2026-02-24T07:46:25.745621+0000","last_undegraded":"2026-02-24T07:46:25.745621+0000","last_fullsized":"2026-02-24T07:46:25.745621+0000","mapping_epoch":42,"log_start":"45'10638","ondisk_log_start":"45'10638","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_clean_scrub_stamp":"2026-02-24T07:46:01.867949+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:47:06.105346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":697368576,"num_objects":10641,"num_object_clones":0,"num_object_copies":31923,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10641,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10641,"num_write_kb":681024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.a","version":"45'10671","reported_seq":10688,"reported_epoch":45,"state":"active+clean","last_fresh":"2026-02-24T07:46:25.701605+0000","last_change":"2026-02-24T07:46:02.882761+0000","last_active":"2026-02-24T07:46:25.701605+0000","last_peered":"2026-02-24T07:46:25.701605+0000","last_clean":"2026-02-24T07:46:25.701605+0000","last_became_active":"2026-02-24T07:46:02.882274+0000","last_became_peered":"2026-02-24T07:46:02.882274+0000","last_unstale":"2026-02-24T07:46:25.701605+0000","last_undegraded":"2026-02-24T07:46:25.701605+0000","last_fullsized":"2026-02-24T07:46:25.701605+0000","mapping_epoch":42,"log_start":"45'10668","ondisk_log_start":"45'10668","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_clean_scrub_stamp":"2026-02-24T07:46:01.867949+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:47:31.776406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":699334656,"num_objects":10671,"num_object_clones":0,"num_object_copies":32013,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10671,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10671,"num_write_kb":682944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"45'10666","reported_seq":10682,"reported_epoch":45,"state":"active+clean","last_fresh":"2026-02-24T07:46:25.707219+0000","last_change":"2026-02-24T07:46:02.882303+0000","last_active":"2026-02-24T07:46:25.707219+0000","last_peered":"2026-02-24T07:46:25.707219+0000","last_clean":"2026-02-24T07:46:25.707219+0000","last_became_active":"2026-02-24T07:46:02.881951+0000","last_became_peered":"2026-02-24T07:46:02.881951+0000","last_unstale":"2026-02-24T07:46:25.707219+0000","last_undegraded":"2026-02-24T07:46:25.707219+0000","last_fullsized":"2026-02-24T07:46:25.707219+0000","mapping_epoch":42,"log_start":"45'10663","ondisk_log_start":"45'10663","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_clean_scrub_stamp":"2026-02-24T07:46:01.867949+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:47:04.345724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":699006976,"num_objects":10666,"num_object_clones":0,"num_object_copies":31998,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10666,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10666,"num_write_kb":682624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"45'10598","reported_seq":10615,"reported_epoch":45,"state":"active+clean","last_fresh":"2026-02-24T07:46:25.744845+0000","last_change":"2026-02-24T07:46:02.881463+0000","last_active":"2026-02-24T07:46:25.744845+0000","last_peered":"2026-02-24T07:46:25.744845+0000","last_clean":"2026-02-24T07:46:25.744845+0000","last_became_active":"2026-02-24T07:46:02.881345+0000","last_became_peered":"2026-02-24T07:46:02.881345+0000","last_unstale":"2026-02-24T07:46:25.744845+0000","last_undegraded":"2026-02-24T07:46:25.744845+0000","last_fullsized":"2026-02-24T07:46:25.744845+0000","mapping_epoch":42,"log_start":"45'10595","ondisk_log_start":"45'10595","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_clean_scrub_stamp":"2026-02-24T07:46:01.867949+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:47:17.135287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":694550528,"num_objects":10598,"num_object_clones":0,"num_object_copies":31794,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10598,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10598,"num_write_kb":678272,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"45'10555","reported_seq":10572,"reported_epoch":45,"state":"active+clean","last_fresh":"2026-02-24T07:46:25.749206+0000","last_change":"2026-02-24T07:46:02.880582+0000","last_active":"2026-02-24T07:46:25.749206+0000","last_peered":"2026-02-24T07:46:25.749206+0000","last_clean":"2026-02-24T07:46:25.749206+0000","last_became_active":"2026-02-24T07:46:02.880323+0000","last_became_peered":"2026-02-24T07:46:02.880323+0000","last_unstale":"2026-02-24T07:46:25.749206+0000","last_undegraded":"2026-02-24T07:46:25.749206+0000","last_fullsized":"2026-02-24T07:46:25.749206+0000","mapping_epoch":42,"log_start":"45'10552","ondisk_log_start":"45'10552","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_clean_scrub_stamp":"2026-02-24T07:46:01.867949+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:47:31.420697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":691732480,"num_objects":10555,"num_object_clones":0,"num_object_copies":31665,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10555,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10555,"num_write_kb":675520,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.6","version":"45'10732","reported_seq":10749,"reported_epoch":45,"state":"active+clean","last_fresh":"2026-02-24T07:46:25.695869+0000","last_change":"2026-02-24T07:46:02.880170+0000","last_active":"2026-02-24T07:46:25.695869+0000","last_peered":"2026-02-24T07:46:25.695869+0000","last_clean":"2026-02-24T07:46:25.695869+0000","last_became_active":"2026-02-24T07:46:02.880005+0000","last_became_peered":"2026-02-24T07:46:02.880005+0000","last_unstale":"2026-02-24T07:46:25.695869+0000","last_undegraded":"2026-02-24T07:46:25.695869+0000","last_fullsized":"2026-02-24T07:46:25.695869+0000","mapping_epoch":42,"log_start":"45'10729","ondisk_log_start":"45'10729","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_clean_scrub_stamp":"2026-02-24T07:46:01.867949+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:47:14.923924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":703332352,"num_objects":10732,"num_object_clones":0,"num_object_copies":32196,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10732,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10732,"num_write_kb":686848,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"45'10671","reported_seq":10688,"reported_epoch":45,"state":"active+clean","last_fresh":"2026-02-24T07:46:25.678143+0000","last_change":"2026-02-24T07:46:02.881931+0000","last_active":"2026-02-24T07:46:25.678143+0000","last_peered":"2026-02-24T07:46:25.678143+0000","last_clean":"2026-02-24T07:46:25.678143+0000","last_became_active":"2026-02-24T07:46:02.881420+0000","last_became_peered":"2026-02-24T07:46:02.881420+0000","last_unstale":"2026-02-24T07:46:25.678143+0000","last_undegraded":"2026-02-24T07:46:25.678143+0000","last_fullsized":"2026-02-24T07:46:25.678143+0000","mapping_epoch":42,"log_start":"45'10668","ondisk_log_start":"45'10668","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_clean_scrub_stamp":"2026-02-24T07:46:01.867949+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:47:09.535393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":699334656,"num_objects":10671,"num_object_clones":0,"num_object_copies":32013,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10671,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10671,"num_write_kb":682944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.4","version":"45'10455","reported_seq":10472,"reported_epoch":45,"state":"active+clean","last_fresh":"2026-02-24T07:46:25.746086+0000","last_change":"2026-02-24T07:46:02.880661+0000","last_active":"2026-02-24T07:46:25.746086+0000","last_peered":"2026-02-24T07:46:25.746086+0000","last_clean":"2026-02-24T07:46:25.746086+0000","last_became_active":"2026-02-24T07:46:02.880345+0000","last_became_peered":"2026-02-24T07:46:02.880345+0000","last_unstale":"2026-02-24T07:46:25.746086+0000","last_undegraded":"2026-02-24T07:46:25.746086+0000","last_fullsized":"2026-02-24T07:46:25.746086+0000","mapping_epoch":42,"log_start":"45'10452","ondisk_log_start":"45'10452","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_clean_scrub_stamp":"2026-02-24T07:46:01.867949+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:47:27.577904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":685178880,"num_objects":10455,"num_object_clones":0,"num_object_copies":31365,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10455,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10455,"num_write_kb":669120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"45'10612","reported_seq":10629,"reported_epoch":45,"state":"active+clean","last_fresh":"2026-02-24T07:46:25.680143+0000","last_change":"2026-02-24T07:46:02.881591+0000","last_active":"2026-02-24T07:46:25.680143+0000","last_peered":"2026-02-24T07:46:25.680143+0000","last_clean":"2026-02-24T07:46:25.680143+0000","last_became_active":"2026-02-24T07:46:02.881035+0000","last_became_peered":"2026-02-24T07:46:02.881035+0000","last_unstale":"2026-02-24T07:46:25.680143+0000","last_undegraded":"2026-02-24T07:46:25.680143+0000","last_fullsized":"2026-02-24T07:46:25.680143+0000","mapping_epoch":42,"log_start":"45'10609","ondisk_log_start":"45'10609","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_clean_scrub_stamp":"2026-02-24T07:46:01.867949+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:47:05.565071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":695468032,"num_objects":10612,"num_object_clones":0,"num_object_copies":31836,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10612,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10612,"num_write_kb":679168,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"45'10785","reported_seq":10802,"reported_epoch":45,"state":"active+clean","last_fresh":"2026-02-24T07:46:25.695923+0000","last_change":"2026-02-24T07:46:02.879955+0000","last_active":"2026-02-24T07:46:25.695923+0000","last_peered":"2026-02-24T07:46:25.695923+0000","last_clean":"2026-02-24T07:46:25.695923+0000","last_became_active":"2026-02-24T07:46:02.879831+0000","last_became_peered":"2026-02-24T07:46:02.879831+0000","last_unstale":"2026-02-24T07:46:25.695923+0000","last_undegraded":"2026-02-24T07:46:25.695923+0000","last_fullsized":"2026-02-24T07:46:25.695923+0000","mapping_epoch":42,"log_start":"45'10782","ondisk_log_start":"45'10782","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_clean_scrub_stamp":"2026-02-24T07:46:01.867949+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:47:19.157247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":706805760,"num_objects":10785,"num_object_clones":0,"num_object_copies":32355,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10785,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10785,"num_write_kb":690240,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.0","version":"45'10564","reported_seq":10581,"reported_epoch":45,"state":"active+clean","last_fresh":"2026-02-24T07:46:25.747665+0000","last_change":"2026-02-24T07:46:02.880585+0000","last_active":"2026-02-24T07:46:25.747665+0000","last_peered":"2026-02-24T07:46:25.747665+0000","last_clean":"2026-02-24T07:46:25.747665+0000","last_became_active":"2026-02-24T07:46:02.880333+0000","last_became_peered":"2026-02-24T07:46:02.880333+0000","last_unstale":"2026-02-24T07:46:25.747665+0000","last_undegraded":"2026-02-24T07:46:25.747665+0000","last_fullsized":"2026-02-24T07:46:25.747665+0000","mapping_epoch":42,"log_start":"45'10561","ondisk_log_start":"45'10561","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_clean_scrub_stamp":"2026-02-24T07:46:01.867949+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:47:04.574448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":692322304,"num_objects":10564,"num_object_clones":0,"num_object_copies":31692,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10564,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10564,"num_write_kb":676096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2","version":"45'10693","reported_seq":10709,"reported_epoch":45,"state":"active+clean","last_fresh":"2026-02-24T07:46:25.747640+0000","last_change":"2026-02-24T07:46:02.880566+0000","last_active":"2026-02-24T07:46:25.747640+0000","last_peered":"2026-02-24T07:46:25.747640+0000","last_clean":"2026-02-24T07:46:25.747640+0000","last_became_active":"2026-02-24T07:46:02.880185+0000","last_became_peered":"2026-02-24T07:46:02.880185+0000","last_unstale":"2026-02-24T07:46:25.747640+0000","last_undegraded":"2026-02-24T07:46:25.747640+0000","last_fullsized":"2026-02-24T07:46:25.747640+0000","mapping_epoch":42,"log_start":"45'10690","ondisk_log_start":"45'10690","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_clean_scrub_stamp":"2026-02-24T07:46:01.867949+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:47:23.244686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":700776448,"num_objects":10693,"num_object_clones":0,"num_object_copies":32079,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10693,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10693,"num_write_kb":684352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"29'99","reported_seq":228,"reported_epoch":45,"state":"active+clean","last_fresh":"2026-02-24T07:46:11.479829+0000","last_change":"2026-02-24T07:45:26.125455+0000","last_active":"2026-02-24T07:46:11.479829+0000","last_peered":"2026-02-24T07:46:11.479829+0000","last_clean":"2026-02-24T07:46:11.479829+0000","last_became_active":"2026-02-24T07:42:54.186719+0000","last_became_peered":"2026-02-24T07:42:54.186719+0000","last_unstale":"2026-02-24T07:46:11.479829+0000","last_undegraded":"2026-02-24T07:46:11.479829+0000","last_fullsized":"2026-02-24T07:46:11.479829+0000","mapping_epoch":27,"log_start":"29'96","ondisk_log_start":"29'96","created":20,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'99","last_scrub_stamp":"2026-02-24T07:45:26.125426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:42:43.640776+0000","last_clean_scrub_stamp":"2026-02-24T07:45:26.125426+0000","objects_scrubbed":2,"log_size":3,"log_dups_size":96,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:46:34.126365+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":240,"num_write":135,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":16,"stat_sum":{"num_bytes":11148001280,"num_objects":170105,"num_object_clones":0,"num_object_copies":510315,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":170105,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":170105,"num_write_kb":10886720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":33432141824,"data_stored":33432141824,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":48,"ondisk_log_size":48,"up":48,"acting":48,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":240,"num_write":135,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":2288,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":3,"up_from":25,"seq":107374182445,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":8009096,"kb_used_data":6623924,"kb_used_omap":8,"kb_used_meta":1385143,"kb_avail":725994104,"statfs":{"total":751619276800,"available":743417962496,"internally_reserved":0,"allocated":6782898176,"data_stored":6782609532,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9064,"internal_metadata":1418386584},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[4],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":19,"seq":81604378671,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":10439880,"kb_used_data":9422772,"kb_used_omap":8,"kb_used_meta":1017079,"kb_avail":723563320,"statfs":{"total":751619276800,"available":740928839680,"internally_reserved":0,"allocated":9648918528,"data_stored":9648629884,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":8485,"internal_metadata":1041489627},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542192,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":8282180,"kb_used_data":7218480,"kb_used_omap":9,"kb_used_meta":1063670,"kb_avail":725721020,"statfs":{"total":751619276800,"available":743138324480,"internally_reserved":0,"allocated":7391723520,"data_stored":7391438428,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9350,"internal_metadata":1089198970},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,3],"upper_bound":4},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705715,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":9624712,"kb_used_data":8643572,"kb_used_omap":8,"kb_used_meta":981111,"kb_avail":724378488,"statfs":{"total":751619276800,"available":741763571712,"internally_reserved":0,"allocated":8851017728,"data_stored":8850729084,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":8362,"internal_metadata":1004658518},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":2288,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":9035251712,"data_stored":9035251712,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":7668498432,"data_stored":7668498432,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":9758179328,"data_stored":9758179328,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":6970212352,"data_stored":6970212352,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-24T07:46:29.674 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph osd pool set .mgr pgp_num 121 2026-02-24T07:46:29.811 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2026-02-24T07:46:29.852 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:46:29.972 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:46:29.974 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 26 16 204053 204037 490.429 494.812 0.00202868 0.00203772 2026-02-24T07:46:29.975 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:46:29.973+0000 7fb3ed83b640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:46:29.976 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:46:29.973+0000 7fb3ed83b640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:46:29.998 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:46:29 trial035 ceph-mon[39284]: purged_snaps scrub starts 2026-02-24T07:46:29.999 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:46:29 trial035 ceph-mon[39284]: purged_snaps scrub ok 2026-02-24T07:46:30.098 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:46:30.095+0000 7f2182065640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:46:30.098 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:46:30.096+0000 7f2182065640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:46:30.107 INFO:teuthology.orchestra.run.trial035.stderr:Error EPERM: crimson_allow_pg_split is false; pgp_num increase denied 2026-02-24T07:46:30.113 INFO:teuthology.orchestra.run.trial035.stdout:osd.0: {} 2026-02-24T07:46:30.114 INFO:teuthology.orchestra.run.trial035.stderr:osd.0: osd_enable_op_tracker = '' 2026-02-24T07:46:30.129 INFO:teuthology.orchestra.run.trial035.stdout:osd.1: {} 2026-02-24T07:46:30.129 INFO:teuthology.orchestra.run.trial035.stderr:osd.1: osd_enable_op_tracker = '' 2026-02-24T07:46:30.144 INFO:teuthology.orchestra.run.trial035.stdout:osd.2: {} 2026-02-24T07:46:30.145 INFO:teuthology.orchestra.run.trial035.stderr:osd.2: osd_enable_op_tracker = '' 2026-02-24T07:46:30.146 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-24T07:46:30.157 INFO:teuthology.orchestra.run.trial035.stdout:osd.3: {} 2026-02-24T07:46:30.157 INFO:teuthology.orchestra.run.trial035.stderr:osd.3: osd_enable_op_tracker = '' 2026-02-24T07:46:30.975 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 27 16 212112 212096 490.919 503.688 0.00118306 0.00203571 2026-02-24T07:46:31.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:46:30 trial035 ceph-mon[39284]: from='client.14642 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2026-02-24T07:46:31.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:46:30 trial035 ceph-mon[39284]: pgmap v124: 17 pgs: 17 active+clean; 10 GiB data, 35 GiB used, 2.7 TiB / 2.7 TiB avail; 394 MiB/s wr, 6.30k op/s 2026-02-24T07:46:31.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:46:30 trial035 ceph-mon[39284]: from='mgr.14284 10.20.193.35:0/3681373222' entity='mgr.x' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2026-02-24T07:46:31.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:46:30 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/19040206' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2026-02-24T07:46:31.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:46:30 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/3131212722' entity='client.admin' cmd={"prefix": "osd pool set", "pool": ".mgr", "var": "pgp_num", "val": "121"} : dispatch 2026-02-24T07:46:31.975 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 28 16 220023 220007 491.043 494.438 0.0024383 0.00203518 2026-02-24T07:46:32.198 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2026-02-24T07:46:32.346 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:46:32.469 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:46:32.466+0000 7f3275307640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:46:32.469 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:46:32.466+0000 7f3275307640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:46:32.483 INFO:teuthology.orchestra.run.trial035.stdout:osd.0: {} 2026-02-24T07:46:32.483 INFO:teuthology.orchestra.run.trial035.stderr:osd.0: osd_enable_op_tracker = '' 2026-02-24T07:46:32.498 INFO:teuthology.orchestra.run.trial035.stdout:osd.1: {} 2026-02-24T07:46:32.498 INFO:teuthology.orchestra.run.trial035.stderr:osd.1: osd_enable_op_tracker = '' 2026-02-24T07:46:32.513 INFO:teuthology.orchestra.run.trial035.stdout:osd.2: {} 2026-02-24T07:46:32.514 INFO:teuthology.orchestra.run.trial035.stderr:osd.2: osd_enable_op_tracker = '' 2026-02-24T07:46:32.525 INFO:teuthology.orchestra.run.trial035.stdout:osd.3: {} 2026-02-24T07:46:32.525 INFO:teuthology.orchestra.run.trial035.stderr:osd.3: osd_enable_op_tracker = '' 2026-02-24T07:46:32.974 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 29 16 227824 227808 490.922 487.562 0.000883122 0.00203567 2026-02-24T07:46:33.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:46:32 trial035 ceph-mon[39284]: pgmap v125: 17 pgs: 17 active+clean; 13 GiB data, 42 GiB used, 2.7 TiB / 2.7 TiB avail; 602 MiB/s wr, 9.63k op/s 2026-02-24T07:46:33.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:46:32 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/4196653544' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2026-02-24T07:46:33.975 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 30 16 235761 235745 491.092 496.062 0.00180993 0.00203502 2026-02-24T07:46:34.567 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2026-02-24T07:46:34.726 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:46:34.975 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 31 16 243734 243718 491.324 498.312 0.00246672 0.00203251 2026-02-24T07:46:35.147 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2026-02-24T07:46:35.148 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2026-02-24T07:46:35.148 INFO:tasks.thrashosds.thrasher:Growing pool unique_pool_1 2026-02-24T07:46:35.148 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph pg dump --format=json 2026-02-24T07:46:35.172 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:46:34 trial035 ceph-mon[39284]: pgmap v126: 17 pgs: 17 active+clean; 13 GiB data, 42 GiB used, 2.7 TiB / 2.7 TiB avail; 394 MiB/s wr, 6.30k op/s 2026-02-24T07:46:35.328 INFO:teuthology.orchestra.run.trial035.stdout:osd.0: {} 2026-02-24T07:46:35.328 INFO:teuthology.orchestra.run.trial035.stdout:osd.1: {} 2026-02-24T07:46:35.328 INFO:teuthology.orchestra.run.trial035.stdout:osd.2: {} 2026-02-24T07:46:35.328 INFO:teuthology.orchestra.run.trial035.stdout:osd.3: {} 2026-02-24T07:46:35.328 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:46:35.252+0000 7f8b27eb4640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:46:35.329 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:46:35.252+0000 7f8b27eb4640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:46:35.329 INFO:teuthology.orchestra.run.trial035.stderr:osd.0: osd_enable_op_tracker = '' 2026-02-24T07:46:35.329 INFO:teuthology.orchestra.run.trial035.stderr:osd.1: osd_enable_op_tracker = '' 2026-02-24T07:46:35.329 INFO:teuthology.orchestra.run.trial035.stderr:osd.2: osd_enable_op_tracker = '' 2026-02-24T07:46:35.329 INFO:teuthology.orchestra.run.trial035.stderr:osd.3: osd_enable_op_tracker = '' 2026-02-24T07:46:35.975 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 32 16 249709 249693 487.639 373.438 0.00156974 0.00204942 2026-02-24T07:46:36.032 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:46:35 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/3814190169' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2026-02-24T07:46:36.115 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:46:36.479 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:46:36.432+0000 7f0f1044f640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:46:36.479 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:46:36.432+0000 7f0f1044f640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:46:36.596 INFO:teuthology.orchestra.run.trial035.stdout: 2026-02-24T07:46:36.597 INFO:teuthology.orchestra.run.trial035.stderr:dumped all 2026-02-24T07:46:36.876 INFO:teuthology.orchestra.run.trial035.stdout:{"pg_ready":true,"pg_map":{"version":127,"stamp":"2026-02-24T07:46:35.686621+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":14391968288,"num_objects":219597,"num_object_clones":0,"num_object_copies":658791,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":219597,"num_whiteouts":0,"num_read":136,"num_read_kb":240,"num_write":219730,"num_write_kb":14056230,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":52,"ondisk_log_size":52,"up":51,"acting":51,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":51,"num_osds":4,"num_per_pool_osds":4,"num_per_pool_omap_osds":4,"kb":2936012800,"kb_used":46659228,"kb_used_data":41593996,"kb_used_omap":35,"kb_used_meta":5065116,"kb_avail":2889353572,"statfs":{"total":3006477107200,"available":2958698057728,"internally_reserved":0,"allocated":42592251904,"data_stored":42591100880,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":35909,"internal_metadata":5186679739},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[5,3,1],"upper_bound":8},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":5583994880,"num_objects":85205,"num_object_clones":0,"num_object_copies":255615,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":85205,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":85205,"num_write_kb":5453120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001636"},"pg_stats":[{"pgid":"3.f","version":"45'15513","reported_seq":15530,"reported_epoch":45,"state":"active+clean","last_fresh":"2026-02-24T07:46:35.666332+0000","last_change":"2026-02-24T07:46:02.882095+0000","last_active":"2026-02-24T07:46:35.666332+0000","last_peered":"2026-02-24T07:46:35.666332+0000","last_clean":"2026-02-24T07:46:35.666332+0000","last_became_active":"2026-02-24T07:46:02.881625+0000","last_became_peered":"2026-02-24T07:46:02.881625+0000","last_unstale":"2026-02-24T07:46:35.666332+0000","last_undegraded":"2026-02-24T07:46:35.666332+0000","last_fullsized":"2026-02-24T07:46:35.666332+0000","mapping_epoch":42,"log_start":"45'15510","ondisk_log_start":"45'15510","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_clean_scrub_stamp":"2026-02-24T07:46:01.867949+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:47:09.267374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1016659968,"num_objects":15513,"num_object_clones":0,"num_object_copies":46539,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15513,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15513,"num_write_kb":992832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.e","version":"45'15692","reported_seq":15709,"reported_epoch":45,"state":"active+clean","last_fresh":"2026-02-24T07:46:35.667131+0000","last_change":"2026-02-24T07:46:02.881462+0000","last_active":"2026-02-24T07:46:35.667131+0000","last_peered":"2026-02-24T07:46:35.667131+0000","last_clean":"2026-02-24T07:46:35.667131+0000","last_became_active":"2026-02-24T07:46:02.881083+0000","last_became_peered":"2026-02-24T07:46:02.881083+0000","last_unstale":"2026-02-24T07:46:35.667131+0000","last_undegraded":"2026-02-24T07:46:35.667131+0000","last_fullsized":"2026-02-24T07:46:35.667131+0000","mapping_epoch":42,"log_start":"45'15689","ondisk_log_start":"45'15689","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_clean_scrub_stamp":"2026-02-24T07:46:01.867949+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:47:29.030602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1028390912,"num_objects":15692,"num_object_clones":0,"num_object_copies":47076,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15692,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15692,"num_write_kb":1004288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"45'13137","reported_seq":13153,"reported_epoch":45,"state":"active+clean","last_fresh":"2026-02-24T07:46:30.707134+0000","last_change":"2026-02-24T07:46:02.883952+0000","last_active":"2026-02-24T07:46:30.707134+0000","last_peered":"2026-02-24T07:46:30.707134+0000","last_clean":"2026-02-24T07:46:30.707134+0000","last_became_active":"2026-02-24T07:46:02.883799+0000","last_became_peered":"2026-02-24T07:46:02.883799+0000","last_unstale":"2026-02-24T07:46:30.707134+0000","last_undegraded":"2026-02-24T07:46:30.707134+0000","last_fullsized":"2026-02-24T07:46:30.707134+0000","mapping_epoch":42,"log_start":"45'13134","ondisk_log_start":"45'13134","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_clean_scrub_stamp":"2026-02-24T07:46:01.867949+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:47:02.402624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":860946432,"num_objects":13137,"num_object_clones":0,"num_object_copies":39411,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13137,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13137,"num_write_kb":840768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"45'12862","reported_seq":12879,"reported_epoch":45,"state":"active+clean","last_fresh":"2026-02-24T07:46:30.704211+0000","last_change":"2026-02-24T07:46:02.880375+0000","last_active":"2026-02-24T07:46:30.704211+0000","last_peered":"2026-02-24T07:46:30.704211+0000","last_clean":"2026-02-24T07:46:30.704211+0000","last_became_active":"2026-02-24T07:46:02.880060+0000","last_became_peered":"2026-02-24T07:46:02.880060+0000","last_unstale":"2026-02-24T07:46:30.704211+0000","last_undegraded":"2026-02-24T07:46:30.704211+0000","last_fullsized":"2026-02-24T07:46:30.704211+0000","mapping_epoch":42,"log_start":"45'12859","ondisk_log_start":"45'12859","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_clean_scrub_stamp":"2026-02-24T07:46:01.867949+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:47:04.123135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":842924032,"num_objects":12862,"num_object_clones":0,"num_object_copies":38586,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12862,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12862,"num_write_kb":823168,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"45'13197","reported_seq":13213,"reported_epoch":45,"state":"active+clean","last_fresh":"2026-02-24T07:46:30.747622+0000","last_change":"2026-02-24T07:46:02.880509+0000","last_active":"2026-02-24T07:46:30.747622+0000","last_peered":"2026-02-24T07:46:30.747622+0000","last_clean":"2026-02-24T07:46:30.747622+0000","last_became_active":"2026-02-24T07:46:02.880171+0000","last_became_peered":"2026-02-24T07:46:02.880171+0000","last_unstale":"2026-02-24T07:46:30.747622+0000","last_undegraded":"2026-02-24T07:46:30.747622+0000","last_fullsized":"2026-02-24T07:46:30.747622+0000","mapping_epoch":42,"log_start":"45'13194","ondisk_log_start":"45'13194","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_clean_scrub_stamp":"2026-02-24T07:46:01.867949+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:47:06.105346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":864878592,"num_objects":13197,"num_object_clones":0,"num_object_copies":39591,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13197,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13197,"num_write_kb":844608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.a","version":"45'13081","reported_seq":13098,"reported_epoch":45,"state":"active+clean","last_fresh":"2026-02-24T07:46:30.705394+0000","last_change":"2026-02-24T07:46:02.882761+0000","last_active":"2026-02-24T07:46:30.705394+0000","last_peered":"2026-02-24T07:46:30.705394+0000","last_clean":"2026-02-24T07:46:30.705394+0000","last_became_active":"2026-02-24T07:46:02.882274+0000","last_became_peered":"2026-02-24T07:46:02.882274+0000","last_unstale":"2026-02-24T07:46:30.705394+0000","last_undegraded":"2026-02-24T07:46:30.705394+0000","last_fullsized":"2026-02-24T07:46:30.705394+0000","mapping_epoch":42,"log_start":"45'13078","ondisk_log_start":"45'13078","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_clean_scrub_stamp":"2026-02-24T07:46:01.867949+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:47:31.776406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":857276416,"num_objects":13081,"num_object_clones":0,"num_object_copies":39243,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13081,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13081,"num_write_kb":837184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"45'13235","reported_seq":13252,"reported_epoch":45,"state":"active+clean","last_fresh":"2026-02-24T07:46:30.702304+0000","last_change":"2026-02-24T07:46:02.882303+0000","last_active":"2026-02-24T07:46:30.702304+0000","last_peered":"2026-02-24T07:46:30.702304+0000","last_clean":"2026-02-24T07:46:30.702304+0000","last_became_active":"2026-02-24T07:46:02.881951+0000","last_became_peered":"2026-02-24T07:46:02.881951+0000","last_unstale":"2026-02-24T07:46:30.702304+0000","last_undegraded":"2026-02-24T07:46:30.702304+0000","last_fullsized":"2026-02-24T07:46:30.702304+0000","mapping_epoch":42,"log_start":"45'13232","ondisk_log_start":"45'13232","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_clean_scrub_stamp":"2026-02-24T07:46:01.867949+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:47:04.345724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":867368960,"num_objects":13235,"num_object_clones":0,"num_object_copies":39705,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13235,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13235,"num_write_kb":847040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"45'13082","reported_seq":13099,"reported_epoch":45,"state":"active+clean","last_fresh":"2026-02-24T07:46:30.749680+0000","last_change":"2026-02-24T07:46:02.881463+0000","last_active":"2026-02-24T07:46:30.749680+0000","last_peered":"2026-02-24T07:46:30.749680+0000","last_clean":"2026-02-24T07:46:30.749680+0000","last_became_active":"2026-02-24T07:46:02.881345+0000","last_became_peered":"2026-02-24T07:46:02.881345+0000","last_unstale":"2026-02-24T07:46:30.749680+0000","last_undegraded":"2026-02-24T07:46:30.749680+0000","last_fullsized":"2026-02-24T07:46:30.749680+0000","mapping_epoch":42,"log_start":"45'13079","ondisk_log_start":"45'13079","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_clean_scrub_stamp":"2026-02-24T07:46:01.867949+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:47:17.135287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":857341952,"num_objects":13082,"num_object_clones":0,"num_object_copies":39246,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13082,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13082,"num_write_kb":837248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"45'13044","reported_seq":13061,"reported_epoch":45,"state":"active+clean","last_fresh":"2026-02-24T07:46:30.749937+0000","last_change":"2026-02-24T07:46:02.880582+0000","last_active":"2026-02-24T07:46:30.749937+0000","last_peered":"2026-02-24T07:46:30.749937+0000","last_clean":"2026-02-24T07:46:30.749937+0000","last_became_active":"2026-02-24T07:46:02.880323+0000","last_became_peered":"2026-02-24T07:46:02.880323+0000","last_unstale":"2026-02-24T07:46:30.749937+0000","last_undegraded":"2026-02-24T07:46:30.749937+0000","last_fullsized":"2026-02-24T07:46:30.749937+0000","mapping_epoch":42,"log_start":"45'13041","ondisk_log_start":"45'13041","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_clean_scrub_stamp":"2026-02-24T07:46:01.867949+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:47:31.420697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":854851584,"num_objects":13044,"num_object_clones":0,"num_object_copies":39132,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13044,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13044,"num_write_kb":834816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.6","version":"45'13212","reported_seq":13229,"reported_epoch":45,"state":"active+clean","last_fresh":"2026-02-24T07:46:30.698591+0000","last_change":"2026-02-24T07:46:02.880170+0000","last_active":"2026-02-24T07:46:30.698591+0000","last_peered":"2026-02-24T07:46:30.698591+0000","last_clean":"2026-02-24T07:46:30.698591+0000","last_became_active":"2026-02-24T07:46:02.880005+0000","last_became_peered":"2026-02-24T07:46:02.880005+0000","last_unstale":"2026-02-24T07:46:30.698591+0000","last_undegraded":"2026-02-24T07:46:30.698591+0000","last_fullsized":"2026-02-24T07:46:30.698591+0000","mapping_epoch":42,"log_start":"45'13209","ondisk_log_start":"45'13209","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_clean_scrub_stamp":"2026-02-24T07:46:01.867949+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:47:14.923924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":865861632,"num_objects":13212,"num_object_clones":0,"num_object_copies":39636,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13212,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13212,"num_write_kb":845568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"45'15617","reported_seq":15634,"reported_epoch":45,"state":"active+clean","last_fresh":"2026-02-24T07:46:35.664325+0000","last_change":"2026-02-24T07:46:02.881931+0000","last_active":"2026-02-24T07:46:35.664325+0000","last_peered":"2026-02-24T07:46:35.664325+0000","last_clean":"2026-02-24T07:46:35.664325+0000","last_became_active":"2026-02-24T07:46:02.881420+0000","last_became_peered":"2026-02-24T07:46:02.881420+0000","last_unstale":"2026-02-24T07:46:35.664325+0000","last_undegraded":"2026-02-24T07:46:35.664325+0000","last_fullsized":"2026-02-24T07:46:35.664325+0000","mapping_epoch":42,"log_start":"45'15614","ondisk_log_start":"45'15614","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_clean_scrub_stamp":"2026-02-24T07:46:01.867949+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:47:09.535393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1023475712,"num_objects":15617,"num_object_clones":0,"num_object_copies":46851,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15617,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15617,"num_write_kb":999488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.4","version":"45'12874","reported_seq":12891,"reported_epoch":45,"state":"active+clean","last_fresh":"2026-02-24T07:46:30.747209+0000","last_change":"2026-02-24T07:46:02.880661+0000","last_active":"2026-02-24T07:46:30.747209+0000","last_peered":"2026-02-24T07:46:30.747209+0000","last_clean":"2026-02-24T07:46:30.747209+0000","last_became_active":"2026-02-24T07:46:02.880345+0000","last_became_peered":"2026-02-24T07:46:02.880345+0000","last_unstale":"2026-02-24T07:46:30.747209+0000","last_undegraded":"2026-02-24T07:46:30.747209+0000","last_fullsized":"2026-02-24T07:46:30.747209+0000","mapping_epoch":42,"log_start":"45'12870","ondisk_log_start":"45'12870","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_clean_scrub_stamp":"2026-02-24T07:46:01.867949+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":3000,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:47:27.577904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":843710464,"num_objects":12874,"num_object_clones":0,"num_object_copies":38622,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12874,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12874,"num_write_kb":823936,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"45'15492","reported_seq":15509,"reported_epoch":45,"state":"active+clean","last_fresh":"2026-02-24T07:46:35.662772+0000","last_change":"2026-02-24T07:46:02.881591+0000","last_active":"2026-02-24T07:46:35.662772+0000","last_peered":"2026-02-24T07:46:35.662772+0000","last_clean":"2026-02-24T07:46:35.662772+0000","last_became_active":"2026-02-24T07:46:02.881035+0000","last_became_peered":"2026-02-24T07:46:02.881035+0000","last_unstale":"2026-02-24T07:46:35.662772+0000","last_undegraded":"2026-02-24T07:46:35.662772+0000","last_fullsized":"2026-02-24T07:46:35.662772+0000","mapping_epoch":42,"log_start":"45'15489","ondisk_log_start":"45'15489","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_clean_scrub_stamp":"2026-02-24T07:46:01.867949+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:47:05.565071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1015283712,"num_objects":15492,"num_object_clones":0,"num_object_copies":46476,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15492,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15492,"num_write_kb":991488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"45'13280","reported_seq":13297,"reported_epoch":45,"state":"active+clean","last_fresh":"2026-02-24T07:46:30.696302+0000","last_change":"2026-02-24T07:46:02.879955+0000","last_active":"2026-02-24T07:46:30.696302+0000","last_peered":"2026-02-24T07:46:30.696302+0000","last_clean":"2026-02-24T07:46:30.696302+0000","last_became_active":"2026-02-24T07:46:02.879831+0000","last_became_peered":"2026-02-24T07:46:02.879831+0000","last_unstale":"2026-02-24T07:46:30.696302+0000","last_undegraded":"2026-02-24T07:46:30.696302+0000","last_fullsized":"2026-02-24T07:46:30.696302+0000","mapping_epoch":42,"log_start":"45'13277","ondisk_log_start":"45'13277","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_clean_scrub_stamp":"2026-02-24T07:46:01.867949+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:47:19.157247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":870318080,"num_objects":13280,"num_object_clones":0,"num_object_copies":39840,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13280,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13280,"num_write_kb":849920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.0","version":"45'13082","reported_seq":13098,"reported_epoch":45,"state":"active+clean","last_fresh":"2026-02-24T07:46:30.747535+0000","last_change":"2026-02-24T07:46:02.880585+0000","last_active":"2026-02-24T07:46:30.747535+0000","last_peered":"2026-02-24T07:46:30.747535+0000","last_clean":"2026-02-24T07:46:30.747535+0000","last_became_active":"2026-02-24T07:46:02.880333+0000","last_became_peered":"2026-02-24T07:46:02.880333+0000","last_unstale":"2026-02-24T07:46:30.747535+0000","last_undegraded":"2026-02-24T07:46:30.747535+0000","last_fullsized":"2026-02-24T07:46:30.747535+0000","mapping_epoch":42,"log_start":"45'13079","ondisk_log_start":"45'13079","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_clean_scrub_stamp":"2026-02-24T07:46:01.867949+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:47:04.574448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":857341952,"num_objects":13082,"num_object_clones":0,"num_object_copies":39246,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13082,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13082,"num_write_kb":837248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2","version":"45'13195","reported_seq":13212,"reported_epoch":45,"state":"active+clean","last_fresh":"2026-02-24T07:46:30.745591+0000","last_change":"2026-02-24T07:46:02.880566+0000","last_active":"2026-02-24T07:46:30.745591+0000","last_peered":"2026-02-24T07:46:30.745591+0000","last_clean":"2026-02-24T07:46:30.745591+0000","last_became_active":"2026-02-24T07:46:02.880185+0000","last_became_peered":"2026-02-24T07:46:02.880185+0000","last_unstale":"2026-02-24T07:46:30.745591+0000","last_undegraded":"2026-02-24T07:46:30.745591+0000","last_fullsized":"2026-02-24T07:46:30.745591+0000","mapping_epoch":42,"log_start":"45'13192","ondisk_log_start":"45'13192","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_clean_scrub_stamp":"2026-02-24T07:46:01.867949+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:47:23.244686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":864747520,"num_objects":13195,"num_object_clones":0,"num_object_copies":39585,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13195,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13195,"num_write_kb":844480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"29'99","reported_seq":228,"reported_epoch":45,"state":"active+clean","last_fresh":"2026-02-24T07:46:11.479829+0000","last_change":"2026-02-24T07:45:26.125455+0000","last_active":"2026-02-24T07:46:11.479829+0000","last_peered":"2026-02-24T07:46:11.479829+0000","last_clean":"2026-02-24T07:46:11.479829+0000","last_became_active":"2026-02-24T07:42:54.186719+0000","last_became_peered":"2026-02-24T07:42:54.186719+0000","last_unstale":"2026-02-24T07:46:11.479829+0000","last_undegraded":"2026-02-24T07:46:11.479829+0000","last_fullsized":"2026-02-24T07:46:11.479829+0000","mapping_epoch":27,"log_start":"29'96","ondisk_log_start":"29'96","created":20,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'99","last_scrub_stamp":"2026-02-24T07:45:26.125426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:42:43.640776+0000","last_clean_scrub_stamp":"2026-02-24T07:45:26.125426+0000","objects_scrubbed":2,"log_size":3,"log_dups_size":96,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:46:34.126365+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":240,"num_write":135,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":16,"stat_sum":{"num_bytes":14391377920,"num_objects":219595,"num_object_clones":0,"num_object_copies":658785,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":219595,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":219595,"num_write_kb":14054080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":43478286336,"data_stored":43478286336,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":49,"ondisk_log_size":49,"up":48,"acting":48,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":240,"num_write":135,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":2280,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":3,"up_from":25,"seq":107374182446,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":9617736,"kb_used_data":8160372,"kb_used_omap":9,"kb_used_meta":1457334,"kb_avail":724385464,"statfs":{"total":751619276800,"available":741770715136,"internally_reserved":0,"allocated":8356220928,"data_stored":8355932284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9712,"internal_metadata":1492310544},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":19,"seq":81604378673,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":15109128,"kb_used_data":13809908,"kb_used_omap":8,"kb_used_meta":1299191,"kb_avail":718894072,"statfs":{"total":751619276800,"available":736147529728,"internally_reserved":0,"allocated":14141345792,"data_stored":14141057148,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":8485,"internal_metadata":1330372315},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542193,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":10134980,"kb_used_data":8951856,"kb_used_omap":9,"kb_used_meta":1183094,"kb_avail":723868220,"statfs":{"total":751619276800,"available":741241057280,"internally_reserved":0,"allocated":9166700544,"data_stored":9166415452,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9350,"internal_metadata":1211489146},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,3,1],"upper_bound":8},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705716,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":11797384,"kb_used_data":10671860,"kb_used_omap":8,"kb_used_meta":1125495,"kb_avail":722205816,"statfs":{"total":751619276800,"available":739538755584,"internally_reserved":0,"allocated":10927984640,"data_stored":10927695996,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":8362,"internal_metadata":1152507734},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":2280,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":11162550272,"data_stored":11162550272,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":9458548736,"data_stored":9458548736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":14254604288,"data_stored":14254604288,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8602583040,"data_stored":8602583040,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-24T07:46:36.878 INFO:tasks.cephadm.ceph_manager.ceph:increase pool size by 10 2026-02-24T07:46:36.878 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph osd pool set unique_pool_1 pg_num 26 2026-02-24T07:46:36.975 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 33 16 256084 256068 484.935 398.438 0.00317194 0.00206086 2026-02-24T07:46:37.027 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:46:37.145 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:46:36 trial035 ceph-mon[39284]: pgmap v127: 17 pgs: 17 active+clean; 13 GiB data, 44 GiB used, 2.7 TiB / 2.7 TiB avail; 444 MiB/s wr, 7.10k op/s 2026-02-24T07:46:37.145 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:46:36 trial035 ceph-mon[39284]: purged_snaps scrub starts 2026-02-24T07:46:37.145 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:46:36 trial035 ceph-mon[39284]: purged_snaps scrub ok 2026-02-24T07:46:37.145 INFO:journalctl@ceph.mgr.x.trial035.stdout:Feb 24 07:46:36 trial035 ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-mgr-x[39621]: ::ffff:10.20.193.91 - - [24/Feb/2026:07:46:36] "GET /metrics HTTP/1.1" 200 4226 "" "Prometheus/3.6.0" 2026-02-24T07:46:37.151 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:46:37.150+0000 7f1297a8b640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:46:37.152 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:46:37.150+0000 7f1297a8b640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:46:37.293 INFO:teuthology.orchestra.run.trial035.stderr:Error EPERM: crimson_allow_pg_split is false; pg_num increase denied for crimson pool 2026-02-24T07:46:37.330 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-24T07:46:37.356 INFO:journalctl@ceph.prometheus.a.trial091.stdout:Feb 24 07:46:36 trial091 ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-prometheus-a[66007]: time=2026-02-24T07:46:36.973Z level=ERROR source=scrape.go:1630 msg="Failed to determine correct type of scrape target." component="scrape manager" scrape_pool=ceph target=http://trial035:9283/metrics content_type="text/html;charset=utf-8" fallback_media_type="" err="received unsupported Content-Type \"text/html;charset=utf-8\" and no fallback_scrape_protocol specified for target" 2026-02-24T07:46:37.975 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 34 16 264322 264306 485.814 514.875 0.00167356 0.00205713 2026-02-24T07:46:38.058 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2026-02-24T07:46:38.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:46:37 trial035 ceph-mon[39284]: from='client.14652 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2026-02-24T07:46:38.205 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:46:37 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/1168953337' entity='client.admin' cmd={"prefix": "osd pool set", "pool": "unique_pool_1", "var": "pg_num", "val": "26"} : dispatch 2026-02-24T07:46:38.206 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:46:38.329 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:46:38.327+0000 7fc012666640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:46:38.329 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:46:38.327+0000 7fc012666640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:46:38.343 INFO:teuthology.orchestra.run.trial035.stdout:osd.0: {} 2026-02-24T07:46:38.344 INFO:teuthology.orchestra.run.trial035.stderr:osd.0: osd_enable_op_tracker = '' 2026-02-24T07:46:38.357 INFO:teuthology.orchestra.run.trial035.stdout:osd.1: {} 2026-02-24T07:46:38.358 INFO:teuthology.orchestra.run.trial035.stderr:osd.1: osd_enable_op_tracker = '' 2026-02-24T07:46:38.373 INFO:teuthology.orchestra.run.trial035.stdout:osd.2: {} 2026-02-24T07:46:38.374 INFO:teuthology.orchestra.run.trial035.stderr:osd.2: osd_enable_op_tracker = '' 2026-02-24T07:46:38.387 INFO:teuthology.orchestra.run.trial035.stdout:osd.3: {} 2026-02-24T07:46:38.387 INFO:teuthology.orchestra.run.trial035.stderr:osd.3: osd_enable_op_tracker = '' 2026-02-24T07:46:38.975 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 35 16 272315 272299 486.206 499.562 0.00167843 0.00205546 2026-02-24T07:46:39.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:46:38 trial035 ceph-mon[39284]: pgmap v128: 17 pgs: 17 active+clean; 15 GiB data, 50 GiB used, 2.7 TiB / 2.7 TiB avail; 547 MiB/s wr, 8.76k op/s 2026-02-24T07:46:39.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:46:38 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/518583782' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2026-02-24T07:46:39.975 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 36 16 280371 280355 486.685 503.5 0.00312817 0.00205345 2026-02-24T07:46:40.428 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2026-02-24T07:46:40.571 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:46:40.690 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:46:40.688+0000 7fe01fd5b640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:46:40.690 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:46:40.689+0000 7fe01fd5b640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:46:40.706 INFO:teuthology.orchestra.run.trial035.stdout:osd.0: {} 2026-02-24T07:46:40.706 INFO:teuthology.orchestra.run.trial035.stderr:osd.0: osd_enable_op_tracker = '' 2026-02-24T07:46:40.722 INFO:teuthology.orchestra.run.trial035.stdout:osd.1: {} 2026-02-24T07:46:40.723 INFO:teuthology.orchestra.run.trial035.stderr:osd.1: osd_enable_op_tracker = '' 2026-02-24T07:46:40.737 INFO:teuthology.orchestra.run.trial035.stdout:osd.2: {} 2026-02-24T07:46:40.738 INFO:teuthology.orchestra.run.trial035.stderr:osd.2: osd_enable_op_tracker = '' 2026-02-24T07:46:40.750 INFO:teuthology.orchestra.run.trial035.stdout:osd.3: {} 2026-02-24T07:46:40.750 INFO:teuthology.orchestra.run.trial035.stderr:osd.3: osd_enable_op_tracker = '' 2026-02-24T07:46:40.976 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 37 16 288772 288756 487.721 525.062 0.00158658 0.0020491 2026-02-24T07:46:41.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:46:40 trial035 ceph-mon[39284]: pgmap v129: 17 pgs: 17 active+clean; 15 GiB data, 50 GiB used, 2.7 TiB / 2.7 TiB avail; 408 MiB/s wr, 6.52k op/s 2026-02-24T07:46:41.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:46:40 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/3662635838' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2026-02-24T07:46:41.975 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 38 16 297537 297521 489.301 547.812 0.00277093 0.00204245 2026-02-24T07:46:42.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:46:41 trial035 ceph-mon[39284]: 1.0 scrub starts 2026-02-24T07:46:42.205 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:46:41 trial035 ceph-mon[39284]: 1.0 scrub ok 2026-02-24T07:46:42.331 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2026-02-24T07:46:42.332 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2026-02-24T07:46:42.332 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph pg dump --format=json 2026-02-24T07:46:42.470 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:46:42.794 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2026-02-24T07:46:42.942 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:46:42 trial035 ceph-mon[39284]: pgmap v130: 17 pgs: 17 active+clean; 17 GiB data, 57 GiB used, 2.7 TiB / 2.7 TiB avail; 606 MiB/s wr, 9.70k op/s 2026-02-24T07:46:42.950 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:46:42.975 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 39 16 305152 305136 488.958 475.938 0.00169502 0.00204391 2026-02-24T07:46:43.066 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:46:42.998+0000 7f8613360640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:46:43.066 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:46:42.998+0000 7f8613360640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:46:43.137 INFO:teuthology.orchestra.run.trial035.stdout: 2026-02-24T07:46:43.138 INFO:teuthology.orchestra.run.trial035.stderr:dumped all 2026-02-24T07:46:43.696 INFO:teuthology.orchestra.run.trial035.stdout:{"pg_ready":true,"pg_map":{"version":130,"stamp":"2026-02-24T07:46:41.687427+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":18780652064,"num_objects":286563,"num_object_clones":0,"num_object_copies":859689,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":286563,"num_whiteouts":0,"num_read":136,"num_read_kb":240,"num_write":286696,"num_write_kb":18342054,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":52,"ondisk_log_size":52,"up":51,"acting":51,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":51,"num_osds":4,"num_per_pool_osds":4,"num_per_pool_omap_osds":4,"kb":2936012800,"kb_used":59521500,"kb_used_data":54164556,"kb_used_omap":35,"kb_used_meta":5356764,"kb_avail":2876491300,"statfs":{"total":3006477107200,"available":2945527091200,"internally_reserved":0,"allocated":55464505344,"data_stored":55463354320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":36546,"internal_metadata":5485326654},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[11,8],"upper_bound":4},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":7632060416,"num_objects":116456,"num_object_clones":0,"num_object_copies":349368,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":116456,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":116456,"num_write_kb":7453184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001652"},"pg_stats":[{"pgid":"3.f","version":"45'17960","reported_seq":17977,"reported_epoch":45,"state":"active+clean","last_fresh":"2026-02-24T07:46:40.678415+0000","last_change":"2026-02-24T07:46:02.882095+0000","last_active":"2026-02-24T07:46:40.678415+0000","last_peered":"2026-02-24T07:46:40.678415+0000","last_clean":"2026-02-24T07:46:40.678415+0000","last_became_active":"2026-02-24T07:46:02.881625+0000","last_became_peered":"2026-02-24T07:46:02.881625+0000","last_unstale":"2026-02-24T07:46:40.678415+0000","last_undegraded":"2026-02-24T07:46:40.678415+0000","last_fullsized":"2026-02-24T07:46:40.678415+0000","mapping_epoch":42,"log_start":"45'17957","ondisk_log_start":"45'17957","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_clean_scrub_stamp":"2026-02-24T07:46:01.867949+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:47:09.267374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1177026560,"num_objects":17960,"num_object_clones":0,"num_object_copies":53880,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17960,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17960,"num_write_kb":1149440,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.e","version":"45'18144","reported_seq":18160,"reported_epoch":45,"state":"active+clean","last_fresh":"2026-02-24T07:46:40.682273+0000","last_change":"2026-02-24T07:46:02.881462+0000","last_active":"2026-02-24T07:46:40.682273+0000","last_peered":"2026-02-24T07:46:40.682273+0000","last_clean":"2026-02-24T07:46:40.682273+0000","last_became_active":"2026-02-24T07:46:02.881083+0000","last_became_peered":"2026-02-24T07:46:02.881083+0000","last_unstale":"2026-02-24T07:46:40.682273+0000","last_undegraded":"2026-02-24T07:46:40.682273+0000","last_fullsized":"2026-02-24T07:46:40.682273+0000","mapping_epoch":42,"log_start":"45'18141","ondisk_log_start":"45'18141","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_clean_scrub_stamp":"2026-02-24T07:46:01.867949+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:47:29.030602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1189085184,"num_objects":18144,"num_object_clones":0,"num_object_copies":54432,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18144,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18144,"num_write_kb":1161216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"45'17915","reported_seq":17932,"reported_epoch":45,"state":"active+clean","last_fresh":"2026-02-24T07:46:40.705281+0000","last_change":"2026-02-24T07:46:02.883952+0000","last_active":"2026-02-24T07:46:40.705281+0000","last_peered":"2026-02-24T07:46:40.705281+0000","last_clean":"2026-02-24T07:46:40.705281+0000","last_became_active":"2026-02-24T07:46:02.883799+0000","last_became_peered":"2026-02-24T07:46:02.883799+0000","last_unstale":"2026-02-24T07:46:40.705281+0000","last_undegraded":"2026-02-24T07:46:40.705281+0000","last_fullsized":"2026-02-24T07:46:40.705281+0000","mapping_epoch":42,"log_start":"45'17912","ondisk_log_start":"45'17912","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_clean_scrub_stamp":"2026-02-24T07:46:01.867949+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:47:02.402624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1174077440,"num_objects":17915,"num_object_clones":0,"num_object_copies":53745,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17915,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17915,"num_write_kb":1146560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"45'17645","reported_seq":17662,"reported_epoch":45,"state":"active+clean","last_fresh":"2026-02-24T07:46:40.707447+0000","last_change":"2026-02-24T07:46:02.880375+0000","last_active":"2026-02-24T07:46:40.707447+0000","last_peered":"2026-02-24T07:46:40.707447+0000","last_clean":"2026-02-24T07:46:40.707447+0000","last_became_active":"2026-02-24T07:46:02.880060+0000","last_became_peered":"2026-02-24T07:46:02.880060+0000","last_unstale":"2026-02-24T07:46:40.707447+0000","last_undegraded":"2026-02-24T07:46:40.707447+0000","last_fullsized":"2026-02-24T07:46:40.707447+0000","mapping_epoch":42,"log_start":"45'17642","ondisk_log_start":"45'17642","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_clean_scrub_stamp":"2026-02-24T07:46:01.867949+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:47:04.123135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1156382720,"num_objects":17645,"num_object_clones":0,"num_object_copies":52935,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17645,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17645,"num_write_kb":1129280,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"45'17945","reported_seq":17962,"reported_epoch":45,"state":"active+clean","last_fresh":"2026-02-24T07:46:40.743315+0000","last_change":"2026-02-24T07:46:02.880509+0000","last_active":"2026-02-24T07:46:40.743315+0000","last_peered":"2026-02-24T07:46:40.743315+0000","last_clean":"2026-02-24T07:46:40.743315+0000","last_became_active":"2026-02-24T07:46:02.880171+0000","last_became_peered":"2026-02-24T07:46:02.880171+0000","last_unstale":"2026-02-24T07:46:40.743315+0000","last_undegraded":"2026-02-24T07:46:40.743315+0000","last_fullsized":"2026-02-24T07:46:40.743315+0000","mapping_epoch":42,"log_start":"45'17942","ondisk_log_start":"45'17942","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_clean_scrub_stamp":"2026-02-24T07:46:01.867949+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:47:06.105346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1176043520,"num_objects":17945,"num_object_clones":0,"num_object_copies":53835,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17945,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17945,"num_write_kb":1148480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.a","version":"45'17703","reported_seq":17720,"reported_epoch":45,"state":"active+clean","last_fresh":"2026-02-24T07:46:40.707375+0000","last_change":"2026-02-24T07:46:02.882761+0000","last_active":"2026-02-24T07:46:40.707375+0000","last_peered":"2026-02-24T07:46:40.707375+0000","last_clean":"2026-02-24T07:46:40.707375+0000","last_became_active":"2026-02-24T07:46:02.882274+0000","last_became_peered":"2026-02-24T07:46:02.882274+0000","last_unstale":"2026-02-24T07:46:40.707375+0000","last_undegraded":"2026-02-24T07:46:40.707375+0000","last_fullsized":"2026-02-24T07:46:40.707375+0000","mapping_epoch":42,"log_start":"45'17700","ondisk_log_start":"45'17700","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_clean_scrub_stamp":"2026-02-24T07:46:01.867949+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:47:31.776406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1160183808,"num_objects":17703,"num_object_clones":0,"num_object_copies":53109,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17703,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17703,"num_write_kb":1132992,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"45'18067","reported_seq":18084,"reported_epoch":45,"state":"active+clean","last_fresh":"2026-02-24T07:46:40.705332+0000","last_change":"2026-02-24T07:46:02.882303+0000","last_active":"2026-02-24T07:46:40.705332+0000","last_peered":"2026-02-24T07:46:40.705332+0000","last_clean":"2026-02-24T07:46:40.705332+0000","last_became_active":"2026-02-24T07:46:02.881951+0000","last_became_peered":"2026-02-24T07:46:02.881951+0000","last_unstale":"2026-02-24T07:46:40.705332+0000","last_undegraded":"2026-02-24T07:46:40.705332+0000","last_fullsized":"2026-02-24T07:46:40.705332+0000","mapping_epoch":42,"log_start":"45'18064","ondisk_log_start":"45'18064","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_clean_scrub_stamp":"2026-02-24T07:46:01.867949+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:47:04.345724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1184038912,"num_objects":18067,"num_object_clones":0,"num_object_copies":54201,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18067,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18067,"num_write_kb":1156288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"45'17917","reported_seq":17934,"reported_epoch":45,"state":"active+clean","last_fresh":"2026-02-24T07:46:40.749155+0000","last_change":"2026-02-24T07:46:02.881463+0000","last_active":"2026-02-24T07:46:40.749155+0000","last_peered":"2026-02-24T07:46:40.749155+0000","last_clean":"2026-02-24T07:46:40.749155+0000","last_became_active":"2026-02-24T07:46:02.881345+0000","last_became_peered":"2026-02-24T07:46:02.881345+0000","last_unstale":"2026-02-24T07:46:40.749155+0000","last_undegraded":"2026-02-24T07:46:40.749155+0000","last_fullsized":"2026-02-24T07:46:40.749155+0000","mapping_epoch":42,"log_start":"45'17913","ondisk_log_start":"45'17913","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_clean_scrub_stamp":"2026-02-24T07:46:01.867949+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":3000,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:47:17.135287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1174208512,"num_objects":17917,"num_object_clones":0,"num_object_copies":53751,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17917,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17917,"num_write_kb":1146688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"45'17778","reported_seq":17795,"reported_epoch":45,"state":"active+clean","last_fresh":"2026-02-24T07:46:40.744576+0000","last_change":"2026-02-24T07:46:02.880582+0000","last_active":"2026-02-24T07:46:40.744576+0000","last_peered":"2026-02-24T07:46:40.744576+0000","last_clean":"2026-02-24T07:46:40.744576+0000","last_became_active":"2026-02-24T07:46:02.880323+0000","last_became_peered":"2026-02-24T07:46:02.880323+0000","last_unstale":"2026-02-24T07:46:40.744576+0000","last_undegraded":"2026-02-24T07:46:40.744576+0000","last_fullsized":"2026-02-24T07:46:40.744576+0000","mapping_epoch":42,"log_start":"45'17775","ondisk_log_start":"45'17775","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_clean_scrub_stamp":"2026-02-24T07:46:01.867949+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:47:31.420697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1165099008,"num_objects":17778,"num_object_clones":0,"num_object_copies":53334,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17778,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17778,"num_write_kb":1137792,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.6","version":"45'18035","reported_seq":18052,"reported_epoch":45,"state":"active+clean","last_fresh":"2026-02-24T07:46:40.702960+0000","last_change":"2026-02-24T07:46:02.880170+0000","last_active":"2026-02-24T07:46:40.702960+0000","last_peered":"2026-02-24T07:46:40.702960+0000","last_clean":"2026-02-24T07:46:40.702960+0000","last_became_active":"2026-02-24T07:46:02.880005+0000","last_became_peered":"2026-02-24T07:46:02.880005+0000","last_unstale":"2026-02-24T07:46:40.702960+0000","last_undegraded":"2026-02-24T07:46:40.702960+0000","last_fullsized":"2026-02-24T07:46:40.702960+0000","mapping_epoch":42,"log_start":"45'18032","ondisk_log_start":"45'18032","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_clean_scrub_stamp":"2026-02-24T07:46:01.867949+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:47:14.923924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1181941760,"num_objects":18035,"num_object_clones":0,"num_object_copies":54105,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18035,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18035,"num_write_kb":1154240,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"45'18062","reported_seq":18079,"reported_epoch":45,"state":"active+clean","last_fresh":"2026-02-24T07:46:40.680151+0000","last_change":"2026-02-24T07:46:02.881931+0000","last_active":"2026-02-24T07:46:40.680151+0000","last_peered":"2026-02-24T07:46:40.680151+0000","last_clean":"2026-02-24T07:46:40.680151+0000","last_became_active":"2026-02-24T07:46:02.881420+0000","last_became_peered":"2026-02-24T07:46:02.881420+0000","last_unstale":"2026-02-24T07:46:40.680151+0000","last_undegraded":"2026-02-24T07:46:40.680151+0000","last_fullsized":"2026-02-24T07:46:40.680151+0000","mapping_epoch":42,"log_start":"45'18059","ondisk_log_start":"45'18059","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_clean_scrub_stamp":"2026-02-24T07:46:01.867949+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:47:09.535393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1183711232,"num_objects":18062,"num_object_clones":0,"num_object_copies":54186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18062,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18062,"num_write_kb":1155968,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.4","version":"45'17636","reported_seq":17653,"reported_epoch":45,"state":"active+clean","last_fresh":"2026-02-24T07:46:40.748783+0000","last_change":"2026-02-24T07:46:02.880661+0000","last_active":"2026-02-24T07:46:40.748783+0000","last_peered":"2026-02-24T07:46:40.748783+0000","last_clean":"2026-02-24T07:46:40.748783+0000","last_became_active":"2026-02-24T07:46:02.880345+0000","last_became_peered":"2026-02-24T07:46:02.880345+0000","last_unstale":"2026-02-24T07:46:40.748783+0000","last_undegraded":"2026-02-24T07:46:40.748783+0000","last_fullsized":"2026-02-24T07:46:40.748783+0000","mapping_epoch":42,"log_start":"45'17633","ondisk_log_start":"45'17633","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_clean_scrub_stamp":"2026-02-24T07:46:01.867949+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:47:27.577904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1155792896,"num_objects":17636,"num_object_clones":0,"num_object_copies":52908,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17636,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17636,"num_write_kb":1128704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"45'17867","reported_seq":17883,"reported_epoch":45,"state":"active+clean","last_fresh":"2026-02-24T07:46:40.677364+0000","last_change":"2026-02-24T07:46:02.881591+0000","last_active":"2026-02-24T07:46:40.677364+0000","last_peered":"2026-02-24T07:46:40.677364+0000","last_clean":"2026-02-24T07:46:40.677364+0000","last_became_active":"2026-02-24T07:46:02.881035+0000","last_became_peered":"2026-02-24T07:46:02.881035+0000","last_unstale":"2026-02-24T07:46:40.677364+0000","last_undegraded":"2026-02-24T07:46:40.677364+0000","last_fullsized":"2026-02-24T07:46:40.677364+0000","mapping_epoch":42,"log_start":"45'17864","ondisk_log_start":"45'17864","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_clean_scrub_stamp":"2026-02-24T07:46:01.867949+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:47:05.565071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170931712,"num_objects":17867,"num_object_clones":0,"num_object_copies":53601,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17867,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17867,"num_write_kb":1143488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"45'17986","reported_seq":18003,"reported_epoch":45,"state":"active+clean","last_fresh":"2026-02-24T07:46:40.698001+0000","last_change":"2026-02-24T07:46:02.879955+0000","last_active":"2026-02-24T07:46:40.698001+0000","last_peered":"2026-02-24T07:46:40.698001+0000","last_clean":"2026-02-24T07:46:40.698001+0000","last_became_active":"2026-02-24T07:46:02.879831+0000","last_became_peered":"2026-02-24T07:46:02.879831+0000","last_unstale":"2026-02-24T07:46:40.698001+0000","last_undegraded":"2026-02-24T07:46:40.698001+0000","last_fullsized":"2026-02-24T07:46:40.698001+0000","mapping_epoch":42,"log_start":"45'17983","ondisk_log_start":"45'17983","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_clean_scrub_stamp":"2026-02-24T07:46:01.867949+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:47:19.157247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1178730496,"num_objects":17986,"num_object_clones":0,"num_object_copies":53958,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17986,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17986,"num_write_kb":1151104,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.0","version":"45'17805","reported_seq":17822,"reported_epoch":45,"state":"active+clean","last_fresh":"2026-02-24T07:46:40.749807+0000","last_change":"2026-02-24T07:46:02.880585+0000","last_active":"2026-02-24T07:46:40.749807+0000","last_peered":"2026-02-24T07:46:40.749807+0000","last_clean":"2026-02-24T07:46:40.749807+0000","last_became_active":"2026-02-24T07:46:02.880333+0000","last_became_peered":"2026-02-24T07:46:02.880333+0000","last_unstale":"2026-02-24T07:46:40.749807+0000","last_undegraded":"2026-02-24T07:46:40.749807+0000","last_fullsized":"2026-02-24T07:46:40.749807+0000","mapping_epoch":42,"log_start":"45'17802","ondisk_log_start":"45'17802","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_clean_scrub_stamp":"2026-02-24T07:46:01.867949+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:47:04.574448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1166868480,"num_objects":17805,"num_object_clones":0,"num_object_copies":53415,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17805,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17805,"num_write_kb":1139520,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2","version":"45'18096","reported_seq":18113,"reported_epoch":45,"state":"active+clean","last_fresh":"2026-02-24T07:46:40.746330+0000","last_change":"2026-02-24T07:46:02.880566+0000","last_active":"2026-02-24T07:46:40.746330+0000","last_peered":"2026-02-24T07:46:40.746330+0000","last_clean":"2026-02-24T07:46:40.746330+0000","last_became_active":"2026-02-24T07:46:02.880185+0000","last_became_peered":"2026-02-24T07:46:02.880185+0000","last_unstale":"2026-02-24T07:46:40.746330+0000","last_undegraded":"2026-02-24T07:46:40.746330+0000","last_fullsized":"2026-02-24T07:46:40.746330+0000","mapping_epoch":42,"log_start":"45'18093","ondisk_log_start":"45'18093","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_clean_scrub_stamp":"2026-02-24T07:46:01.867949+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:47:23.244686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1185939456,"num_objects":18096,"num_object_clones":0,"num_object_copies":54288,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18096,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18096,"num_write_kb":1158144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"29'99","reported_seq":228,"reported_epoch":45,"state":"active+clean","last_fresh":"2026-02-24T07:46:11.479829+0000","last_change":"2026-02-24T07:45:26.125455+0000","last_active":"2026-02-24T07:46:11.479829+0000","last_peered":"2026-02-24T07:46:11.479829+0000","last_clean":"2026-02-24T07:46:11.479829+0000","last_became_active":"2026-02-24T07:42:54.186719+0000","last_became_peered":"2026-02-24T07:42:54.186719+0000","last_unstale":"2026-02-24T07:46:11.479829+0000","last_undegraded":"2026-02-24T07:46:11.479829+0000","last_fullsized":"2026-02-24T07:46:11.479829+0000","mapping_epoch":27,"log_start":"29'96","ondisk_log_start":"29'96","created":20,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'99","last_scrub_stamp":"2026-02-24T07:45:26.125426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:42:43.640776+0000","last_clean_scrub_stamp":"2026-02-24T07:45:26.125426+0000","objects_scrubbed":2,"log_size":3,"log_dups_size":96,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:46:34.126365+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":240,"num_write":135,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":16,"stat_sum":{"num_bytes":18780061696,"num_objects":286561,"num_object_clones":0,"num_object_copies":859683,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":286561,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":286561,"num_write_kb":18339904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":56326750208,"data_stored":56326750208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":49,"ondisk_log_size":49,"up":48,"acting":48,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":240,"num_write":135,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":2280,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":3,"up_from":25,"seq":107374182448,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":12430152,"kb_used_data":11233908,"kb_used_omap":9,"kb_used_meta":1196214,"kb_avail":721573048,"statfs":{"total":751619276800,"available":738890801152,"internally_reserved":0,"allocated":11503521792,"data_stored":11503233148,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9712,"internal_metadata":1224923664},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":19,"seq":81604378674,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":17437064,"kb_used_data":15975412,"kb_used_omap":8,"kb_used_meta":1461623,"kb_avail":716566136,"statfs":{"total":751619276800,"available":733763723264,"internally_reserved":0,"allocated":16358821888,"data_stored":16358533244,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":8485,"internal_metadata":1496702683},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,4],"upper_bound":4},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542195,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":13685764,"kb_used_data":12322160,"kb_used_omap":9,"kb_used_meta":1363574,"kb_avail":720317436,"statfs":{"total":751619276800,"available":737605054464,"internally_reserved":0,"allocated":12617891840,"data_stored":12617606748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9987,"internal_metadata":1396300029},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[6,2],"upper_bound":4},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705718,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":15968520,"kb_used_data":14633076,"kb_used_omap":8,"kb_used_meta":1335351,"kb_avail":718034680,"statfs":{"total":751619276800,"available":735267512320,"internally_reserved":0,"allocated":14984269824,"data_stored":14983981180,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":8362,"internal_metadata":1367400278},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,2],"upper_bound":4},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":2280,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":15237644288,"data_stored":15237644288,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12899057664,"data_stored":12899057664,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16442851328,"data_stored":16442851328,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":11747196928,"data_stored":11747196928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-24T07:46:43.792 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:46:43.790+0000 7f21b4b02640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:46:43.793 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:46:43.791+0000 7f21b4b02640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:46:43.807 INFO:teuthology.orchestra.run.trial035.stdout:osd.0: {} 2026-02-24T07:46:43.807 INFO:teuthology.orchestra.run.trial035.stderr:osd.0: osd_enable_op_tracker = '' 2026-02-24T07:46:43.823 INFO:teuthology.orchestra.run.trial035.stdout:osd.1: {} 2026-02-24T07:46:43.823 INFO:teuthology.orchestra.run.trial035.stderr:osd.1: osd_enable_op_tracker = '' 2026-02-24T07:46:43.839 INFO:teuthology.orchestra.run.trial035.stdout:osd.2: {} 2026-02-24T07:46:43.839 INFO:teuthology.orchestra.run.trial035.stderr:osd.2: osd_enable_op_tracker = '' 2026-02-24T07:46:43.851 INFO:teuthology.orchestra.run.trial035.stdout:osd.3: {} 2026-02-24T07:46:43.852 INFO:teuthology.orchestra.run.trial035.stderr:osd.3: osd_enable_op_tracker = '' 2026-02-24T07:46:43.954 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:46:43 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/1049732256' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2026-02-24T07:46:43.976 INFO:tasks.radosbench.radosbench.0.trial035.stdout:2026-02-24T07:46:43.975205+0000 min lat: 0.000742459 max lat: 0.0389727 avg lat: 0.00205234 2026-02-24T07:46:43.976 INFO:tasks.radosbench.radosbench.0.trial035.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2026-02-24T07:46:43.976 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 40 16 311689 311673 486.947 408.562 0.00225591 0.00205234 2026-02-24T07:46:44.975 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 41 16 319693 319677 487.271 500.25 0.00205451 0.00205099 2026-02-24T07:46:45.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:46:44 trial035 ceph-mon[39284]: from='client.14660 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2026-02-24T07:46:45.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:46:44 trial035 ceph-mon[39284]: pgmap v131: 17 pgs: 17 active+clean; 17 GiB data, 57 GiB used, 2.7 TiB / 2.7 TiB avail; 399 MiB/s wr, 6.38k op/s 2026-02-24T07:46:45.205 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:46:44 trial035 ceph-mon[39284]: from='mgr.14284 10.20.193.35:0/3681373222' entity='mgr.x' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2026-02-24T07:46:45.892 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2026-02-24T07:46:45.976 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 42 15 327609 327594 487.449 494.812 0.00112155 0.00205018 2026-02-24T07:46:46.040 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:46:46.153 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:46:46.151+0000 7f9d28d5f640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:46:46.153 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:46:46.151+0000 7f9d28d5f640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:46:46.167 INFO:teuthology.orchestra.run.trial035.stdout:osd.0: {} 2026-02-24T07:46:46.167 INFO:teuthology.orchestra.run.trial035.stderr:osd.0: osd_enable_op_tracker = '' 2026-02-24T07:46:46.181 INFO:teuthology.orchestra.run.trial035.stdout:osd.1: {} 2026-02-24T07:46:46.181 INFO:teuthology.orchestra.run.trial035.stderr:osd.1: osd_enable_op_tracker = '' 2026-02-24T07:46:46.194 INFO:teuthology.orchestra.run.trial035.stdout:osd.2: {} 2026-02-24T07:46:46.194 INFO:teuthology.orchestra.run.trial035.stderr:osd.2: osd_enable_op_tracker = '' 2026-02-24T07:46:46.207 INFO:teuthology.orchestra.run.trial035.stdout:osd.3: {} 2026-02-24T07:46:46.207 INFO:teuthology.orchestra.run.trial035.stderr:osd.3: osd_enable_op_tracker = '' 2026-02-24T07:46:46.976 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 43 16 335552 335536 487.656 496.375 0.00160381 0.00204937 2026-02-24T07:46:47.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:46:46 trial035 ceph-mon[39284]: pgmap v132: 17 pgs: 17 active+clean; 18 GiB data, 59 GiB used, 2.7 TiB / 2.7 TiB avail; 449 MiB/s wr, 7.19k op/s 2026-02-24T07:46:47.205 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:46:46 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/4013906522' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2026-02-24T07:46:47.205 INFO:journalctl@ceph.mgr.x.trial035.stdout:Feb 24 07:46:46 trial035 ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-mgr-x[39621]: ::ffff:10.20.193.91 - - [24/Feb/2026:07:46:46] "GET /metrics HTTP/1.1" 200 4212 "" "Prometheus/3.6.0" 2026-02-24T07:46:47.266 INFO:journalctl@ceph.prometheus.a.trial091.stdout:Feb 24 07:46:46 trial091 ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-prometheus-a[66007]: time=2026-02-24T07:46:46.973Z level=ERROR source=scrape.go:1630 msg="Failed to determine correct type of scrape target." component="scrape manager" scrape_pool=ceph target=http://trial035:9283/metrics content_type="text/html;charset=utf-8" fallback_media_type="" err="received unsupported Content-Type \"text/html;charset=utf-8\" and no fallback_scrape_protocol specified for target" 2026-02-24T07:46:47.976 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 44 16 343757 343741 488.227 512.812 0.00349802 0.00204696 2026-02-24T07:46:48.252 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2026-02-24T07:46:48.394 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:46:48.698 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2026-02-24T07:46:48.698 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2026-02-24T07:46:48.698 INFO:tasks.thrashosds.thrasher:Growing pool unique_pool_1 2026-02-24T07:46:48.698 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph pg dump --format=json 2026-02-24T07:46:48.809 INFO:teuthology.orchestra.run.trial035.stdout:osd.0: {} 2026-02-24T07:46:48.809 INFO:teuthology.orchestra.run.trial035.stdout:osd.1: {} 2026-02-24T07:46:48.809 INFO:teuthology.orchestra.run.trial035.stdout:osd.2: {} 2026-02-24T07:46:48.809 INFO:teuthology.orchestra.run.trial035.stdout:osd.3: {} 2026-02-24T07:46:48.810 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:46:48.744+0000 7fa003c40640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:46:48.810 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:46:48.744+0000 7fa003c40640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:46:48.810 INFO:teuthology.orchestra.run.trial035.stderr:osd.0: osd_enable_op_tracker = '' 2026-02-24T07:46:48.810 INFO:teuthology.orchestra.run.trial035.stderr:osd.1: osd_enable_op_tracker = '' 2026-02-24T07:46:48.810 INFO:teuthology.orchestra.run.trial035.stderr:osd.2: osd_enable_op_tracker = '' 2026-02-24T07:46:48.810 INFO:teuthology.orchestra.run.trial035.stderr:osd.3: osd_enable_op_tracker = '' 2026-02-24T07:46:48.976 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 45 16 350435 350419 486.652 417.375 0.00167545 0.00205305 2026-02-24T07:46:49.124 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:46:48 trial035 ceph-mon[39284]: pgmap v133: 17 pgs: 17 active+clean; 20 GiB data, 64 GiB used, 2.7 TiB / 2.7 TiB avail; 552 MiB/s wr, 8.83k op/s 2026-02-24T07:46:49.124 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:46:48 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/2664583708' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2026-02-24T07:46:49.507 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:46:49.976 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 46 16 356425 356409 484.21 374.375 0.0313468 0.00206338 2026-02-24T07:46:50.083 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:46:50.081+0000 7f76c76b5640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:46:50.084 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:46:50.081+0000 7f76c76b5640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:46:50.236 INFO:teuthology.orchestra.run.trial035.stdout: 2026-02-24T07:46:50.237 INFO:teuthology.orchestra.run.trial035.stderr:dumped all 2026-02-24T07:46:50.286 INFO:teuthology.orchestra.run.trial035.stdout:{"pg_ready":true,"pg_map":{"version":134,"stamp":"2026-02-24T07:46:49.688513+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":21338980896,"num_objects":325600,"num_object_clones":0,"num_object_copies":976800,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":325600,"num_whiteouts":0,"num_read":136,"num_read_kb":240,"num_write":325733,"num_write_kb":20840422,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":52,"ondisk_log_size":52,"up":51,"acting":51,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":51,"num_osds":4,"num_per_pool_osds":4,"num_per_pool_omap_osds":4,"kb":2936012800,"kb_used":67508188,"kb_used_data":61718156,"kb_used_omap":35,"kb_used_meta":5789916,"kb_avail":2868504612,"statfs":{"total":3006477107200,"available":2937348722688,"internally_reserved":0,"allocated":63199391744,"data_stored":63198240720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":36618,"internal_metadata":5928874230},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[4,2],"upper_bound":4},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":5061410816,"num_objects":77231,"num_object_clones":0,"num_object_copies":231693,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":77231,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":77231,"num_write_kb":4942784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001662"},"pg_stats":[{"pgid":"3.f","version":"45'20366","reported_seq":20382,"reported_epoch":45,"state":"active+clean","last_fresh":"2026-02-24T07:46:45.679818+0000","last_change":"2026-02-24T07:46:02.882095+0000","last_active":"2026-02-24T07:46:45.679818+0000","last_peered":"2026-02-24T07:46:45.679818+0000","last_clean":"2026-02-24T07:46:45.679818+0000","last_became_active":"2026-02-24T07:46:02.881625+0000","last_became_peered":"2026-02-24T07:46:02.881625+0000","last_unstale":"2026-02-24T07:46:45.679818+0000","last_undegraded":"2026-02-24T07:46:45.679818+0000","last_fullsized":"2026-02-24T07:46:45.679818+0000","mapping_epoch":42,"log_start":"45'20363","ondisk_log_start":"45'20363","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_clean_scrub_stamp":"2026-02-24T07:46:01.867949+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:47:09.267374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1334706176,"num_objects":20366,"num_object_clones":0,"num_object_copies":61098,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20366,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20366,"num_write_kb":1303424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.e","version":"45'20589","reported_seq":20606,"reported_epoch":45,"state":"active+clean","last_fresh":"2026-02-24T07:46:45.680410+0000","last_change":"2026-02-24T07:46:02.881462+0000","last_active":"2026-02-24T07:46:45.680410+0000","last_peered":"2026-02-24T07:46:45.680410+0000","last_clean":"2026-02-24T07:46:45.680410+0000","last_became_active":"2026-02-24T07:46:02.881083+0000","last_became_peered":"2026-02-24T07:46:02.881083+0000","last_unstale":"2026-02-24T07:46:45.680410+0000","last_undegraded":"2026-02-24T07:46:45.680410+0000","last_fullsized":"2026-02-24T07:46:45.680410+0000","mapping_epoch":42,"log_start":"45'20586","ondisk_log_start":"45'20586","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_clean_scrub_stamp":"2026-02-24T07:46:01.867949+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:47:29.030602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1349320704,"num_objects":20589,"num_object_clones":0,"num_object_copies":61767,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20589,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20589,"num_write_kb":1317696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"45'20378","reported_seq":20395,"reported_epoch":45,"state":"active+clean","last_fresh":"2026-02-24T07:46:45.705283+0000","last_change":"2026-02-24T07:46:02.883952+0000","last_active":"2026-02-24T07:46:45.705283+0000","last_peered":"2026-02-24T07:46:45.705283+0000","last_clean":"2026-02-24T07:46:45.705283+0000","last_became_active":"2026-02-24T07:46:02.883799+0000","last_became_peered":"2026-02-24T07:46:02.883799+0000","last_unstale":"2026-02-24T07:46:45.705283+0000","last_undegraded":"2026-02-24T07:46:45.705283+0000","last_fullsized":"2026-02-24T07:46:45.705283+0000","mapping_epoch":42,"log_start":"45'20375","ondisk_log_start":"45'20375","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_clean_scrub_stamp":"2026-02-24T07:46:01.867949+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:47:02.402624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1335492608,"num_objects":20378,"num_object_clones":0,"num_object_copies":61134,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20378,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20378,"num_write_kb":1304192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"45'20083","reported_seq":20100,"reported_epoch":45,"state":"active+clean","last_fresh":"2026-02-24T07:46:45.706672+0000","last_change":"2026-02-24T07:46:02.880375+0000","last_active":"2026-02-24T07:46:45.706672+0000","last_peered":"2026-02-24T07:46:45.706672+0000","last_clean":"2026-02-24T07:46:45.706672+0000","last_became_active":"2026-02-24T07:46:02.880060+0000","last_became_peered":"2026-02-24T07:46:02.880060+0000","last_unstale":"2026-02-24T07:46:45.706672+0000","last_undegraded":"2026-02-24T07:46:45.706672+0000","last_fullsized":"2026-02-24T07:46:45.706672+0000","mapping_epoch":42,"log_start":"45'20080","ondisk_log_start":"45'20080","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_clean_scrub_stamp":"2026-02-24T07:46:01.867949+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:47:04.123135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1316159488,"num_objects":20083,"num_object_clones":0,"num_object_copies":60249,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20083,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20083,"num_write_kb":1285312,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"45'20368","reported_seq":20384,"reported_epoch":45,"state":"active+clean","last_fresh":"2026-02-24T07:46:45.747212+0000","last_change":"2026-02-24T07:46:02.880509+0000","last_active":"2026-02-24T07:46:45.747212+0000","last_peered":"2026-02-24T07:46:45.747212+0000","last_clean":"2026-02-24T07:46:45.747212+0000","last_became_active":"2026-02-24T07:46:02.880171+0000","last_became_peered":"2026-02-24T07:46:02.880171+0000","last_unstale":"2026-02-24T07:46:45.747212+0000","last_undegraded":"2026-02-24T07:46:45.747212+0000","last_fullsized":"2026-02-24T07:46:45.747212+0000","mapping_epoch":42,"log_start":"45'20365","ondisk_log_start":"45'20365","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_clean_scrub_stamp":"2026-02-24T07:46:01.867949+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:47:06.105346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1334837248,"num_objects":20368,"num_object_clones":0,"num_object_copies":61104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20368,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20368,"num_write_kb":1303552,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.a","version":"45'20205","reported_seq":20222,"reported_epoch":45,"state":"active+clean","last_fresh":"2026-02-24T07:46:45.706058+0000","last_change":"2026-02-24T07:46:02.882761+0000","last_active":"2026-02-24T07:46:45.706058+0000","last_peered":"2026-02-24T07:46:45.706058+0000","last_clean":"2026-02-24T07:46:45.706058+0000","last_became_active":"2026-02-24T07:46:02.882274+0000","last_became_peered":"2026-02-24T07:46:02.882274+0000","last_unstale":"2026-02-24T07:46:45.706058+0000","last_undegraded":"2026-02-24T07:46:45.706058+0000","last_fullsized":"2026-02-24T07:46:45.706058+0000","mapping_epoch":42,"log_start":"45'20202","ondisk_log_start":"45'20202","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_clean_scrub_stamp":"2026-02-24T07:46:01.867949+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:47:31.776406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1324154880,"num_objects":20205,"num_object_clones":0,"num_object_copies":60615,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20205,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20205,"num_write_kb":1293120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"45'20555","reported_seq":20571,"reported_epoch":45,"state":"active+clean","last_fresh":"2026-02-24T07:46:45.706753+0000","last_change":"2026-02-24T07:46:02.882303+0000","last_active":"2026-02-24T07:46:45.706753+0000","last_peered":"2026-02-24T07:46:45.706753+0000","last_clean":"2026-02-24T07:46:45.706753+0000","last_became_active":"2026-02-24T07:46:02.881951+0000","last_became_peered":"2026-02-24T07:46:02.881951+0000","last_unstale":"2026-02-24T07:46:45.706753+0000","last_undegraded":"2026-02-24T07:46:45.706753+0000","last_fullsized":"2026-02-24T07:46:45.706753+0000","mapping_epoch":42,"log_start":"45'20552","ondisk_log_start":"45'20552","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_clean_scrub_stamp":"2026-02-24T07:46:01.867949+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:47:04.345724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1347092480,"num_objects":20555,"num_object_clones":0,"num_object_copies":61665,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20555,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20555,"num_write_kb":1315520,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"45'20366","reported_seq":20383,"reported_epoch":45,"state":"active+clean","last_fresh":"2026-02-24T07:46:45.744101+0000","last_change":"2026-02-24T07:46:02.881463+0000","last_active":"2026-02-24T07:46:45.744101+0000","last_peered":"2026-02-24T07:46:45.744101+0000","last_clean":"2026-02-24T07:46:45.744101+0000","last_became_active":"2026-02-24T07:46:02.881345+0000","last_became_peered":"2026-02-24T07:46:02.881345+0000","last_unstale":"2026-02-24T07:46:45.744101+0000","last_undegraded":"2026-02-24T07:46:45.744101+0000","last_fullsized":"2026-02-24T07:46:45.744101+0000","mapping_epoch":42,"log_start":"45'20363","ondisk_log_start":"45'20363","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_clean_scrub_stamp":"2026-02-24T07:46:01.867949+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:47:17.135287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1334706176,"num_objects":20366,"num_object_clones":0,"num_object_copies":61098,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20366,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20366,"num_write_kb":1303424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"45'20198","reported_seq":20215,"reported_epoch":45,"state":"active+clean","last_fresh":"2026-02-24T07:46:45.739475+0000","last_change":"2026-02-24T07:46:02.880582+0000","last_active":"2026-02-24T07:46:45.739475+0000","last_peered":"2026-02-24T07:46:45.739475+0000","last_clean":"2026-02-24T07:46:45.739475+0000","last_became_active":"2026-02-24T07:46:02.880323+0000","last_became_peered":"2026-02-24T07:46:02.880323+0000","last_unstale":"2026-02-24T07:46:45.739475+0000","last_undegraded":"2026-02-24T07:46:45.739475+0000","last_fullsized":"2026-02-24T07:46:45.739475+0000","mapping_epoch":42,"log_start":"45'20195","ondisk_log_start":"45'20195","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_clean_scrub_stamp":"2026-02-24T07:46:01.867949+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:47:31.420697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1323696128,"num_objects":20198,"num_object_clones":0,"num_object_copies":60594,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20198,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20198,"num_write_kb":1292672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.6","version":"45'20488","reported_seq":20505,"reported_epoch":45,"state":"active+clean","last_fresh":"2026-02-24T07:46:45.701101+0000","last_change":"2026-02-24T07:46:02.880170+0000","last_active":"2026-02-24T07:46:45.701101+0000","last_peered":"2026-02-24T07:46:45.701101+0000","last_clean":"2026-02-24T07:46:45.701101+0000","last_became_active":"2026-02-24T07:46:02.880005+0000","last_became_peered":"2026-02-24T07:46:02.880005+0000","last_unstale":"2026-02-24T07:46:45.701101+0000","last_undegraded":"2026-02-24T07:46:45.701101+0000","last_fullsized":"2026-02-24T07:46:45.701101+0000","mapping_epoch":42,"log_start":"45'20485","ondisk_log_start":"45'20485","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_clean_scrub_stamp":"2026-02-24T07:46:01.867949+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:47:14.923924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1342701568,"num_objects":20488,"num_object_clones":0,"num_object_copies":61464,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20488,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20488,"num_write_kb":1311232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"45'20474","reported_seq":20491,"reported_epoch":45,"state":"active+clean","last_fresh":"2026-02-24T07:46:45.678049+0000","last_change":"2026-02-24T07:46:02.881931+0000","last_active":"2026-02-24T07:46:45.678049+0000","last_peered":"2026-02-24T07:46:45.678049+0000","last_clean":"2026-02-24T07:46:45.678049+0000","last_became_active":"2026-02-24T07:46:02.881420+0000","last_became_peered":"2026-02-24T07:46:02.881420+0000","last_unstale":"2026-02-24T07:46:45.678049+0000","last_undegraded":"2026-02-24T07:46:45.678049+0000","last_fullsized":"2026-02-24T07:46:45.678049+0000","mapping_epoch":42,"log_start":"45'20471","ondisk_log_start":"45'20471","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_clean_scrub_stamp":"2026-02-24T07:46:01.867949+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:47:09.535393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1341784064,"num_objects":20474,"num_object_clones":0,"num_object_copies":61422,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20474,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20474,"num_write_kb":1310336,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.4","version":"45'20059","reported_seq":20073,"reported_epoch":45,"state":"active+clean","last_fresh":"2026-02-24T07:46:45.749610+0000","last_change":"2026-02-24T07:46:02.880661+0000","last_active":"2026-02-24T07:46:45.749610+0000","last_peered":"2026-02-24T07:46:45.749610+0000","last_clean":"2026-02-24T07:46:45.749610+0000","last_became_active":"2026-02-24T07:46:02.880345+0000","last_became_peered":"2026-02-24T07:46:02.880345+0000","last_unstale":"2026-02-24T07:46:45.749610+0000","last_undegraded":"2026-02-24T07:46:45.749610+0000","last_fullsized":"2026-02-24T07:46:45.749610+0000","mapping_epoch":42,"log_start":"45'20055","ondisk_log_start":"45'20055","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_clean_scrub_stamp":"2026-02-24T07:46:01.867949+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":3000,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:47:27.577904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1314586624,"num_objects":20059,"num_object_clones":0,"num_object_copies":60177,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20059,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20059,"num_write_kb":1283776,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"45'20281","reported_seq":20298,"reported_epoch":45,"state":"active+clean","last_fresh":"2026-02-24T07:46:45.679377+0000","last_change":"2026-02-24T07:46:02.881591+0000","last_active":"2026-02-24T07:46:45.679377+0000","last_peered":"2026-02-24T07:46:45.679377+0000","last_clean":"2026-02-24T07:46:45.679377+0000","last_became_active":"2026-02-24T07:46:02.881035+0000","last_became_peered":"2026-02-24T07:46:02.881035+0000","last_unstale":"2026-02-24T07:46:45.679377+0000","last_undegraded":"2026-02-24T07:46:45.679377+0000","last_fullsized":"2026-02-24T07:46:45.679377+0000","mapping_epoch":42,"log_start":"45'20278","ondisk_log_start":"45'20278","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_clean_scrub_stamp":"2026-02-24T07:46:01.867949+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:47:05.565071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1329135616,"num_objects":20281,"num_object_clones":0,"num_object_copies":60843,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20281,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20281,"num_write_kb":1297984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"45'20447","reported_seq":20464,"reported_epoch":45,"state":"active+clean","last_fresh":"2026-02-24T07:46:45.700220+0000","last_change":"2026-02-24T07:46:02.879955+0000","last_active":"2026-02-24T07:46:45.700220+0000","last_peered":"2026-02-24T07:46:45.700220+0000","last_clean":"2026-02-24T07:46:45.700220+0000","last_became_active":"2026-02-24T07:46:02.879831+0000","last_became_peered":"2026-02-24T07:46:02.879831+0000","last_unstale":"2026-02-24T07:46:45.700220+0000","last_undegraded":"2026-02-24T07:46:45.700220+0000","last_fullsized":"2026-02-24T07:46:45.700220+0000","mapping_epoch":42,"log_start":"45'20444","ondisk_log_start":"45'20444","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_clean_scrub_stamp":"2026-02-24T07:46:01.867949+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:47:19.157247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1340014592,"num_objects":20447,"num_object_clones":0,"num_object_copies":61341,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20447,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20447,"num_write_kb":1308608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.0","version":"45'20203","reported_seq":20219,"reported_epoch":45,"state":"active+clean","last_fresh":"2026-02-24T07:46:45.748302+0000","last_change":"2026-02-24T07:46:02.880585+0000","last_active":"2026-02-24T07:46:45.748302+0000","last_peered":"2026-02-24T07:46:45.748302+0000","last_clean":"2026-02-24T07:46:45.748302+0000","last_became_active":"2026-02-24T07:46:02.880333+0000","last_became_peered":"2026-02-24T07:46:02.880333+0000","last_unstale":"2026-02-24T07:46:45.748302+0000","last_undegraded":"2026-02-24T07:46:45.748302+0000","last_fullsized":"2026-02-24T07:46:45.748302+0000","mapping_epoch":42,"log_start":"45'20200","ondisk_log_start":"45'20200","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_clean_scrub_stamp":"2026-02-24T07:46:01.867949+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:47:04.574448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1324023808,"num_objects":20203,"num_object_clones":0,"num_object_copies":60609,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20203,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20203,"num_write_kb":1292992,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2","version":"45'20538","reported_seq":20555,"reported_epoch":45,"state":"active+clean","last_fresh":"2026-02-24T07:46:45.749676+0000","last_change":"2026-02-24T07:46:02.880566+0000","last_active":"2026-02-24T07:46:45.749676+0000","last_peered":"2026-02-24T07:46:45.749676+0000","last_clean":"2026-02-24T07:46:45.749676+0000","last_became_active":"2026-02-24T07:46:02.880185+0000","last_became_peered":"2026-02-24T07:46:02.880185+0000","last_unstale":"2026-02-24T07:46:45.749676+0000","last_undegraded":"2026-02-24T07:46:45.749676+0000","last_fullsized":"2026-02-24T07:46:45.749676+0000","mapping_epoch":42,"log_start":"45'20535","ondisk_log_start":"45'20535","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_clean_scrub_stamp":"2026-02-24T07:46:01.867949+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:47:23.244686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1345978368,"num_objects":20538,"num_object_clones":0,"num_object_copies":61614,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20538,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20538,"num_write_kb":1314432,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"29'99","reported_seq":236,"reported_epoch":45,"state":"active+clean","last_fresh":"2026-02-24T07:46:41.007499+0000","last_change":"2026-02-24T07:46:41.007499+0000","last_active":"2026-02-24T07:46:41.007499+0000","last_peered":"2026-02-24T07:46:41.007499+0000","last_clean":"2026-02-24T07:46:41.007499+0000","last_became_active":"2026-02-24T07:42:54.186719+0000","last_became_peered":"2026-02-24T07:42:54.186719+0000","last_unstale":"2026-02-24T07:46:41.007499+0000","last_undegraded":"2026-02-24T07:46:41.007499+0000","last_fullsized":"2026-02-24T07:46:41.007499+0000","mapping_epoch":27,"log_start":"29'96","ondisk_log_start":"29'96","created":20,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'99","last_scrub_stamp":"2026-02-24T07:46:41.007434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:42:43.640776+0000","last_clean_scrub_stamp":"2026-02-24T07:46:41.007434+0000","objects_scrubbed":2,"log_size":3,"log_dups_size":96,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:47:53.486511+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":240,"num_write":135,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":16,"stat_sum":{"num_bytes":21338390528,"num_objects":325598,"num_object_clones":0,"num_object_copies":976794,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":325598,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":325598,"num_write_kb":20838272,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":64000229376,"data_stored":64000229376,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":49,"ondisk_log_size":49,"up":48,"acting":48,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":240,"num_write":135,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":2120,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":3,"up_from":25,"seq":107374182449,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":14082632,"kb_used_data":12814196,"kb_used_omap":9,"kb_used_meta":1268406,"kb_avail":719920568,"statfs":{"total":751619276800,"available":737198661632,"internally_reserved":0,"allocated":13121736704,"data_stored":13121448060,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9784,"internal_metadata":1298848200},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":19,"seq":81604378675,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":19721416,"kb_used_data":18151476,"kb_used_omap":8,"kb_used_meta":1569911,"kb_avail":714281784,"statfs":{"total":751619276800,"available":731424546816,"internally_reserved":0,"allocated":18587111424,"data_stored":18586822780,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":8485,"internal_metadata":1607589595},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,2],"upper_bound":4},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542196,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":15528644,"kb_used_data":14074800,"kb_used_omap":9,"kb_used_meta":1453814,"kb_avail":718474556,"statfs":{"total":751619276800,"available":735717945344,"internally_reserved":0,"allocated":14412595200,"data_stored":14412310108,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9987,"internal_metadata":1488705789},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705719,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":18175496,"kb_used_data":16677684,"kb_used_omap":8,"kb_used_meta":1497783,"kb_avail":715827704,"statfs":{"total":751619276800,"available":733007568896,"internally_reserved":0,"allocated":17077948416,"data_stored":17077659772,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":8362,"internal_metadata":1533730646},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":2120,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":17312514048,"data_stored":17312514048,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":14657388544,"data_stored":14657388544,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":18683133952,"data_stored":18683133952,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":13347192832,"data_stored":13347192832,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-24T07:46:50.288 INFO:tasks.cephadm.ceph_manager.ceph:increase pool size by 10 2026-02-24T07:46:50.288 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph osd pool set unique_pool_1 pg_num 36 2026-02-24T07:46:50.440 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:46:50.569 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:46:50.566+0000 7fd52632b640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:46:50.569 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:46:50.566+0000 7fd52632b640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:46:50.719 INFO:teuthology.orchestra.run.trial035.stderr:Error EPERM: crimson_allow_pg_split is false; pg_num increase denied for crimson pool 2026-02-24T07:46:50.793 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-24T07:46:50.976 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 47 16 364271 364255 484.34 490.375 0.00107798 0.00206341 2026-02-24T07:46:51.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:46:50 trial035 ceph-mon[39284]: pgmap v134: 17 pgs: 17 active+clean; 20 GiB data, 64 GiB used, 2.7 TiB / 2.7 TiB avail; 402 MiB/s wr, 6.43k op/s 2026-02-24T07:46:51.205 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:46:50 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/3511945597' entity='client.admin' cmd={"prefix": "osd pool set", "pool": "unique_pool_1", "var": "pg_num", "val": "36"} : dispatch 2026-02-24T07:46:51.593 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2026-02-24T07:46:51.743 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:46:51.874 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:46:51.871+0000 7f49f7397640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:46:51.875 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:46:51.872+0000 7f49f7397640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:46:51.889 INFO:teuthology.orchestra.run.trial035.stdout:osd.0: {} 2026-02-24T07:46:51.890 INFO:teuthology.orchestra.run.trial035.stderr:osd.0: osd_enable_op_tracker = '' 2026-02-24T07:46:51.907 INFO:teuthology.orchestra.run.trial035.stdout:osd.1: {} 2026-02-24T07:46:51.907 INFO:teuthology.orchestra.run.trial035.stderr:osd.1: osd_enable_op_tracker = '' 2026-02-24T07:46:51.920 INFO:teuthology.orchestra.run.trial035.stdout:osd.2: {} 2026-02-24T07:46:51.920 INFO:teuthology.orchestra.run.trial035.stderr:osd.2: osd_enable_op_tracker = '' 2026-02-24T07:46:51.933 INFO:teuthology.orchestra.run.trial035.stdout:osd.3: {} 2026-02-24T07:46:51.933 INFO:teuthology.orchestra.run.trial035.stderr:osd.3: osd_enable_op_tracker = '' 2026-02-24T07:46:51.976 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 48 16 372286 372270 484.685 500.938 0.00155648 0.00206194 2026-02-24T07:46:52.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:46:51 trial035 ceph-mon[39284]: from='client.14668 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2026-02-24T07:46:52.976 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 49 16 380470 380454 485.232 511.5 0.00159973 0.00205961 2026-02-24T07:46:53.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:46:52 trial035 ceph-mon[39284]: pgmap v135: 17 pgs: 17 active+clean; 22 GiB data, 72 GiB used, 2.7 TiB / 2.7 TiB avail; 593 MiB/s wr, 9.48k op/s 2026-02-24T07:46:53.205 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:46:52 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/3959244925' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2026-02-24T07:46:53.975 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2026-02-24T07:46:53.977 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 50 15 388382 388367 485.418 494.562 0.00215438 0.00205884 2026-02-24T07:46:54.118 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:46:54.248 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:46:54.245+0000 7f473b4f7640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:46:54.249 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:46:54.246+0000 7f473b4f7640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:46:54.262 INFO:teuthology.orchestra.run.trial035.stdout:osd.0: {} 2026-02-24T07:46:54.263 INFO:teuthology.orchestra.run.trial035.stderr:osd.0: osd_enable_op_tracker = '' 2026-02-24T07:46:54.275 INFO:teuthology.orchestra.run.trial035.stdout:osd.1: {} 2026-02-24T07:46:54.276 INFO:teuthology.orchestra.run.trial035.stderr:osd.1: osd_enable_op_tracker = '' 2026-02-24T07:46:54.288 INFO:teuthology.orchestra.run.trial035.stdout:osd.2: {} 2026-02-24T07:46:54.288 INFO:teuthology.orchestra.run.trial035.stderr:osd.2: osd_enable_op_tracker = '' 2026-02-24T07:46:54.299 INFO:teuthology.orchestra.run.trial035.stdout:osd.3: {} 2026-02-24T07:46:54.300 INFO:teuthology.orchestra.run.trial035.stderr:osd.3: osd_enable_op_tracker = '' 2026-02-24T07:46:54.977 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 51 16 396457 396441 485.794 504.625 0.00131709 0.00205723 2026-02-24T07:46:55.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:46:54 trial035 ceph-mon[39284]: pgmap v136: 17 pgs: 17 active+clean; 22 GiB data, 72 GiB used, 2.7 TiB / 2.7 TiB avail; 394 MiB/s wr, 6.30k op/s 2026-02-24T07:46:55.205 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:46:54 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/1749755125' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2026-02-24T07:46:55.794 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2026-02-24T07:46:55.794 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2026-02-24T07:46:55.794 INFO:tasks.thrashosds.thrasher:fixing pg num pool unique_pool_1 2026-02-24T07:46:55.795 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph pg dump --format=json 2026-02-24T07:46:55.941 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:46:55.976 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 52 15 404555 404540 486.185 506.188 0.00214635 0.00205558 2026-02-24T07:46:56.113 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:46:55 trial035 ceph-mon[39284]: pgmap v137: 17 pgs: 17 active+clean; 23 GiB data, 74 GiB used, 2.7 TiB / 2.7 TiB avail; 447 MiB/s wr, 7.15k op/s 2026-02-24T07:46:56.341 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2026-02-24T07:46:56.556 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:46:56.469+0000 7f4305b66640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:46:56.556 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:46:56.469+0000 7f4305b66640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:46:56.561 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:46:56.619 INFO:teuthology.orchestra.run.trial035.stdout: 2026-02-24T07:46:56.620 INFO:teuthology.orchestra.run.trial035.stderr:dumped all 2026-02-24T07:46:56.977 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 53 16 410975 410959 484.581 401.188 0.026384 0.00206175 2026-02-24T07:46:57.125 INFO:teuthology.orchestra.run.trial035.stdout:osd.0: {} 2026-02-24T07:46:57.125 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:46:57.097+0000 7f86172a9640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:46:57.125 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:46:57.097+0000 7f86172a9640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:46:57.125 INFO:teuthology.orchestra.run.trial035.stderr:osd.0: osd_enable_op_tracker = '' 2026-02-24T07:46:57.136 INFO:teuthology.orchestra.run.trial035.stdout:osd.1: {} 2026-02-24T07:46:57.136 INFO:teuthology.orchestra.run.trial035.stderr:osd.1: osd_enable_op_tracker = '' 2026-02-24T07:46:57.153 INFO:teuthology.orchestra.run.trial035.stdout:osd.2: {} 2026-02-24T07:46:57.153 INFO:teuthology.orchestra.run.trial035.stderr:osd.2: osd_enable_op_tracker = '' 2026-02-24T07:46:57.159 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:46:56 trial035 ceph-mon[39284]: from='client.14676 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2026-02-24T07:46:57.159 INFO:journalctl@ceph.mgr.x.trial035.stdout:Feb 24 07:46:56 trial035 ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-mgr-x[39621]: ::ffff:10.20.193.91 - - [24/Feb/2026:07:46:56] "GET /metrics HTTP/1.1" 200 4212 "" "Prometheus/3.6.0" 2026-02-24T07:46:57.166 INFO:teuthology.orchestra.run.trial035.stdout:osd.3: {} 2026-02-24T07:46:57.166 INFO:teuthology.orchestra.run.trial035.stderr:osd.3: osd_enable_op_tracker = '' 2026-02-24T07:46:57.196 INFO:teuthology.orchestra.run.trial035.stdout:{"pg_ready":true,"pg_map":{"version":137,"stamp":"2026-02-24T07:46:55.689291+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":24405738016,"num_objects":372395,"num_object_clones":0,"num_object_copies":1117185,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":372395,"num_whiteouts":0,"num_read":136,"num_read_kb":240,"num_write":372528,"num_write_kb":23835302,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":53,"ondisk_log_size":53,"up":51,"acting":51,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":51,"num_osds":4,"num_per_pool_osds":4,"num_per_pool_omap_osds":4,"kb":2936012800,"kb_used":77465756,"kb_used_data":71062092,"kb_used_omap":35,"kb_used_meta":6403548,"kb_avail":2858547044,"statfs":{"total":3006477107200,"available":2927152173056,"internally_reserved":0,"allocated":72767582208,"data_stored":72766431184,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":36618,"internal_metadata":6557233398},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,2,4],"upper_bound":8},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":5625085952,"num_objects":85832,"num_object_clones":0,"num_object_copies":257496,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":85832,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":85832,"num_write_kb":5493248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001575"},"pg_stats":[{"pgid":"3.f","version":"45'25127","reported_seq":25144,"reported_epoch":45,"state":"active+clean","last_fresh":"2026-02-24T07:46:55.682877+0000","last_change":"2026-02-24T07:46:02.882095+0000","last_active":"2026-02-24T07:46:55.682877+0000","last_peered":"2026-02-24T07:46:55.682877+0000","last_clean":"2026-02-24T07:46:55.682877+0000","last_became_active":"2026-02-24T07:46:02.881625+0000","last_became_peered":"2026-02-24T07:46:02.881625+0000","last_unstale":"2026-02-24T07:46:55.682877+0000","last_undegraded":"2026-02-24T07:46:55.682877+0000","last_fullsized":"2026-02-24T07:46:55.682877+0000","mapping_epoch":42,"log_start":"45'25124","ondisk_log_start":"45'25124","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_clean_scrub_stamp":"2026-02-24T07:46:01.867949+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:47:09.267374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1646723072,"num_objects":25127,"num_object_clones":0,"num_object_copies":75381,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25127,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":25127,"num_write_kb":1608128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.e","version":"45'25490","reported_seq":25506,"reported_epoch":45,"state":"active+clean","last_fresh":"2026-02-24T07:46:55.679406+0000","last_change":"2026-02-24T07:46:02.881462+0000","last_active":"2026-02-24T07:46:55.679406+0000","last_peered":"2026-02-24T07:46:55.679406+0000","last_clean":"2026-02-24T07:46:55.679406+0000","last_became_active":"2026-02-24T07:46:02.881083+0000","last_became_peered":"2026-02-24T07:46:02.881083+0000","last_unstale":"2026-02-24T07:46:55.679406+0000","last_undegraded":"2026-02-24T07:46:55.679406+0000","last_fullsized":"2026-02-24T07:46:55.679406+0000","mapping_epoch":42,"log_start":"45'25487","ondisk_log_start":"45'25487","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_clean_scrub_stamp":"2026-02-24T07:46:01.867949+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:47:29.030602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1670512640,"num_objects":25490,"num_object_clones":0,"num_object_copies":76470,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25490,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":25490,"num_write_kb":1631360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"45'22606","reported_seq":22623,"reported_epoch":45,"state":"active+clean","last_fresh":"2026-02-24T07:46:50.700711+0000","last_change":"2026-02-24T07:46:02.883952+0000","last_active":"2026-02-24T07:46:50.700711+0000","last_peered":"2026-02-24T07:46:50.700711+0000","last_clean":"2026-02-24T07:46:50.700711+0000","last_became_active":"2026-02-24T07:46:02.883799+0000","last_became_peered":"2026-02-24T07:46:02.883799+0000","last_unstale":"2026-02-24T07:46:50.700711+0000","last_undegraded":"2026-02-24T07:46:50.700711+0000","last_fullsized":"2026-02-24T07:46:50.700711+0000","mapping_epoch":42,"log_start":"45'22603","ondisk_log_start":"45'22603","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_clean_scrub_stamp":"2026-02-24T07:46:01.867949+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:47:02.402624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1481506816,"num_objects":22606,"num_object_clones":0,"num_object_copies":67818,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22606,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22606,"num_write_kb":1446784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"45'22329","reported_seq":22346,"reported_epoch":45,"state":"active+clean","last_fresh":"2026-02-24T07:46:50.704911+0000","last_change":"2026-02-24T07:46:02.880375+0000","last_active":"2026-02-24T07:46:50.704911+0000","last_peered":"2026-02-24T07:46:50.704911+0000","last_clean":"2026-02-24T07:46:50.704911+0000","last_became_active":"2026-02-24T07:46:02.880060+0000","last_became_peered":"2026-02-24T07:46:02.880060+0000","last_unstale":"2026-02-24T07:46:50.704911+0000","last_undegraded":"2026-02-24T07:46:50.704911+0000","last_fullsized":"2026-02-24T07:46:50.704911+0000","mapping_epoch":42,"log_start":"45'22326","ondisk_log_start":"45'22326","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_clean_scrub_stamp":"2026-02-24T07:46:01.867949+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:47:04.123135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1463353344,"num_objects":22329,"num_object_clones":0,"num_object_copies":66987,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22329,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22329,"num_write_kb":1429056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"45'22610","reported_seq":22627,"reported_epoch":45,"state":"active+clean","last_fresh":"2026-02-24T07:46:50.747232+0000","last_change":"2026-02-24T07:46:02.880509+0000","last_active":"2026-02-24T07:46:50.747232+0000","last_peered":"2026-02-24T07:46:50.747232+0000","last_clean":"2026-02-24T07:46:50.747232+0000","last_became_active":"2026-02-24T07:46:02.880171+0000","last_became_peered":"2026-02-24T07:46:02.880171+0000","last_unstale":"2026-02-24T07:46:50.747232+0000","last_undegraded":"2026-02-24T07:46:50.747232+0000","last_fullsized":"2026-02-24T07:46:50.747232+0000","mapping_epoch":42,"log_start":"45'22607","ondisk_log_start":"45'22607","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_clean_scrub_stamp":"2026-02-24T07:46:01.867949+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:47:06.105346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1481768960,"num_objects":22610,"num_object_clones":0,"num_object_copies":67830,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22610,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22610,"num_write_kb":1447040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.a","version":"45'22472","reported_seq":22489,"reported_epoch":45,"state":"active+clean","last_fresh":"2026-02-24T07:46:50.704509+0000","last_change":"2026-02-24T07:46:02.882761+0000","last_active":"2026-02-24T07:46:50.704509+0000","last_peered":"2026-02-24T07:46:50.704509+0000","last_clean":"2026-02-24T07:46:50.704509+0000","last_became_active":"2026-02-24T07:46:02.882274+0000","last_became_peered":"2026-02-24T07:46:02.882274+0000","last_unstale":"2026-02-24T07:46:50.704509+0000","last_undegraded":"2026-02-24T07:46:50.704509+0000","last_fullsized":"2026-02-24T07:46:50.704509+0000","mapping_epoch":42,"log_start":"45'22469","ondisk_log_start":"45'22469","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_clean_scrub_stamp":"2026-02-24T07:46:01.867949+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:47:31.776406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1472724992,"num_objects":22472,"num_object_clones":0,"num_object_copies":67416,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22472,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22472,"num_write_kb":1438208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"45'22955","reported_seq":22972,"reported_epoch":45,"state":"active+clean","last_fresh":"2026-02-24T07:46:50.705894+0000","last_change":"2026-02-24T07:46:02.882303+0000","last_active":"2026-02-24T07:46:50.705894+0000","last_peered":"2026-02-24T07:46:50.705894+0000","last_clean":"2026-02-24T07:46:50.705894+0000","last_became_active":"2026-02-24T07:46:02.881951+0000","last_became_peered":"2026-02-24T07:46:02.881951+0000","last_unstale":"2026-02-24T07:46:50.705894+0000","last_undegraded":"2026-02-24T07:46:50.705894+0000","last_fullsized":"2026-02-24T07:46:50.705894+0000","mapping_epoch":42,"log_start":"45'22952","ondisk_log_start":"45'22952","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_clean_scrub_stamp":"2026-02-24T07:46:01.867949+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:47:04.345724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1504378880,"num_objects":22955,"num_object_clones":0,"num_object_copies":68865,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22955,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22955,"num_write_kb":1469120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"45'22649","reported_seq":22666,"reported_epoch":45,"state":"active+clean","last_fresh":"2026-02-24T07:46:50.747109+0000","last_change":"2026-02-24T07:46:02.881463+0000","last_active":"2026-02-24T07:46:50.747109+0000","last_peered":"2026-02-24T07:46:50.747109+0000","last_clean":"2026-02-24T07:46:50.747109+0000","last_became_active":"2026-02-24T07:46:02.881345+0000","last_became_peered":"2026-02-24T07:46:02.881345+0000","last_unstale":"2026-02-24T07:46:50.747109+0000","last_undegraded":"2026-02-24T07:46:50.747109+0000","last_fullsized":"2026-02-24T07:46:50.747109+0000","mapping_epoch":42,"log_start":"45'22646","ondisk_log_start":"45'22646","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_clean_scrub_stamp":"2026-02-24T07:46:01.867949+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:47:17.135287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1484324864,"num_objects":22649,"num_object_clones":0,"num_object_copies":67947,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22649,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22649,"num_write_kb":1449536,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"45'22520","reported_seq":22537,"reported_epoch":45,"state":"active+clean","last_fresh":"2026-02-24T07:46:50.743838+0000","last_change":"2026-02-24T07:46:02.880582+0000","last_active":"2026-02-24T07:46:50.743838+0000","last_peered":"2026-02-24T07:46:50.743838+0000","last_clean":"2026-02-24T07:46:50.743838+0000","last_became_active":"2026-02-24T07:46:02.880323+0000","last_became_peered":"2026-02-24T07:46:02.880323+0000","last_unstale":"2026-02-24T07:46:50.743838+0000","last_undegraded":"2026-02-24T07:46:50.743838+0000","last_fullsized":"2026-02-24T07:46:50.743838+0000","mapping_epoch":42,"log_start":"45'22517","ondisk_log_start":"45'22517","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_clean_scrub_stamp":"2026-02-24T07:46:01.867949+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:47:31.420697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1475870720,"num_objects":22520,"num_object_clones":0,"num_object_copies":67560,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22520,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22520,"num_write_kb":1441280,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.6","version":"45'22852","reported_seq":22869,"reported_epoch":45,"state":"active+clean","last_fresh":"2026-02-24T07:46:50.703853+0000","last_change":"2026-02-24T07:46:02.880170+0000","last_active":"2026-02-24T07:46:50.703853+0000","last_peered":"2026-02-24T07:46:50.703853+0000","last_clean":"2026-02-24T07:46:50.703853+0000","last_became_active":"2026-02-24T07:46:02.880005+0000","last_became_peered":"2026-02-24T07:46:02.880005+0000","last_unstale":"2026-02-24T07:46:50.703853+0000","last_undegraded":"2026-02-24T07:46:50.703853+0000","last_fullsized":"2026-02-24T07:46:50.703853+0000","mapping_epoch":42,"log_start":"45'22847","ondisk_log_start":"45'22847","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_clean_scrub_stamp":"2026-02-24T07:46:01.867949+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":3000,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:47:14.923924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1497628672,"num_objects":22852,"num_object_clones":0,"num_object_copies":68556,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22852,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22852,"num_write_kb":1462528,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"45'25288","reported_seq":25305,"reported_epoch":45,"state":"active+clean","last_fresh":"2026-02-24T07:46:55.682733+0000","last_change":"2026-02-24T07:46:02.881931+0000","last_active":"2026-02-24T07:46:55.682733+0000","last_peered":"2026-02-24T07:46:55.682733+0000","last_clean":"2026-02-24T07:46:55.682733+0000","last_became_active":"2026-02-24T07:46:02.881420+0000","last_became_peered":"2026-02-24T07:46:02.881420+0000","last_unstale":"2026-02-24T07:46:55.682733+0000","last_undegraded":"2026-02-24T07:46:55.682733+0000","last_fullsized":"2026-02-24T07:46:55.682733+0000","mapping_epoch":42,"log_start":"45'25285","ondisk_log_start":"45'25285","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_clean_scrub_stamp":"2026-02-24T07:46:01.867949+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:47:09.535393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1657274368,"num_objects":25288,"num_object_clones":0,"num_object_copies":75864,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25288,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":25288,"num_write_kb":1618432,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.4","version":"45'22351","reported_seq":22367,"reported_epoch":45,"state":"active+clean","last_fresh":"2026-02-24T07:46:50.750156+0000","last_change":"2026-02-24T07:46:02.880661+0000","last_active":"2026-02-24T07:46:50.750156+0000","last_peered":"2026-02-24T07:46:50.750156+0000","last_clean":"2026-02-24T07:46:50.750156+0000","last_became_active":"2026-02-24T07:46:02.880345+0000","last_became_peered":"2026-02-24T07:46:02.880345+0000","last_unstale":"2026-02-24T07:46:50.750156+0000","last_undegraded":"2026-02-24T07:46:50.750156+0000","last_fullsized":"2026-02-24T07:46:50.750156+0000","mapping_epoch":42,"log_start":"45'22348","ondisk_log_start":"45'22348","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_clean_scrub_stamp":"2026-02-24T07:46:01.867949+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:47:27.577904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1464795136,"num_objects":22351,"num_object_clones":0,"num_object_copies":67053,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22351,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22351,"num_write_kb":1430464,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"45'25141","reported_seq":25158,"reported_epoch":45,"state":"active+clean","last_fresh":"2026-02-24T07:46:55.678624+0000","last_change":"2026-02-24T07:46:02.881591+0000","last_active":"2026-02-24T07:46:55.678624+0000","last_peered":"2026-02-24T07:46:55.678624+0000","last_clean":"2026-02-24T07:46:55.678624+0000","last_became_active":"2026-02-24T07:46:02.881035+0000","last_became_peered":"2026-02-24T07:46:02.881035+0000","last_unstale":"2026-02-24T07:46:55.678624+0000","last_undegraded":"2026-02-24T07:46:55.678624+0000","last_fullsized":"2026-02-24T07:46:55.678624+0000","mapping_epoch":42,"log_start":"45'25138","ondisk_log_start":"45'25138","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_clean_scrub_stamp":"2026-02-24T07:46:01.867949+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:47:05.565071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1647640576,"num_objects":25141,"num_object_clones":0,"num_object_copies":75423,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25141,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":25141,"num_write_kb":1609024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"45'22735","reported_seq":22752,"reported_epoch":45,"state":"active+clean","last_fresh":"2026-02-24T07:46:50.696836+0000","last_change":"2026-02-24T07:46:02.879955+0000","last_active":"2026-02-24T07:46:50.696836+0000","last_peered":"2026-02-24T07:46:50.696836+0000","last_clean":"2026-02-24T07:46:50.696836+0000","last_became_active":"2026-02-24T07:46:02.879831+0000","last_became_peered":"2026-02-24T07:46:02.879831+0000","last_unstale":"2026-02-24T07:46:50.696836+0000","last_undegraded":"2026-02-24T07:46:50.696836+0000","last_fullsized":"2026-02-24T07:46:50.696836+0000","mapping_epoch":42,"log_start":"45'22732","ondisk_log_start":"45'22732","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_clean_scrub_stamp":"2026-02-24T07:46:01.867949+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:47:19.157247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1489960960,"num_objects":22735,"num_object_clones":0,"num_object_copies":68205,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22735,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22735,"num_write_kb":1455040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.0","version":"45'22456","reported_seq":22472,"reported_epoch":45,"state":"active+clean","last_fresh":"2026-02-24T07:46:50.751276+0000","last_change":"2026-02-24T07:46:02.880585+0000","last_active":"2026-02-24T07:46:50.751276+0000","last_peered":"2026-02-24T07:46:50.751276+0000","last_clean":"2026-02-24T07:46:50.751276+0000","last_became_active":"2026-02-24T07:46:02.880333+0000","last_became_peered":"2026-02-24T07:46:02.880333+0000","last_unstale":"2026-02-24T07:46:50.751276+0000","last_undegraded":"2026-02-24T07:46:50.751276+0000","last_fullsized":"2026-02-24T07:46:50.751276+0000","mapping_epoch":42,"log_start":"45'22453","ondisk_log_start":"45'22453","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_clean_scrub_stamp":"2026-02-24T07:46:01.867949+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:47:04.574448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1471676416,"num_objects":22456,"num_object_clones":0,"num_object_copies":67368,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22456,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22456,"num_write_kb":1437184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2","version":"45'22812","reported_seq":22829,"reported_epoch":45,"state":"active+clean","last_fresh":"2026-02-24T07:46:50.748822+0000","last_change":"2026-02-24T07:46:02.880566+0000","last_active":"2026-02-24T07:46:50.748822+0000","last_peered":"2026-02-24T07:46:50.748822+0000","last_clean":"2026-02-24T07:46:50.748822+0000","last_became_active":"2026-02-24T07:46:02.880185+0000","last_became_peered":"2026-02-24T07:46:02.880185+0000","last_unstale":"2026-02-24T07:46:50.748822+0000","last_undegraded":"2026-02-24T07:46:50.748822+0000","last_fullsized":"2026-02-24T07:46:50.748822+0000","mapping_epoch":42,"log_start":"45'22809","ondisk_log_start":"45'22809","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_clean_scrub_stamp":"2026-02-24T07:46:01.867949+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:47:23.244686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1495007232,"num_objects":22812,"num_object_clones":0,"num_object_copies":68436,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22812,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22812,"num_write_kb":1459968,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"29'99","reported_seq":236,"reported_epoch":45,"state":"active+clean","last_fresh":"2026-02-24T07:46:41.007499+0000","last_change":"2026-02-24T07:46:41.007499+0000","last_active":"2026-02-24T07:46:41.007499+0000","last_peered":"2026-02-24T07:46:41.007499+0000","last_clean":"2026-02-24T07:46:41.007499+0000","last_became_active":"2026-02-24T07:42:54.186719+0000","last_became_peered":"2026-02-24T07:42:54.186719+0000","last_unstale":"2026-02-24T07:46:41.007499+0000","last_undegraded":"2026-02-24T07:46:41.007499+0000","last_fullsized":"2026-02-24T07:46:41.007499+0000","mapping_epoch":27,"log_start":"29'96","ondisk_log_start":"29'96","created":20,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'99","last_scrub_stamp":"2026-02-24T07:46:41.007434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:42:43.640776+0000","last_clean_scrub_stamp":"2026-02-24T07:46:41.007434+0000","objects_scrubbed":2,"log_size":3,"log_dups_size":96,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:47:53.486511+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":240,"num_write":135,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":16,"stat_sum":{"num_bytes":24405147648,"num_objects":372393,"num_object_clones":0,"num_object_copies":1117179,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":372393,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":372393,"num_write_kb":23833152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":73516711936,"data_stored":73516711936,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":50,"ondisk_log_size":50,"up":48,"acting":48,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":240,"num_write":135,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":2120,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":3,"up_from":25,"seq":107374182450,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":15636936,"kb_used_data":14296308,"kb_used_omap":9,"kb_used_meta":1340598,"kb_avail":718366264,"statfs":{"total":751619276800,"available":735607054336,"internally_reserved":0,"allocated":14639419392,"data_stored":14639130748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9784,"internal_metadata":1372772808},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":19,"seq":81604378677,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":24328776,"kb_used_data":22470068,"kb_used_omap":8,"kb_used_meta":1858679,"kb_avail":709674424,"statfs":{"total":751619276800,"available":726706610176,"internally_reserved":0,"allocated":23009349632,"data_stored":23009060988,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":8485,"internal_metadata":1903288027},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542197,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":17304964,"kb_used_data":15706736,"kb_used_omap":9,"kb_used_meta":1598198,"kb_avail":716698236,"statfs":{"total":751619276800,"available":733898993664,"internally_reserved":0,"allocated":16083697664,"data_stored":16083412572,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9987,"internal_metadata":1636555005},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,2,4],"upper_bound":8},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705720,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":20195080,"kb_used_data":18588980,"kb_used_omap":8,"kb_used_meta":1606071,"kb_avail":713808120,"statfs":{"total":751619276800,"available":730939514880,"internally_reserved":0,"allocated":19035115520,"data_stored":19034826876,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":8362,"internal_metadata":1644617558},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":2120,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":19262341120,"data_stored":19262341120,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16306405376,"data_stored":16306405376,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":23099932672,"data_stored":23099932672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":14848032768,"data_stored":14848032768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-24T07:46:57.198 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph osd pool set unique_pool_1 pgp_num 36 2026-02-24T07:46:57.343 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:46:57.356 INFO:journalctl@ceph.prometheus.a.trial091.stdout:Feb 24 07:46:56 trial091 ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-prometheus-a[66007]: time=2026-02-24T07:46:56.973Z level=ERROR source=scrape.go:1630 msg="Failed to determine correct type of scrape target." component="scrape manager" scrape_pool=ceph target=http://trial035:9283/metrics content_type="text/html;charset=utf-8" fallback_media_type="" err="received unsupported Content-Type \"text/html;charset=utf-8\" and no fallback_scrape_protocol specified for target" 2026-02-24T07:46:57.475 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:46:57.472+0000 7f5a5662e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:46:57.475 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:46:57.473+0000 7f5a5662e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:46:57.621 INFO:teuthology.orchestra.run.trial035.stderr:Error EPERM: crimson_allow_pg_split is false; pgp_num increase denied 2026-02-24T07:46:57.661 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-24T07:46:57.977 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 54 15 418723 418708 484.575 484.312 0.00189246 0.00206242 2026-02-24T07:46:58.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:46:57 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/2617494582' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2026-02-24T07:46:58.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:46:57 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/3824231010' entity='client.admin' cmd={"prefix": "osd pool set", "pool": "unique_pool_1", "var": "pgp_num", "val": "36"} : dispatch 2026-02-24T07:46:58.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:46:57 trial035 ceph-mon[39284]: pgmap v138: 17 pgs: 17 active+clean; 25 GiB data, 79 GiB used, 2.7 TiB / 2.7 TiB avail; 553 MiB/s wr, 8.85k op/s 2026-02-24T07:46:58.976 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 55 15 426909 426894 485.066 511.625 0.00160371 0.00206032 2026-02-24T07:46:59.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:46:58 trial035 ceph-mon[39284]: purged_snaps scrub starts 2026-02-24T07:46:59.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:46:58 trial035 ceph-mon[39284]: purged_snaps scrub ok 2026-02-24T07:46:59.210 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2026-02-24T07:46:59.354 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:46:59.490 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:46:59.488+0000 7f803973c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:46:59.491 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:46:59.489+0000 7f803973c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:46:59.506 INFO:teuthology.orchestra.run.trial035.stdout:osd.0: {} 2026-02-24T07:46:59.506 INFO:teuthology.orchestra.run.trial035.stderr:osd.0: osd_enable_op_tracker = '' 2026-02-24T07:46:59.523 INFO:teuthology.orchestra.run.trial035.stdout:osd.1: {} 2026-02-24T07:46:59.524 INFO:teuthology.orchestra.run.trial035.stderr:osd.1: osd_enable_op_tracker = '' 2026-02-24T07:46:59.537 INFO:teuthology.orchestra.run.trial035.stdout:osd.2: {} 2026-02-24T07:46:59.537 INFO:teuthology.orchestra.run.trial035.stderr:osd.2: osd_enable_op_tracker = '' 2026-02-24T07:46:59.551 INFO:teuthology.orchestra.run.trial035.stdout:osd.3: {} 2026-02-24T07:46:59.551 INFO:teuthology.orchestra.run.trial035.stderr:osd.3: osd_enable_op_tracker = '' 2026-02-24T07:46:59.977 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 56 16 434828 434812 485.241 494.875 0.00164819 0.00205959 2026-02-24T07:47:00.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:46:59 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/3283074912' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2026-02-24T07:47:00.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:46:59 trial035 ceph-mon[39284]: pgmap v139: 17 pgs: 17 active+clean; 25 GiB data, 79 GiB used, 2.7 TiB / 2.7 TiB avail; 401 MiB/s wr, 6.41k op/s 2026-02-24T07:47:00.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:46:59 trial035 ceph-mon[39284]: from='mgr.14284 10.20.193.35:0/3681373222' entity='mgr.x' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2026-02-24T07:47:00.977 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 57 16 442773 442757 485.439 496.562 0.00214197 0.00205875 2026-02-24T07:47:01.593 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2026-02-24T07:47:01.738 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:47:01.861 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:47:01.858+0000 7fbf6cf8e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:47:01.861 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:47:01.858+0000 7fbf6cf8e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:47:01.875 INFO:teuthology.orchestra.run.trial035.stdout:osd.0: {} 2026-02-24T07:47:01.875 INFO:teuthology.orchestra.run.trial035.stderr:osd.0: osd_enable_op_tracker = '' 2026-02-24T07:47:01.888 INFO:teuthology.orchestra.run.trial035.stdout:osd.1: {} 2026-02-24T07:47:01.888 INFO:teuthology.orchestra.run.trial035.stderr:osd.1: osd_enable_op_tracker = '' 2026-02-24T07:47:01.900 INFO:teuthology.orchestra.run.trial035.stdout:osd.2: {} 2026-02-24T07:47:01.901 INFO:teuthology.orchestra.run.trial035.stderr:osd.2: osd_enable_op_tracker = '' 2026-02-24T07:47:01.914 INFO:teuthology.orchestra.run.trial035.stdout:osd.3: {} 2026-02-24T07:47:01.914 INFO:teuthology.orchestra.run.trial035.stderr:osd.3: osd_enable_op_tracker = '' 2026-02-24T07:47:01.977 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 58 16 450461 450445 485.353 480.5 0.00123983 0.00205909 2026-02-24T07:47:02.661 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2026-02-24T07:47:02.662 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2026-02-24T07:47:02.662 INFO:tasks.thrashosds.thrasher:Growing pool .mgr 2026-02-24T07:47:02.662 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph pg dump --format=json 2026-02-24T07:47:02.798 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:47:02.954 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:47:02 trial035 ceph-mon[39284]: pgmap v140: 17 pgs: 17 active+clean; 27 GiB data, 87 GiB used, 2.6 TiB / 2.7 TiB avail; 600 MiB/s wr, 9.59k op/s 2026-02-24T07:47:02.955 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:47:02 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/125284272' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2026-02-24T07:47:02.977 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 59 16 458422 458406 485.559 497.562 0.00206673 0.00205824 2026-02-24T07:47:03.378 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:47:03.313+0000 7f9c64ed7640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:47:03.378 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:47:03.313+0000 7f9c64ed7640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:47:03.457 INFO:teuthology.orchestra.run.trial035.stdout: 2026-02-24T07:47:03.458 INFO:teuthology.orchestra.run.trial035.stderr:dumped all 2026-02-24T07:47:03.957 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2026-02-24T07:47:03.977 INFO:tasks.radosbench.radosbench.0.trial035.stdout:2026-02-24T07:47:03.976766+0000 min lat: 0.00073176 max lat: 0.0419194 avg lat: 0.00206574 2026-02-24T07:47:03.977 INFO:tasks.radosbench.radosbench.0.trial035.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2026-02-24T07:47:03.977 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 60 16 464496 464480 483.793 379.625 0.00765673 0.00206574 2026-02-24T07:47:04.180 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:47:04.328 INFO:teuthology.orchestra.run.trial035.stdout:{"pg_ready":true,"pg_map":{"version":140,"stamp":"2026-02-24T07:47:01.690117+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":28883485216,"num_objects":440720,"num_object_clones":0,"num_object_copies":1322160,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":440720,"num_whiteouts":0,"num_read":136,"num_read_kb":240,"num_write":440853,"num_write_kb":28208102,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":53,"ondisk_log_size":53,"up":51,"acting":51,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":51,"num_osds":4,"num_per_pool_osds":4,"num_per_pool_omap_osds":4,"kb":2936012800,"kb_used":90775644,"kb_used_data":83850060,"kb_used_omap":33,"kb_used_meta":6925406,"kb_avail":2845237156,"statfs":{"total":3006477107200,"available":2913522847744,"internally_reserved":0,"allocated":85862461440,"data_stored":85861310416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":34090,"internal_metadata":7091616470},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[8,5],"upper_bound":4},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":7544504320,"num_objects":115120,"num_object_clones":0,"num_object_copies":345360,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":115120,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":115120,"num_write_kb":7367680,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001603"},"pg_stats":[{"pgid":"3.f","version":"45'27511","reported_seq":27527,"reported_epoch":45,"state":"active+clean","last_fresh":"2026-02-24T07:47:00.682785+0000","last_change":"2026-02-24T07:46:02.882095+0000","last_active":"2026-02-24T07:47:00.682785+0000","last_peered":"2026-02-24T07:47:00.682785+0000","last_clean":"2026-02-24T07:47:00.682785+0000","last_became_active":"2026-02-24T07:46:02.881625+0000","last_became_peered":"2026-02-24T07:46:02.881625+0000","last_unstale":"2026-02-24T07:47:00.682785+0000","last_undegraded":"2026-02-24T07:47:00.682785+0000","last_fullsized":"2026-02-24T07:47:00.682785+0000","mapping_epoch":42,"log_start":"45'27508","ondisk_log_start":"45'27508","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_clean_scrub_stamp":"2026-02-24T07:46:01.867949+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:47:09.267374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1802960896,"num_objects":27511,"num_object_clones":0,"num_object_copies":82533,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27511,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":27511,"num_write_kb":1760704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.e","version":"45'27829","reported_seq":27846,"reported_epoch":45,"state":"active+clean","last_fresh":"2026-02-24T07:47:00.680952+0000","last_change":"2026-02-24T07:46:02.881462+0000","last_active":"2026-02-24T07:47:00.680952+0000","last_peered":"2026-02-24T07:47:00.680952+0000","last_clean":"2026-02-24T07:47:00.680952+0000","last_became_active":"2026-02-24T07:46:02.881083+0000","last_became_peered":"2026-02-24T07:46:02.881083+0000","last_unstale":"2026-02-24T07:47:00.680952+0000","last_undegraded":"2026-02-24T07:47:00.680952+0000","last_fullsized":"2026-02-24T07:47:00.680952+0000","mapping_epoch":42,"log_start":"45'27826","ondisk_log_start":"45'27826","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_clean_scrub_stamp":"2026-02-24T07:46:01.867949+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:47:29.030602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1823801344,"num_objects":27829,"num_object_clones":0,"num_object_copies":83487,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27829,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":27829,"num_write_kb":1781056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"45'27442","reported_seq":27459,"reported_epoch":45,"state":"active+clean","last_fresh":"2026-02-24T07:47:00.708404+0000","last_change":"2026-02-24T07:46:02.883952+0000","last_active":"2026-02-24T07:47:00.708404+0000","last_peered":"2026-02-24T07:47:00.708404+0000","last_clean":"2026-02-24T07:47:00.708404+0000","last_became_active":"2026-02-24T07:46:02.883799+0000","last_became_peered":"2026-02-24T07:46:02.883799+0000","last_unstale":"2026-02-24T07:47:00.708404+0000","last_undegraded":"2026-02-24T07:47:00.708404+0000","last_fullsized":"2026-02-24T07:47:00.708404+0000","mapping_epoch":42,"log_start":"45'27439","ondisk_log_start":"45'27439","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_clean_scrub_stamp":"2026-02-24T07:46:01.867949+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:47:02.402624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1798438912,"num_objects":27442,"num_object_clones":0,"num_object_copies":82326,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27442,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":27442,"num_write_kb":1756288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"45'27145","reported_seq":27162,"reported_epoch":45,"state":"active+clean","last_fresh":"2026-02-24T07:47:00.708454+0000","last_change":"2026-02-24T07:46:02.880375+0000","last_active":"2026-02-24T07:47:00.708454+0000","last_peered":"2026-02-24T07:47:00.708454+0000","last_clean":"2026-02-24T07:47:00.708454+0000","last_became_active":"2026-02-24T07:46:02.880060+0000","last_became_peered":"2026-02-24T07:46:02.880060+0000","last_unstale":"2026-02-24T07:47:00.708454+0000","last_undegraded":"2026-02-24T07:47:00.708454+0000","last_fullsized":"2026-02-24T07:47:00.708454+0000","mapping_epoch":42,"log_start":"45'27142","ondisk_log_start":"45'27142","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_clean_scrub_stamp":"2026-02-24T07:46:01.867949+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:47:04.123135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1778974720,"num_objects":27145,"num_object_clones":0,"num_object_copies":81435,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27145,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":27145,"num_write_kb":1737280,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"45'27530","reported_seq":27547,"reported_epoch":45,"state":"active+clean","last_fresh":"2026-02-24T07:47:00.746486+0000","last_change":"2026-02-24T07:46:02.880509+0000","last_active":"2026-02-24T07:47:00.746486+0000","last_peered":"2026-02-24T07:47:00.746486+0000","last_clean":"2026-02-24T07:47:00.746486+0000","last_became_active":"2026-02-24T07:46:02.880171+0000","last_became_peered":"2026-02-24T07:46:02.880171+0000","last_unstale":"2026-02-24T07:47:00.746486+0000","last_undegraded":"2026-02-24T07:47:00.746486+0000","last_fullsized":"2026-02-24T07:47:00.746486+0000","mapping_epoch":42,"log_start":"45'27527","ondisk_log_start":"45'27527","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_clean_scrub_stamp":"2026-02-24T07:46:01.867949+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:47:06.105346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1804206080,"num_objects":27530,"num_object_clones":0,"num_object_copies":82590,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27530,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":27530,"num_write_kb":1761920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.a","version":"45'27390","reported_seq":27407,"reported_epoch":45,"state":"active+clean","last_fresh":"2026-02-24T07:47:00.707003+0000","last_change":"2026-02-24T07:46:02.882761+0000","last_active":"2026-02-24T07:47:00.707003+0000","last_peered":"2026-02-24T07:47:00.707003+0000","last_clean":"2026-02-24T07:47:00.707003+0000","last_became_active":"2026-02-24T07:46:02.882274+0000","last_became_peered":"2026-02-24T07:46:02.882274+0000","last_unstale":"2026-02-24T07:47:00.707003+0000","last_undegraded":"2026-02-24T07:47:00.707003+0000","last_fullsized":"2026-02-24T07:47:00.707003+0000","mapping_epoch":42,"log_start":"45'27387","ondisk_log_start":"45'27387","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_clean_scrub_stamp":"2026-02-24T07:46:01.867949+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:47:31.776406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1795031040,"num_objects":27390,"num_object_clones":0,"num_object_copies":82170,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27390,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":27390,"num_write_kb":1752960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"45'27880","reported_seq":27897,"reported_epoch":45,"state":"active+clean","last_fresh":"2026-02-24T07:47:00.705143+0000","last_change":"2026-02-24T07:46:02.882303+0000","last_active":"2026-02-24T07:47:00.705143+0000","last_peered":"2026-02-24T07:47:00.705143+0000","last_clean":"2026-02-24T07:47:00.705143+0000","last_became_active":"2026-02-24T07:46:02.881951+0000","last_became_peered":"2026-02-24T07:46:02.881951+0000","last_unstale":"2026-02-24T07:47:00.705143+0000","last_undegraded":"2026-02-24T07:47:00.705143+0000","last_fullsized":"2026-02-24T07:47:00.705143+0000","mapping_epoch":42,"log_start":"45'27877","ondisk_log_start":"45'27877","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_clean_scrub_stamp":"2026-02-24T07:46:01.867949+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:47:04.345724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1827143680,"num_objects":27880,"num_object_clones":0,"num_object_copies":83640,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27880,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":27880,"num_write_kb":1784320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"45'27639","reported_seq":27652,"reported_epoch":45,"state":"active+clean","last_fresh":"2026-02-24T07:47:00.752237+0000","last_change":"2026-02-24T07:46:02.881463+0000","last_active":"2026-02-24T07:47:00.752237+0000","last_peered":"2026-02-24T07:47:00.752237+0000","last_clean":"2026-02-24T07:47:00.752237+0000","last_became_active":"2026-02-24T07:46:02.881345+0000","last_became_peered":"2026-02-24T07:46:02.881345+0000","last_unstale":"2026-02-24T07:47:00.752237+0000","last_undegraded":"2026-02-24T07:47:00.752237+0000","last_fullsized":"2026-02-24T07:47:00.752237+0000","mapping_epoch":42,"log_start":"45'27634","ondisk_log_start":"45'27634","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_clean_scrub_stamp":"2026-02-24T07:46:01.867949+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":3000,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:47:17.135287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1811349504,"num_objects":27639,"num_object_clones":0,"num_object_copies":82917,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27639,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":27639,"num_write_kb":1768896,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"45'27421","reported_seq":27438,"reported_epoch":45,"state":"active+clean","last_fresh":"2026-02-24T07:47:00.752148+0000","last_change":"2026-02-24T07:46:02.880582+0000","last_active":"2026-02-24T07:47:00.752148+0000","last_peered":"2026-02-24T07:47:00.752148+0000","last_clean":"2026-02-24T07:47:00.752148+0000","last_became_active":"2026-02-24T07:46:02.880323+0000","last_became_peered":"2026-02-24T07:46:02.880323+0000","last_unstale":"2026-02-24T07:47:00.752148+0000","last_undegraded":"2026-02-24T07:47:00.752148+0000","last_fullsized":"2026-02-24T07:47:00.752148+0000","mapping_epoch":42,"log_start":"45'27418","ondisk_log_start":"45'27418","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_clean_scrub_stamp":"2026-02-24T07:46:01.867949+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:47:31.420697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1797062656,"num_objects":27421,"num_object_clones":0,"num_object_copies":82263,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27421,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":27421,"num_write_kb":1754944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.6","version":"45'27665","reported_seq":27682,"reported_epoch":45,"state":"active+clean","last_fresh":"2026-02-24T07:47:00.703670+0000","last_change":"2026-02-24T07:46:02.880170+0000","last_active":"2026-02-24T07:47:00.703670+0000","last_peered":"2026-02-24T07:47:00.703670+0000","last_clean":"2026-02-24T07:47:00.703670+0000","last_became_active":"2026-02-24T07:46:02.880005+0000","last_became_peered":"2026-02-24T07:46:02.880005+0000","last_unstale":"2026-02-24T07:47:00.703670+0000","last_undegraded":"2026-02-24T07:47:00.703670+0000","last_fullsized":"2026-02-24T07:47:00.703670+0000","mapping_epoch":42,"log_start":"45'27662","ondisk_log_start":"45'27662","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_clean_scrub_stamp":"2026-02-24T07:46:01.867949+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:47:14.923924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1813053440,"num_objects":27665,"num_object_clones":0,"num_object_copies":82995,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27665,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":27665,"num_write_kb":1770560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"45'27715","reported_seq":27732,"reported_epoch":45,"state":"active+clean","last_fresh":"2026-02-24T07:47:00.682027+0000","last_change":"2026-02-24T07:46:02.881931+0000","last_active":"2026-02-24T07:47:00.682027+0000","last_peered":"2026-02-24T07:47:00.682027+0000","last_clean":"2026-02-24T07:47:00.682027+0000","last_became_active":"2026-02-24T07:46:02.881420+0000","last_became_peered":"2026-02-24T07:46:02.881420+0000","last_unstale":"2026-02-24T07:47:00.682027+0000","last_undegraded":"2026-02-24T07:47:00.682027+0000","last_fullsized":"2026-02-24T07:47:00.682027+0000","mapping_epoch":42,"log_start":"45'27712","ondisk_log_start":"45'27712","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_clean_scrub_stamp":"2026-02-24T07:46:01.867949+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:47:09.535393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1816330240,"num_objects":27715,"num_object_clones":0,"num_object_copies":83145,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27715,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":27715,"num_write_kb":1773760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.4","version":"45'27274","reported_seq":27291,"reported_epoch":45,"state":"active+clean","last_fresh":"2026-02-24T07:47:00.750578+0000","last_change":"2026-02-24T07:46:02.880661+0000","last_active":"2026-02-24T07:47:00.750578+0000","last_peered":"2026-02-24T07:47:00.750578+0000","last_clean":"2026-02-24T07:47:00.750578+0000","last_became_active":"2026-02-24T07:46:02.880345+0000","last_became_peered":"2026-02-24T07:46:02.880345+0000","last_unstale":"2026-02-24T07:47:00.750578+0000","last_undegraded":"2026-02-24T07:47:00.750578+0000","last_fullsized":"2026-02-24T07:47:00.750578+0000","mapping_epoch":42,"log_start":"45'27271","ondisk_log_start":"45'27271","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_clean_scrub_stamp":"2026-02-24T07:46:01.867949+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:47:27.577904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1787428864,"num_objects":27274,"num_object_clones":0,"num_object_copies":81822,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27274,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":27274,"num_write_kb":1745536,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"45'27572","reported_seq":27589,"reported_epoch":45,"state":"active+clean","last_fresh":"2026-02-24T07:47:00.682649+0000","last_change":"2026-02-24T07:46:02.881591+0000","last_active":"2026-02-24T07:47:00.682649+0000","last_peered":"2026-02-24T07:47:00.682649+0000","last_clean":"2026-02-24T07:47:00.682649+0000","last_became_active":"2026-02-24T07:46:02.881035+0000","last_became_peered":"2026-02-24T07:46:02.881035+0000","last_unstale":"2026-02-24T07:47:00.682649+0000","last_undegraded":"2026-02-24T07:47:00.682649+0000","last_fullsized":"2026-02-24T07:47:00.682649+0000","mapping_epoch":42,"log_start":"45'27569","ondisk_log_start":"45'27569","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_clean_scrub_stamp":"2026-02-24T07:46:01.867949+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:47:05.565071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1806958592,"num_objects":27572,"num_object_clones":0,"num_object_copies":82716,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27572,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":27572,"num_write_kb":1764608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"45'27691","reported_seq":27708,"reported_epoch":45,"state":"active+clean","last_fresh":"2026-02-24T07:47:00.703829+0000","last_change":"2026-02-24T07:46:02.879955+0000","last_active":"2026-02-24T07:47:00.703829+0000","last_peered":"2026-02-24T07:47:00.703829+0000","last_clean":"2026-02-24T07:47:00.703829+0000","last_became_active":"2026-02-24T07:46:02.879831+0000","last_became_peered":"2026-02-24T07:46:02.879831+0000","last_unstale":"2026-02-24T07:47:00.703829+0000","last_undegraded":"2026-02-24T07:47:00.703829+0000","last_fullsized":"2026-02-24T07:47:00.703829+0000","mapping_epoch":42,"log_start":"45'27688","ondisk_log_start":"45'27688","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_clean_scrub_stamp":"2026-02-24T07:46:01.867949+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:47:19.157247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1814757376,"num_objects":27691,"num_object_clones":0,"num_object_copies":83073,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27691,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":27691,"num_write_kb":1772224,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.0","version":"45'27329","reported_seq":27346,"reported_epoch":45,"state":"active+clean","last_fresh":"2026-02-24T07:47:00.747108+0000","last_change":"2026-02-24T07:46:02.880585+0000","last_active":"2026-02-24T07:47:00.747108+0000","last_peered":"2026-02-24T07:47:00.747108+0000","last_clean":"2026-02-24T07:47:00.747108+0000","last_became_active":"2026-02-24T07:46:02.880333+0000","last_became_peered":"2026-02-24T07:46:02.880333+0000","last_unstale":"2026-02-24T07:47:00.747108+0000","last_undegraded":"2026-02-24T07:47:00.747108+0000","last_fullsized":"2026-02-24T07:47:00.747108+0000","mapping_epoch":42,"log_start":"45'27326","ondisk_log_start":"45'27326","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_clean_scrub_stamp":"2026-02-24T07:46:01.867949+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:47:04.574448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1791033344,"num_objects":27329,"num_object_clones":0,"num_object_copies":81987,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27329,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":27329,"num_write_kb":1749056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2","version":"45'27685","reported_seq":27702,"reported_epoch":45,"state":"active+clean","last_fresh":"2026-02-24T07:47:00.742007+0000","last_change":"2026-02-24T07:46:02.880566+0000","last_active":"2026-02-24T07:47:00.742007+0000","last_peered":"2026-02-24T07:47:00.742007+0000","last_clean":"2026-02-24T07:47:00.742007+0000","last_became_active":"2026-02-24T07:46:02.880185+0000","last_became_peered":"2026-02-24T07:46:02.880185+0000","last_unstale":"2026-02-24T07:47:00.742007+0000","last_undegraded":"2026-02-24T07:47:00.742007+0000","last_fullsized":"2026-02-24T07:47:00.742007+0000","mapping_epoch":42,"log_start":"45'27682","ondisk_log_start":"45'27682","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_clean_scrub_stamp":"2026-02-24T07:46:01.867949+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:47:23.244686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1814364160,"num_objects":27685,"num_object_clones":0,"num_object_copies":83055,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27685,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":27685,"num_write_kb":1771840,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"29'99","reported_seq":237,"reported_epoch":45,"state":"active+clean","last_fresh":"2026-02-24T07:46:41.007543+0000","last_change":"2026-02-24T07:46:41.007499+0000","last_active":"2026-02-24T07:46:41.007543+0000","last_peered":"2026-02-24T07:46:41.007543+0000","last_clean":"2026-02-24T07:46:41.007543+0000","last_became_active":"2026-02-24T07:42:54.186719+0000","last_became_peered":"2026-02-24T07:42:54.186719+0000","last_unstale":"2026-02-24T07:46:41.007543+0000","last_undegraded":"2026-02-24T07:46:41.007543+0000","last_fullsized":"2026-02-24T07:46:41.007543+0000","mapping_epoch":27,"log_start":"29'96","ondisk_log_start":"29'96","created":20,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'99","last_scrub_stamp":"2026-02-24T07:46:41.007434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:42:43.640776+0000","last_clean_scrub_stamp":"2026-02-24T07:46:41.007434+0000","objects_scrubbed":2,"log_size":3,"log_dups_size":96,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:47:53.486511+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":240,"num_write":135,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":16,"stat_sum":{"num_bytes":28882894848,"num_objects":440718,"num_object_clones":0,"num_object_copies":1322154,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":440718,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":440718,"num_write_kb":28205952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":86639181824,"data_stored":86639181824,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":50,"ondisk_log_size":50,"up":48,"acting":48,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":240,"num_write":135,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":2120,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":3,"up_from":25,"seq":107374182452,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":18945096,"kb_used_data":17387892,"kb_used_omap":9,"kb_used_meta":1557174,"kb_avail":715058104,"statfs":{"total":751619276800,"available":732219498496,"internally_reserved":0,"allocated":17805201408,"data_stored":17804912764,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9784,"internal_metadata":1594546632},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,2],"upper_bound":4},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":19,"seq":81604378678,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":26527368,"kb_used_data":24602740,"kb_used_omap":8,"kb_used_meta":1924535,"kb_avail":707475832,"statfs":{"total":751619276800,"available":724455251968,"internally_reserved":0,"allocated":25193205760,"data_stored":25192917116,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":8485,"internal_metadata":1970724571},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,1],"upper_bound":4},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542199,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":20798532,"kb_used_data":19213552,"kb_used_omap":6,"kb_used_meta":1584953,"kb_avail":713204668,"statfs":{"total":751619276800,"available":730321580032,"internally_reserved":0,"allocated":19674677248,"data_stored":19674392156,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6807,"internal_metadata":1622992233},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705722,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":24504648,"kb_used_data":22645876,"kb_used_omap":8,"kb_used_meta":1858743,"kb_avail":709498552,"statfs":{"total":751619276800,"available":726526517248,"internally_reserved":0,"allocated":23189377024,"data_stored":23189088380,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9014,"internal_metadata":1903353034},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,2],"upper_bound":4},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":2120,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":23443931136,"data_stored":23443931136,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":19839909888,"data_stored":19839909888,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":25293094912,"data_stored":25293094912,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":18062245888,"data_stored":18062245888,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-24T07:47:04.330 INFO:tasks.cephadm.ceph_manager.ceph:increase pool size by 10 2026-02-24T07:47:04.331 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph osd pool set .mgr pg_num 131 2026-02-24T07:47:04.524 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:47:04.844 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:47:04.842+0000 7f550bb87640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:47:04.844 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:47:04.842+0000 7f550bb87640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:47:04.860 INFO:teuthology.orchestra.run.trial035.stdout:osd.0: {} 2026-02-24T07:47:04.860 INFO:teuthology.orchestra.run.trial035.stderr:osd.0: osd_enable_op_tracker = '' 2026-02-24T07:47:04.874 INFO:teuthology.orchestra.run.trial035.stdout:osd.1: {} 2026-02-24T07:47:04.874 INFO:teuthology.orchestra.run.trial035.stderr:osd.1: osd_enable_op_tracker = '' 2026-02-24T07:47:04.888 INFO:teuthology.orchestra.run.trial035.stdout:osd.2: {} 2026-02-24T07:47:04.888 INFO:teuthology.orchestra.run.trial035.stderr:osd.2: osd_enable_op_tracker = '' 2026-02-24T07:47:04.901 INFO:teuthology.orchestra.run.trial035.stdout:osd.3: {} 2026-02-24T07:47:04.902 INFO:teuthology.orchestra.run.trial035.stderr:osd.3: osd_enable_op_tracker = '' 2026-02-24T07:47:04.918 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:47:04 trial035 ceph-mon[39284]: from='client.14686 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2026-02-24T07:47:04.918 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:47:04 trial035 ceph-mon[39284]: pgmap v141: 17 pgs: 17 active+clean; 27 GiB data, 87 GiB used, 2.6 TiB / 2.7 TiB avail; 409 MiB/s wr, 6.54k op/s 2026-02-24T07:47:04.919 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:47:04 trial035 ceph-mon[39284]: 3.d scrub starts 2026-02-24T07:47:04.977 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 61 16 470581 470565 482.096 380.312 0.0015745 0.00207299 2026-02-24T07:47:04.980 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:47:04.977+0000 7f17a72d2640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:47:04.980 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:47:04.978+0000 7f17a72d2640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:47:05.116 INFO:teuthology.orchestra.run.trial035.stderr:Error EPERM: crimson_allow_pg_split is false; pg_num increase denied for crimson pool 2026-02-24T07:47:05.154 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-24T07:47:05.977 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 62 16 476938 476922 480.728 397.312 0.00152483 0.00207886 2026-02-24T07:47:06.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:47:05 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/2052532527' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2026-02-24T07:47:06.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:47:05 trial035 ceph-mon[39284]: 3.c scrub starts 2026-02-24T07:47:06.205 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:47:05 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/513387555' entity='client.admin' cmd={"prefix": "osd pool set", "pool": ".mgr", "var": "pg_num", "val": "131"} : dispatch 2026-02-24T07:47:06.950 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2026-02-24T07:47:06.977 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 63 16 482932 482916 479.044 374.625 0.00148092 0.00208622 2026-02-24T07:47:07.096 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:47:07.123 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:47:06 trial035 ceph-mon[39284]: 3.0 scrub starts 2026-02-24T07:47:07.123 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:47:06 trial035 ceph-mon[39284]: pgmap v142: 17 pgs: 17 active+clean; 27 GiB data, 88 GiB used, 2.6 TiB / 2.7 TiB avail; 454 MiB/s wr, 7.27k op/s 2026-02-24T07:47:07.123 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:47:06 trial035 ceph-mon[39284]: from='mgr.14284 10.20.193.35:0/3681373222' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2026-02-24T07:47:07.124 INFO:journalctl@ceph.mgr.x.trial035.stdout:Feb 24 07:47:06 trial035 ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-mgr-x[39621]: ::ffff:10.20.193.91 - - [24/Feb/2026:07:47:06] "GET /metrics HTTP/1.1" 200 4215 "" "Prometheus/3.6.0" 2026-02-24T07:47:07.218 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:47:07.215+0000 7fe247389640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:47:07.218 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:47:07.216+0000 7fe247389640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:47:07.233 INFO:teuthology.orchestra.run.trial035.stdout:osd.0: {} 2026-02-24T07:47:07.233 INFO:teuthology.orchestra.run.trial035.stderr:osd.0: osd_enable_op_tracker = '' 2026-02-24T07:47:07.249 INFO:teuthology.orchestra.run.trial035.stdout:osd.1: {} 2026-02-24T07:47:07.249 INFO:teuthology.orchestra.run.trial035.stderr:osd.1: osd_enable_op_tracker = '' 2026-02-24T07:47:07.264 INFO:teuthology.orchestra.run.trial035.stdout:osd.2: {} 2026-02-24T07:47:07.264 INFO:teuthology.orchestra.run.trial035.stderr:osd.2: osd_enable_op_tracker = '' 2026-02-24T07:47:07.277 INFO:teuthology.orchestra.run.trial035.stdout:osd.3: {} 2026-02-24T07:47:07.277 INFO:teuthology.orchestra.run.trial035.stderr:osd.3: osd_enable_op_tracker = '' 2026-02-24T07:47:07.356 INFO:journalctl@ceph.prometheus.a.trial091.stdout:Feb 24 07:47:06 trial091 ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-prometheus-a[66007]: time=2026-02-24T07:47:06.973Z level=ERROR source=scrape.go:1630 msg="Failed to determine correct type of scrape target." component="scrape manager" scrape_pool=ceph target=http://trial035:9283/metrics content_type="text/html;charset=utf-8" fallback_media_type="" err="received unsupported Content-Type \"text/html;charset=utf-8\" and no fallback_scrape_protocol specified for target" 2026-02-24T07:47:07.977 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 64 15 488780 488765 477.27 365.562 0.00199135 0.00209397 2026-02-24T07:47:08.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:47:07 trial035 ceph-mon[39284]: from='mgr.14284 10.20.193.35:0/3681373222' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-24T07:47:08.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:47:07 trial035 ceph-mon[39284]: from='mgr.14284 10.20.193.35:0/3681373222' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2026-02-24T07:47:08.205 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:47:07 trial035 ceph-mon[39284]: from='mgr.14284 10.20.193.35:0/3681373222' entity='mgr.x' 2026-02-24T07:47:08.205 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:47:07 trial035 ceph-mon[39284]: from='mgr.14284 10.20.193.35:0/3681373222' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2026-02-24T07:47:08.205 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:47:07 trial035 ceph-mon[39284]: 3.d scrub ok 2026-02-24T07:47:08.205 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:47:07 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/885556223' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2026-02-24T07:47:08.978 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 65 16 495314 495298 476.209 408.312 0.00249063 0.00209864 2026-02-24T07:47:09.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:47:08 trial035 ceph-mon[39284]: 3.b scrub starts 2026-02-24T07:47:09.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:47:08 trial035 ceph-mon[39284]: 3.c scrub ok 2026-02-24T07:47:09.205 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:47:08 trial035 ceph-mon[39284]: 3.9 scrub starts 2026-02-24T07:47:09.205 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:47:08 trial035 ceph-mon[39284]: pgmap v143: 17 pgs: 3 active+clean+scrubbing, 14 active+clean; 29 GiB data, 93 GiB used, 2.6 TiB / 2.7 TiB avail; 536 MiB/s wr, 8.58k op/s 2026-02-24T07:47:09.321 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2026-02-24T07:47:09.464 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:47:09.589 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:47:09.587+0000 7f43e80d8640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:47:09.589 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:47:09.588+0000 7f43e80d8640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:47:09.603 INFO:teuthology.orchestra.run.trial035.stdout:osd.0: {} 2026-02-24T07:47:09.603 INFO:teuthology.orchestra.run.trial035.stderr:osd.0: osd_enable_op_tracker = '' 2026-02-24T07:47:09.618 INFO:teuthology.orchestra.run.trial035.stdout:osd.1: {} 2026-02-24T07:47:09.618 INFO:teuthology.orchestra.run.trial035.stderr:osd.1: osd_enable_op_tracker = '' 2026-02-24T07:47:09.634 INFO:teuthology.orchestra.run.trial035.stdout:osd.2: {} 2026-02-24T07:47:09.635 INFO:teuthology.orchestra.run.trial035.stderr:osd.2: osd_enable_op_tracker = '' 2026-02-24T07:47:09.649 INFO:teuthology.orchestra.run.trial035.stdout:osd.3: {} 2026-02-24T07:47:09.649 INFO:teuthology.orchestra.run.trial035.stderr:osd.3: osd_enable_op_tracker = '' 2026-02-24T07:47:09.954 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:47:09 trial035 ceph-mon[39284]: 3.0 scrub ok 2026-02-24T07:47:09.954 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:47:09 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/2080488153' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2026-02-24T07:47:09.977 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 66 16 501995 501979 475.319 417.562 0.00148388 0.00210257 2026-02-24T07:47:10.155 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2026-02-24T07:47:10.155 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2026-02-24T07:47:10.156 INFO:tasks.thrashosds.thrasher:fixing pg num pool .mgr 2026-02-24T07:47:10.156 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph pg dump --format=json 2026-02-24T07:47:10.305 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:47:10.841 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:47:10.792+0000 7fa5e6bd7640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:47:10.841 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:47:10.792+0000 7fa5e6bd7640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:47:10.965 INFO:teuthology.orchestra.run.trial035.stdout: 2026-02-24T07:47:10.966 INFO:teuthology.orchestra.run.trial035.stderr:dumped all 2026-02-24T07:47:10.977 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:47:10 trial035 ceph-mon[39284]: pgmap v144: 17 pgs: 3 active+clean+scrubbing, 14 active+clean; 29 GiB data, 93 GiB used, 2.6 TiB / 2.7 TiB avail; 379 MiB/s wr, 6.07k op/s 2026-02-24T07:47:10.977 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:47:10 trial035 ceph-mon[39284]: 3.9 scrub ok 2026-02-24T07:47:10.978 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 67 16 508457 508441 474.251 403.875 0.00199958 0.00210733 2026-02-24T07:47:11.446 INFO:teuthology.orchestra.run.trial035.stdout:{"pg_ready":true,"pg_map":{"version":144,"stamp":"2026-02-24T07:47:09.691225+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":31153586720,"num_objects":475359,"num_object_clones":0,"num_object_copies":1426077,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":475359,"num_whiteouts":0,"num_read":136,"num_read_kb":240,"num_write":475492,"num_write_kb":30424998,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":52,"ondisk_log_size":52,"up":51,"acting":51,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":51,"num_osds":4,"num_per_pool_osds":4,"num_per_pool_omap_osds":4,"kb":2936012800,"kb_used":97378012,"kb_used_data":90639436,"kb_used_omap":34,"kb_used_meta":6738461,"kb_avail":2838634788,"statfs":{"total":3006477107200,"available":2906762022912,"internally_reserved":0,"allocated":92814782464,"data_stored":92813631440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":35170,"internal_metadata":6900184734},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[8,0,0,1],"upper_bound":16},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":4774232064,"num_objects":72849,"num_object_clones":0,"num_object_copies":218547,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":72849,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":72849,"num_write_kb":4662336,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001661"},"pg_stats":[{"pgid":"3.f","version":"45'29642","reported_seq":29659,"reported_epoch":45,"state":"active+clean","last_fresh":"2026-02-24T07:47:05.684318+0000","last_change":"2026-02-24T07:46:02.882095+0000","last_active":"2026-02-24T07:47:05.684318+0000","last_peered":"2026-02-24T07:47:05.684318+0000","last_clean":"2026-02-24T07:47:05.684318+0000","last_became_active":"2026-02-24T07:46:02.881625+0000","last_became_peered":"2026-02-24T07:46:02.881625+0000","last_unstale":"2026-02-24T07:47:05.684318+0000","last_undegraded":"2026-02-24T07:47:05.684318+0000","last_fullsized":"2026-02-24T07:47:05.684318+0000","mapping_epoch":42,"log_start":"45'29639","ondisk_log_start":"45'29639","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_clean_scrub_stamp":"2026-02-24T07:46:01.867949+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:47:09.267374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1942618112,"num_objects":29642,"num_object_clones":0,"num_object_copies":88926,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29642,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":29642,"num_write_kb":1897088,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.e","version":"45'30089","reported_seq":30106,"reported_epoch":45,"state":"active+clean","last_fresh":"2026-02-24T07:47:05.680658+0000","last_change":"2026-02-24T07:46:02.881462+0000","last_active":"2026-02-24T07:47:05.680658+0000","last_peered":"2026-02-24T07:47:05.680658+0000","last_clean":"2026-02-24T07:47:05.680658+0000","last_became_active":"2026-02-24T07:46:02.881083+0000","last_became_peered":"2026-02-24T07:46:02.881083+0000","last_unstale":"2026-02-24T07:47:05.680658+0000","last_undegraded":"2026-02-24T07:47:05.680658+0000","last_fullsized":"2026-02-24T07:47:05.680658+0000","mapping_epoch":42,"log_start":"45'30086","ondisk_log_start":"45'30086","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_clean_scrub_stamp":"2026-02-24T07:46:01.867949+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:47:29.030602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1971912704,"num_objects":30089,"num_object_clones":0,"num_object_copies":90267,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30089,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":30089,"num_write_kb":1925696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"45'29638","reported_seq":29820,"reported_epoch":45,"state":"active+clean+scrubbing","last_fresh":"2026-02-24T07:47:05.709584+0000","last_change":"2026-02-24T07:47:03.898569+0000","last_active":"2026-02-24T07:47:05.709584+0000","last_peered":"2026-02-24T07:47:05.709584+0000","last_clean":"2026-02-24T07:47:05.709584+0000","last_became_active":"2026-02-24T07:46:02.883799+0000","last_became_peered":"2026-02-24T07:46:02.883799+0000","last_unstale":"2026-02-24T07:47:05.709584+0000","last_undegraded":"2026-02-24T07:47:05.709584+0000","last_fullsized":"2026-02-24T07:47:05.709584+0000","mapping_epoch":42,"log_start":"45'29634","ondisk_log_start":"45'29634","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_clean_scrub_stamp":"2026-02-24T07:46:01.867949+0000","objects_scrubbed":16299,"log_size":4,"log_dups_size":3000,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"scrubbing for 2s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1942355968,"num_objects":29638,"num_object_clones":0,"num_object_copies":88914,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29638,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":29638,"num_write_kb":1896832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"45'29247","reported_seq":29355,"reported_epoch":45,"state":"active+clean+scrubbing","last_fresh":"2026-02-24T07:47:05.711256+0000","last_change":"2026-02-24T07:47:04.849903+0000","last_active":"2026-02-24T07:47:05.711256+0000","last_peered":"2026-02-24T07:47:05.711256+0000","last_clean":"2026-02-24T07:47:05.711256+0000","last_became_active":"2026-02-24T07:46:02.880060+0000","last_became_peered":"2026-02-24T07:46:02.880060+0000","last_unstale":"2026-02-24T07:47:05.711256+0000","last_undegraded":"2026-02-24T07:47:05.711256+0000","last_fullsized":"2026-02-24T07:47:05.711256+0000","mapping_epoch":42,"log_start":"45'29244","ondisk_log_start":"45'29244","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_clean_scrub_stamp":"2026-02-24T07:46:01.867949+0000","objects_scrubbed":8699,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"scrubbing for 1s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1916731392,"num_objects":29247,"num_object_clones":0,"num_object_copies":87741,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29247,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":29247,"num_write_kb":1871808,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"45'29689","reported_seq":29706,"reported_epoch":45,"state":"active+clean","last_fresh":"2026-02-24T07:47:05.742016+0000","last_change":"2026-02-24T07:46:02.880509+0000","last_active":"2026-02-24T07:47:05.742016+0000","last_peered":"2026-02-24T07:47:05.742016+0000","last_clean":"2026-02-24T07:47:05.742016+0000","last_became_active":"2026-02-24T07:46:02.880171+0000","last_became_peered":"2026-02-24T07:46:02.880171+0000","last_unstale":"2026-02-24T07:47:05.742016+0000","last_undegraded":"2026-02-24T07:47:05.742016+0000","last_fullsized":"2026-02-24T07:47:05.742016+0000","mapping_epoch":42,"log_start":"45'29686","ondisk_log_start":"45'29686","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_clean_scrub_stamp":"2026-02-24T07:46:01.867949+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:47:06.105346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1945698304,"num_objects":29689,"num_object_clones":0,"num_object_copies":89067,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29689,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":29689,"num_write_kb":1900096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.a","version":"45'29537","reported_seq":29554,"reported_epoch":45,"state":"active+clean","last_fresh":"2026-02-24T07:47:05.710384+0000","last_change":"2026-02-24T07:46:02.882761+0000","last_active":"2026-02-24T07:47:05.710384+0000","last_peered":"2026-02-24T07:47:05.710384+0000","last_clean":"2026-02-24T07:47:05.710384+0000","last_became_active":"2026-02-24T07:46:02.882274+0000","last_became_peered":"2026-02-24T07:46:02.882274+0000","last_unstale":"2026-02-24T07:47:05.710384+0000","last_undegraded":"2026-02-24T07:47:05.710384+0000","last_fullsized":"2026-02-24T07:47:05.710384+0000","mapping_epoch":42,"log_start":"45'29534","ondisk_log_start":"45'29534","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_clean_scrub_stamp":"2026-02-24T07:46:01.867949+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:47:31.776406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1935736832,"num_objects":29537,"num_object_clones":0,"num_object_copies":88611,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29537,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":29537,"num_write_kb":1890368,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"45'30072","reported_seq":30089,"reported_epoch":45,"state":"active+clean","last_fresh":"2026-02-24T07:47:05.711644+0000","last_change":"2026-02-24T07:46:02.882303+0000","last_active":"2026-02-24T07:47:05.711644+0000","last_peered":"2026-02-24T07:47:05.711644+0000","last_clean":"2026-02-24T07:47:05.711644+0000","last_became_active":"2026-02-24T07:46:02.881951+0000","last_became_peered":"2026-02-24T07:46:02.881951+0000","last_unstale":"2026-02-24T07:47:05.711644+0000","last_undegraded":"2026-02-24T07:47:05.711644+0000","last_fullsized":"2026-02-24T07:47:05.711644+0000","mapping_epoch":42,"log_start":"45'30069","ondisk_log_start":"45'30069","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_clean_scrub_stamp":"2026-02-24T07:46:01.867949+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1970798592,"num_objects":30072,"num_object_clones":0,"num_object_copies":90216,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30072,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":30072,"num_write_kb":1924608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"45'29799","reported_seq":29816,"reported_epoch":45,"state":"active+clean","last_fresh":"2026-02-24T07:47:05.754143+0000","last_change":"2026-02-24T07:46:02.881463+0000","last_active":"2026-02-24T07:47:05.754143+0000","last_peered":"2026-02-24T07:47:05.754143+0000","last_clean":"2026-02-24T07:47:05.754143+0000","last_became_active":"2026-02-24T07:46:02.881345+0000","last_became_peered":"2026-02-24T07:46:02.881345+0000","last_unstale":"2026-02-24T07:47:05.754143+0000","last_undegraded":"2026-02-24T07:47:05.754143+0000","last_fullsized":"2026-02-24T07:47:05.754143+0000","mapping_epoch":42,"log_start":"45'29796","ondisk_log_start":"45'29796","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_clean_scrub_stamp":"2026-02-24T07:46:01.867949+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:47:17.135287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1952907264,"num_objects":29799,"num_object_clones":0,"num_object_copies":89397,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29799,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":29799,"num_write_kb":1907136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"45'29554","reported_seq":29571,"reported_epoch":45,"state":"active+clean","last_fresh":"2026-02-24T07:47:05.749064+0000","last_change":"2026-02-24T07:46:02.880582+0000","last_active":"2026-02-24T07:47:05.749064+0000","last_peered":"2026-02-24T07:47:05.749064+0000","last_clean":"2026-02-24T07:47:05.749064+0000","last_became_active":"2026-02-24T07:46:02.880323+0000","last_became_peered":"2026-02-24T07:46:02.880323+0000","last_unstale":"2026-02-24T07:47:05.749064+0000","last_undegraded":"2026-02-24T07:47:05.749064+0000","last_fullsized":"2026-02-24T07:47:05.749064+0000","mapping_epoch":42,"log_start":"45'29551","ondisk_log_start":"45'29551","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_clean_scrub_stamp":"2026-02-24T07:46:01.867949+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:47:31.420697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1936850944,"num_objects":29554,"num_object_clones":0,"num_object_copies":88662,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29554,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":29554,"num_write_kb":1891456,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.6","version":"45'29823","reported_seq":29840,"reported_epoch":45,"state":"active+clean","last_fresh":"2026-02-24T07:47:05.703919+0000","last_change":"2026-02-24T07:46:02.880170+0000","last_active":"2026-02-24T07:47:05.703919+0000","last_peered":"2026-02-24T07:47:05.703919+0000","last_clean":"2026-02-24T07:47:05.703919+0000","last_became_active":"2026-02-24T07:46:02.880005+0000","last_became_peered":"2026-02-24T07:46:02.880005+0000","last_unstale":"2026-02-24T07:47:05.703919+0000","last_undegraded":"2026-02-24T07:47:05.703919+0000","last_fullsized":"2026-02-24T07:47:05.703919+0000","mapping_epoch":42,"log_start":"45'29820","ondisk_log_start":"45'29820","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_clean_scrub_stamp":"2026-02-24T07:46:01.867949+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:47:14.923924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1954480128,"num_objects":29823,"num_object_clones":0,"num_object_copies":89469,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29823,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":29823,"num_write_kb":1908672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"45'29851","reported_seq":29868,"reported_epoch":45,"state":"active+clean","last_fresh":"2026-02-24T07:47:05.684142+0000","last_change":"2026-02-24T07:46:02.881931+0000","last_active":"2026-02-24T07:47:05.684142+0000","last_peered":"2026-02-24T07:47:05.684142+0000","last_clean":"2026-02-24T07:47:05.684142+0000","last_became_active":"2026-02-24T07:46:02.881420+0000","last_became_peered":"2026-02-24T07:46:02.881420+0000","last_unstale":"2026-02-24T07:47:05.684142+0000","last_undegraded":"2026-02-24T07:47:05.684142+0000","last_fullsized":"2026-02-24T07:47:05.684142+0000","mapping_epoch":42,"log_start":"45'29848","ondisk_log_start":"45'29848","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_clean_scrub_stamp":"2026-02-24T07:46:01.867949+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:47:09.535393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1956315136,"num_objects":29851,"num_object_clones":0,"num_object_copies":89553,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29851,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":29851,"num_write_kb":1910464,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.4","version":"45'29402","reported_seq":29419,"reported_epoch":45,"state":"active+clean","last_fresh":"2026-02-24T07:47:05.750774+0000","last_change":"2026-02-24T07:46:02.880661+0000","last_active":"2026-02-24T07:47:05.750774+0000","last_peered":"2026-02-24T07:47:05.750774+0000","last_clean":"2026-02-24T07:47:05.750774+0000","last_became_active":"2026-02-24T07:46:02.880345+0000","last_became_peered":"2026-02-24T07:46:02.880345+0000","last_unstale":"2026-02-24T07:47:05.750774+0000","last_undegraded":"2026-02-24T07:47:05.750774+0000","last_fullsized":"2026-02-24T07:47:05.750774+0000","mapping_epoch":42,"log_start":"45'29399","ondisk_log_start":"45'29399","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_clean_scrub_stamp":"2026-02-24T07:46:01.867949+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:47:27.577904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1926889472,"num_objects":29402,"num_object_clones":0,"num_object_copies":88206,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29402,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":29402,"num_write_kb":1881728,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"45'29749","reported_seq":29766,"reported_epoch":45,"state":"active+clean","last_fresh":"2026-02-24T07:47:05.680255+0000","last_change":"2026-02-24T07:46:02.881591+0000","last_active":"2026-02-24T07:47:05.680255+0000","last_peered":"2026-02-24T07:47:05.680255+0000","last_clean":"2026-02-24T07:47:05.680255+0000","last_became_active":"2026-02-24T07:46:02.881035+0000","last_became_peered":"2026-02-24T07:46:02.881035+0000","last_unstale":"2026-02-24T07:47:05.680255+0000","last_undegraded":"2026-02-24T07:47:05.680255+0000","last_fullsized":"2026-02-24T07:47:05.680255+0000","mapping_epoch":42,"log_start":"45'29746","ondisk_log_start":"45'29746","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_clean_scrub_stamp":"2026-02-24T07:46:01.867949+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1949630464,"num_objects":29749,"num_object_clones":0,"num_object_copies":89247,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29749,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":29749,"num_write_kb":1903936,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"45'29901","reported_seq":29917,"reported_epoch":45,"state":"active+clean","last_fresh":"2026-02-24T07:47:05.705152+0000","last_change":"2026-02-24T07:46:02.879955+0000","last_active":"2026-02-24T07:47:05.705152+0000","last_peered":"2026-02-24T07:47:05.705152+0000","last_clean":"2026-02-24T07:47:05.705152+0000","last_became_active":"2026-02-24T07:46:02.879831+0000","last_became_peered":"2026-02-24T07:46:02.879831+0000","last_unstale":"2026-02-24T07:47:05.705152+0000","last_undegraded":"2026-02-24T07:47:05.705152+0000","last_fullsized":"2026-02-24T07:47:05.705152+0000","mapping_epoch":42,"log_start":"45'29898","ondisk_log_start":"45'29898","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_clean_scrub_stamp":"2026-02-24T07:46:01.867949+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:47:19.157247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1959591936,"num_objects":29901,"num_object_clones":0,"num_object_copies":89703,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29901,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":29901,"num_write_kb":1913664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.0","version":"45'29534","reported_seq":29604,"reported_epoch":45,"state":"active+clean+scrubbing","last_fresh":"2026-02-24T07:47:05.754049+0000","last_change":"2026-02-24T07:47:05.326458+0000","last_active":"2026-02-24T07:47:05.754049+0000","last_peered":"2026-02-24T07:47:05.754049+0000","last_clean":"2026-02-24T07:47:05.754049+0000","last_became_active":"2026-02-24T07:46:02.880333+0000","last_became_peered":"2026-02-24T07:46:02.880333+0000","last_unstale":"2026-02-24T07:47:05.754049+0000","last_undegraded":"2026-02-24T07:47:05.754049+0000","last_fullsized":"2026-02-24T07:47:05.754049+0000","mapping_epoch":42,"log_start":"45'29531","ondisk_log_start":"45'29531","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_clean_scrub_stamp":"2026-02-24T07:46:01.867949+0000","objects_scrubbed":4999,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"scrubbing for 1s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1935540224,"num_objects":29534,"num_object_clones":0,"num_object_copies":88602,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29534,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":29534,"num_write_kb":1890176,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2","version":"45'29830","reported_seq":29847,"reported_epoch":45,"state":"active+clean","last_fresh":"2026-02-24T07:47:05.743974+0000","last_change":"2026-02-24T07:46:02.880566+0000","last_active":"2026-02-24T07:47:05.743974+0000","last_peered":"2026-02-24T07:47:05.743974+0000","last_clean":"2026-02-24T07:47:05.743974+0000","last_became_active":"2026-02-24T07:46:02.880185+0000","last_became_peered":"2026-02-24T07:46:02.880185+0000","last_unstale":"2026-02-24T07:47:05.743974+0000","last_undegraded":"2026-02-24T07:47:05.743974+0000","last_fullsized":"2026-02-24T07:47:05.743974+0000","mapping_epoch":42,"log_start":"45'29827","ondisk_log_start":"45'29827","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_clean_scrub_stamp":"2026-02-24T07:46:01.867949+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:47:23.244686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1954938880,"num_objects":29830,"num_object_clones":0,"num_object_copies":89490,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29830,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":29830,"num_write_kb":1909120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"29'99","reported_seq":237,"reported_epoch":45,"state":"active+clean","last_fresh":"2026-02-24T07:46:41.007543+0000","last_change":"2026-02-24T07:46:41.007499+0000","last_active":"2026-02-24T07:46:41.007543+0000","last_peered":"2026-02-24T07:46:41.007543+0000","last_clean":"2026-02-24T07:46:41.007543+0000","last_became_active":"2026-02-24T07:42:54.186719+0000","last_became_peered":"2026-02-24T07:42:54.186719+0000","last_unstale":"2026-02-24T07:46:41.007543+0000","last_undegraded":"2026-02-24T07:46:41.007543+0000","last_fullsized":"2026-02-24T07:46:41.007543+0000","mapping_epoch":27,"log_start":"29'96","ondisk_log_start":"29'96","created":20,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'99","last_scrub_stamp":"2026-02-24T07:46:41.007434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:42:43.640776+0000","last_clean_scrub_stamp":"2026-02-24T07:46:41.007434+0000","objects_scrubbed":2,"log_size":3,"log_dups_size":96,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:47:53.486511+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":240,"num_write":135,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":16,"stat_sum":{"num_bytes":31152996352,"num_objects":475357,"num_object_clones":0,"num_object_copies":1426071,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":475357,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":475357,"num_write_kb":30422848,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":93454139392,"data_stored":93454139392,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":49,"ondisk_log_size":49,"up":48,"acting":48,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":240,"num_write":135,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":2120,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":3,"up_from":25,"seq":107374182453,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":20427848,"kb_used_data":18798452,"kb_used_omap":9,"kb_used_meta":1629366,"kb_avail":713575352,"statfs":{"total":751619276800,"available":730701160448,"internally_reserved":0,"allocated":19249614848,"data_stored":19249326204,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9784,"internal_metadata":1668471240},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":19,"seq":81604378679,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":28336328,"kb_used_data":26556084,"kb_used_omap":9,"kb_used_meta":1780214,"kb_avail":705666872,"statfs":{"total":751619276800,"available":722602876928,"internally_reserved":0,"allocated":27193430016,"data_stored":27193141372,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9565,"internal_metadata":1822939811},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[5,0,0,1],"upper_bound":16},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542200,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":22459012,"kb_used_data":20783792,"kb_used_omap":6,"kb_used_meta":1675193,"kb_avail":711544188,"statfs":{"total":751619276800,"available":728621248512,"internally_reserved":0,"allocated":21282603008,"data_stored":21282317916,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6807,"internal_metadata":1715397993},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705723,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":26154824,"kb_used_data":24501108,"kb_used_omap":8,"kb_used_meta":1653687,"kb_avail":707848376,"statfs":{"total":751619276800,"available":724836737024,"internally_reserved":0,"allocated":25089134592,"data_stored":25088845948,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9014,"internal_metadata":1693375690},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":2120,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":25286606848,"data_stored":25286606848,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":21398749184,"data_stored":21398749184,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":27288862720,"data_stored":27288862720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":19479920640,"data_stored":19479920640,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-24T07:47:11.449 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph osd pool set .mgr pgp_num 131 2026-02-24T07:47:11.582 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:47:11.694 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2026-02-24T07:47:11.720 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:47:11.719+0000 7fbb3d4a2640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:47:11.720 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:47:11.719+0000 7fbb3d4a2640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:47:11.853 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:47:11.872 INFO:teuthology.orchestra.run.trial035.stderr:Error EPERM: crimson_allow_pg_split is false; pgp_num increase denied 2026-02-24T07:47:11.872 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:47:11 trial035 ceph-mon[39284]: 3.b scrub ok 2026-02-24T07:47:11.916 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-24T07:47:11.969 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:47:11.968+0000 7fafa515a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:47:11.969 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:47:11.968+0000 7fafa515a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:47:11.978 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 68 16 514976 514960 473.268 407.438 0.000970464 0.0021117 2026-02-24T07:47:11.985 INFO:teuthology.orchestra.run.trial035.stdout:osd.0: {} 2026-02-24T07:47:11.985 INFO:teuthology.orchestra.run.trial035.stderr:osd.0: osd_enable_op_tracker = '' 2026-02-24T07:47:12.001 INFO:teuthology.orchestra.run.trial035.stdout:osd.1: {} 2026-02-24T07:47:12.001 INFO:teuthology.orchestra.run.trial035.stderr:osd.1: osd_enable_op_tracker = '' 2026-02-24T07:47:12.014 INFO:teuthology.orchestra.run.trial035.stdout:osd.2: {} 2026-02-24T07:47:12.014 INFO:teuthology.orchestra.run.trial035.stderr:osd.2: osd_enable_op_tracker = '' 2026-02-24T07:47:12.028 INFO:teuthology.orchestra.run.trial035.stdout:osd.3: {} 2026-02-24T07:47:12.028 INFO:teuthology.orchestra.run.trial035.stderr:osd.3: osd_enable_op_tracker = '' 2026-02-24T07:47:12.978 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 69 15 522438 522423 473.169 466.438 0.00128535 0.00211216 2026-02-24T07:47:13.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:47:12 trial035 ceph-mon[39284]: from='client.14696 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2026-02-24T07:47:13.205 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:47:12 trial035 ceph-mon[39284]: 3.3 scrub starts 2026-02-24T07:47:13.205 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:47:12 trial035 ceph-mon[39284]: pgmap v145: 17 pgs: 17 active+clean; 31 GiB data, 99 GiB used, 2.6 TiB / 2.7 TiB avail; 543 MiB/s wr, 8.68k op/s 2026-02-24T07:47:13.205 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:47:12 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/130339708' entity='client.admin' cmd={"prefix": "osd pool set", "pool": ".mgr", "var": "pgp_num", "val": "131"} : dispatch 2026-02-24T07:47:13.205 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:47:12 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/2448701083' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2026-02-24T07:47:13.978 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 70 15 529942 529927 473.109 469 0.00190919 0.00211241 2026-02-24T07:47:14.072 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2026-02-24T07:47:14.097 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:47:13 trial035 ceph-mon[39284]: 3.f scrub starts 2026-02-24T07:47:14.211 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:47:14.332 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:47:14.330+0000 7f331a91a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:47:14.332 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:47:14.331+0000 7f331a91a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:47:14.347 INFO:teuthology.orchestra.run.trial035.stdout:osd.0: {} 2026-02-24T07:47:14.347 INFO:teuthology.orchestra.run.trial035.stderr:osd.0: osd_enable_op_tracker = '' 2026-02-24T07:47:14.361 INFO:teuthology.orchestra.run.trial035.stdout:osd.1: {} 2026-02-24T07:47:14.361 INFO:teuthology.orchestra.run.trial035.stderr:osd.1: osd_enable_op_tracker = '' 2026-02-24T07:47:14.374 INFO:teuthology.orchestra.run.trial035.stdout:osd.2: {} 2026-02-24T07:47:14.374 INFO:teuthology.orchestra.run.trial035.stderr:osd.2: osd_enable_op_tracker = '' 2026-02-24T07:47:14.387 INFO:teuthology.orchestra.run.trial035.stdout:osd.3: {} 2026-02-24T07:47:14.387 INFO:teuthology.orchestra.run.trial035.stderr:osd.3: osd_enable_op_tracker = '' 2026-02-24T07:47:14.978 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 71 16 537091 537075 472.737 446.75 0.00143665 0.00211408 2026-02-24T07:47:15.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:47:14 trial035 ceph-mon[39284]: pgmap v146: 17 pgs: 17 active+clean; 31 GiB data, 99 GiB used, 2.6 TiB / 2.7 TiB avail; 344 MiB/s wr, 5.50k op/s 2026-02-24T07:47:15.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:47:14 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/2405054957' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2026-02-24T07:47:15.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:47:14 trial035 ceph-mon[39284]: 3.3 scrub ok 2026-02-24T07:47:15.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:47:14 trial035 ceph-mon[39284]: 3.5 scrub starts 2026-02-24T07:47:15.978 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 72 15 544278 544263 472.41 449.25 0.00157459 0.00211553 2026-02-24T07:47:16.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:47:15 trial035 ceph-mon[39284]: from='mgr.14284 10.20.193.35:0/3681373222' entity='mgr.x' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2026-02-24T07:47:16.429 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2026-02-24T07:47:16.572 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:47:16.711 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:47:16.710+0000 7fccd78da640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:47:16.711 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:47:16.710+0000 7fccd78da640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:47:16.726 INFO:teuthology.orchestra.run.trial035.stdout:osd.0: {} 2026-02-24T07:47:16.726 INFO:teuthology.orchestra.run.trial035.stderr:osd.0: osd_enable_op_tracker = '' 2026-02-24T07:47:16.740 INFO:teuthology.orchestra.run.trial035.stdout:osd.1: {} 2026-02-24T07:47:16.740 INFO:teuthology.orchestra.run.trial035.stderr:osd.1: osd_enable_op_tracker = '' 2026-02-24T07:47:16.754 INFO:teuthology.orchestra.run.trial035.stdout:osd.2: {} 2026-02-24T07:47:16.754 INFO:teuthology.orchestra.run.trial035.stderr:osd.2: osd_enable_op_tracker = '' 2026-02-24T07:47:16.768 INFO:teuthology.orchestra.run.trial035.stdout:osd.3: {} 2026-02-24T07:47:16.768 INFO:teuthology.orchestra.run.trial035.stderr:osd.3: osd_enable_op_tracker = '' 2026-02-24T07:47:16.917 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2026-02-24T07:47:16.917 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2026-02-24T07:47:16.918 INFO:tasks.thrashosds.thrasher:fixing pg num pool .mgr 2026-02-24T07:47:16.918 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph pg dump --format=json 2026-02-24T07:47:16.974 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:47:16 trial035 ceph-mon[39284]: pgmap v147: 17 pgs: 17 active+clean; 31 GiB data, 99 GiB used, 2.6 TiB / 2.7 TiB avail; 344 MiB/s wr, 5.50k op/s 2026-02-24T07:47:16.974 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:47:16 trial035 ceph-mon[39284]: 3.f scrub ok 2026-02-24T07:47:16.974 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:47:16 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/4189617145' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2026-02-24T07:47:16.978 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 73 16 550938 550922 471.64 416.188 0.00974194 0.00211888 2026-02-24T07:47:17.264 INFO:journalctl@ceph.prometheus.a.trial091.stdout:Feb 24 07:47:16 trial091 ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-prometheus-a[66007]: time=2026-02-24T07:47:16.973Z level=ERROR source=scrape.go:1630 msg="Failed to determine correct type of scrape target." component="scrape manager" scrape_pool=ceph target=http://trial035:9283/metrics content_type="text/html;charset=utf-8" fallback_media_type="" err="received unsupported Content-Type \"text/html;charset=utf-8\" and no fallback_scrape_protocol specified for target" 2026-02-24T07:47:17.266 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:47:17.322 INFO:journalctl@ceph.mgr.x.trial035.stdout:Feb 24 07:47:16 trial035 ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-mgr-x[39621]: ::ffff:10.20.193.91 - - [24/Feb/2026:07:47:16] "GET /metrics HTTP/1.1" 200 4221 "" "Prometheus/3.6.0" 2026-02-24T07:47:17.730 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:47:17.703+0000 7f0d691c9640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:47:17.730 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:47:17.703+0000 7f0d691c9640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:47:17.845 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:47:17 trial035 ceph-mon[39284]: 3.5 scrub ok 2026-02-24T07:47:17.845 INFO:teuthology.orchestra.run.trial035.stdout: 2026-02-24T07:47:17.845 INFO:teuthology.orchestra.run.trial035.stderr:dumped all 2026-02-24T07:47:17.893 INFO:teuthology.orchestra.run.trial035.stdout:{"pg_ready":true,"pg_map":{"version":148,"stamp":"2026-02-24T07:47:17.692306+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":35555967520,"num_objects":542534,"num_object_clones":0,"num_object_copies":1627602,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":542534,"num_whiteouts":0,"num_read":136,"num_read_kb":240,"num_write":542667,"num_write_kb":34724198,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":51,"ondisk_log_size":51,"up":51,"acting":51,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":51,"num_osds":4,"num_per_pool_osds":4,"num_per_pool_omap_osds":4,"kb":2936012800,"kb_used":110923036,"kb_used_data":103462860,"kb_used_omap":34,"kb_used_meta":7460061,"kb_avail":2825089764,"statfs":{"total":3006477107200,"available":2892891918336,"internally_reserved":0,"allocated":105945968640,"data_stored":105944817616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":35170,"internal_metadata":7639103134},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[8,1],"upper_bound":4},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":6102056960,"num_objects":93110,"num_object_clones":0,"num_object_copies":279330,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":93110,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":93110,"num_write_kb":5959040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001618"},"pg_stats":[{"pgid":"3.f","version":"45'33813","reported_seq":34092,"reported_epoch":45,"state":"active+clean+scrubbing","last_fresh":"2026-02-24T07:47:15.691117+0000","last_change":"2026-02-24T07:47:13.392294+0000","last_active":"2026-02-24T07:47:15.691117+0000","last_peered":"2026-02-24T07:47:15.691117+0000","last_clean":"2026-02-24T07:47:15.691117+0000","last_became_active":"2026-02-24T07:46:02.881625+0000","last_became_peered":"2026-02-24T07:46:02.881625+0000","last_unstale":"2026-02-24T07:47:15.691117+0000","last_undegraded":"2026-02-24T07:47:15.691117+0000","last_fullsized":"2026-02-24T07:47:15.691117+0000","mapping_epoch":42,"log_start":"45'33810","ondisk_log_start":"45'33810","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_clean_scrub_stamp":"2026-02-24T07:46:01.867949+0000","objects_scrubbed":25799,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"scrubbing for 3s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2215968768,"num_objects":33813,"num_object_clones":0,"num_object_copies":101439,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33813,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":33813,"num_write_kb":2164032,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.e","version":"45'34243","reported_seq":34260,"reported_epoch":45,"state":"active+clean","last_fresh":"2026-02-24T07:47:15.686263+0000","last_change":"2026-02-24T07:46:02.881462+0000","last_active":"2026-02-24T07:47:15.686263+0000","last_peered":"2026-02-24T07:47:15.686263+0000","last_clean":"2026-02-24T07:47:15.686263+0000","last_became_active":"2026-02-24T07:46:02.881083+0000","last_became_peered":"2026-02-24T07:46:02.881083+0000","last_unstale":"2026-02-24T07:47:15.686263+0000","last_undegraded":"2026-02-24T07:47:15.686263+0000","last_fullsized":"2026-02-24T07:47:15.686263+0000","mapping_epoch":42,"log_start":"45'34240","ondisk_log_start":"45'34240","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_clean_scrub_stamp":"2026-02-24T07:46:01.867949+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:47:29.030602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2244149248,"num_objects":34243,"num_object_clones":0,"num_object_copies":102729,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34243,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":34243,"num_write_kb":2191552,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"45'33840","reported_seq":34162,"reported_epoch":45,"state":"active+clean","last_fresh":"2026-02-24T07:47:15.704523+0000","last_change":"2026-02-24T07:47:06.968813+0000","last_active":"2026-02-24T07:47:15.704523+0000","last_peered":"2026-02-24T07:47:15.704523+0000","last_clean":"2026-02-24T07:47:15.704523+0000","last_became_active":"2026-02-24T07:46:02.883799+0000","last_became_peered":"2026-02-24T07:46:02.883799+0000","last_unstale":"2026-02-24T07:47:15.704523+0000","last_undegraded":"2026-02-24T07:47:15.704523+0000","last_fullsized":"2026-02-24T07:47:15.704523+0000","mapping_epoch":42,"log_start":"45'33837","ondisk_log_start":"45'33837","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"45'30090","last_scrub_stamp":"2026-02-24T07:47:06.968732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_clean_scrub_stamp":"2026-02-24T07:47:06.968732+0000","objects_scrubbed":29542,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:48:25.198085+0000","scrub_duration":3070,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2217738240,"num_objects":33840,"num_object_clones":0,"num_object_copies":101520,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33840,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":33840,"num_write_kb":2165760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"45'33346","reported_seq":33668,"reported_epoch":45,"state":"active+clean","last_fresh":"2026-02-24T07:47:15.712047+0000","last_change":"2026-02-24T07:47:07.546834+0000","last_active":"2026-02-24T07:47:15.712047+0000","last_peered":"2026-02-24T07:47:15.712047+0000","last_clean":"2026-02-24T07:47:15.712047+0000","last_became_active":"2026-02-24T07:46:02.880060+0000","last_became_peered":"2026-02-24T07:46:02.880060+0000","last_unstale":"2026-02-24T07:47:15.712047+0000","last_undegraded":"2026-02-24T07:47:15.712047+0000","last_fullsized":"2026-02-24T07:47:15.712047+0000","mapping_epoch":42,"log_start":"45'33343","ondisk_log_start":"45'33343","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"45'29910","last_scrub_stamp":"2026-02-24T07:47:07.546803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_clean_scrub_stamp":"2026-02-24T07:47:07.546803+0000","objects_scrubbed":29435,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:48:27.524515+0000","scrub_duration":2696,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2185363456,"num_objects":33346,"num_object_clones":0,"num_object_copies":100038,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33346,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":33346,"num_write_kb":2134144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"45'33947","reported_seq":34281,"reported_epoch":45,"state":"active+clean","last_fresh":"2026-02-24T07:47:15.755820+0000","last_change":"2026-02-24T07:47:10.376412+0000","last_active":"2026-02-24T07:47:15.755820+0000","last_peered":"2026-02-24T07:47:15.755820+0000","last_clean":"2026-02-24T07:47:15.755820+0000","last_became_active":"2026-02-24T07:46:02.880171+0000","last_became_peered":"2026-02-24T07:46:02.880171+0000","last_unstale":"2026-02-24T07:47:15.755820+0000","last_undegraded":"2026-02-24T07:47:15.755820+0000","last_fullsized":"2026-02-24T07:47:15.755820+0000","mapping_epoch":42,"log_start":"45'33944","ondisk_log_start":"45'33944","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"45'31519","last_scrub_stamp":"2026-02-24T07:47:10.376345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_clean_scrub_stamp":"2026-02-24T07:47:10.376345+0000","objects_scrubbed":30810,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:48:13.377288+0000","scrub_duration":4056,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2224750592,"num_objects":33947,"num_object_clones":0,"num_object_copies":101841,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33947,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":33947,"num_write_kb":2172608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.a","version":"45'33592","reported_seq":33609,"reported_epoch":45,"state":"active+clean","last_fresh":"2026-02-24T07:47:15.714245+0000","last_change":"2026-02-24T07:46:02.882761+0000","last_active":"2026-02-24T07:47:15.714245+0000","last_peered":"2026-02-24T07:47:15.714245+0000","last_clean":"2026-02-24T07:47:15.714245+0000","last_became_active":"2026-02-24T07:46:02.882274+0000","last_became_peered":"2026-02-24T07:46:02.882274+0000","last_unstale":"2026-02-24T07:47:15.714245+0000","last_undegraded":"2026-02-24T07:47:15.714245+0000","last_fullsized":"2026-02-24T07:47:15.714245+0000","mapping_epoch":42,"log_start":"45'33589","ondisk_log_start":"45'33589","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_clean_scrub_stamp":"2026-02-24T07:46:01.867949+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:47:31.776406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2201485312,"num_objects":33592,"num_object_clones":0,"num_object_copies":100776,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33592,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":33592,"num_write_kb":2149888,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"45'34349","reported_seq":34690,"reported_epoch":45,"state":"active+clean","last_fresh":"2026-02-24T07:47:15.714312+0000","last_change":"2026-02-24T07:47:10.197277+0000","last_active":"2026-02-24T07:47:15.714312+0000","last_peered":"2026-02-24T07:47:15.714312+0000","last_clean":"2026-02-24T07:47:15.714312+0000","last_became_active":"2026-02-24T07:46:02.881951+0000","last_became_peered":"2026-02-24T07:46:02.881951+0000","last_unstale":"2026-02-24T07:47:15.714312+0000","last_undegraded":"2026-02-24T07:47:15.714312+0000","last_fullsized":"2026-02-24T07:47:15.714312+0000","mapping_epoch":42,"log_start":"45'34346","ondisk_log_start":"45'34346","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"45'31847","last_scrub_stamp":"2026-02-24T07:47:10.197239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_clean_scrub_stamp":"2026-02-24T07:47:10.197239+0000","objects_scrubbed":31279,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:48:27.205650+0000","scrub_duration":3289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2251096064,"num_objects":34349,"num_object_clones":0,"num_object_copies":103047,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34349,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":34349,"num_write_kb":2198336,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"45'33955","reported_seq":33972,"reported_epoch":45,"state":"active+clean","last_fresh":"2026-02-24T07:47:15.755339+0000","last_change":"2026-02-24T07:46:02.881463+0000","last_active":"2026-02-24T07:47:15.755339+0000","last_peered":"2026-02-24T07:47:15.755339+0000","last_clean":"2026-02-24T07:47:15.755339+0000","last_became_active":"2026-02-24T07:46:02.881345+0000","last_became_peered":"2026-02-24T07:46:02.881345+0000","last_unstale":"2026-02-24T07:47:15.755339+0000","last_undegraded":"2026-02-24T07:47:15.755339+0000","last_fullsized":"2026-02-24T07:47:15.755339+0000","mapping_epoch":42,"log_start":"45'33952","ondisk_log_start":"45'33952","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_clean_scrub_stamp":"2026-02-24T07:46:01.867949+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:47:17.135287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2225274880,"num_objects":33955,"num_object_clones":0,"num_object_copies":101865,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33955,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":33955,"num_write_kb":2173120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"45'33753","reported_seq":33770,"reported_epoch":45,"state":"active+clean","last_fresh":"2026-02-24T07:47:15.752351+0000","last_change":"2026-02-24T07:46:02.880582+0000","last_active":"2026-02-24T07:47:15.752351+0000","last_peered":"2026-02-24T07:47:15.752351+0000","last_clean":"2026-02-24T07:47:15.752351+0000","last_became_active":"2026-02-24T07:46:02.880323+0000","last_became_peered":"2026-02-24T07:46:02.880323+0000","last_unstale":"2026-02-24T07:47:15.752351+0000","last_undegraded":"2026-02-24T07:47:15.752351+0000","last_fullsized":"2026-02-24T07:47:15.752351+0000","mapping_epoch":42,"log_start":"45'33750","ondisk_log_start":"45'33750","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_clean_scrub_stamp":"2026-02-24T07:46:01.867949+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:47:31.420697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2212036608,"num_objects":33753,"num_object_clones":0,"num_object_copies":101259,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33753,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":33753,"num_write_kb":2160192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.6","version":"45'34065","reported_seq":34082,"reported_epoch":45,"state":"active+clean","last_fresh":"2026-02-24T07:47:15.705634+0000","last_change":"2026-02-24T07:46:02.880170+0000","last_active":"2026-02-24T07:47:15.705634+0000","last_peered":"2026-02-24T07:47:15.705634+0000","last_clean":"2026-02-24T07:47:15.705634+0000","last_became_active":"2026-02-24T07:46:02.880005+0000","last_became_peered":"2026-02-24T07:46:02.880005+0000","last_unstale":"2026-02-24T07:47:15.705634+0000","last_undegraded":"2026-02-24T07:47:15.705634+0000","last_fullsized":"2026-02-24T07:47:15.705634+0000","mapping_epoch":42,"log_start":"45'34062","ondisk_log_start":"45'34062","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_clean_scrub_stamp":"2026-02-24T07:46:01.867949+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2232483840,"num_objects":34065,"num_object_clones":0,"num_object_copies":102195,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34065,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":34065,"num_write_kb":2180160,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"45'34114","reported_seq":34276,"reported_epoch":45,"state":"active+clean+scrubbing","last_fresh":"2026-02-24T07:47:15.691418+0000","last_change":"2026-02-24T07:47:14.430082+0000","last_active":"2026-02-24T07:47:15.691418+0000","last_peered":"2026-02-24T07:47:15.691418+0000","last_clean":"2026-02-24T07:47:15.691418+0000","last_became_active":"2026-02-24T07:46:02.881420+0000","last_became_peered":"2026-02-24T07:46:02.881420+0000","last_unstale":"2026-02-24T07:47:15.691418+0000","last_undegraded":"2026-02-24T07:47:15.691418+0000","last_fullsized":"2026-02-24T07:47:15.691418+0000","mapping_epoch":42,"log_start":"45'34111","ondisk_log_start":"45'34111","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_clean_scrub_stamp":"2026-02-24T07:46:01.867949+0000","objects_scrubbed":14099,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"scrubbing for 2s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2235695104,"num_objects":34114,"num_object_clones":0,"num_object_copies":102342,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34114,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":34114,"num_write_kb":2183296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.4","version":"45'33638","reported_seq":33655,"reported_epoch":45,"state":"active+clean","last_fresh":"2026-02-24T07:47:15.756486+0000","last_change":"2026-02-24T07:46:02.880661+0000","last_active":"2026-02-24T07:47:15.756486+0000","last_peered":"2026-02-24T07:47:15.756486+0000","last_clean":"2026-02-24T07:47:15.756486+0000","last_became_active":"2026-02-24T07:46:02.880345+0000","last_became_peered":"2026-02-24T07:46:02.880345+0000","last_unstale":"2026-02-24T07:47:15.756486+0000","last_undegraded":"2026-02-24T07:47:15.756486+0000","last_fullsized":"2026-02-24T07:47:15.756486+0000","mapping_epoch":42,"log_start":"45'33635","ondisk_log_start":"45'33635","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_clean_scrub_stamp":"2026-02-24T07:46:01.867949+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:47:27.577904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2204499968,"num_objects":33638,"num_object_clones":0,"num_object_copies":100914,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33638,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":33638,"num_write_kb":2152832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"45'33911","reported_seq":34265,"reported_epoch":45,"state":"active+clean","last_fresh":"2026-02-24T07:47:15.686493+0000","last_change":"2026-02-24T07:47:14.339634+0000","last_active":"2026-02-24T07:47:15.686493+0000","last_peered":"2026-02-24T07:47:15.686493+0000","last_clean":"2026-02-24T07:47:15.686493+0000","last_became_active":"2026-02-24T07:46:02.881035+0000","last_became_peered":"2026-02-24T07:46:02.881035+0000","last_unstale":"2026-02-24T07:47:15.686493+0000","last_undegraded":"2026-02-24T07:47:15.686493+0000","last_fullsized":"2026-02-24T07:47:15.686493+0000","mapping_epoch":42,"log_start":"45'33908","ondisk_log_start":"45'33908","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"45'33283","last_scrub_stamp":"2026-02-24T07:47:14.339597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_clean_scrub_stamp":"2026-02-24T07:47:14.339597+0000","objects_scrubbed":32630,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:48:22.078719+0000","scrub_duration":2951,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2222391296,"num_objects":33911,"num_object_clones":0,"num_object_copies":101733,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33911,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":33911,"num_write_kb":2170304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"45'34043","reported_seq":34060,"reported_epoch":45,"state":"active+clean","last_fresh":"2026-02-24T07:47:15.704482+0000","last_change":"2026-02-24T07:46:02.879955+0000","last_active":"2026-02-24T07:47:15.704482+0000","last_peered":"2026-02-24T07:47:15.704482+0000","last_clean":"2026-02-24T07:47:15.704482+0000","last_became_active":"2026-02-24T07:46:02.879831+0000","last_became_peered":"2026-02-24T07:46:02.879831+0000","last_unstale":"2026-02-24T07:47:15.704482+0000","last_undegraded":"2026-02-24T07:47:15.704482+0000","last_fullsized":"2026-02-24T07:47:15.704482+0000","mapping_epoch":42,"log_start":"45'34040","ondisk_log_start":"45'34040","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_clean_scrub_stamp":"2026-02-24T07:46:01.867949+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:47:19.157247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2231042048,"num_objects":34043,"num_object_clones":0,"num_object_copies":102129,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34043,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":34043,"num_write_kb":2178752,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.0","version":"45'33722","reported_seq":34049,"reported_epoch":45,"state":"active+clean","last_fresh":"2026-02-24T07:47:15.750273+0000","last_change":"2026-02-24T07:47:08.376360+0000","last_active":"2026-02-24T07:47:15.750273+0000","last_peered":"2026-02-24T07:47:15.750273+0000","last_clean":"2026-02-24T07:47:15.750273+0000","last_became_active":"2026-02-24T07:46:02.880333+0000","last_became_peered":"2026-02-24T07:46:02.880333+0000","last_unstale":"2026-02-24T07:47:15.750273+0000","last_undegraded":"2026-02-24T07:47:15.750273+0000","last_fullsized":"2026-02-24T07:47:15.750273+0000","mapping_epoch":42,"log_start":"45'33719","ondisk_log_start":"45'33719","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"45'30511","last_scrub_stamp":"2026-02-24T07:47:08.376304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_clean_scrub_stamp":"2026-02-24T07:47:08.376304+0000","objects_scrubbed":29912,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:48:24.552234+0000","scrub_duration":3049,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2210004992,"num_objects":33722,"num_object_clones":0,"num_object_copies":101166,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33722,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":33722,"num_write_kb":2158208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2","version":"45'34201","reported_seq":34218,"reported_epoch":45,"state":"active+clean","last_fresh":"2026-02-24T07:47:15.744224+0000","last_change":"2026-02-24T07:46:02.880566+0000","last_active":"2026-02-24T07:47:15.744224+0000","last_peered":"2026-02-24T07:47:15.744224+0000","last_clean":"2026-02-24T07:47:15.744224+0000","last_became_active":"2026-02-24T07:46:02.880185+0000","last_became_peered":"2026-02-24T07:46:02.880185+0000","last_unstale":"2026-02-24T07:47:15.744224+0000","last_undegraded":"2026-02-24T07:47:15.744224+0000","last_fullsized":"2026-02-24T07:47:15.744224+0000","mapping_epoch":42,"log_start":"45'34198","ondisk_log_start":"45'34198","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_clean_scrub_stamp":"2026-02-24T07:46:01.867949+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:47:23.244686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2241396736,"num_objects":34201,"num_object_clones":0,"num_object_copies":102603,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34201,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":34201,"num_write_kb":2188864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"29'99","reported_seq":237,"reported_epoch":45,"state":"active+clean","last_fresh":"2026-02-24T07:46:41.007543+0000","last_change":"2026-02-24T07:46:41.007499+0000","last_active":"2026-02-24T07:46:41.007543+0000","last_peered":"2026-02-24T07:46:41.007543+0000","last_clean":"2026-02-24T07:46:41.007543+0000","last_became_active":"2026-02-24T07:42:54.186719+0000","last_became_peered":"2026-02-24T07:42:54.186719+0000","last_unstale":"2026-02-24T07:46:41.007543+0000","last_undegraded":"2026-02-24T07:46:41.007543+0000","last_fullsized":"2026-02-24T07:46:41.007543+0000","mapping_epoch":27,"log_start":"29'96","ondisk_log_start":"29'96","created":20,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'99","last_scrub_stamp":"2026-02-24T07:46:41.007434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:42:43.640776+0000","last_clean_scrub_stamp":"2026-02-24T07:46:41.007434+0000","objects_scrubbed":2,"log_size":3,"log_dups_size":96,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:47:53.486511+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":240,"num_write":135,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":16,"stat_sum":{"num_bytes":35555377152,"num_objects":542532,"num_object_clones":0,"num_object_copies":1627596,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":542532,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":542532,"num_write_kb":34722048,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":106657021952,"data_stored":106657021952,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":48,"ondisk_log_size":48,"up":48,"acting":48,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":240,"num_write":135,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":2120,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":3,"up_from":25,"seq":107374182455,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":23221448,"kb_used_data":21447988,"kb_used_omap":9,"kb_used_meta":1773430,"kb_avail":710781752,"statfs":{"total":751619276800,"available":727840514048,"internally_reserved":0,"allocated":21962739712,"data_stored":21962451068,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9784,"internal_metadata":1815992776},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":19,"seq":81604378681,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":32332168,"kb_used_data":30317300,"kb_used_omap":9,"kb_used_meta":2014838,"kb_avail":701671032,"statfs":{"total":751619276800,"available":718511136768,"internally_reserved":0,"allocated":31044915200,"data_stored":31044626556,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9565,"internal_metadata":2063194787},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542202,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":25577476,"kb_used_data":23739824,"kb_used_omap":6,"kb_used_meta":1837625,"kb_avail":708425724,"statfs":{"total":751619276800,"available":725427941376,"internally_reserved":0,"allocated":24309579776,"data_stored":24309294684,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6807,"internal_metadata":1881728361},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[6,1],"upper_bound":4},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705725,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":29791944,"kb_used_data":27957748,"kb_used_omap":8,"kb_used_meta":1834167,"kb_avail":704211256,"statfs":{"total":751619276800,"available":721112326144,"internally_reserved":0,"allocated":28628733952,"data_stored":28628445308,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9014,"internal_metadata":1878187210},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":2120,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28862316544,"data_stored":28862316544,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":24419500032,"data_stored":24419500032,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":31146704896,"data_stored":31146704896,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":22228500480,"data_stored":22228500480,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-24T07:47:17.893 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph osd pool set .mgr pgp_num 131 2026-02-24T07:47:17.978 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 74 15 556628 556613 470.072 355.688 0.00200889 0.00212608 2026-02-24T07:47:18.044 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:47:18.179 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:47:18.178+0000 7f8eb6050640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:47:18.179 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:47:18.178+0000 7f8eb6050640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:47:18.322 INFO:teuthology.orchestra.run.trial035.stderr:Error EPERM: crimson_allow_pg_split is false; pgp_num increase denied 2026-02-24T07:47:18.365 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-24T07:47:18.979 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 75 16 564004 563988 469.95 460.938 0.00149683 0.00212659 2026-02-24T07:47:19.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:47:18 trial035 ceph-mon[39284]: pgmap v148: 17 pgs: 2 active+clean+scrubbing, 15 active+clean; 33 GiB data, 106 GiB used, 2.6 TiB / 2.7 TiB avail; 485 MiB/s wr, 7.76k op/s 2026-02-24T07:47:19.205 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:47:18 trial035 ceph-mon[39284]: from='client.14706 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2026-02-24T07:47:19.205 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:47:18 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/1144597610' entity='client.admin' cmd={"prefix": "osd pool set", "pool": ".mgr", "var": "pgp_num", "val": "131"} : dispatch 2026-02-24T07:47:19.346 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2026-02-24T07:47:19.499 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:47:19.625 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:47:19.623+0000 7f53e28a1640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:47:19.625 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:47:19.623+0000 7f53e28a1640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:47:19.643 INFO:teuthology.orchestra.run.trial035.stdout:osd.0: {} 2026-02-24T07:47:19.643 INFO:teuthology.orchestra.run.trial035.stderr:osd.0: osd_enable_op_tracker = '' 2026-02-24T07:47:19.658 INFO:teuthology.orchestra.run.trial035.stdout:osd.1: {} 2026-02-24T07:47:19.659 INFO:teuthology.orchestra.run.trial035.stderr:osd.1: osd_enable_op_tracker = '' 2026-02-24T07:47:19.671 INFO:teuthology.orchestra.run.trial035.stdout:osd.2: {} 2026-02-24T07:47:19.672 INFO:teuthology.orchestra.run.trial035.stderr:osd.2: osd_enable_op_tracker = '' 2026-02-24T07:47:19.683 INFO:teuthology.orchestra.run.trial035.stdout:osd.3: {} 2026-02-24T07:47:19.683 INFO:teuthology.orchestra.run.trial035.stderr:osd.3: osd_enable_op_tracker = '' 2026-02-24T07:47:19.954 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:47:19 trial035 ceph-mon[39284]: 3.8 scrub starts 2026-02-24T07:47:19.955 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:47:19 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/3535789251' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2026-02-24T07:47:19.979 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 76 15 570873 570858 469.416 429.375 0.00106904 0.00212906 2026-02-24T07:47:20.979 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 77 15 577410 577395 468.625 408.562 0.0010614 0.00213261 2026-02-24T07:47:21.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:47:20 trial035 ceph-mon[39284]: pgmap v149: 17 pgs: 2 active+clean+scrubbing, 15 active+clean; 33 GiB data, 106 GiB used, 2.6 TiB / 2.7 TiB avail; 350 MiB/s wr, 5.60k op/s 2026-02-24T07:47:21.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:47:20 trial035 ceph-mon[39284]: 3.6 scrub starts 2026-02-24T07:47:21.727 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2026-02-24T07:47:21.883 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:47:21.979 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 78 16 583810 583794 467.744 399.938 0.00438363 0.00213657 2026-02-24T07:47:22.040 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:47:22.037+0000 7efc32389640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:47:22.040 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:47:22.038+0000 7efc32389640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:47:22.041 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:47:21 trial035 ceph-mon[39284]: 3.1 scrub starts 2026-02-24T07:47:22.058 INFO:teuthology.orchestra.run.trial035.stdout:osd.0: {} 2026-02-24T07:47:22.059 INFO:teuthology.orchestra.run.trial035.stderr:osd.0: osd_enable_op_tracker = '' 2026-02-24T07:47:22.074 INFO:teuthology.orchestra.run.trial035.stdout:osd.1: {} 2026-02-24T07:47:22.075 INFO:teuthology.orchestra.run.trial035.stderr:osd.1: osd_enable_op_tracker = '' 2026-02-24T07:47:22.089 INFO:teuthology.orchestra.run.trial035.stdout:osd.2: {} 2026-02-24T07:47:22.089 INFO:teuthology.orchestra.run.trial035.stderr:osd.2: osd_enable_op_tracker = '' 2026-02-24T07:47:22.103 INFO:teuthology.orchestra.run.trial035.stdout:osd.3: {} 2026-02-24T07:47:22.104 INFO:teuthology.orchestra.run.trial035.stderr:osd.3: osd_enable_op_tracker = '' 2026-02-24T07:47:22.979 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 79 16 590354 590338 467 409 0.00241567 0.00214004 2026-02-24T07:47:23.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:47:22 trial035 ceph-mon[39284]: pgmap v150: 17 pgs: 2 active+clean+scrubbing, 15 active+clean; 35 GiB data, 112 GiB used, 2.6 TiB / 2.7 TiB avail; 523 MiB/s wr, 8.36k op/s 2026-02-24T07:47:23.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:47:22 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/2354449164' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2026-02-24T07:47:23.366 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2026-02-24T07:47:23.366 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2026-02-24T07:47:23.366 INFO:tasks.thrashosds.thrasher:Growing pool .mgr 2026-02-24T07:47:23.367 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph pg dump --format=json 2026-02-24T07:47:23.510 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:47:23.809 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:47:23.797+0000 7fda0669d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:47:23.809 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:47:23.798+0000 7fda0669d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:47:23.943 INFO:teuthology.orchestra.run.trial035.stdout: 2026-02-24T07:47:23.943 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:47:23 trial035 ceph-mon[39284]: 3.8 scrub ok 2026-02-24T07:47:23.944 INFO:teuthology.orchestra.run.trial035.stderr:dumped all 2026-02-24T07:47:23.979 INFO:tasks.radosbench.radosbench.0.trial035.stdout:2026-02-24T07:47:23.978539+0000 min lat: 0.000682899 max lat: 0.0419194 avg lat: 0.00214548 2026-02-24T07:47:23.979 INFO:tasks.radosbench.radosbench.0.trial035.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2026-02-24T07:47:23.979 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 80 16 596319 596303 465.822 372.812 0.00217722 0.00214548 2026-02-24T07:47:24.254 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2026-02-24T07:47:24.383 INFO:teuthology.orchestra.run.trial035.stdout:{"pg_ready":true,"pg_map":{"version":151,"stamp":"2026-02-24T07:47:23.693170+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":37730320928,"num_objects":575712,"num_object_clones":0,"num_object_copies":1727136,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":575712,"num_whiteouts":0,"num_read":136,"num_read_kb":240,"num_write":575845,"num_write_kb":36847590,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":51,"ondisk_log_size":51,"up":51,"acting":51,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":51,"num_osds":4,"num_per_pool_osds":4,"num_per_pool_omap_osds":4,"kb":2936012800,"kb_used":117760284,"kb_used_data":109848844,"kb_used_omap":34,"kb_used_meta":7911261,"kb_avail":2818252516,"statfs":{"total":3006477107200,"available":2885890576384,"internally_reserved":0,"allocated":112485216256,"data_stored":112484065232,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":35170,"internal_metadata":8101131934},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[6,1],"upper_bound":4},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":4521787392,"num_objects":68997,"num_object_clones":0,"num_object_copies":206991,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":68997,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":68997,"num_write_kb":4415808,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001683"},"pg_stats":[{"pgid":"3.f","version":"45'35844","reported_seq":36206,"reported_epoch":45,"state":"active+clean","last_fresh":"2026-02-24T07:47:20.687449+0000","last_change":"2026-02-24T07:47:16.466592+0000","last_active":"2026-02-24T07:47:20.687449+0000","last_peered":"2026-02-24T07:47:20.687449+0000","last_clean":"2026-02-24T07:47:20.687449+0000","last_became_active":"2026-02-24T07:46:02.881625+0000","last_became_peered":"2026-02-24T07:46:02.881625+0000","last_unstale":"2026-02-24T07:47:20.687449+0000","last_undegraded":"2026-02-24T07:47:20.687449+0000","last_fullsized":"2026-02-24T07:47:20.687449+0000","mapping_epoch":42,"log_start":"45'35841","ondisk_log_start":"45'35841","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"45'34153","last_scrub_stamp":"2026-02-24T07:47:16.466530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_clean_scrub_stamp":"2026-02-24T07:47:16.466530+0000","objects_scrubbed":33463,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:48:23.487779+0000","scrub_duration":3074,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2349072384,"num_objects":35844,"num_object_clones":0,"num_object_copies":107532,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35844,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":35844,"num_write_kb":2294016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.e","version":"45'36365","reported_seq":36382,"reported_epoch":45,"state":"active+clean","last_fresh":"2026-02-24T07:47:20.691092+0000","last_change":"2026-02-24T07:46:02.881462+0000","last_active":"2026-02-24T07:47:20.691092+0000","last_peered":"2026-02-24T07:47:20.691092+0000","last_clean":"2026-02-24T07:47:20.691092+0000","last_became_active":"2026-02-24T07:46:02.881083+0000","last_became_peered":"2026-02-24T07:46:02.881083+0000","last_unstale":"2026-02-24T07:47:20.691092+0000","last_undegraded":"2026-02-24T07:47:20.691092+0000","last_fullsized":"2026-02-24T07:47:20.691092+0000","mapping_epoch":42,"log_start":"45'36362","ondisk_log_start":"45'36362","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_clean_scrub_stamp":"2026-02-24T07:46:01.867949+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:47:29.030602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2383216640,"num_objects":36365,"num_object_clones":0,"num_object_copies":109095,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36365,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":36365,"num_write_kb":2327360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"45'35938","reported_seq":36261,"reported_epoch":45,"state":"active+clean","last_fresh":"2026-02-24T07:47:20.706407+0000","last_change":"2026-02-24T07:47:06.968813+0000","last_active":"2026-02-24T07:47:20.706407+0000","last_peered":"2026-02-24T07:47:20.706407+0000","last_clean":"2026-02-24T07:47:20.706407+0000","last_became_active":"2026-02-24T07:46:02.883799+0000","last_became_peered":"2026-02-24T07:46:02.883799+0000","last_unstale":"2026-02-24T07:47:20.706407+0000","last_undegraded":"2026-02-24T07:47:20.706407+0000","last_fullsized":"2026-02-24T07:47:20.706407+0000","mapping_epoch":42,"log_start":"45'35935","ondisk_log_start":"45'35935","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"45'30090","last_scrub_stamp":"2026-02-24T07:47:06.968732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_clean_scrub_stamp":"2026-02-24T07:47:06.968732+0000","objects_scrubbed":29542,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:48:25.198085+0000","scrub_duration":3070,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2355232768,"num_objects":35938,"num_object_clones":0,"num_object_copies":107814,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35938,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":35938,"num_write_kb":2300032,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"45'35411","reported_seq":35732,"reported_epoch":45,"state":"active+clean","last_fresh":"2026-02-24T07:47:20.717202+0000","last_change":"2026-02-24T07:47:07.546834+0000","last_active":"2026-02-24T07:47:20.717202+0000","last_peered":"2026-02-24T07:47:20.717202+0000","last_clean":"2026-02-24T07:47:20.717202+0000","last_became_active":"2026-02-24T07:46:02.880060+0000","last_became_peered":"2026-02-24T07:46:02.880060+0000","last_unstale":"2026-02-24T07:47:20.717202+0000","last_undegraded":"2026-02-24T07:47:20.717202+0000","last_fullsized":"2026-02-24T07:47:20.717202+0000","mapping_epoch":42,"log_start":"45'35408","ondisk_log_start":"45'35408","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"45'29910","last_scrub_stamp":"2026-02-24T07:47:07.546803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_clean_scrub_stamp":"2026-02-24T07:47:07.546803+0000","objects_scrubbed":29435,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:48:27.524515+0000","scrub_duration":2696,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2320695296,"num_objects":35411,"num_object_clones":0,"num_object_copies":106233,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35411,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":35411,"num_write_kb":2266304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"45'36019","reported_seq":36353,"reported_epoch":45,"state":"active+clean","last_fresh":"2026-02-24T07:47:20.756832+0000","last_change":"2026-02-24T07:47:10.376412+0000","last_active":"2026-02-24T07:47:20.756832+0000","last_peered":"2026-02-24T07:47:20.756832+0000","last_clean":"2026-02-24T07:47:20.756832+0000","last_became_active":"2026-02-24T07:46:02.880171+0000","last_became_peered":"2026-02-24T07:46:02.880171+0000","last_unstale":"2026-02-24T07:47:20.756832+0000","last_undegraded":"2026-02-24T07:47:20.756832+0000","last_fullsized":"2026-02-24T07:47:20.756832+0000","mapping_epoch":42,"log_start":"45'36016","ondisk_log_start":"45'36016","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"45'31519","last_scrub_stamp":"2026-02-24T07:47:10.376345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_clean_scrub_stamp":"2026-02-24T07:47:10.376345+0000","objects_scrubbed":30810,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:48:13.377288+0000","scrub_duration":4056,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2360541184,"num_objects":36019,"num_object_clones":0,"num_object_copies":108057,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36019,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":36019,"num_write_kb":2305216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.a","version":"45'35604","reported_seq":35621,"reported_epoch":45,"state":"active+clean","last_fresh":"2026-02-24T07:47:20.705241+0000","last_change":"2026-02-24T07:46:02.882761+0000","last_active":"2026-02-24T07:47:20.705241+0000","last_peered":"2026-02-24T07:47:20.705241+0000","last_clean":"2026-02-24T07:47:20.705241+0000","last_became_active":"2026-02-24T07:46:02.882274+0000","last_became_peered":"2026-02-24T07:46:02.882274+0000","last_unstale":"2026-02-24T07:47:20.705241+0000","last_undegraded":"2026-02-24T07:47:20.705241+0000","last_fullsized":"2026-02-24T07:47:20.705241+0000","mapping_epoch":42,"log_start":"45'35601","ondisk_log_start":"45'35601","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_clean_scrub_stamp":"2026-02-24T07:46:01.867949+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:47:31.776406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2333343744,"num_objects":35604,"num_object_clones":0,"num_object_copies":106812,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35604,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":35604,"num_write_kb":2278656,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"45'36411","reported_seq":36752,"reported_epoch":45,"state":"active+clean","last_fresh":"2026-02-24T07:47:20.717025+0000","last_change":"2026-02-24T07:47:10.197277+0000","last_active":"2026-02-24T07:47:20.717025+0000","last_peered":"2026-02-24T07:47:20.717025+0000","last_clean":"2026-02-24T07:47:20.717025+0000","last_became_active":"2026-02-24T07:46:02.881951+0000","last_became_peered":"2026-02-24T07:46:02.881951+0000","last_unstale":"2026-02-24T07:47:20.717025+0000","last_undegraded":"2026-02-24T07:47:20.717025+0000","last_fullsized":"2026-02-24T07:47:20.717025+0000","mapping_epoch":42,"log_start":"45'36408","ondisk_log_start":"45'36408","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"45'31847","last_scrub_stamp":"2026-02-24T07:47:10.197239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_clean_scrub_stamp":"2026-02-24T07:47:10.197239+0000","objects_scrubbed":31279,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:48:27.205650+0000","scrub_duration":3289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2386231296,"num_objects":36411,"num_object_clones":0,"num_object_copies":109233,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36411,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":36411,"num_write_kb":2330304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"45'36100","reported_seq":36318,"reported_epoch":45,"state":"active+clean+scrubbing","last_fresh":"2026-02-24T07:47:20.759079+0000","last_change":"2026-02-24T07:47:18.399241+0000","last_active":"2026-02-24T07:47:20.759079+0000","last_peered":"2026-02-24T07:47:20.759079+0000","last_clean":"2026-02-24T07:47:20.759079+0000","last_became_active":"2026-02-24T07:46:02.881345+0000","last_became_peered":"2026-02-24T07:46:02.881345+0000","last_unstale":"2026-02-24T07:47:20.759079+0000","last_undegraded":"2026-02-24T07:47:20.759079+0000","last_fullsized":"2026-02-24T07:47:20.759079+0000","mapping_epoch":42,"log_start":"45'36097","ondisk_log_start":"45'36097","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_clean_scrub_stamp":"2026-02-24T07:46:01.867949+0000","objects_scrubbed":19799,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"scrubbing for 3s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2365849600,"num_objects":36100,"num_object_clones":0,"num_object_copies":108300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36100,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":36100,"num_write_kb":2310400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"45'35739","reported_seq":35756,"reported_epoch":45,"state":"active+clean","last_fresh":"2026-02-24T07:47:20.755418+0000","last_change":"2026-02-24T07:46:02.880582+0000","last_active":"2026-02-24T07:47:20.755418+0000","last_peered":"2026-02-24T07:47:20.755418+0000","last_clean":"2026-02-24T07:47:20.755418+0000","last_became_active":"2026-02-24T07:46:02.880323+0000","last_became_peered":"2026-02-24T07:46:02.880323+0000","last_unstale":"2026-02-24T07:47:20.755418+0000","last_undegraded":"2026-02-24T07:47:20.755418+0000","last_fullsized":"2026-02-24T07:47:20.755418+0000","mapping_epoch":42,"log_start":"45'35736","ondisk_log_start":"45'35736","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_clean_scrub_stamp":"2026-02-24T07:46:01.867949+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:47:31.420697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2342191104,"num_objects":35739,"num_object_clones":0,"num_object_copies":107217,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35739,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":35739,"num_write_kb":2287296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.6","version":"45'36163","reported_seq":36259,"reported_epoch":45,"state":"active+clean+scrubbing","last_fresh":"2026-02-24T07:47:20.702123+0000","last_change":"2026-02-24T07:47:20.020799+0000","last_active":"2026-02-24T07:47:20.702123+0000","last_peered":"2026-02-24T07:47:20.702123+0000","last_clean":"2026-02-24T07:47:20.702123+0000","last_became_active":"2026-02-24T07:46:02.880005+0000","last_became_peered":"2026-02-24T07:46:02.880005+0000","last_unstale":"2026-02-24T07:47:20.702123+0000","last_undegraded":"2026-02-24T07:47:20.702123+0000","last_fullsized":"2026-02-24T07:47:20.702123+0000","mapping_epoch":42,"log_start":"45'36160","ondisk_log_start":"45'36160","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_clean_scrub_stamp":"2026-02-24T07:46:01.867949+0000","objects_scrubbed":7799,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"scrubbing for 1s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2369978368,"num_objects":36163,"num_object_clones":0,"num_object_copies":108489,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36163,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":36163,"num_write_kb":2314432,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"45'36244","reported_seq":36612,"reported_epoch":45,"state":"active+clean","last_fresh":"2026-02-24T07:47:20.692618+0000","last_change":"2026-02-24T07:47:17.546625+0000","last_active":"2026-02-24T07:47:20.692618+0000","last_peered":"2026-02-24T07:47:20.692618+0000","last_clean":"2026-02-24T07:47:20.692618+0000","last_became_active":"2026-02-24T07:46:02.881420+0000","last_became_peered":"2026-02-24T07:46:02.881420+0000","last_unstale":"2026-02-24T07:47:20.692618+0000","last_undegraded":"2026-02-24T07:47:20.692618+0000","last_fullsized":"2026-02-24T07:47:20.692618+0000","mapping_epoch":42,"log_start":"45'36241","ondisk_log_start":"45'36241","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"45'34854","last_scrub_stamp":"2026-02-24T07:47:17.546590+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_clean_scrub_stamp":"2026-02-24T07:47:17.546590+0000","objects_scrubbed":34222,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:48:38.813192+0000","scrub_duration":3116,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2375286784,"num_objects":36244,"num_object_clones":0,"num_object_copies":108732,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36244,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":36244,"num_write_kb":2319616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.4","version":"45'35761","reported_seq":35778,"reported_epoch":45,"state":"active+clean","last_fresh":"2026-02-24T07:47:20.752528+0000","last_change":"2026-02-24T07:46:02.880661+0000","last_active":"2026-02-24T07:47:20.752528+0000","last_peered":"2026-02-24T07:47:20.752528+0000","last_clean":"2026-02-24T07:47:20.752528+0000","last_became_active":"2026-02-24T07:46:02.880345+0000","last_became_peered":"2026-02-24T07:46:02.880345+0000","last_unstale":"2026-02-24T07:47:20.752528+0000","last_undegraded":"2026-02-24T07:47:20.752528+0000","last_fullsized":"2026-02-24T07:47:20.752528+0000","mapping_epoch":42,"log_start":"45'35758","ondisk_log_start":"45'35758","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_clean_scrub_stamp":"2026-02-24T07:46:01.867949+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:47:27.577904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2343632896,"num_objects":35761,"num_object_clones":0,"num_object_copies":107283,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35761,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":35761,"num_write_kb":2288704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"45'35964","reported_seq":36318,"reported_epoch":45,"state":"active+clean","last_fresh":"2026-02-24T07:47:20.690859+0000","last_change":"2026-02-24T07:47:14.339634+0000","last_active":"2026-02-24T07:47:20.690859+0000","last_peered":"2026-02-24T07:47:20.690859+0000","last_clean":"2026-02-24T07:47:20.690859+0000","last_became_active":"2026-02-24T07:46:02.881035+0000","last_became_peered":"2026-02-24T07:46:02.881035+0000","last_unstale":"2026-02-24T07:47:20.690859+0000","last_undegraded":"2026-02-24T07:47:20.690859+0000","last_fullsized":"2026-02-24T07:47:20.690859+0000","mapping_epoch":42,"log_start":"45'35961","ondisk_log_start":"45'35961","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"45'33283","last_scrub_stamp":"2026-02-24T07:47:14.339597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_clean_scrub_stamp":"2026-02-24T07:47:14.339597+0000","objects_scrubbed":32630,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:48:22.078719+0000","scrub_duration":2951,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2356936704,"num_objects":35964,"num_object_clones":0,"num_object_copies":107892,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35964,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":35964,"num_write_kb":2301696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"45'36032","reported_seq":36049,"reported_epoch":45,"state":"active+clean","last_fresh":"2026-02-24T07:47:20.706953+0000","last_change":"2026-02-24T07:46:02.879955+0000","last_active":"2026-02-24T07:47:20.706953+0000","last_peered":"2026-02-24T07:47:20.706953+0000","last_clean":"2026-02-24T07:47:20.706953+0000","last_became_active":"2026-02-24T07:46:02.879831+0000","last_became_peered":"2026-02-24T07:46:02.879831+0000","last_unstale":"2026-02-24T07:47:20.706953+0000","last_undegraded":"2026-02-24T07:47:20.706953+0000","last_fullsized":"2026-02-24T07:47:20.706953+0000","mapping_epoch":42,"log_start":"45'36029","ondisk_log_start":"45'36029","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_clean_scrub_stamp":"2026-02-24T07:46:01.867949+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2361393152,"num_objects":36032,"num_object_clones":0,"num_object_copies":108096,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36032,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":36032,"num_write_kb":2306048,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.0","version":"45'35884","reported_seq":36210,"reported_epoch":45,"state":"active+clean","last_fresh":"2026-02-24T07:47:20.758459+0000","last_change":"2026-02-24T07:47:08.376360+0000","last_active":"2026-02-24T07:47:20.758459+0000","last_peered":"2026-02-24T07:47:20.758459+0000","last_clean":"2026-02-24T07:47:20.758459+0000","last_became_active":"2026-02-24T07:46:02.880333+0000","last_became_peered":"2026-02-24T07:46:02.880333+0000","last_unstale":"2026-02-24T07:47:20.758459+0000","last_undegraded":"2026-02-24T07:47:20.758459+0000","last_fullsized":"2026-02-24T07:47:20.758459+0000","mapping_epoch":42,"log_start":"45'35881","ondisk_log_start":"45'35881","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"45'30511","last_scrub_stamp":"2026-02-24T07:47:08.376304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_clean_scrub_stamp":"2026-02-24T07:47:08.376304+0000","objects_scrubbed":29912,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:48:24.552234+0000","scrub_duration":3049,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2351693824,"num_objects":35884,"num_object_clones":0,"num_object_copies":107652,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35884,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":35884,"num_write_kb":2296576,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2","version":"45'36231","reported_seq":36248,"reported_epoch":45,"state":"active+clean","last_fresh":"2026-02-24T07:47:20.750520+0000","last_change":"2026-02-24T07:46:02.880566+0000","last_active":"2026-02-24T07:47:20.750520+0000","last_peered":"2026-02-24T07:47:20.750520+0000","last_clean":"2026-02-24T07:47:20.750520+0000","last_became_active":"2026-02-24T07:46:02.880185+0000","last_became_peered":"2026-02-24T07:46:02.880185+0000","last_unstale":"2026-02-24T07:47:20.750520+0000","last_undegraded":"2026-02-24T07:47:20.750520+0000","last_fullsized":"2026-02-24T07:47:20.750520+0000","mapping_epoch":42,"log_start":"45'36228","ondisk_log_start":"45'36228","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_clean_scrub_stamp":"2026-02-24T07:46:01.867949+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:47:23.244686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2374434816,"num_objects":36231,"num_object_clones":0,"num_object_copies":108693,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36231,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":36231,"num_write_kb":2318784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"29'99","reported_seq":237,"reported_epoch":45,"state":"active+clean","last_fresh":"2026-02-24T07:46:41.007543+0000","last_change":"2026-02-24T07:46:41.007499+0000","last_active":"2026-02-24T07:46:41.007543+0000","last_peered":"2026-02-24T07:46:41.007543+0000","last_clean":"2026-02-24T07:46:41.007543+0000","last_became_active":"2026-02-24T07:42:54.186719+0000","last_became_peered":"2026-02-24T07:42:54.186719+0000","last_unstale":"2026-02-24T07:46:41.007543+0000","last_undegraded":"2026-02-24T07:46:41.007543+0000","last_fullsized":"2026-02-24T07:46:41.007543+0000","mapping_epoch":27,"log_start":"29'96","ondisk_log_start":"29'96","created":20,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'99","last_scrub_stamp":"2026-02-24T07:46:41.007434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:42:43.640776+0000","last_clean_scrub_stamp":"2026-02-24T07:46:41.007434+0000","objects_scrubbed":2,"log_size":3,"log_dups_size":96,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:47:53.486511+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":240,"num_write":135,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":16,"stat_sum":{"num_bytes":37729730560,"num_objects":575710,"num_object_clones":0,"num_object_copies":1727130,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":575710,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":575710,"num_write_kb":36845440,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":113179099136,"data_stored":113179099136,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":48,"ondisk_log_size":48,"up":48,"acting":48,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":240,"num_write":135,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":2120,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":3,"up_from":25,"seq":107374182456,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":24641992,"kb_used_data":22796276,"kb_used_omap":9,"kb_used_meta":1845622,"kb_avail":709361208,"statfs":{"total":751619276800,"available":726385876992,"internally_reserved":0,"allocated":23343386624,"data_stored":23343097980,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9784,"internal_metadata":1889917384},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":19,"seq":81604378682,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":34279432,"kb_used_data":32156276,"kb_used_omap":9,"kb_used_meta":2123126,"kb_avail":699723768,"statfs":{"total":751619276800,"available":716517138432,"internally_reserved":0,"allocated":32928026624,"data_stored":32927737980,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9565,"internal_metadata":2174081699},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[6,1],"upper_bound":4},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542203,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":27179780,"kb_used_data":25215792,"kb_used_omap":6,"kb_used_meta":1963961,"kb_avail":706823420,"statfs":{"total":751619276800,"available":723787182080,"internally_reserved":0,"allocated":25820971008,"data_stored":25820685916,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6807,"internal_metadata":2011096425},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705726,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":31659080,"kb_used_data":29680500,"kb_used_omap":8,"kb_used_meta":1978551,"kb_avail":702344120,"statfs":{"total":751619276800,"available":719200378880,"internally_reserved":0,"allocated":30392832000,"data_stored":30392543356,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9014,"internal_metadata":2026036426},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":2120,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":30631985152,"data_stored":30631985152,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":25914572800,"data_stored":25914572800,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":33054720000,"data_stored":33054720000,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":23577821184,"data_stored":23577821184,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-24T07:47:24.386 INFO:tasks.cephadm.ceph_manager.ceph:increase pool size by 10 2026-02-24T07:47:24.386 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph osd pool set .mgr pg_num 141 2026-02-24T07:47:24.394 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:47:24.516 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:47:24.514+0000 7f3bda427640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:47:24.517 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:47:24.514+0000 7f3bda427640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:47:24.532 INFO:teuthology.orchestra.run.trial035.stdout:osd.0: {} 2026-02-24T07:47:24.532 INFO:teuthology.orchestra.run.trial035.stderr:osd.0: osd_enable_op_tracker = '' 2026-02-24T07:47:24.534 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:47:24.550 INFO:teuthology.orchestra.run.trial035.stdout:osd.1: {} 2026-02-24T07:47:24.550 INFO:teuthology.orchestra.run.trial035.stderr:osd.1: osd_enable_op_tracker = '' 2026-02-24T07:47:24.564 INFO:teuthology.orchestra.run.trial035.stdout:osd.2: {} 2026-02-24T07:47:24.564 INFO:teuthology.orchestra.run.trial035.stderr:osd.2: osd_enable_op_tracker = '' 2026-02-24T07:47:24.578 INFO:teuthology.orchestra.run.trial035.stdout:osd.3: {} 2026-02-24T07:47:24.578 INFO:teuthology.orchestra.run.trial035.stderr:osd.3: osd_enable_op_tracker = '' 2026-02-24T07:47:24.660 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:47:24.657+0000 7fb00e099640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:47:24.660 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:47:24.658+0000 7fb00e099640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:47:24.816 INFO:teuthology.orchestra.run.trial035.stderr:Error EPERM: crimson_allow_pg_split is false; pg_num increase denied for crimson pool 2026-02-24T07:47:24.856 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-24T07:47:24.979 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 81 16 601917 601901 464.39 349.875 0.00214852 0.00215208 2026-02-24T07:47:25.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:47:24 trial035 ceph-mon[39284]: 3.6 scrub ok 2026-02-24T07:47:25.205 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:47:24 trial035 ceph-mon[39284]: 3.2 scrub starts 2026-02-24T07:47:25.205 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:47:24 trial035 ceph-mon[39284]: pgmap v151: 17 pgs: 2 active+clean+scrubbing, 15 active+clean; 35 GiB data, 112 GiB used, 2.6 TiB / 2.7 TiB avail; 359 MiB/s wr, 5.75k op/s 2026-02-24T07:47:25.205 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:47:24 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/3022414364' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2026-02-24T07:47:25.205 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:47:24 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/2676465828' entity='client.admin' cmd={"prefix": "osd pool set", "pool": ".mgr", "var": "pg_num", "val": "141"} : dispatch 2026-02-24T07:47:25.979 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 82 16 609229 609213 464.3 457 0.0014462 0.00215251 2026-02-24T07:47:26.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:47:25 trial035 ceph-mon[39284]: from='client.14714 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2026-02-24T07:47:26.205 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:47:25 trial035 ceph-mon[39284]: 3.1 scrub ok 2026-02-24T07:47:26.623 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2026-02-24T07:47:26.763 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:47:26.955 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:47:26 trial035 ceph-mon[39284]: pgmap v152: 17 pgs: 2 active+clean+scrubbing, 15 active+clean; 35 GiB data, 112 GiB used, 2.6 TiB / 2.7 TiB avail; 359 MiB/s wr, 5.75k op/s 2026-02-24T07:47:26.979 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 83 16 616064 616048 463.852 427.188 0.00197815 0.00215459 2026-02-24T07:47:27.166 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:47:27.163+0000 7f6d4cac6640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:47:27.166 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:47:27.164+0000 7f6d4cac6640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:47:27.180 INFO:teuthology.orchestra.run.trial035.stdout:osd.0: {} 2026-02-24T07:47:27.180 INFO:teuthology.orchestra.run.trial035.stderr:osd.0: osd_enable_op_tracker = '' 2026-02-24T07:47:27.194 INFO:teuthology.orchestra.run.trial035.stdout:osd.1: {} 2026-02-24T07:47:27.195 INFO:teuthology.orchestra.run.trial035.stderr:osd.1: osd_enable_op_tracker = '' 2026-02-24T07:47:27.207 INFO:teuthology.orchestra.run.trial035.stdout:osd.2: {} 2026-02-24T07:47:27.207 INFO:teuthology.orchestra.run.trial035.stderr:osd.2: osd_enable_op_tracker = '' 2026-02-24T07:47:27.220 INFO:teuthology.orchestra.run.trial035.stdout:osd.3: {} 2026-02-24T07:47:27.220 INFO:teuthology.orchestra.run.trial035.stderr:osd.3: osd_enable_op_tracker = '' 2026-02-24T07:47:27.228 INFO:journalctl@ceph.mgr.x.trial035.stdout:Feb 24 07:47:26 trial035 ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-mgr-x[39621]: ::ffff:10.20.193.91 - - [24/Feb/2026:07:47:26] "GET /metrics HTTP/1.1" 200 4221 "" "Prometheus/3.6.0" 2026-02-24T07:47:27.356 INFO:journalctl@ceph.prometheus.a.trial091.stdout:Feb 24 07:47:26 trial091 ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-prometheus-a[66007]: time=2026-02-24T07:47:26.973Z level=ERROR source=scrape.go:1630 msg="Failed to determine correct type of scrape target." component="scrape manager" scrape_pool=ceph target=http://trial035:9283/metrics content_type="text/html;charset=utf-8" fallback_media_type="" err="received unsupported Content-Type \"text/html;charset=utf-8\" and no fallback_scrape_protocol specified for target" 2026-02-24T07:47:27.979 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 84 16 623538 623522 463.891 467.125 0.00270951 0.00215443 2026-02-24T07:47:28.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:47:27 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/361360330' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2026-02-24T07:47:28.979 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 85 15 632159 632144 464.773 538.875 0.00123088 0.00215033 2026-02-24T07:47:29.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:47:28 trial035 ceph-mon[39284]: pgmap v153: 17 pgs: 1 active+clean+scrubbing, 16 active+clean; 37 GiB data, 118 GiB used, 2.6 TiB / 2.7 TiB avail; 524 MiB/s wr, 8.38k op/s 2026-02-24T07:47:29.205 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:47:28 trial035 ceph-mon[39284]: 3.2 scrub ok 2026-02-24T07:47:29.205 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:47:28 trial035 ceph-mon[39284]: osdmap e46: 4 total, 4 up, 4 in 2026-02-24T07:47:29.263 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2026-02-24T07:47:29.408 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:47:29.530 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:47:29.527+0000 7f41f289a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:47:29.530 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:47:29.527+0000 7f41f289a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:47:29.544 INFO:teuthology.orchestra.run.trial035.stdout:osd.0: {} 2026-02-24T07:47:29.545 INFO:teuthology.orchestra.run.trial035.stderr:osd.0: osd_enable_op_tracker = '' 2026-02-24T07:47:29.560 INFO:teuthology.orchestra.run.trial035.stdout:osd.1: {} 2026-02-24T07:47:29.560 INFO:teuthology.orchestra.run.trial035.stderr:osd.1: osd_enable_op_tracker = '' 2026-02-24T07:47:29.573 INFO:teuthology.orchestra.run.trial035.stdout:osd.2: {} 2026-02-24T07:47:29.573 INFO:teuthology.orchestra.run.trial035.stderr:osd.2: osd_enable_op_tracker = '' 2026-02-24T07:47:29.588 INFO:teuthology.orchestra.run.trial035.stdout:osd.3: {} 2026-02-24T07:47:29.588 INFO:teuthology.orchestra.run.trial035.stderr:osd.3: osd_enable_op_tracker = '' 2026-02-24T07:47:29.857 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2026-02-24T07:47:29.858 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2026-02-24T07:47:29.858 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph pg dump --format=json 2026-02-24T07:47:29.979 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 86 16 639100 639084 464.411 433.75 0.00101519 0.00215198 2026-02-24T07:47:29.998 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:47:30.115 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:47:30.112+0000 7fc315c50640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:47:30.116 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:47:30.113+0000 7fc315c50640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:47:30.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:47:29 trial035 ceph-mon[39284]: 3.e scrub starts 2026-02-24T07:47:30.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:47:29 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/3741697018' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2026-02-24T07:47:30.257 INFO:teuthology.orchestra.run.trial035.stdout: 2026-02-24T07:47:30.258 INFO:teuthology.orchestra.run.trial035.stderr:dumped all 2026-02-24T07:47:30.299 INFO:teuthology.orchestra.run.trial035.stdout:{"pg_ready":true,"pg_map":{"version":155,"stamp":"2026-02-24T07:47:29.694020+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":39804273184,"num_objects":607358,"num_object_clones":0,"num_object_copies":1822074,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":607358,"num_whiteouts":0,"num_read":136,"num_read_kb":240,"num_write":607491,"num_write_kb":38872934,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":51,"ondisk_log_size":51,"up":51,"acting":51,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":51,"num_osds":4,"num_per_pool_osds":4,"num_per_pool_omap_osds":4,"kb":2936012800,"kb_used":123993628,"kb_used_data":115829132,"kb_used_omap":35,"kb_used_meta":8164380,"kb_avail":2812019172,"statfs":{"total":3006477107200,"available":2879507632128,"internally_reserved":0,"allocated":118609031168,"data_stored":118607880144,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":36054,"internal_metadata":8360325930},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[14,2,3,2],"upper_bound":16},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":4248305664,"num_objects":64824,"num_object_clones":0,"num_object_copies":194472,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64824,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":64824,"num_write_kb":4148736,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.001424"},"pg_stats":[{"pgid":"3.f","version":"45'37841","reported_seq":38203,"reported_epoch":45,"state":"active+clean","last_fresh":"2026-02-24T07:47:25.692516+0000","last_change":"2026-02-24T07:47:16.466592+0000","last_active":"2026-02-24T07:47:25.692516+0000","last_peered":"2026-02-24T07:47:25.692516+0000","last_clean":"2026-02-24T07:47:25.692516+0000","last_became_active":"2026-02-24T07:46:02.881625+0000","last_became_peered":"2026-02-24T07:46:02.881625+0000","last_unstale":"2026-02-24T07:47:25.692516+0000","last_undegraded":"2026-02-24T07:47:25.692516+0000","last_fullsized":"2026-02-24T07:47:25.692516+0000","mapping_epoch":42,"log_start":"45'37838","ondisk_log_start":"45'37838","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"45'34153","last_scrub_stamp":"2026-02-24T07:47:16.466530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_clean_scrub_stamp":"2026-02-24T07:47:16.466530+0000","objects_scrubbed":33463,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:48:23.487779+0000","scrub_duration":3074,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2479947776,"num_objects":37841,"num_object_clones":0,"num_object_copies":113523,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37841,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":37841,"num_write_kb":2421824,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.e","version":"45'38388","reported_seq":38405,"reported_epoch":45,"state":"active+clean","last_fresh":"2026-02-24T07:47:25.691992+0000","last_change":"2026-02-24T07:46:02.881462+0000","last_active":"2026-02-24T07:47:25.691992+0000","last_peered":"2026-02-24T07:47:25.691992+0000","last_clean":"2026-02-24T07:47:25.691992+0000","last_became_active":"2026-02-24T07:46:02.881083+0000","last_became_peered":"2026-02-24T07:46:02.881083+0000","last_unstale":"2026-02-24T07:47:25.691992+0000","last_undegraded":"2026-02-24T07:47:25.691992+0000","last_fullsized":"2026-02-24T07:47:25.691992+0000","mapping_epoch":42,"log_start":"45'38385","ondisk_log_start":"45'38385","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_clean_scrub_stamp":"2026-02-24T07:46:01.867949+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:47:29.030602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2515795968,"num_objects":38388,"num_object_clones":0,"num_object_copies":115164,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38388,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":38388,"num_write_kb":2456832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"45'37956","reported_seq":38279,"reported_epoch":45,"state":"active+clean","last_fresh":"2026-02-24T07:47:25.714767+0000","last_change":"2026-02-24T07:47:06.968813+0000","last_active":"2026-02-24T07:47:25.714767+0000","last_peered":"2026-02-24T07:47:25.714767+0000","last_clean":"2026-02-24T07:47:25.714767+0000","last_became_active":"2026-02-24T07:46:02.883799+0000","last_became_peered":"2026-02-24T07:46:02.883799+0000","last_unstale":"2026-02-24T07:47:25.714767+0000","last_undegraded":"2026-02-24T07:47:25.714767+0000","last_fullsized":"2026-02-24T07:47:25.714767+0000","mapping_epoch":42,"log_start":"45'37953","ondisk_log_start":"45'37953","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"45'30090","last_scrub_stamp":"2026-02-24T07:47:06.968732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_clean_scrub_stamp":"2026-02-24T07:47:06.968732+0000","objects_scrubbed":29542,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:48:25.198085+0000","scrub_duration":3070,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2487484416,"num_objects":37956,"num_object_clones":0,"num_object_copies":113868,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37956,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":37956,"num_write_kb":2429184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"45'37344","reported_seq":37666,"reported_epoch":45,"state":"active+clean","last_fresh":"2026-02-24T07:47:25.715549+0000","last_change":"2026-02-24T07:47:07.546834+0000","last_active":"2026-02-24T07:47:25.715549+0000","last_peered":"2026-02-24T07:47:25.715549+0000","last_clean":"2026-02-24T07:47:25.715549+0000","last_became_active":"2026-02-24T07:46:02.880060+0000","last_became_peered":"2026-02-24T07:46:02.880060+0000","last_unstale":"2026-02-24T07:47:25.715549+0000","last_undegraded":"2026-02-24T07:47:25.715549+0000","last_fullsized":"2026-02-24T07:47:25.715549+0000","mapping_epoch":42,"log_start":"45'37341","ondisk_log_start":"45'37341","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"45'29910","last_scrub_stamp":"2026-02-24T07:47:07.546803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_clean_scrub_stamp":"2026-02-24T07:47:07.546803+0000","objects_scrubbed":29435,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:48:27.524515+0000","scrub_duration":2696,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2447376384,"num_objects":37344,"num_object_clones":0,"num_object_copies":112032,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37344,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":37344,"num_write_kb":2390016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"45'37999","reported_seq":38333,"reported_epoch":45,"state":"active+clean","last_fresh":"2026-02-24T07:47:25.764913+0000","last_change":"2026-02-24T07:47:10.376412+0000","last_active":"2026-02-24T07:47:25.764913+0000","last_peered":"2026-02-24T07:47:25.764913+0000","last_clean":"2026-02-24T07:47:25.764913+0000","last_became_active":"2026-02-24T07:46:02.880171+0000","last_became_peered":"2026-02-24T07:46:02.880171+0000","last_unstale":"2026-02-24T07:47:25.764913+0000","last_undegraded":"2026-02-24T07:47:25.764913+0000","last_fullsized":"2026-02-24T07:47:25.764913+0000","mapping_epoch":42,"log_start":"45'37996","ondisk_log_start":"45'37996","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"45'31519","last_scrub_stamp":"2026-02-24T07:47:10.376345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_clean_scrub_stamp":"2026-02-24T07:47:10.376345+0000","objects_scrubbed":30810,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:48:13.377288+0000","scrub_duration":4056,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2490302464,"num_objects":37999,"num_object_clones":0,"num_object_copies":113997,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37999,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":37999,"num_write_kb":2431936,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.a","version":"45'37599","reported_seq":37616,"reported_epoch":45,"state":"active+clean","last_fresh":"2026-02-24T07:47:25.712697+0000","last_change":"2026-02-24T07:46:02.882761+0000","last_active":"2026-02-24T07:47:25.712697+0000","last_peered":"2026-02-24T07:47:25.712697+0000","last_clean":"2026-02-24T07:47:25.712697+0000","last_became_active":"2026-02-24T07:46:02.882274+0000","last_became_peered":"2026-02-24T07:46:02.882274+0000","last_unstale":"2026-02-24T07:47:25.712697+0000","last_undegraded":"2026-02-24T07:47:25.712697+0000","last_fullsized":"2026-02-24T07:47:25.712697+0000","mapping_epoch":42,"log_start":"45'37596","ondisk_log_start":"45'37596","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_clean_scrub_stamp":"2026-02-24T07:46:01.867949+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:47:31.776406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2464088064,"num_objects":37599,"num_object_clones":0,"num_object_copies":112797,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37599,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":37599,"num_write_kb":2406336,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"45'38403","reported_seq":38744,"reported_epoch":45,"state":"active+clean","last_fresh":"2026-02-24T07:47:25.712523+0000","last_change":"2026-02-24T07:47:10.197277+0000","last_active":"2026-02-24T07:47:25.712523+0000","last_peered":"2026-02-24T07:47:25.712523+0000","last_clean":"2026-02-24T07:47:25.712523+0000","last_became_active":"2026-02-24T07:46:02.881951+0000","last_became_peered":"2026-02-24T07:46:02.881951+0000","last_unstale":"2026-02-24T07:47:25.712523+0000","last_undegraded":"2026-02-24T07:47:25.712523+0000","last_fullsized":"2026-02-24T07:47:25.712523+0000","mapping_epoch":42,"log_start":"45'38400","ondisk_log_start":"45'38400","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"45'31847","last_scrub_stamp":"2026-02-24T07:47:10.197239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_clean_scrub_stamp":"2026-02-24T07:47:10.197239+0000","objects_scrubbed":31279,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:48:27.205650+0000","scrub_duration":3289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2516779008,"num_objects":38403,"num_object_clones":0,"num_object_copies":115209,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38403,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":38403,"num_write_kb":2457792,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"45'38070","reported_seq":38457,"reported_epoch":45,"state":"active+clean","last_fresh":"2026-02-24T07:47:25.763639+0000","last_change":"2026-02-24T07:47:22.289650+0000","last_active":"2026-02-24T07:47:25.763639+0000","last_peered":"2026-02-24T07:47:25.763639+0000","last_clean":"2026-02-24T07:47:25.763639+0000","last_became_active":"2026-02-24T07:46:02.881345+0000","last_became_peered":"2026-02-24T07:46:02.881345+0000","last_unstale":"2026-02-24T07:47:25.763639+0000","last_undegraded":"2026-02-24T07:47:25.763639+0000","last_fullsized":"2026-02-24T07:47:25.763639+0000","mapping_epoch":42,"log_start":"45'38067","ondisk_log_start":"45'38067","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"45'36703","last_scrub_stamp":"2026-02-24T07:47:22.289617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_clean_scrub_stamp":"2026-02-24T07:47:22.289617+0000","objects_scrubbed":35960,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:48:31.332102+0000","scrub_duration":3891,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2494955520,"num_objects":38070,"num_object_clones":0,"num_object_copies":114210,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38070,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":38070,"num_write_kb":2436480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"45'37757","reported_seq":37774,"reported_epoch":45,"state":"active+clean","last_fresh":"2026-02-24T07:47:25.765295+0000","last_change":"2026-02-24T07:46:02.880582+0000","last_active":"2026-02-24T07:47:25.765295+0000","last_peered":"2026-02-24T07:47:25.765295+0000","last_clean":"2026-02-24T07:47:25.765295+0000","last_became_active":"2026-02-24T07:46:02.880323+0000","last_became_peered":"2026-02-24T07:46:02.880323+0000","last_unstale":"2026-02-24T07:47:25.765295+0000","last_undegraded":"2026-02-24T07:47:25.765295+0000","last_fullsized":"2026-02-24T07:47:25.765295+0000","mapping_epoch":42,"log_start":"45'37754","ondisk_log_start":"45'37754","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_clean_scrub_stamp":"2026-02-24T07:46:01.867949+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:47:31.420697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2474442752,"num_objects":37757,"num_object_clones":0,"num_object_copies":113271,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37757,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":37757,"num_write_kb":2416448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.6","version":"45'38085","reported_seq":38477,"reported_epoch":45,"state":"active+clean","last_fresh":"2026-02-24T07:47:25.707793+0000","last_change":"2026-02-24T07:47:23.432795+0000","last_active":"2026-02-24T07:47:25.707793+0000","last_peered":"2026-02-24T07:47:25.707793+0000","last_clean":"2026-02-24T07:47:25.707793+0000","last_became_active":"2026-02-24T07:46:02.880005+0000","last_became_peered":"2026-02-24T07:46:02.880005+0000","last_unstale":"2026-02-24T07:47:25.707793+0000","last_undegraded":"2026-02-24T07:47:25.707793+0000","last_fullsized":"2026-02-24T07:47:25.707793+0000","mapping_epoch":42,"log_start":"45'38082","ondisk_log_start":"45'38082","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"45'37219","last_scrub_stamp":"2026-02-24T07:47:23.432744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_clean_scrub_stamp":"2026-02-24T07:47:23.432744+0000","objects_scrubbed":36510,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:48:51.822832+0000","scrub_duration":3412,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2495938560,"num_objects":38085,"num_object_clones":0,"num_object_copies":114255,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38085,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":38085,"num_write_kb":2437440,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"45'38212","reported_seq":38581,"reported_epoch":45,"state":"active+clean","last_fresh":"2026-02-24T07:47:25.689880+0000","last_change":"2026-02-24T07:47:17.546625+0000","last_active":"2026-02-24T07:47:25.689880+0000","last_peered":"2026-02-24T07:47:25.689880+0000","last_clean":"2026-02-24T07:47:25.689880+0000","last_became_active":"2026-02-24T07:46:02.881420+0000","last_became_peered":"2026-02-24T07:46:02.881420+0000","last_unstale":"2026-02-24T07:47:25.689880+0000","last_undegraded":"2026-02-24T07:47:25.689880+0000","last_fullsized":"2026-02-24T07:47:25.689880+0000","mapping_epoch":42,"log_start":"45'38209","ondisk_log_start":"45'38209","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"45'34854","last_scrub_stamp":"2026-02-24T07:47:17.546590+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_clean_scrub_stamp":"2026-02-24T07:47:17.546590+0000","objects_scrubbed":34222,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:48:38.813192+0000","scrub_duration":3116,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2504261632,"num_objects":38212,"num_object_clones":0,"num_object_copies":114636,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38212,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":38212,"num_write_kb":2445568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.4","version":"45'37744","reported_seq":37761,"reported_epoch":45,"state":"active+clean","last_fresh":"2026-02-24T07:47:25.762678+0000","last_change":"2026-02-24T07:46:02.880661+0000","last_active":"2026-02-24T07:47:25.762678+0000","last_peered":"2026-02-24T07:47:25.762678+0000","last_clean":"2026-02-24T07:47:25.762678+0000","last_became_active":"2026-02-24T07:46:02.880345+0000","last_became_peered":"2026-02-24T07:46:02.880345+0000","last_unstale":"2026-02-24T07:47:25.762678+0000","last_undegraded":"2026-02-24T07:47:25.762678+0000","last_fullsized":"2026-02-24T07:47:25.762678+0000","mapping_epoch":42,"log_start":"45'37741","ondisk_log_start":"45'37741","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_clean_scrub_stamp":"2026-02-24T07:46:01.867949+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:47:27.577904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2473590784,"num_objects":37744,"num_object_clones":0,"num_object_copies":113232,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37744,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":37744,"num_write_kb":2415616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"45'37979","reported_seq":38333,"reported_epoch":45,"state":"active+clean","last_fresh":"2026-02-24T07:47:25.691101+0000","last_change":"2026-02-24T07:47:14.339634+0000","last_active":"2026-02-24T07:47:25.691101+0000","last_peered":"2026-02-24T07:47:25.691101+0000","last_clean":"2026-02-24T07:47:25.691101+0000","last_became_active":"2026-02-24T07:46:02.881035+0000","last_became_peered":"2026-02-24T07:46:02.881035+0000","last_unstale":"2026-02-24T07:47:25.691101+0000","last_undegraded":"2026-02-24T07:47:25.691101+0000","last_fullsized":"2026-02-24T07:47:25.691101+0000","mapping_epoch":42,"log_start":"45'37976","ondisk_log_start":"45'37976","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"45'33283","last_scrub_stamp":"2026-02-24T07:47:14.339597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_clean_scrub_stamp":"2026-02-24T07:47:14.339597+0000","objects_scrubbed":32630,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:48:22.078719+0000","scrub_duration":2951,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2488991744,"num_objects":37979,"num_object_clones":0,"num_object_copies":113937,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37979,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":37979,"num_write_kb":2430656,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"45'37980","reported_seq":38375,"reported_epoch":45,"state":"active+clean","last_fresh":"2026-02-24T07:47:25.704856+0000","last_change":"2026-02-24T07:47:24.807734+0000","last_active":"2026-02-24T07:47:25.704856+0000","last_peered":"2026-02-24T07:47:25.704856+0000","last_clean":"2026-02-24T07:47:25.704856+0000","last_became_active":"2026-02-24T07:46:02.879831+0000","last_became_peered":"2026-02-24T07:46:02.879831+0000","last_unstale":"2026-02-24T07:47:25.704856+0000","last_undegraded":"2026-02-24T07:47:25.704856+0000","last_fullsized":"2026-02-24T07:47:25.704856+0000","mapping_epoch":42,"log_start":"45'37977","ondisk_log_start":"45'37977","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"45'37558","last_scrub_stamp":"2026-02-24T07:47:24.807659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_clean_scrub_stamp":"2026-02-24T07:47:24.807659+0000","objects_scrubbed":36869,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:48:39.172465+0000","scrub_duration":3748,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2489057280,"num_objects":37980,"num_object_clones":0,"num_object_copies":113940,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37980,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":37980,"num_write_kb":2430720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.0","version":"45'37818","reported_seq":38145,"reported_epoch":45,"state":"active+clean","last_fresh":"2026-02-24T07:47:25.762376+0000","last_change":"2026-02-24T07:47:08.376360+0000","last_active":"2026-02-24T07:47:25.762376+0000","last_peered":"2026-02-24T07:47:25.762376+0000","last_clean":"2026-02-24T07:47:25.762376+0000","last_became_active":"2026-02-24T07:46:02.880333+0000","last_became_peered":"2026-02-24T07:46:02.880333+0000","last_unstale":"2026-02-24T07:47:25.762376+0000","last_undegraded":"2026-02-24T07:47:25.762376+0000","last_fullsized":"2026-02-24T07:47:25.762376+0000","mapping_epoch":42,"log_start":"45'37815","ondisk_log_start":"45'37815","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"45'30511","last_scrub_stamp":"2026-02-24T07:47:08.376304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_clean_scrub_stamp":"2026-02-24T07:47:08.376304+0000","objects_scrubbed":29912,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:48:24.552234+0000","scrub_duration":3049,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2478440448,"num_objects":37818,"num_object_clones":0,"num_object_copies":113454,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37818,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":37818,"num_write_kb":2420352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2","version":"45'38181","reported_seq":38388,"reported_epoch":45,"state":"active+clean+scrubbing","last_fresh":"2026-02-24T07:47:25.766013+0000","last_change":"2026-02-24T07:47:23.519206+0000","last_active":"2026-02-24T07:47:25.766013+0000","last_peered":"2026-02-24T07:47:25.766013+0000","last_clean":"2026-02-24T07:47:25.766013+0000","last_became_active":"2026-02-24T07:46:02.880185+0000","last_became_peered":"2026-02-24T07:46:02.880185+0000","last_unstale":"2026-02-24T07:47:25.766013+0000","last_undegraded":"2026-02-24T07:47:25.766013+0000","last_fullsized":"2026-02-24T07:47:25.766013+0000","mapping_epoch":42,"log_start":"45'38178","ondisk_log_start":"45'38178","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_clean_scrub_stamp":"2026-02-24T07:46:01.867949+0000","objects_scrubbed":18649,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"scrubbing for 3s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2502230016,"num_objects":38181,"num_object_clones":0,"num_object_copies":114543,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38181,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":38181,"num_write_kb":2443584,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"29'99","reported_seq":237,"reported_epoch":45,"state":"active+clean","last_fresh":"2026-02-24T07:46:41.007543+0000","last_change":"2026-02-24T07:46:41.007499+0000","last_active":"2026-02-24T07:46:41.007543+0000","last_peered":"2026-02-24T07:46:41.007543+0000","last_clean":"2026-02-24T07:46:41.007543+0000","last_became_active":"2026-02-24T07:42:54.186719+0000","last_became_peered":"2026-02-24T07:42:54.186719+0000","last_unstale":"2026-02-24T07:46:41.007543+0000","last_undegraded":"2026-02-24T07:46:41.007543+0000","last_fullsized":"2026-02-24T07:46:41.007543+0000","mapping_epoch":27,"log_start":"29'96","ondisk_log_start":"29'96","created":20,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'99","last_scrub_stamp":"2026-02-24T07:46:41.007434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:42:43.640776+0000","last_clean_scrub_stamp":"2026-02-24T07:46:41.007434+0000","objects_scrubbed":2,"log_size":3,"log_dups_size":96,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:47:53.486511+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":240,"num_write":135,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":16,"stat_sum":{"num_bytes":39803682816,"num_objects":607356,"num_object_clones":0,"num_object_copies":1822068,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":607356,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":607356,"num_write_kb":38870784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":119396892672,"data_stored":119396892672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":48,"ondisk_log_size":48,"up":48,"acting":48,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":240,"num_write":135,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":3,"up_from":25,"seq":107374182457,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":25983816,"kb_used_data":24011828,"kb_used_omap":9,"kb_used_meta":1971958,"kb_avail":708019384,"statfs":{"total":751619276800,"available":725011849216,"internally_reserved":0,"allocated":24588111872,"data_stored":24587823228,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9784,"internal_metadata":2019285448},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":19,"seq":81604378683,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":36050504,"kb_used_data":33854772,"kb_used_omap":9,"kb_used_meta":2195702,"kb_avail":697952696,"statfs":{"total":751619276800,"available":714703560704,"internally_reserved":0,"allocated":34667286528,"data_stored":34666997884,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9565,"internal_metadata":2248399523},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[6,1],"upper_bound":4},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542204,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":28579716,"kb_used_data":26633712,"kb_used_omap":7,"kb_used_meta":1945976,"kb_avail":705423484,"statfs":{"total":751619276800,"available":722353647616,"internally_reserved":0,"allocated":27272921088,"data_stored":27272635996,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":7691,"internal_metadata":1992679925},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705727,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":33379592,"kb_used_data":31328820,"kb_used_omap":8,"kb_used_meta":2050743,"kb_avail":700623608,"statfs":{"total":751619276800,"available":717438574592,"internally_reserved":0,"allocated":32080711680,"data_stored":32080423036,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9014,"internal_metadata":2099961034},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[6,1,3,2],"upper_bound":16},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":32313114624,"data_stored":32313114624,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":27334148096,"data_stored":27334148096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":34867773440,"data_stored":34867773440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":24881856512,"data_stored":24881856512,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-24T07:47:30.980 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 87 16 645565 645549 463.717 404.062 0.00109531 0.00215519 2026-02-24T07:47:31.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:47:30 trial035 ceph-mon[39284]: 3.4 scrub starts 2026-02-24T07:47:31.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:47:30 trial035 ceph-mon[39284]: pgmap v155: 17 pgs: 1 active+clean+scrubbing, 16 active+clean; 37 GiB data, 118 GiB used, 2.6 TiB / 2.7 TiB avail; 405 MiB/s wr, 6.48k op/s 2026-02-24T07:47:31.205 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:47:30 trial035 ceph-mon[39284]: from='mgr.14284 10.20.193.35:0/3681373222' entity='mgr.x' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2026-02-24T07:47:31.632 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2026-02-24T07:47:31.777 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:47:31.896 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:47:31.892+0000 7f08eca14640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:47:31.896 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:47:31.893+0000 7f08eca14640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:47:31.910 INFO:teuthology.orchestra.run.trial035.stdout:osd.0: {} 2026-02-24T07:47:31.910 INFO:teuthology.orchestra.run.trial035.stderr:osd.0: osd_enable_op_tracker = '' 2026-02-24T07:47:31.911 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:47:31 trial035 ceph-mon[39284]: from='client.14724 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2026-02-24T07:47:31.923 INFO:teuthology.orchestra.run.trial035.stdout:osd.1: {} 2026-02-24T07:47:31.924 INFO:teuthology.orchestra.run.trial035.stderr:osd.1: osd_enable_op_tracker = '' 2026-02-24T07:47:31.937 INFO:teuthology.orchestra.run.trial035.stdout:osd.2: {} 2026-02-24T07:47:31.937 INFO:teuthology.orchestra.run.trial035.stderr:osd.2: osd_enable_op_tracker = '' 2026-02-24T07:47:31.950 INFO:teuthology.orchestra.run.trial035.stdout:osd.3: {} 2026-02-24T07:47:31.950 INFO:teuthology.orchestra.run.trial035.stderr:osd.3: osd_enable_op_tracker = '' 2026-02-24T07:47:31.980 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 88 16 651698 651682 462.803 383.312 0.00296102 0.00215947 2026-02-24T07:47:32.980 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 89 16 657340 657324 461.565 352.625 0.00182532 0.00216529 2026-02-24T07:47:33.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:47:32 trial035 ceph-mon[39284]: pgmap v156: 17 pgs: 2 active+clean+scrubbing, 15 active+clean; 39 GiB data, 124 GiB used, 2.6 TiB / 2.7 TiB avail; 427 MiB/s wr, 6.84k op/s 2026-02-24T07:47:33.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:47:32 trial035 ceph-mon[39284]: 3.a scrub starts 2026-02-24T07:47:33.205 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:47:32 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/3237230866' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2026-02-24T07:47:33.980 INFO:tasks.radosbench.radosbench.0.trial035.stdout:Total time run: 90.003 2026-02-24T07:47:33.980 INFO:tasks.radosbench.radosbench.0.trial035.stdout:Total writes made: 663507 2026-02-24T07:47:33.980 INFO:tasks.radosbench.radosbench.0.trial035.stdout:Write size: 65536 2026-02-24T07:47:33.980 INFO:tasks.radosbench.radosbench.0.trial035.stdout:Object size: 65536 2026-02-24T07:47:33.980 INFO:tasks.radosbench.radosbench.0.trial035.stdout:Bandwidth (MB/sec): 460.754 2026-02-24T07:47:33.980 INFO:tasks.radosbench.radosbench.0.trial035.stdout:Stddev Bandwidth: 54.7692 2026-02-24T07:47:33.980 INFO:tasks.radosbench.radosbench.0.trial035.stdout:Max bandwidth (MB/sec): 547.812 2026-02-24T07:47:33.980 INFO:tasks.radosbench.radosbench.0.trial035.stdout:Min bandwidth (MB/sec): 349.875 2026-02-24T07:47:33.980 INFO:tasks.radosbench.radosbench.0.trial035.stdout:Average IOPS: 7372 2026-02-24T07:47:33.981 INFO:tasks.radosbench.radosbench.0.trial035.stdout:Stddev IOPS: 876.306 2026-02-24T07:47:33.981 INFO:tasks.radosbench.radosbench.0.trial035.stdout:Max IOPS: 8765 2026-02-24T07:47:33.981 INFO:tasks.radosbench.radosbench.0.trial035.stdout:Min IOPS: 5598 2026-02-24T07:47:33.981 INFO:tasks.radosbench.radosbench.0.trial035.stdout:Average Latency(s): 0.00216909 2026-02-24T07:47:33.981 INFO:tasks.radosbench.radosbench.0.trial035.stdout:Stddev Latency(s): 0.00191611 2026-02-24T07:47:33.981 INFO:tasks.radosbench.radosbench.0.trial035.stdout:Max latency(s): 0.0419194 2026-02-24T07:47:33.981 INFO:tasks.radosbench.radosbench.0.trial035.stdout:Min latency(s): 0.000682899 2026-02-24T07:47:33.981 INFO:tasks.radosbench.radosbench.0.trial035.stdout:Cleaning up (deleting benchmark objects) 2026-02-24T07:47:33.994 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2026-02-24T07:47:34.142 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:47:34.166 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:47:33 trial035 ceph-mon[39284]: 3.7 scrub starts 2026-02-24T07:47:34.166 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:47:33 trial035 ceph-mon[39284]: 3.e scrub ok 2026-02-24T07:47:34.266 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:47:34.264+0000 7fcacb94b640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:47:34.266 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:47:34.264+0000 7fcacb94b640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:47:34.281 INFO:teuthology.orchestra.run.trial035.stdout:osd.0: {} 2026-02-24T07:47:34.281 INFO:teuthology.orchestra.run.trial035.stderr:osd.0: osd_enable_op_tracker = '' 2026-02-24T07:47:34.297 INFO:teuthology.orchestra.run.trial035.stdout:osd.1: {} 2026-02-24T07:47:34.297 INFO:teuthology.orchestra.run.trial035.stderr:osd.1: osd_enable_op_tracker = '' 2026-02-24T07:47:34.311 INFO:teuthology.orchestra.run.trial035.stdout:osd.2: {} 2026-02-24T07:47:34.312 INFO:teuthology.orchestra.run.trial035.stderr:osd.2: osd_enable_op_tracker = '' 2026-02-24T07:47:34.324 INFO:teuthology.orchestra.run.trial035.stdout:osd.3: {} 2026-02-24T07:47:34.325 INFO:teuthology.orchestra.run.trial035.stderr:osd.3: osd_enable_op_tracker = '' 2026-02-24T07:47:35.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:47:34 trial035 ceph-mon[39284]: 3.4 scrub ok 2026-02-24T07:47:35.205 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:47:34 trial035 ceph-mon[39284]: pgmap v157: 17 pgs: 2 active+clean+scrubbing, 15 active+clean; 39 GiB data, 124 GiB used, 2.6 TiB / 2.7 TiB avail; 427 MiB/s wr, 6.84k op/s 2026-02-24T07:47:35.205 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:47:34 trial035 ceph-mon[39284]: purged_snaps scrub starts 2026-02-24T07:47:35.205 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:47:34 trial035 ceph-mon[39284]: purged_snaps scrub ok 2026-02-24T07:47:35.205 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:47:34 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/215710018' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2026-02-24T07:47:35.301 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2026-02-24T07:47:35.302 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2026-02-24T07:47:35.302 INFO:tasks.thrashosds.thrasher:Growing pool .mgr 2026-02-24T07:47:35.302 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph pg dump --format=json 2026-02-24T07:47:35.444 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:47:35.572 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:47:35.570+0000 7f46b7f5e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:47:35.572 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:47:35.570+0000 7f46b7f5e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:47:35.712 INFO:teuthology.orchestra.run.trial035.stdout: 2026-02-24T07:47:35.713 INFO:teuthology.orchestra.run.trial035.stderr:dumped all 2026-02-24T07:47:35.757 INFO:teuthology.orchestra.run.trial035.stdout:{"pg_ready":true,"pg_map":{"version":158,"stamp":"2026-02-24T07:47:35.694867+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":42212262432,"num_objects":644101,"num_object_clones":0,"num_object_copies":1932303,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":644101,"num_whiteouts":0,"num_read":136,"num_read_kb":240,"num_write":644234,"num_write_kb":41224486,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":51,"ondisk_log_size":51,"up":51,"acting":51,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":51,"num_osds":4,"num_per_pool_osds":4,"num_per_pool_omap_osds":4,"kb":2936012800,"kb_used":129817612,"kb_used_data":122912060,"kb_used_omap":33,"kb_used_meta":6905438,"kb_avail":2806195188,"statfs":{"total":3006477107200,"available":2873543872512,"internally_reserved":0,"allocated":125861949440,"data_stored":125860767068,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":34095,"internal_metadata":7071169233},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[6,3,1,2],"upper_bound":16},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":4481941504,"num_objects":68389,"num_object_clones":0,"num_object_copies":205167,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":68389,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":68389,"num_write_kb":4376896,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.001431"},"pg_stats":[{"pgid":"3.f","version":"46'40145","reported_seq":40509,"reported_epoch":46,"state":"active+clean","last_fresh":"2026-02-24T07:47:30.697850+0000","last_change":"2026-02-24T07:47:16.466592+0000","last_active":"2026-02-24T07:47:30.697850+0000","last_peered":"2026-02-24T07:47:30.697850+0000","last_clean":"2026-02-24T07:47:30.697850+0000","last_became_active":"2026-02-24T07:46:02.881625+0000","last_became_peered":"2026-02-24T07:46:02.881625+0000","last_unstale":"2026-02-24T07:47:30.697850+0000","last_undegraded":"2026-02-24T07:47:30.697850+0000","last_fullsized":"2026-02-24T07:47:30.697850+0000","mapping_epoch":42,"log_start":"46'40142","ondisk_log_start":"46'40142","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"45'34153","last_scrub_stamp":"2026-02-24T07:47:16.466530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_clean_scrub_stamp":"2026-02-24T07:47:16.466530+0000","objects_scrubbed":33463,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:48:23.487779+0000","scrub_duration":3074,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2630942720,"num_objects":40145,"num_object_clones":0,"num_object_copies":120435,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40145,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":40145,"num_write_kb":2569280,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.e","version":"46'40649","reported_seq":40844,"reported_epoch":46,"state":"active+clean+scrubbing","last_fresh":"2026-02-24T07:47:30.693230+0000","last_change":"2026-02-24T07:47:29.130393+0000","last_active":"2026-02-24T07:47:30.693230+0000","last_peered":"2026-02-24T07:47:30.693230+0000","last_clean":"2026-02-24T07:47:30.693230+0000","last_became_active":"2026-02-24T07:46:02.881083+0000","last_became_peered":"2026-02-24T07:46:02.881083+0000","last_unstale":"2026-02-24T07:47:30.693230+0000","last_undegraded":"2026-02-24T07:47:30.693230+0000","last_fullsized":"2026-02-24T07:47:30.693230+0000","mapping_epoch":42,"log_start":"46'40646","ondisk_log_start":"46'40646","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_clean_scrub_stamp":"2026-02-24T07:46:01.867949+0000","objects_scrubbed":17199,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"scrubbing for 2s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2663972864,"num_objects":40649,"num_object_clones":0,"num_object_copies":121947,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40649,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":40649,"num_write_kb":2601536,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"46'40338","reported_seq":40663,"reported_epoch":46,"state":"active+clean","last_fresh":"2026-02-24T07:47:30.713443+0000","last_change":"2026-02-24T07:47:06.968813+0000","last_active":"2026-02-24T07:47:30.713443+0000","last_peered":"2026-02-24T07:47:30.713443+0000","last_clean":"2026-02-24T07:47:30.713443+0000","last_became_active":"2026-02-24T07:46:02.883799+0000","last_became_peered":"2026-02-24T07:46:02.883799+0000","last_unstale":"2026-02-24T07:47:30.713443+0000","last_undegraded":"2026-02-24T07:47:30.713443+0000","last_fullsized":"2026-02-24T07:47:30.713443+0000","mapping_epoch":42,"log_start":"46'40335","ondisk_log_start":"46'40335","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"45'30090","last_scrub_stamp":"2026-02-24T07:47:06.968732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_clean_scrub_stamp":"2026-02-24T07:47:06.968732+0000","objects_scrubbed":29542,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:48:25.198085+0000","scrub_duration":3070,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2643591168,"num_objects":40338,"num_object_clones":0,"num_object_copies":121014,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40338,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":40338,"num_write_kb":2581632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"46'39584","reported_seq":39908,"reported_epoch":46,"state":"active+clean","last_fresh":"2026-02-24T07:47:30.718164+0000","last_change":"2026-02-24T07:47:07.546834+0000","last_active":"2026-02-24T07:47:30.718164+0000","last_peered":"2026-02-24T07:47:30.718164+0000","last_clean":"2026-02-24T07:47:30.718164+0000","last_became_active":"2026-02-24T07:46:02.880060+0000","last_became_peered":"2026-02-24T07:46:02.880060+0000","last_unstale":"2026-02-24T07:47:30.718164+0000","last_undegraded":"2026-02-24T07:47:30.718164+0000","last_fullsized":"2026-02-24T07:47:30.718164+0000","mapping_epoch":42,"log_start":"46'39581","ondisk_log_start":"46'39581","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"45'29910","last_scrub_stamp":"2026-02-24T07:47:07.546803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_clean_scrub_stamp":"2026-02-24T07:47:07.546803+0000","objects_scrubbed":29435,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:48:27.524515+0000","scrub_duration":2696,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2594177024,"num_objects":39584,"num_object_clones":0,"num_object_copies":118752,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39584,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":39584,"num_write_kb":2533376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"46'40262","reported_seq":40598,"reported_epoch":46,"state":"active+clean","last_fresh":"2026-02-24T07:47:30.755791+0000","last_change":"2026-02-24T07:47:10.376412+0000","last_active":"2026-02-24T07:47:30.755791+0000","last_peered":"2026-02-24T07:47:30.755791+0000","last_clean":"2026-02-24T07:47:30.755791+0000","last_became_active":"2026-02-24T07:46:02.880171+0000","last_became_peered":"2026-02-24T07:46:02.880171+0000","last_unstale":"2026-02-24T07:47:30.755791+0000","last_undegraded":"2026-02-24T07:47:30.755791+0000","last_fullsized":"2026-02-24T07:47:30.755791+0000","mapping_epoch":42,"log_start":"46'40259","ondisk_log_start":"46'40259","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"45'31519","last_scrub_stamp":"2026-02-24T07:47:10.376345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_clean_scrub_stamp":"2026-02-24T07:47:10.376345+0000","objects_scrubbed":30810,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:48:13.377288+0000","scrub_duration":4056,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2638610432,"num_objects":40262,"num_object_clones":0,"num_object_copies":120786,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40262,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":40262,"num_write_kb":2576768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.a","version":"46'39880","reported_seq":39899,"reported_epoch":46,"state":"active+clean","last_fresh":"2026-02-24T07:47:30.717940+0000","last_change":"2026-02-24T07:46:02.882761+0000","last_active":"2026-02-24T07:47:30.717940+0000","last_peered":"2026-02-24T07:47:30.717940+0000","last_clean":"2026-02-24T07:47:30.717940+0000","last_became_active":"2026-02-24T07:46:02.882274+0000","last_became_peered":"2026-02-24T07:46:02.882274+0000","last_unstale":"2026-02-24T07:47:30.717940+0000","last_undegraded":"2026-02-24T07:47:30.717940+0000","last_fullsized":"2026-02-24T07:47:30.717940+0000","mapping_epoch":42,"log_start":"46'39877","ondisk_log_start":"46'39877","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_clean_scrub_stamp":"2026-02-24T07:46:01.867949+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:47:31.776406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2613575680,"num_objects":39880,"num_object_clones":0,"num_object_copies":119640,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39880,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":39880,"num_write_kb":2552320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"46'40678","reported_seq":41021,"reported_epoch":46,"state":"active+clean","last_fresh":"2026-02-24T07:47:30.711893+0000","last_change":"2026-02-24T07:47:10.197277+0000","last_active":"2026-02-24T07:47:30.711893+0000","last_peered":"2026-02-24T07:47:30.711893+0000","last_clean":"2026-02-24T07:47:30.711893+0000","last_became_active":"2026-02-24T07:46:02.881951+0000","last_became_peered":"2026-02-24T07:46:02.881951+0000","last_unstale":"2026-02-24T07:47:30.711893+0000","last_undegraded":"2026-02-24T07:47:30.711893+0000","last_fullsized":"2026-02-24T07:47:30.711893+0000","mapping_epoch":42,"log_start":"46'40675","ondisk_log_start":"46'40675","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"45'31847","last_scrub_stamp":"2026-02-24T07:47:10.197239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_clean_scrub_stamp":"2026-02-24T07:47:10.197239+0000","objects_scrubbed":31279,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:48:27.205650+0000","scrub_duration":3289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2665873408,"num_objects":40678,"num_object_clones":0,"num_object_copies":122034,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40678,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":40678,"num_write_kb":2603392,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"46'40371","reported_seq":40760,"reported_epoch":46,"state":"active+clean","last_fresh":"2026-02-24T07:47:30.758023+0000","last_change":"2026-02-24T07:47:22.289650+0000","last_active":"2026-02-24T07:47:30.758023+0000","last_peered":"2026-02-24T07:47:30.758023+0000","last_clean":"2026-02-24T07:47:30.758023+0000","last_became_active":"2026-02-24T07:46:02.881345+0000","last_became_peered":"2026-02-24T07:46:02.881345+0000","last_unstale":"2026-02-24T07:47:30.758023+0000","last_undegraded":"2026-02-24T07:47:30.758023+0000","last_fullsized":"2026-02-24T07:47:30.758023+0000","mapping_epoch":42,"log_start":"46'40368","ondisk_log_start":"46'40368","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"45'36703","last_scrub_stamp":"2026-02-24T07:47:22.289617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_clean_scrub_stamp":"2026-02-24T07:47:22.289617+0000","objects_scrubbed":35960,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:48:31.332102+0000","scrub_duration":3891,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2645753856,"num_objects":40371,"num_object_clones":0,"num_object_copies":121113,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40371,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":40371,"num_write_kb":2583744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"46'40057","reported_seq":40076,"reported_epoch":46,"state":"active+clean","last_fresh":"2026-02-24T07:47:30.762445+0000","last_change":"2026-02-24T07:46:02.880582+0000","last_active":"2026-02-24T07:47:30.762445+0000","last_peered":"2026-02-24T07:47:30.762445+0000","last_clean":"2026-02-24T07:47:30.762445+0000","last_became_active":"2026-02-24T07:46:02.880323+0000","last_became_peered":"2026-02-24T07:46:02.880323+0000","last_unstale":"2026-02-24T07:47:30.762445+0000","last_undegraded":"2026-02-24T07:47:30.762445+0000","last_fullsized":"2026-02-24T07:47:30.762445+0000","mapping_epoch":42,"log_start":"46'40054","ondisk_log_start":"46'40054","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_clean_scrub_stamp":"2026-02-24T07:46:01.867949+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:47:31.420697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2625175552,"num_objects":40057,"num_object_clones":0,"num_object_copies":120171,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40057,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":40057,"num_write_kb":2563648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.6","version":"46'40300","reported_seq":40693,"reported_epoch":46,"state":"active+clean","last_fresh":"2026-02-24T07:47:30.704836+0000","last_change":"2026-02-24T07:47:23.432795+0000","last_active":"2026-02-24T07:47:30.704836+0000","last_peered":"2026-02-24T07:47:30.704836+0000","last_clean":"2026-02-24T07:47:30.704836+0000","last_became_active":"2026-02-24T07:46:02.880005+0000","last_became_peered":"2026-02-24T07:46:02.880005+0000","last_unstale":"2026-02-24T07:47:30.704836+0000","last_undegraded":"2026-02-24T07:47:30.704836+0000","last_fullsized":"2026-02-24T07:47:30.704836+0000","mapping_epoch":42,"log_start":"46'40297","ondisk_log_start":"46'40297","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"45'37219","last_scrub_stamp":"2026-02-24T07:47:23.432744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_clean_scrub_stamp":"2026-02-24T07:47:23.432744+0000","objects_scrubbed":36510,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:48:51.822832+0000","scrub_duration":3412,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2641100800,"num_objects":40300,"num_object_clones":0,"num_object_copies":120900,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40300,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":40300,"num_write_kb":2579200,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"46'40599","reported_seq":40970,"reported_epoch":46,"state":"active+clean","last_fresh":"2026-02-24T07:47:30.694103+0000","last_change":"2026-02-24T07:47:17.546625+0000","last_active":"2026-02-24T07:47:30.694103+0000","last_peered":"2026-02-24T07:47:30.694103+0000","last_clean":"2026-02-24T07:47:30.694103+0000","last_became_active":"2026-02-24T07:46:02.881420+0000","last_became_peered":"2026-02-24T07:46:02.881420+0000","last_unstale":"2026-02-24T07:47:30.694103+0000","last_undegraded":"2026-02-24T07:47:30.694103+0000","last_fullsized":"2026-02-24T07:47:30.694103+0000","mapping_epoch":42,"log_start":"46'40596","ondisk_log_start":"46'40596","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"45'34854","last_scrub_stamp":"2026-02-24T07:47:17.546590+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_clean_scrub_stamp":"2026-02-24T07:47:17.546590+0000","objects_scrubbed":34222,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:48:38.813192+0000","scrub_duration":3116,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2660696064,"num_objects":40599,"num_object_clones":0,"num_object_copies":121797,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40599,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":40599,"num_write_kb":2598336,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.4","version":"46'40030","reported_seq":40161,"reported_epoch":46,"state":"active+clean+scrubbing","last_fresh":"2026-02-24T07:47:30.766347+0000","last_change":"2026-02-24T07:47:29.547712+0000","last_active":"2026-02-24T07:47:30.766347+0000","last_peered":"2026-02-24T07:47:30.766347+0000","last_clean":"2026-02-24T07:47:30.766347+0000","last_became_active":"2026-02-24T07:46:02.880345+0000","last_became_peered":"2026-02-24T07:46:02.880345+0000","last_unstale":"2026-02-24T07:47:30.766347+0000","last_undegraded":"2026-02-24T07:47:30.766347+0000","last_fullsized":"2026-02-24T07:47:30.766347+0000","mapping_epoch":42,"log_start":"46'40027","ondisk_log_start":"46'40027","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_clean_scrub_stamp":"2026-02-24T07:46:01.867949+0000","objects_scrubbed":10949,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"scrubbing for 2s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2623406080,"num_objects":40030,"num_object_clones":0,"num_object_copies":120090,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40030,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":40030,"num_write_kb":2561920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"46'40316","reported_seq":40672,"reported_epoch":46,"state":"active+clean","last_fresh":"2026-02-24T07:47:30.692812+0000","last_change":"2026-02-24T07:47:14.339634+0000","last_active":"2026-02-24T07:47:30.692812+0000","last_peered":"2026-02-24T07:47:30.692812+0000","last_clean":"2026-02-24T07:47:30.692812+0000","last_became_active":"2026-02-24T07:46:02.881035+0000","last_became_peered":"2026-02-24T07:46:02.881035+0000","last_unstale":"2026-02-24T07:47:30.692812+0000","last_undegraded":"2026-02-24T07:47:30.692812+0000","last_fullsized":"2026-02-24T07:47:30.692812+0000","mapping_epoch":42,"log_start":"46'40313","ondisk_log_start":"46'40313","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"45'33283","last_scrub_stamp":"2026-02-24T07:47:14.339597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_clean_scrub_stamp":"2026-02-24T07:47:14.339597+0000","objects_scrubbed":32630,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:48:22.078719+0000","scrub_duration":2951,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2642149376,"num_objects":40316,"num_object_clones":0,"num_object_copies":120948,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40316,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":40316,"num_write_kb":2580224,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"46'40228","reported_seq":40625,"reported_epoch":46,"state":"active+clean","last_fresh":"2026-02-24T07:47:30.703467+0000","last_change":"2026-02-24T07:47:24.807734+0000","last_active":"2026-02-24T07:47:30.703467+0000","last_peered":"2026-02-24T07:47:30.703467+0000","last_clean":"2026-02-24T07:47:30.703467+0000","last_became_active":"2026-02-24T07:46:02.879831+0000","last_became_peered":"2026-02-24T07:46:02.879831+0000","last_unstale":"2026-02-24T07:47:30.703467+0000","last_undegraded":"2026-02-24T07:47:30.703467+0000","last_fullsized":"2026-02-24T07:47:30.703467+0000","mapping_epoch":42,"log_start":"46'40225","ondisk_log_start":"46'40225","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"45'37558","last_scrub_stamp":"2026-02-24T07:47:24.807659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_clean_scrub_stamp":"2026-02-24T07:47:24.807659+0000","objects_scrubbed":36869,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:48:39.172465+0000","scrub_duration":3748,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2636382208,"num_objects":40228,"num_object_clones":0,"num_object_copies":120684,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40228,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":40228,"num_write_kb":2574592,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.0","version":"46'40154","reported_seq":40482,"reported_epoch":46,"state":"active+clean","last_fresh":"2026-02-24T07:47:30.765219+0000","last_change":"2026-02-24T07:47:08.376360+0000","last_active":"2026-02-24T07:47:30.765219+0000","last_peered":"2026-02-24T07:47:30.765219+0000","last_clean":"2026-02-24T07:47:30.765219+0000","last_became_active":"2026-02-24T07:46:02.880333+0000","last_became_peered":"2026-02-24T07:46:02.880333+0000","last_unstale":"2026-02-24T07:47:30.765219+0000","last_undegraded":"2026-02-24T07:47:30.765219+0000","last_fullsized":"2026-02-24T07:47:30.765219+0000","mapping_epoch":42,"log_start":"46'40151","ondisk_log_start":"46'40151","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"45'30511","last_scrub_stamp":"2026-02-24T07:47:08.376304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_clean_scrub_stamp":"2026-02-24T07:47:08.376304+0000","objects_scrubbed":29912,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:48:24.552234+0000","scrub_duration":3049,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2631532544,"num_objects":40154,"num_object_clones":0,"num_object_copies":120462,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40154,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":40154,"num_write_kb":2569856,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2","version":"46'40508","reported_seq":40919,"reported_epoch":46,"state":"active+clean","last_fresh":"2026-02-24T07:47:30.760338+0000","last_change":"2026-02-24T07:47:27.784936+0000","last_active":"2026-02-24T07:47:30.760338+0000","last_peered":"2026-02-24T07:47:30.760338+0000","last_clean":"2026-02-24T07:47:30.760338+0000","last_became_active":"2026-02-24T07:46:02.880185+0000","last_became_peered":"2026-02-24T07:46:02.880185+0000","last_unstale":"2026-02-24T07:47:30.760338+0000","last_undegraded":"2026-02-24T07:47:30.760338+0000","last_fullsized":"2026-02-24T07:47:30.760338+0000","mapping_epoch":42,"log_start":"46'40505","ondisk_log_start":"46'40505","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"45'39074","last_scrub_stamp":"2026-02-24T07:47:27.784888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_clean_scrub_stamp":"2026-02-24T07:47:27.784888+0000","objects_scrubbed":38200,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:48:47.063599+0000","scrub_duration":4266,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2654732288,"num_objects":40508,"num_object_clones":0,"num_object_copies":121524,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40508,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":40508,"num_write_kb":2592512,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"29'99","reported_seq":239,"reported_epoch":46,"state":"active+clean","last_fresh":"2026-02-24T07:47:27.863858+0000","last_change":"2026-02-24T07:46:41.007499+0000","last_active":"2026-02-24T07:47:27.863858+0000","last_peered":"2026-02-24T07:47:27.863858+0000","last_clean":"2026-02-24T07:47:27.863858+0000","last_became_active":"2026-02-24T07:42:54.186719+0000","last_became_peered":"2026-02-24T07:42:54.186719+0000","last_unstale":"2026-02-24T07:47:27.863858+0000","last_undegraded":"2026-02-24T07:47:27.863858+0000","last_fullsized":"2026-02-24T07:47:27.863858+0000","mapping_epoch":27,"log_start":"29'96","ondisk_log_start":"29'96","created":20,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'99","last_scrub_stamp":"2026-02-24T07:46:41.007434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:42:43.640776+0000","last_clean_scrub_stamp":"2026-02-24T07:46:41.007434+0000","objects_scrubbed":2,"log_size":3,"log_dups_size":96,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:47:53.486511+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":240,"num_write":135,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":16,"stat_sum":{"num_bytes":42211672064,"num_objects":644099,"num_object_clones":0,"num_object_copies":1932297,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":644099,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":644099,"num_write_kb":41222336,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":126626365440,"data_stored":126626365440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":48,"ondisk_log_size":48,"up":48,"acting":48,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":240,"num_write":135,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":3,"up_from":25,"seq":107374182458,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":27286548,"kb_used_data":25512960,"kb_used_omap":7,"kb_used_meta":1773560,"kb_avail":706716652,"statfs":{"total":751619276800,"available":723677851648,"internally_reserved":0,"allocated":26125271040,"data_stored":26124974559,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":8105,"internal_metadata":1816125527},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":19,"seq":81604378684,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":36855380,"kb_used_data":35918720,"kb_used_omap":7,"kb_used_meta":936632,"kb_avail":697147820,"statfs":{"total":751619276800,"available":713879367680,"internally_reserved":0,"allocated":36780769280,"data_stored":36780472799,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":7981,"internal_metadata":959111379},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[4,1,1,2],"upper_bound":16},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542205,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":30269904,"kb_used_data":28233660,"kb_used_omap":8,"kb_used_meta":2036215,"kb_avail":703733296,"statfs":{"total":751619276800,"available":720622895104,"internally_reserved":0,"allocated":28911267840,"data_stored":28910974911,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":8343,"internal_metadata":2085085033},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705728,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":35405780,"kb_used_data":33246720,"kb_used_omap":9,"kb_used_meta":2159030,"kb_avail":698597420,"statfs":{"total":751619276800,"available":715363758080,"internally_reserved":0,"allocated":34044641280,"data_stored":34044344799,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9666,"internal_metadata":2210847294},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,2],"upper_bound":4},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":34272182272,"data_stored":34272182272,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28976807936,"data_stored":28976807936,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":36982095872,"data_stored":36982095872,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":26395279360,"data_stored":26395279360,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-24T07:47:35.759 INFO:tasks.cephadm.ceph_manager.ceph:increase pool size by 10 2026-02-24T07:47:35.759 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph osd pool set .mgr pg_num 151 2026-02-24T07:47:35.901 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:47:35.964 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:47:35 trial035 ceph-mon[39284]: purged_snaps scrub starts 2026-02-24T07:47:35.964 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:47:35 trial035 ceph-mon[39284]: purged_snaps scrub ok 2026-02-24T07:47:36.022 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:47:36.020+0000 7fb80c248640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:47:36.022 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:47:36.020+0000 7fb80c248640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:47:36.161 INFO:teuthology.orchestra.run.trial035.stderr:Error EPERM: crimson_allow_pg_split is false; pg_num increase denied for crimson pool 2026-02-24T07:47:36.200 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-24T07:47:36.368 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2026-02-24T07:47:36.511 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:47:36.633 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:47:36.630+0000 7f0c0af61640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:47:36.633 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:47:36.630+0000 7f0c0af61640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:47:36.648 INFO:teuthology.orchestra.run.trial035.stdout:osd.0: {} 2026-02-24T07:47:36.648 INFO:teuthology.orchestra.run.trial035.stderr:osd.0: osd_enable_op_tracker = '' 2026-02-24T07:47:36.662 INFO:teuthology.orchestra.run.trial035.stdout:osd.1: {} 2026-02-24T07:47:36.662 INFO:teuthology.orchestra.run.trial035.stderr:osd.1: osd_enable_op_tracker = '' 2026-02-24T07:47:36.674 INFO:teuthology.orchestra.run.trial035.stdout:osd.2: {} 2026-02-24T07:47:36.675 INFO:teuthology.orchestra.run.trial035.stderr:osd.2: osd_enable_op_tracker = '' 2026-02-24T07:47:36.687 INFO:teuthology.orchestra.run.trial035.stdout:osd.3: {} 2026-02-24T07:47:36.687 INFO:teuthology.orchestra.run.trial035.stderr:osd.3: osd_enable_op_tracker = '' 2026-02-24T07:47:36.954 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:47:36 trial035 ceph-mon[39284]: 3.a scrub ok 2026-02-24T07:47:36.955 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:47:36 trial035 ceph-mon[39284]: pgmap v158: 17 pgs: 2 active+clean+scrubbing, 15 active+clean; 39 GiB data, 124 GiB used, 2.6 TiB / 2.7 TiB avail; 427 MiB/s wr, 6.84k op/s 2026-02-24T07:47:36.955 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:47:36 trial035 ceph-mon[39284]: from='client.14730 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2026-02-24T07:47:36.955 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:47:36 trial035 ceph-mon[39284]: osdmap e47: 4 total, 4 up, 4 in 2026-02-24T07:47:36.955 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:47:36 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/2595642367' entity='client.admin' cmd={"prefix": "osd pool set", "pool": ".mgr", "var": "pg_num", "val": "151"} : dispatch 2026-02-24T07:47:36.955 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:47:36 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/865054975' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2026-02-24T07:47:37.357 INFO:journalctl@ceph.prometheus.a.trial091.stdout:Feb 24 07:47:36 trial091 ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-prometheus-a[66007]: time=2026-02-24T07:47:36.973Z level=ERROR source=scrape.go:1630 msg="Failed to determine correct type of scrape target." component="scrape manager" scrape_pool=ceph target=http://trial035:9283/metrics content_type="text/html;charset=utf-8" fallback_media_type="" err="received unsupported Content-Type \"text/html;charset=utf-8\" and no fallback_scrape_protocol specified for target" 2026-02-24T07:47:37.454 INFO:journalctl@ceph.mgr.x.trial035.stdout:Feb 24 07:47:36 trial035 ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-mgr-x[39621]: ::ffff:10.20.193.91 - - [24/Feb/2026:07:47:36] "GET /metrics HTTP/1.1" 200 4217 "" "Prometheus/3.6.0" 2026-02-24T07:47:38.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:47:37 trial035 ceph-mon[39284]: 3.7 scrub ok 2026-02-24T07:47:38.726 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2026-02-24T07:47:38.871 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:47:39.013 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:47:38 trial035 ceph-mon[39284]: pgmap v160: 17 pgs: 1 active+clean+scrubbing, 16 active+clean; 40 GiB data, 125 GiB used, 2.6 TiB / 2.7 TiB avail; 104 B/s rd, 357 MiB/s wr, 7.16k op/s 2026-02-24T07:47:39.333 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:47:39.259+0000 7f378151e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:47:39.333 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:47:39.260+0000 7f378151e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:47:39.333 INFO:teuthology.orchestra.run.trial035.stderr:osd.0: osd_enable_op_tracker = '' 2026-02-24T07:47:39.333 INFO:teuthology.orchestra.run.trial035.stderr:osd.1: osd_enable_op_tracker = '' 2026-02-24T07:47:39.334 INFO:teuthology.orchestra.run.trial035.stdout:osd.0: {} 2026-02-24T07:47:39.334 INFO:teuthology.orchestra.run.trial035.stdout:osd.1: {} 2026-02-24T07:47:39.334 INFO:teuthology.orchestra.run.trial035.stdout:osd.2: {} 2026-02-24T07:47:39.334 INFO:teuthology.orchestra.run.trial035.stdout:osd.3: {} 2026-02-24T07:47:39.334 INFO:teuthology.orchestra.run.trial035.stderr:osd.2: osd_enable_op_tracker = '' 2026-02-24T07:47:39.334 INFO:teuthology.orchestra.run.trial035.stderr:osd.3: osd_enable_op_tracker = '' 2026-02-24T07:47:40.042 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:47:39 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/4133562250' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2026-02-24T07:47:41.201 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2026-02-24T07:47:41.202 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2026-02-24T07:47:41.202 INFO:tasks.thrashosds.thrasher:Growing pool .mgr 2026-02-24T07:47:41.202 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph pg dump --format=json 2026-02-24T07:47:41.205 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:47:40 trial035 ceph-mon[39284]: pgmap v161: 17 pgs: 1 active+clean+scrubbing, 16 active+clean; 40 GiB data, 125 GiB used, 2.6 TiB / 2.7 TiB avail; 102 B/s rd, 351 MiB/s wr, 7.04k op/s 2026-02-24T07:47:41.348 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:47:41.474 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:47:41.471+0000 7f1808781640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:47:41.474 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:47:41.471+0000 7f1808781640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:47:41.622 INFO:teuthology.orchestra.run.trial035.stdout: 2026-02-24T07:47:41.623 INFO:teuthology.orchestra.run.trial035.stderr:dumped all 2026-02-24T07:47:41.667 INFO:teuthology.orchestra.run.trial035.stdout:{"pg_ready":true,"pg_map":{"version":161,"stamp":"2026-02-24T07:47:39.695336+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":42548593208,"num_objects":649234,"num_object_clones":0,"num_object_copies":1947702,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":649234,"num_whiteouts":0,"num_read":137,"num_read_kb":241,"num_write":677919,"num_write_kb":42466599,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":52,"ondisk_log_size":52,"up":51,"acting":51,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":51,"num_osds":4,"num_per_pool_osds":4,"num_per_pool_omap_osds":4,"kb":2936012800,"kb_used":131163224,"kb_used_data":125480264,"kb_used_omap":32,"kb_used_meta":5682719,"kb_avail":2804849576,"statfs":{"total":3006477107200,"available":2872165965824,"internally_reserved":0,"allocated":128491790336,"data_stored":128490595748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":32844,"internal_metadata":5819105204},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[5],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2744320024,"num_objects":41876,"num_object_clones":0,"num_object_copies":125628,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41876,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":70428,"num_write_kb":3593665,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.001315"},"pg_stats":[{"pgid":"3.f","version":"46'42193","reported_seq":42557,"reported_epoch":46,"state":"active+clean","last_fresh":"2026-02-24T07:47:35.699221+0000","last_change":"2026-02-24T07:47:16.466592+0000","last_active":"2026-02-24T07:47:35.699221+0000","last_peered":"2026-02-24T07:47:35.699221+0000","last_clean":"2026-02-24T07:47:35.699221+0000","last_became_active":"2026-02-24T07:46:02.881625+0000","last_became_peered":"2026-02-24T07:46:02.881625+0000","last_unstale":"2026-02-24T07:47:35.699221+0000","last_undegraded":"2026-02-24T07:47:35.699221+0000","last_fullsized":"2026-02-24T07:47:35.699221+0000","mapping_epoch":42,"log_start":"46'42190","ondisk_log_start":"46'42190","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"45'34153","last_scrub_stamp":"2026-02-24T07:47:16.466530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_clean_scrub_stamp":"2026-02-24T07:47:16.466530+0000","objects_scrubbed":33463,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:48:23.487779+0000","scrub_duration":3074,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2648768512,"num_objects":40417,"num_object_clones":0,"num_object_copies":121251,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40417,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":42193,"num_write_kb":2643520,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.e","version":"46'42785","reported_seq":43221,"reported_epoch":46,"state":"active+clean","last_fresh":"2026-02-24T07:47:35.697618+0000","last_change":"2026-02-24T07:47:33.099762+0000","last_active":"2026-02-24T07:47:35.697618+0000","last_peered":"2026-02-24T07:47:35.697618+0000","last_clean":"2026-02-24T07:47:35.697618+0000","last_became_active":"2026-02-24T07:46:02.881083+0000","last_became_peered":"2026-02-24T07:46:02.881083+0000","last_unstale":"2026-02-24T07:47:35.697618+0000","last_undegraded":"2026-02-24T07:47:35.697618+0000","last_fullsized":"2026-02-24T07:47:35.697618+0000","mapping_epoch":42,"log_start":"46'42782","ondisk_log_start":"46'42782","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"46'41533","last_scrub_stamp":"2026-02-24T07:47:33.099675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_clean_scrub_stamp":"2026-02-24T07:47:33.099675+0000","objects_scrubbed":40754,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:48:53.832432+0000","scrub_duration":3969,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2687696896,"num_objects":41011,"num_object_clones":0,"num_object_copies":123033,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41011,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":42785,"num_write_kb":2681472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"46'42422","reported_seq":42748,"reported_epoch":46,"state":"active+clean","last_fresh":"2026-02-24T07:47:35.716513+0000","last_change":"2026-02-24T07:47:06.968813+0000","last_active":"2026-02-24T07:47:35.716513+0000","last_peered":"2026-02-24T07:47:35.716513+0000","last_clean":"2026-02-24T07:47:35.716513+0000","last_became_active":"2026-02-24T07:46:02.883799+0000","last_became_peered":"2026-02-24T07:46:02.883799+0000","last_unstale":"2026-02-24T07:47:35.716513+0000","last_undegraded":"2026-02-24T07:47:35.716513+0000","last_fullsized":"2026-02-24T07:47:35.716513+0000","mapping_epoch":42,"log_start":"46'42419","ondisk_log_start":"46'42419","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"45'30090","last_scrub_stamp":"2026-02-24T07:47:06.968732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_clean_scrub_stamp":"2026-02-24T07:47:06.968732+0000","objects_scrubbed":29542,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:48:25.198085+0000","scrub_duration":3070,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2664628248,"num_objects":40660,"num_object_clones":0,"num_object_copies":121980,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40660,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":42422,"num_write_kb":2658561,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"46'41655","reported_seq":41979,"reported_epoch":46,"state":"active+clean","last_fresh":"2026-02-24T07:47:35.719096+0000","last_change":"2026-02-24T07:47:07.546834+0000","last_active":"2026-02-24T07:47:35.719096+0000","last_peered":"2026-02-24T07:47:35.719096+0000","last_clean":"2026-02-24T07:47:35.719096+0000","last_became_active":"2026-02-24T07:46:02.880060+0000","last_became_peered":"2026-02-24T07:46:02.880060+0000","last_unstale":"2026-02-24T07:47:35.719096+0000","last_undegraded":"2026-02-24T07:47:35.719096+0000","last_fullsized":"2026-02-24T07:47:35.719096+0000","mapping_epoch":42,"log_start":"46'41652","ondisk_log_start":"46'41652","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"45'29910","last_scrub_stamp":"2026-02-24T07:47:07.546803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_clean_scrub_stamp":"2026-02-24T07:47:07.546803+0000","objects_scrubbed":29435,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:48:27.524515+0000","scrub_duration":2696,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2620588032,"num_objects":39987,"num_object_clones":0,"num_object_copies":119961,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39987,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":41655,"num_write_kb":2612544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"46'42389","reported_seq":42724,"reported_epoch":46,"state":"active+clean","last_fresh":"2026-02-24T07:47:35.771043+0000","last_change":"2026-02-24T07:47:10.376412+0000","last_active":"2026-02-24T07:47:35.771043+0000","last_peered":"2026-02-24T07:47:35.771043+0000","last_clean":"2026-02-24T07:47:35.771043+0000","last_became_active":"2026-02-24T07:46:02.880171+0000","last_became_peered":"2026-02-24T07:46:02.880171+0000","last_unstale":"2026-02-24T07:47:35.771043+0000","last_undegraded":"2026-02-24T07:47:35.771043+0000","last_fullsized":"2026-02-24T07:47:35.771043+0000","mapping_epoch":42,"log_start":"46'42386","ondisk_log_start":"46'42386","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"45'31519","last_scrub_stamp":"2026-02-24T07:47:10.376345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_clean_scrub_stamp":"2026-02-24T07:47:10.376345+0000","objects_scrubbed":30810,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:48:13.377288+0000","scrub_duration":4056,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2657026048,"num_objects":40543,"num_object_clones":0,"num_object_copies":121629,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40543,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":42389,"num_write_kb":2653824,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.a","version":"46'42009","reported_seq":42445,"reported_epoch":46,"state":"active+clean","last_fresh":"2026-02-24T07:47:35.718137+0000","last_change":"2026-02-24T07:47:35.439589+0000","last_active":"2026-02-24T07:47:35.718137+0000","last_peered":"2026-02-24T07:47:35.718137+0000","last_clean":"2026-02-24T07:47:35.718137+0000","last_became_active":"2026-02-24T07:46:02.882274+0000","last_became_peered":"2026-02-24T07:46:02.882274+0000","last_unstale":"2026-02-24T07:47:35.718137+0000","last_undegraded":"2026-02-24T07:47:35.718137+0000","last_fullsized":"2026-02-24T07:47:35.718137+0000","mapping_epoch":42,"log_start":"46'42006","ondisk_log_start":"46'42006","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"46'41859","last_scrub_stamp":"2026-02-24T07:47:35.439542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_clean_scrub_stamp":"2026-02-24T07:47:35.439542+0000","objects_scrubbed":40699,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:48:38.034606+0000","scrub_duration":3645,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2633039872,"num_objects":40177,"num_object_clones":0,"num_object_copies":120531,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40177,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":42009,"num_write_kb":2629952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"46'42796","reported_seq":43139,"reported_epoch":46,"state":"active+clean","last_fresh":"2026-02-24T07:47:35.717366+0000","last_change":"2026-02-24T07:47:10.197277+0000","last_active":"2026-02-24T07:47:35.717366+0000","last_peered":"2026-02-24T07:47:35.717366+0000","last_clean":"2026-02-24T07:47:35.717366+0000","last_became_active":"2026-02-24T07:46:02.881951+0000","last_became_peered":"2026-02-24T07:46:02.881951+0000","last_unstale":"2026-02-24T07:47:35.717366+0000","last_undegraded":"2026-02-24T07:47:35.717366+0000","last_fullsized":"2026-02-24T07:47:35.717366+0000","mapping_epoch":42,"log_start":"46'42793","ondisk_log_start":"46'42793","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"45'31847","last_scrub_stamp":"2026-02-24T07:47:10.197239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_clean_scrub_stamp":"2026-02-24T07:47:10.197239+0000","objects_scrubbed":31279,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:48:27.205650+0000","scrub_duration":3289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2691432448,"num_objects":41068,"num_object_clones":0,"num_object_copies":123204,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41068,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":42796,"num_write_kb":2683648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"46'42511","reported_seq":42900,"reported_epoch":46,"state":"active+clean","last_fresh":"2026-02-24T07:47:35.768310+0000","last_change":"2026-02-24T07:47:22.289650+0000","last_active":"2026-02-24T07:47:35.768310+0000","last_peered":"2026-02-24T07:47:35.768310+0000","last_clean":"2026-02-24T07:47:35.768310+0000","last_became_active":"2026-02-24T07:46:02.881345+0000","last_became_peered":"2026-02-24T07:46:02.881345+0000","last_unstale":"2026-02-24T07:47:35.768310+0000","last_undegraded":"2026-02-24T07:47:35.768310+0000","last_fullsized":"2026-02-24T07:47:35.768310+0000","mapping_epoch":42,"log_start":"46'42508","ondisk_log_start":"46'42508","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"45'36703","last_scrub_stamp":"2026-02-24T07:47:22.289617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_clean_scrub_stamp":"2026-02-24T07:47:22.289617+0000","objects_scrubbed":35960,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:48:31.332102+0000","scrub_duration":3891,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2668036096,"num_objects":40711,"num_object_clones":0,"num_object_copies":122133,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40711,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":42511,"num_write_kb":2663104,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"46'42167","reported_seq":42581,"reported_epoch":46,"state":"active+clean+scrubbing","last_fresh":"2026-02-24T07:47:35.771122+0000","last_change":"2026-02-24T07:47:32.542448+0000","last_active":"2026-02-24T07:47:35.771122+0000","last_peered":"2026-02-24T07:47:35.771122+0000","last_clean":"2026-02-24T07:47:35.771122+0000","last_became_active":"2026-02-24T07:46:02.880323+0000","last_became_peered":"2026-02-24T07:46:02.880323+0000","last_unstale":"2026-02-24T07:47:35.771122+0000","last_undegraded":"2026-02-24T07:47:35.771122+0000","last_fullsized":"2026-02-24T07:47:35.771122+0000","mapping_epoch":42,"log_start":"46'42163","ondisk_log_start":"46'42163","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_clean_scrub_stamp":"2026-02-24T07:46:01.867949+0000","objects_scrubbed":39299,"log_size":4,"log_dups_size":3000,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"scrubbing for 4s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2641690624,"num_objects":40309,"num_object_clones":0,"num_object_copies":120927,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40309,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":42167,"num_write_kb":2639232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.6","version":"46'42386","reported_seq":42778,"reported_epoch":46,"state":"active+clean","last_fresh":"2026-02-24T07:47:35.704018+0000","last_change":"2026-02-24T07:47:23.432795+0000","last_active":"2026-02-24T07:47:35.704018+0000","last_peered":"2026-02-24T07:47:35.704018+0000","last_clean":"2026-02-24T07:47:35.704018+0000","last_became_active":"2026-02-24T07:46:02.880005+0000","last_became_peered":"2026-02-24T07:46:02.880005+0000","last_unstale":"2026-02-24T07:47:35.704018+0000","last_undegraded":"2026-02-24T07:47:35.704018+0000","last_fullsized":"2026-02-24T07:47:35.704018+0000","mapping_epoch":42,"log_start":"46'42383","ondisk_log_start":"46'42383","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"45'37219","last_scrub_stamp":"2026-02-24T07:47:23.432744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_clean_scrub_stamp":"2026-02-24T07:47:23.432744+0000","objects_scrubbed":36510,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:48:51.822832+0000","scrub_duration":3412,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2663776256,"num_objects":40646,"num_object_clones":0,"num_object_copies":121938,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40646,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":42386,"num_write_kb":2657024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"46'42709","reported_seq":43080,"reported_epoch":46,"state":"active+clean","last_fresh":"2026-02-24T07:47:35.698848+0000","last_change":"2026-02-24T07:47:17.546625+0000","last_active":"2026-02-24T07:47:35.698848+0000","last_peered":"2026-02-24T07:47:35.698848+0000","last_clean":"2026-02-24T07:47:35.698848+0000","last_became_active":"2026-02-24T07:46:02.881420+0000","last_became_peered":"2026-02-24T07:46:02.881420+0000","last_unstale":"2026-02-24T07:47:35.698848+0000","last_undegraded":"2026-02-24T07:47:35.698848+0000","last_fullsized":"2026-02-24T07:47:35.698848+0000","mapping_epoch":42,"log_start":"46'42706","ondisk_log_start":"46'42706","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"45'34854","last_scrub_stamp":"2026-02-24T07:47:17.546590+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_clean_scrub_stamp":"2026-02-24T07:47:17.546590+0000","objects_scrubbed":34222,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:48:38.813192+0000","scrub_duration":3116,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2683371520,"num_objects":40945,"num_object_clones":0,"num_object_copies":122835,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40945,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":42709,"num_write_kb":2676928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.4","version":"46'42138","reported_seq":42571,"reported_epoch":46,"state":"active+clean","last_fresh":"2026-02-24T07:47:35.764808+0000","last_change":"2026-02-24T07:47:33.628245+0000","last_active":"2026-02-24T07:47:35.764808+0000","last_peered":"2026-02-24T07:47:35.764808+0000","last_clean":"2026-02-24T07:47:35.764808+0000","last_became_active":"2026-02-24T07:46:02.880345+0000","last_became_peered":"2026-02-24T07:46:02.880345+0000","last_unstale":"2026-02-24T07:47:35.764808+0000","last_undegraded":"2026-02-24T07:47:35.764808+0000","last_fullsized":"2026-02-24T07:47:35.764808+0000","mapping_epoch":42,"log_start":"46'42135","ondisk_log_start":"46'42135","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"46'41076","last_scrub_stamp":"2026-02-24T07:47:33.628218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_clean_scrub_stamp":"2026-02-24T07:47:33.628218+0000","objects_scrubbed":40364,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:48:34.035175+0000","scrub_duration":4080,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2640183296,"num_objects":40286,"num_object_clones":0,"num_object_copies":120858,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40286,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":42138,"num_write_kb":2637568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"46'42476","reported_seq":42832,"reported_epoch":46,"state":"active+clean","last_fresh":"2026-02-24T07:47:35.694502+0000","last_change":"2026-02-24T07:47:14.339634+0000","last_active":"2026-02-24T07:47:35.694502+0000","last_peered":"2026-02-24T07:47:35.694502+0000","last_clean":"2026-02-24T07:47:35.694502+0000","last_became_active":"2026-02-24T07:46:02.881035+0000","last_became_peered":"2026-02-24T07:46:02.881035+0000","last_unstale":"2026-02-24T07:47:35.694502+0000","last_undegraded":"2026-02-24T07:47:35.694502+0000","last_fullsized":"2026-02-24T07:47:35.694502+0000","mapping_epoch":42,"log_start":"46'42473","ondisk_log_start":"46'42473","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"45'33283","last_scrub_stamp":"2026-02-24T07:47:14.339597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_clean_scrub_stamp":"2026-02-24T07:47:14.339597+0000","objects_scrubbed":32630,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:48:22.078719+0000","scrub_duration":2951,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2666921984,"num_objects":40694,"num_object_clones":0,"num_object_copies":122082,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40694,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":42476,"num_write_kb":2661440,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"46'42284","reported_seq":42681,"reported_epoch":46,"state":"active+clean","last_fresh":"2026-02-24T07:47:35.707159+0000","last_change":"2026-02-24T07:47:24.807734+0000","last_active":"2026-02-24T07:47:35.707159+0000","last_peered":"2026-02-24T07:47:35.707159+0000","last_clean":"2026-02-24T07:47:35.707159+0000","last_became_active":"2026-02-24T07:46:02.879831+0000","last_became_peered":"2026-02-24T07:46:02.879831+0000","last_unstale":"2026-02-24T07:47:35.707159+0000","last_undegraded":"2026-02-24T07:47:35.707159+0000","last_fullsized":"2026-02-24T07:47:35.707159+0000","mapping_epoch":42,"log_start":"46'42281","ondisk_log_start":"46'42281","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"45'37558","last_scrub_stamp":"2026-02-24T07:47:24.807659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_clean_scrub_stamp":"2026-02-24T07:47:24.807659+0000","objects_scrubbed":36869,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:48:39.172465+0000","scrub_duration":3748,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2656698368,"num_objects":40538,"num_object_clones":0,"num_object_copies":121614,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40538,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":42284,"num_write_kb":2650304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.0","version":"46'42231","reported_seq":42560,"reported_epoch":46,"state":"active+clean","last_fresh":"2026-02-24T07:47:35.770215+0000","last_change":"2026-02-24T07:47:08.376360+0000","last_active":"2026-02-24T07:47:35.770215+0000","last_peered":"2026-02-24T07:47:35.770215+0000","last_clean":"2026-02-24T07:47:35.770215+0000","last_became_active":"2026-02-24T07:46:02.880333+0000","last_became_peered":"2026-02-24T07:46:02.880333+0000","last_unstale":"2026-02-24T07:47:35.770215+0000","last_undegraded":"2026-02-24T07:47:35.770215+0000","last_fullsized":"2026-02-24T07:47:35.770215+0000","mapping_epoch":42,"log_start":"46'42228","ondisk_log_start":"46'42228","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"45'30511","last_scrub_stamp":"2026-02-24T07:47:08.376304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_clean_scrub_stamp":"2026-02-24T07:47:08.376304+0000","objects_scrubbed":29912,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:48:24.552234+0000","scrub_duration":3049,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2653880320,"num_objects":40495,"num_object_clones":0,"num_object_copies":121485,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40495,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":42231,"num_write_kb":2647232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2","version":"46'42633","reported_seq":43044,"reported_epoch":46,"state":"active+clean","last_fresh":"2026-02-24T07:47:35.768247+0000","last_change":"2026-02-24T07:47:27.784936+0000","last_active":"2026-02-24T07:47:35.768247+0000","last_peered":"2026-02-24T07:47:35.768247+0000","last_clean":"2026-02-24T07:47:35.768247+0000","last_became_active":"2026-02-24T07:46:02.880185+0000","last_became_peered":"2026-02-24T07:46:02.880185+0000","last_unstale":"2026-02-24T07:47:35.768247+0000","last_undegraded":"2026-02-24T07:47:35.768247+0000","last_fullsized":"2026-02-24T07:47:35.768247+0000","mapping_epoch":42,"log_start":"46'42630","ondisk_log_start":"46'42630","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"45'39074","last_scrub_stamp":"2026-02-24T07:47:27.784888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_clean_scrub_stamp":"2026-02-24T07:47:27.784888+0000","objects_scrubbed":38200,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:48:47.063599+0000","scrub_duration":4266,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2670264320,"num_objects":40745,"num_object_clones":0,"num_object_copies":122235,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40745,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":42633,"num_write_kb":2668096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"29'99","reported_seq":239,"reported_epoch":46,"state":"active+clean","last_fresh":"2026-02-24T07:47:27.863858+0000","last_change":"2026-02-24T07:46:41.007499+0000","last_active":"2026-02-24T07:47:27.863858+0000","last_peered":"2026-02-24T07:47:27.863858+0000","last_clean":"2026-02-24T07:47:27.863858+0000","last_became_active":"2026-02-24T07:42:54.186719+0000","last_became_peered":"2026-02-24T07:42:54.186719+0000","last_unstale":"2026-02-24T07:47:27.863858+0000","last_undegraded":"2026-02-24T07:47:27.863858+0000","last_fullsized":"2026-02-24T07:47:27.863858+0000","mapping_epoch":27,"log_start":"29'96","ondisk_log_start":"29'96","created":20,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'99","last_scrub_stamp":"2026-02-24T07:46:41.007434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:42:43.640776+0000","last_clean_scrub_stamp":"2026-02-24T07:46:41.007434+0000","objects_scrubbed":2,"log_size":3,"log_dups_size":96,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:47:53.486511+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":240,"num_write":135,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":16,"stat_sum":{"num_bytes":42548002840,"num_objects":649232,"num_object_clones":0,"num_object_copies":1947696,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":649232,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":677784,"num_write_kb":42464449,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":127661387776,"data_stored":127661375560,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":49,"ondisk_log_size":49,"up":48,"acting":48,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":240,"num_write":135,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":3,"up_from":25,"seq":107374182459,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":28110232,"kb_used_data":26264388,"kb_used_omap":8,"kb_used_meta":1845751,"kb_avail":705892968,"statfs":{"total":751619276800,"available":722834399232,"internally_reserved":0,"allocated":26894733312,"data_stored":26894432759,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":8757,"internal_metadata":1890049483},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":19,"seq":81604378685,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":37674584,"kb_used_data":36629636,"kb_used_omap":8,"kb_used_meta":1044919,"kb_avail":696328616,"statfs":{"total":751619276800,"available":713040502784,"internally_reserved":0,"allocated":37508747264,"data_stored":37508446711,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":8633,"internal_metadata":1069997639},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[5],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542206,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":30480656,"kb_used_data":28552636,"kb_used_omap":9,"kb_used_meta":1927926,"kb_avail":703522544,"statfs":{"total":751619276800,"available":720407085056,"internally_reserved":0,"allocated":29237899264,"data_stored":29237606335,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9328,"internal_metadata":1974197136},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705729,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":34897752,"kb_used_data":34033604,"kb_used_omap":5,"kb_used_meta":864122,"kb_avail":699105448,"statfs":{"total":751619276800,"available":715883978752,"internally_reserved":0,"allocated":34850410496,"data_stored":34850109943,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6126,"internal_metadata":884860946},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":34569261056,"data_stored":34569256984,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":29181411328,"data_stored":29181411328,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":37300801536,"data_stored":37300797464,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":26609913856,"data_stored":26609909784,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-24T07:47:41.669 INFO:tasks.cephadm.ceph_manager.ceph:increase pool size by 10 2026-02-24T07:47:41.669 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph osd pool set .mgr pg_num 161 2026-02-24T07:47:41.809 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:47:41.928 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:47:41.926+0000 7f64502ee640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:47:41.929 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:47:41.927+0000 7f64502ee640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:47:42.064 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2026-02-24T07:47:42.081 INFO:teuthology.orchestra.run.trial035.stderr:Error EPERM: crimson_allow_pg_split is false; pg_num increase denied for crimson pool 2026-02-24T07:47:42.126 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:47:41 trial035 ceph-mon[39284]: from='client.14738 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2026-02-24T07:47:42.127 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:47:41 trial035 ceph-mon[39284]: pgmap v162: 17 pgs: 17 active+clean; 37 GiB data, 117 GiB used, 2.6 TiB / 2.7 TiB avail; 102 B/s rd, 121 MiB/s wr, 7.80k op/s 2026-02-24T07:47:42.127 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-24T07:47:42.215 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:47:42.342 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:47:42.339+0000 7fd551758640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:47:42.342 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:47:42.340+0000 7fd551758640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:47:42.357 INFO:teuthology.orchestra.run.trial035.stdout:osd.0: {} 2026-02-24T07:47:42.357 INFO:teuthology.orchestra.run.trial035.stderr:osd.0: osd_enable_op_tracker = '' 2026-02-24T07:47:42.371 INFO:teuthology.orchestra.run.trial035.stdout:osd.1: {} 2026-02-24T07:47:42.372 INFO:teuthology.orchestra.run.trial035.stderr:osd.1: osd_enable_op_tracker = '' 2026-02-24T07:47:42.386 INFO:teuthology.orchestra.run.trial035.stdout:osd.2: {} 2026-02-24T07:47:42.386 INFO:teuthology.orchestra.run.trial035.stderr:osd.2: osd_enable_op_tracker = '' 2026-02-24T07:47:42.398 INFO:teuthology.orchestra.run.trial035.stdout:osd.3: {} 2026-02-24T07:47:42.399 INFO:teuthology.orchestra.run.trial035.stderr:osd.3: osd_enable_op_tracker = '' 2026-02-24T07:47:43.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:47:42 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/2868416138' entity='client.admin' cmd={"prefix": "osd pool set", "pool": ".mgr", "var": "pg_num", "val": "161"} : dispatch 2026-02-24T07:47:43.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:47:42 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/4235354626' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2026-02-24T07:47:44.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:47:43 trial035 ceph-mon[39284]: pgmap v163: 17 pgs: 17 active+clean; 37 GiB data, 117 GiB used, 2.6 TiB / 2.7 TiB avail; 102 B/s rd, 121 MiB/s wr, 7.80k op/s 2026-02-24T07:47:44.445 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2026-02-24T07:47:44.582 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:47:44.708 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:47:44.705+0000 7fcc46d99640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:47:44.708 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:47:44.705+0000 7fcc46d99640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:47:44.723 INFO:teuthology.orchestra.run.trial035.stdout:osd.0: {} 2026-02-24T07:47:44.724 INFO:teuthology.orchestra.run.trial035.stderr:osd.0: osd_enable_op_tracker = '' 2026-02-24T07:47:44.741 INFO:teuthology.orchestra.run.trial035.stdout:osd.1: {} 2026-02-24T07:47:44.741 INFO:teuthology.orchestra.run.trial035.stderr:osd.1: osd_enable_op_tracker = '' 2026-02-24T07:47:44.754 INFO:teuthology.orchestra.run.trial035.stdout:osd.2: {} 2026-02-24T07:47:44.755 INFO:teuthology.orchestra.run.trial035.stderr:osd.2: osd_enable_op_tracker = '' 2026-02-24T07:47:44.769 INFO:teuthology.orchestra.run.trial035.stdout:osd.3: {} 2026-02-24T07:47:44.769 INFO:teuthology.orchestra.run.trial035.stderr:osd.3: osd_enable_op_tracker = '' 2026-02-24T07:47:45.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:47:44 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/2189106179' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2026-02-24T07:47:45.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:47:44 trial035 ceph-mon[39284]: from='mgr.14284 10.20.193.35:0/3681373222' entity='mgr.x' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2026-02-24T07:47:46.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:47:45 trial035 ceph-mon[39284]: osdmap e48: 4 total, 4 up, 4 in 2026-02-24T07:47:46.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:47:45 trial035 ceph-mon[39284]: pgmap v165: 17 pgs: 17 active+clean; 37 GiB data, 117 GiB used, 2.6 TiB / 2.7 TiB avail; 0 B/s wr, 4.52k op/s 2026-02-24T07:47:46.811 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2026-02-24T07:47:46.954 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:47:47.083 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:47:47.082+0000 7f8c934bb640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:47:47.084 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:47:47.082+0000 7f8c934bb640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:47:47.098 INFO:teuthology.orchestra.run.trial035.stdout:osd.0: {} 2026-02-24T07:47:47.099 INFO:teuthology.orchestra.run.trial035.stderr:osd.0: osd_enable_op_tracker = '' 2026-02-24T07:47:47.099 INFO:journalctl@ceph.mgr.x.trial035.stdout:Feb 24 07:47:46 trial035 ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-mgr-x[39621]: ::ffff:10.20.193.91 - - [24/Feb/2026:07:47:46] "GET /metrics HTTP/1.1" 200 4240 "" "Prometheus/3.6.0" 2026-02-24T07:47:47.113 INFO:teuthology.orchestra.run.trial035.stdout:osd.1: {} 2026-02-24T07:47:47.113 INFO:teuthology.orchestra.run.trial035.stderr:osd.1: osd_enable_op_tracker = '' 2026-02-24T07:47:47.126 INFO:teuthology.orchestra.run.trial035.stdout:osd.2: {} 2026-02-24T07:47:47.127 INFO:teuthology.orchestra.run.trial035.stderr:osd.2: osd_enable_op_tracker = '' 2026-02-24T07:47:47.129 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2026-02-24T07:47:47.129 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2026-02-24T07:47:47.129 INFO:tasks.thrashosds.thrasher:Growing pool unique_pool_1 2026-02-24T07:47:47.130 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph pg dump --format=json 2026-02-24T07:47:47.140 INFO:teuthology.orchestra.run.trial035.stdout:osd.3: {} 2026-02-24T07:47:47.140 INFO:teuthology.orchestra.run.trial035.stderr:osd.3: osd_enable_op_tracker = '' 2026-02-24T07:47:47.273 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:47:47.356 INFO:journalctl@ceph.prometheus.a.trial091.stdout:Feb 24 07:47:46 trial091 ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-prometheus-a[66007]: time=2026-02-24T07:47:46.973Z level=ERROR source=scrape.go:1630 msg="Failed to determine correct type of scrape target." component="scrape manager" scrape_pool=ceph target=http://trial035:9283/metrics content_type="text/html;charset=utf-8" fallback_media_type="" err="received unsupported Content-Type \"text/html;charset=utf-8\" and no fallback_scrape_protocol specified for target" 2026-02-24T07:47:47.390 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:47:47 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/3232611539' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2026-02-24T07:47:47.396 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:47:47.394+0000 7f6fe12ec640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:47:47.396 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:47:47.395+0000 7f6fe12ec640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:47:47.540 INFO:teuthology.orchestra.run.trial035.stdout: 2026-02-24T07:47:47.541 INFO:teuthology.orchestra.run.trial035.stderr:dumped all 2026-02-24T07:47:47.583 INFO:teuthology.orchestra.run.trial035.stdout:{"pg_ready":true,"pg_map":{"version":165,"stamp":"2026-02-24T07:47:45.696096+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":39640891960,"num_objects":604866,"num_object_clones":0,"num_object_copies":1814598,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":604866,"num_whiteouts":0,"num_read":137,"num_read_kb":241,"num_write":722287,"num_write_kb":42466599,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":51,"ondisk_log_size":51,"up":51,"acting":51,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":51,"num_osds":4,"num_per_pool_osds":4,"num_per_pool_omap_osds":4,"kb":2936012800,"kb_used":123059016,"kb_used_data":117141368,"kb_used_omap":34,"kb_used_meta":5917341,"kb_avail":2812953784,"statfs":{"total":3006477107200,"available":2880464674816,"internally_reserved":0,"allocated":119952760832,"data_stored":119951534896,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":35452,"internal_metadata":6059357572},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,1],"upper_bound":4},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-2907701248,"num_objects":-44368,"num_object_clones":0,"num_object_copies":-133104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-44368,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44368,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.821729"},"pg_stats":[{"pgid":"3.f","version":"47'44926","reported_seq":45292,"reported_epoch":47,"state":"active+clean","last_fresh":"2026-02-24T07:47:40.696655+0000","last_change":"2026-02-24T07:47:16.466592+0000","last_active":"2026-02-24T07:47:40.696655+0000","last_peered":"2026-02-24T07:47:40.696655+0000","last_clean":"2026-02-24T07:47:40.696655+0000","last_became_active":"2026-02-24T07:46:02.881625+0000","last_became_peered":"2026-02-24T07:46:02.881625+0000","last_unstale":"2026-02-24T07:47:40.696655+0000","last_undegraded":"2026-02-24T07:47:40.696655+0000","last_fullsized":"2026-02-24T07:47:40.696655+0000","mapping_epoch":42,"log_start":"47'44923","ondisk_log_start":"47'44923","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"45'34153","last_scrub_stamp":"2026-02-24T07:47:16.466530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_clean_scrub_stamp":"2026-02-24T07:47:16.466530+0000","objects_scrubbed":33463,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:48:23.487779+0000","scrub_duration":3074,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2469658624,"num_objects":37684,"num_object_clones":0,"num_object_copies":113052,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37684,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44926,"num_write_kb":2643520,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.e","version":"47'45634","reported_seq":46072,"reported_epoch":47,"state":"active+clean","last_fresh":"2026-02-24T07:47:40.692009+0000","last_change":"2026-02-24T07:47:33.099762+0000","last_active":"2026-02-24T07:47:40.692009+0000","last_peered":"2026-02-24T07:47:40.692009+0000","last_clean":"2026-02-24T07:47:40.692009+0000","last_became_active":"2026-02-24T07:46:02.881083+0000","last_became_peered":"2026-02-24T07:46:02.881083+0000","last_unstale":"2026-02-24T07:47:40.692009+0000","last_undegraded":"2026-02-24T07:47:40.692009+0000","last_fullsized":"2026-02-24T07:47:40.692009+0000","mapping_epoch":42,"log_start":"47'45631","ondisk_log_start":"47'45631","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"46'41533","last_scrub_stamp":"2026-02-24T07:47:33.099675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_clean_scrub_stamp":"2026-02-24T07:47:33.099675+0000","objects_scrubbed":40754,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:48:53.832432+0000","scrub_duration":3969,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2500984832,"num_objects":38162,"num_object_clones":0,"num_object_copies":114486,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38162,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45634,"num_write_kb":2681472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"47'45149","reported_seq":45477,"reported_epoch":47,"state":"active+clean","last_fresh":"2026-02-24T07:47:40.720245+0000","last_change":"2026-02-24T07:47:06.968813+0000","last_active":"2026-02-24T07:47:40.720245+0000","last_peered":"2026-02-24T07:47:40.720245+0000","last_clean":"2026-02-24T07:47:40.720245+0000","last_became_active":"2026-02-24T07:46:02.883799+0000","last_became_peered":"2026-02-24T07:46:02.883799+0000","last_unstale":"2026-02-24T07:47:40.720245+0000","last_undegraded":"2026-02-24T07:47:40.720245+0000","last_fullsized":"2026-02-24T07:47:40.720245+0000","mapping_epoch":42,"log_start":"47'45146","ondisk_log_start":"47'45146","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"45'30090","last_scrub_stamp":"2026-02-24T07:47:06.968732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_clean_scrub_stamp":"2026-02-24T07:47:06.968732+0000","objects_scrubbed":29542,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:48:25.198085+0000","scrub_duration":3070,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2485911576,"num_objects":37933,"num_object_clones":0,"num_object_copies":113799,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37933,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":45149,"num_write_kb":2658561,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"47'44379","reported_seq":44705,"reported_epoch":47,"state":"active+clean","last_fresh":"2026-02-24T07:47:40.715789+0000","last_change":"2026-02-24T07:47:07.546834+0000","last_active":"2026-02-24T07:47:40.715789+0000","last_peered":"2026-02-24T07:47:40.715789+0000","last_clean":"2026-02-24T07:47:40.715789+0000","last_became_active":"2026-02-24T07:46:02.880060+0000","last_became_peered":"2026-02-24T07:46:02.880060+0000","last_unstale":"2026-02-24T07:47:40.715789+0000","last_undegraded":"2026-02-24T07:47:40.715789+0000","last_fullsized":"2026-02-24T07:47:40.715789+0000","mapping_epoch":42,"log_start":"47'44376","ondisk_log_start":"47'44376","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"45'29910","last_scrub_stamp":"2026-02-24T07:47:07.546803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_clean_scrub_stamp":"2026-02-24T07:47:07.546803+0000","objects_scrubbed":29435,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:48:27.524515+0000","scrub_duration":2696,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2442067968,"num_objects":37263,"num_object_clones":0,"num_object_copies":111789,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37263,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44379,"num_write_kb":2612544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"47'45101","reported_seq":45439,"reported_epoch":47,"state":"active+clean","last_fresh":"2026-02-24T07:47:40.771995+0000","last_change":"2026-02-24T07:47:10.376412+0000","last_active":"2026-02-24T07:47:40.771995+0000","last_peered":"2026-02-24T07:47:40.771995+0000","last_clean":"2026-02-24T07:47:40.771995+0000","last_became_active":"2026-02-24T07:46:02.880171+0000","last_became_peered":"2026-02-24T07:46:02.880171+0000","last_unstale":"2026-02-24T07:47:40.771995+0000","last_undegraded":"2026-02-24T07:47:40.771995+0000","last_fullsized":"2026-02-24T07:47:40.771995+0000","mapping_epoch":42,"log_start":"47'45098","ondisk_log_start":"47'45098","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"45'31519","last_scrub_stamp":"2026-02-24T07:47:10.376345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_clean_scrub_stamp":"2026-02-24T07:47:10.376345+0000","objects_scrubbed":30810,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:48:13.377288+0000","scrub_duration":4056,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2479292416,"num_objects":37831,"num_object_clones":0,"num_object_copies":113493,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37831,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45101,"num_write_kb":2653824,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.a","version":"47'44805","reported_seq":45243,"reported_epoch":47,"state":"active+clean","last_fresh":"2026-02-24T07:47:40.720025+0000","last_change":"2026-02-24T07:47:35.439589+0000","last_active":"2026-02-24T07:47:40.720025+0000","last_peered":"2026-02-24T07:47:40.720025+0000","last_clean":"2026-02-24T07:47:40.720025+0000","last_became_active":"2026-02-24T07:46:02.882274+0000","last_became_peered":"2026-02-24T07:46:02.882274+0000","last_unstale":"2026-02-24T07:47:40.720025+0000","last_undegraded":"2026-02-24T07:47:40.720025+0000","last_fullsized":"2026-02-24T07:47:40.720025+0000","mapping_epoch":42,"log_start":"47'44802","ondisk_log_start":"47'44802","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"46'41859","last_scrub_stamp":"2026-02-24T07:47:35.439542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_clean_scrub_stamp":"2026-02-24T07:47:35.439542+0000","objects_scrubbed":40699,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:48:38.034606+0000","scrub_duration":3645,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2449801216,"num_objects":37381,"num_object_clones":0,"num_object_copies":112143,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37381,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44805,"num_write_kb":2629952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"47'45616","reported_seq":45961,"reported_epoch":47,"state":"active+clean","last_fresh":"2026-02-24T07:47:40.706391+0000","last_change":"2026-02-24T07:47:10.197277+0000","last_active":"2026-02-24T07:47:40.706391+0000","last_peered":"2026-02-24T07:47:40.706391+0000","last_clean":"2026-02-24T07:47:40.706391+0000","last_became_active":"2026-02-24T07:46:02.881951+0000","last_became_peered":"2026-02-24T07:46:02.881951+0000","last_unstale":"2026-02-24T07:47:40.706391+0000","last_undegraded":"2026-02-24T07:47:40.706391+0000","last_fullsized":"2026-02-24T07:47:40.706391+0000","mapping_epoch":42,"log_start":"47'45613","ondisk_log_start":"47'45613","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"45'31847","last_scrub_stamp":"2026-02-24T07:47:10.197239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_clean_scrub_stamp":"2026-02-24T07:47:10.197239+0000","objects_scrubbed":31279,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:48:27.205650+0000","scrub_duration":3289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2506620928,"num_objects":38248,"num_object_clones":0,"num_object_copies":114744,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38248,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45616,"num_write_kb":2683648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"47'45267","reported_seq":45658,"reported_epoch":47,"state":"active+clean","last_fresh":"2026-02-24T07:47:40.770112+0000","last_change":"2026-02-24T07:47:22.289650+0000","last_active":"2026-02-24T07:47:40.770112+0000","last_peered":"2026-02-24T07:47:40.770112+0000","last_clean":"2026-02-24T07:47:40.770112+0000","last_became_active":"2026-02-24T07:46:02.881345+0000","last_became_peered":"2026-02-24T07:46:02.881345+0000","last_unstale":"2026-02-24T07:47:40.770112+0000","last_undegraded":"2026-02-24T07:47:40.770112+0000","last_fullsized":"2026-02-24T07:47:40.770112+0000","mapping_epoch":42,"log_start":"47'45264","ondisk_log_start":"47'45264","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"45'36703","last_scrub_stamp":"2026-02-24T07:47:22.289617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_clean_scrub_stamp":"2026-02-24T07:47:22.289617+0000","objects_scrubbed":35960,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:48:31.332102+0000","scrub_duration":3891,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2487418880,"num_objects":37955,"num_object_clones":0,"num_object_copies":113865,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37955,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45267,"num_write_kb":2663104,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"47'44864","reported_seq":45304,"reported_epoch":47,"state":"active+clean","last_fresh":"2026-02-24T07:47:40.770793+0000","last_change":"2026-02-24T07:47:35.925072+0000","last_active":"2026-02-24T07:47:40.770793+0000","last_peered":"2026-02-24T07:47:40.770793+0000","last_clean":"2026-02-24T07:47:40.770793+0000","last_became_active":"2026-02-24T07:46:02.880323+0000","last_became_peered":"2026-02-24T07:46:02.880323+0000","last_unstale":"2026-02-24T07:47:40.770793+0000","last_undegraded":"2026-02-24T07:47:40.770793+0000","last_fullsized":"2026-02-24T07:47:40.770793+0000","mapping_epoch":42,"log_start":"47'44861","ondisk_log_start":"47'44861","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"47'42248","last_scrub_stamp":"2026-02-24T07:47:35.925000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_clean_scrub_stamp":"2026-02-24T07:47:35.925000+0000","objects_scrubbed":40858,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:48:43.057825+0000","scrub_duration":3382,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2464940032,"num_objects":37612,"num_object_clones":0,"num_object_copies":112836,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37612,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44864,"num_write_kb":2639232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.6","version":"47'45264","reported_seq":45660,"reported_epoch":47,"state":"active+clean","last_fresh":"2026-02-24T07:47:40.709533+0000","last_change":"2026-02-24T07:47:23.432795+0000","last_active":"2026-02-24T07:47:40.709533+0000","last_peered":"2026-02-24T07:47:40.709533+0000","last_clean":"2026-02-24T07:47:40.709533+0000","last_became_active":"2026-02-24T07:46:02.880005+0000","last_became_peered":"2026-02-24T07:46:02.880005+0000","last_unstale":"2026-02-24T07:47:40.709533+0000","last_undegraded":"2026-02-24T07:47:40.709533+0000","last_fullsized":"2026-02-24T07:47:40.709533+0000","mapping_epoch":42,"log_start":"47'45261","ondisk_log_start":"47'45261","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"45'37219","last_scrub_stamp":"2026-02-24T07:47:23.432744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_clean_scrub_stamp":"2026-02-24T07:47:23.432744+0000","objects_scrubbed":36510,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:48:51.822832+0000","scrub_duration":3412,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2475163648,"num_objects":37768,"num_object_clones":0,"num_object_copies":113304,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37768,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45264,"num_write_kb":2657024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"47'45447","reported_seq":45820,"reported_epoch":47,"state":"active+clean","last_fresh":"2026-02-24T07:47:40.695346+0000","last_change":"2026-02-24T07:47:17.546625+0000","last_active":"2026-02-24T07:47:40.695346+0000","last_peered":"2026-02-24T07:47:40.695346+0000","last_clean":"2026-02-24T07:47:40.695346+0000","last_became_active":"2026-02-24T07:46:02.881420+0000","last_became_peered":"2026-02-24T07:46:02.881420+0000","last_unstale":"2026-02-24T07:47:40.695346+0000","last_undegraded":"2026-02-24T07:47:40.695346+0000","last_fullsized":"2026-02-24T07:47:40.695346+0000","mapping_epoch":42,"log_start":"47'45444","ondisk_log_start":"47'45444","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"45'34854","last_scrub_stamp":"2026-02-24T07:47:17.546590+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_clean_scrub_stamp":"2026-02-24T07:47:17.546590+0000","objects_scrubbed":34222,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:48:38.813192+0000","scrub_duration":3116,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2503933952,"num_objects":38207,"num_object_clones":0,"num_object_copies":114621,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38207,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45447,"num_write_kb":2676928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.4","version":"47'44865","reported_seq":45300,"reported_epoch":47,"state":"active+clean","last_fresh":"2026-02-24T07:47:40.765125+0000","last_change":"2026-02-24T07:47:33.628245+0000","last_active":"2026-02-24T07:47:40.765125+0000","last_peered":"2026-02-24T07:47:40.765125+0000","last_clean":"2026-02-24T07:47:40.765125+0000","last_became_active":"2026-02-24T07:46:02.880345+0000","last_became_peered":"2026-02-24T07:46:02.880345+0000","last_unstale":"2026-02-24T07:47:40.765125+0000","last_undegraded":"2026-02-24T07:47:40.765125+0000","last_fullsized":"2026-02-24T07:47:40.765125+0000","mapping_epoch":42,"log_start":"47'44862","ondisk_log_start":"47'44862","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"46'41076","last_scrub_stamp":"2026-02-24T07:47:33.628218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_clean_scrub_stamp":"2026-02-24T07:47:33.628218+0000","objects_scrubbed":40364,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:48:34.035175+0000","scrub_duration":4080,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2461466624,"num_objects":37559,"num_object_clones":0,"num_object_copies":112677,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37559,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44865,"num_write_kb":2637568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"47'45233","reported_seq":45591,"reported_epoch":47,"state":"active+clean","last_fresh":"2026-02-24T07:47:40.697449+0000","last_change":"2026-02-24T07:47:14.339634+0000","last_active":"2026-02-24T07:47:40.697449+0000","last_peered":"2026-02-24T07:47:40.697449+0000","last_clean":"2026-02-24T07:47:40.697449+0000","last_became_active":"2026-02-24T07:46:02.881035+0000","last_became_peered":"2026-02-24T07:46:02.881035+0000","last_unstale":"2026-02-24T07:47:40.697449+0000","last_undegraded":"2026-02-24T07:47:40.697449+0000","last_fullsized":"2026-02-24T07:47:40.697449+0000","mapping_epoch":42,"log_start":"47'45230","ondisk_log_start":"47'45230","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"45'33283","last_scrub_stamp":"2026-02-24T07:47:14.339597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_clean_scrub_stamp":"2026-02-24T07:47:14.339597+0000","objects_scrubbed":32630,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:48:22.078719+0000","scrub_duration":2951,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2486239232,"num_objects":37937,"num_object_clones":0,"num_object_copies":113811,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37937,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45233,"num_write_kb":2661440,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"47'45093","reported_seq":45492,"reported_epoch":47,"state":"active+clean","last_fresh":"2026-02-24T07:47:40.707387+0000","last_change":"2026-02-24T07:47:24.807734+0000","last_active":"2026-02-24T07:47:40.707387+0000","last_peered":"2026-02-24T07:47:40.707387+0000","last_clean":"2026-02-24T07:47:40.707387+0000","last_became_active":"2026-02-24T07:46:02.879831+0000","last_became_peered":"2026-02-24T07:46:02.879831+0000","last_unstale":"2026-02-24T07:47:40.707387+0000","last_undegraded":"2026-02-24T07:47:40.707387+0000","last_fullsized":"2026-02-24T07:47:40.707387+0000","mapping_epoch":42,"log_start":"47'45090","ondisk_log_start":"47'45090","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"45'37558","last_scrub_stamp":"2026-02-24T07:47:24.807659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_clean_scrub_stamp":"2026-02-24T07:47:24.807659+0000","objects_scrubbed":36869,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:48:39.172465+0000","scrub_duration":3748,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2472607744,"num_objects":37729,"num_object_clones":0,"num_object_copies":113187,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37729,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45093,"num_write_kb":2650304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.0","version":"47'45036","reported_seq":45367,"reported_epoch":47,"state":"active+clean","last_fresh":"2026-02-24T07:47:40.770210+0000","last_change":"2026-02-24T07:47:08.376360+0000","last_active":"2026-02-24T07:47:40.770210+0000","last_peered":"2026-02-24T07:47:40.770210+0000","last_clean":"2026-02-24T07:47:40.770210+0000","last_became_active":"2026-02-24T07:46:02.880333+0000","last_became_peered":"2026-02-24T07:46:02.880333+0000","last_unstale":"2026-02-24T07:47:40.770210+0000","last_undegraded":"2026-02-24T07:47:40.770210+0000","last_fullsized":"2026-02-24T07:47:40.770210+0000","mapping_epoch":42,"log_start":"47'45033","ondisk_log_start":"47'45033","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"45'30511","last_scrub_stamp":"2026-02-24T07:47:08.376304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_clean_scrub_stamp":"2026-02-24T07:47:08.376304+0000","objects_scrubbed":29912,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:48:24.552234+0000","scrub_duration":3049,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2470051840,"num_objects":37690,"num_object_clones":0,"num_object_copies":113070,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37690,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45036,"num_write_kb":2647232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2","version":"47'45473","reported_seq":45886,"reported_epoch":47,"state":"active+clean","last_fresh":"2026-02-24T07:47:40.766731+0000","last_change":"2026-02-24T07:47:27.784936+0000","last_active":"2026-02-24T07:47:40.766731+0000","last_peered":"2026-02-24T07:47:40.766731+0000","last_clean":"2026-02-24T07:47:40.766731+0000","last_became_active":"2026-02-24T07:46:02.880185+0000","last_became_peered":"2026-02-24T07:46:02.880185+0000","last_unstale":"2026-02-24T07:47:40.766731+0000","last_undegraded":"2026-02-24T07:47:40.766731+0000","last_fullsized":"2026-02-24T07:47:40.766731+0000","mapping_epoch":42,"log_start":"47'45470","ondisk_log_start":"47'45470","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"45'39074","last_scrub_stamp":"2026-02-24T07:47:27.784888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_clean_scrub_stamp":"2026-02-24T07:47:27.784888+0000","objects_scrubbed":38200,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:48:47.063599+0000","scrub_duration":4266,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2484142080,"num_objects":37905,"num_object_clones":0,"num_object_copies":113715,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37905,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45473,"num_write_kb":2668096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"29'99","reported_seq":241,"reported_epoch":47,"state":"active+clean","last_fresh":"2026-02-24T07:47:35.875622+0000","last_change":"2026-02-24T07:46:41.007499+0000","last_active":"2026-02-24T07:47:35.875622+0000","last_peered":"2026-02-24T07:47:35.875622+0000","last_clean":"2026-02-24T07:47:35.875622+0000","last_became_active":"2026-02-24T07:42:54.186719+0000","last_became_peered":"2026-02-24T07:42:54.186719+0000","last_unstale":"2026-02-24T07:47:35.875622+0000","last_undegraded":"2026-02-24T07:47:35.875622+0000","last_fullsized":"2026-02-24T07:47:35.875622+0000","mapping_epoch":27,"log_start":"29'96","ondisk_log_start":"29'96","created":20,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'99","last_scrub_stamp":"2026-02-24T07:46:41.007434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:42:43.640776+0000","last_clean_scrub_stamp":"2026-02-24T07:46:41.007434+0000","objects_scrubbed":2,"log_size":3,"log_dups_size":96,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:47:53.486511+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":240,"num_write":135,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":16,"stat_sum":{"num_bytes":39640301592,"num_objects":604864,"num_object_clones":0,"num_object_copies":1814592,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":604864,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":722152,"num_write_kb":42464449,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":118938480640,"data_stored":118938468424,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":48,"ondisk_log_size":48,"up":48,"acting":48,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":240,"num_write":135,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":3,"up_from":25,"seq":107374182460,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":26425700,"kb_used_data":24525712,"kb_used_omap":9,"kb_used_meta":1899894,"kb_avail":707577500,"statfs":{"total":751619276800,"available":724559360000,"internally_reserved":0,"allocated":25114329088,"data_stored":25114020698,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9409,"internal_metadata":1945492287},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":19,"seq":81604378686,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":35279844,"kb_used_data":34162704,"kb_used_omap":9,"kb_used_meta":1117110,"kb_avail":698723356,"statfs":{"total":751619276800,"available":715492716544,"internally_reserved":0,"allocated":34982608896,"data_stored":34982300506,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9285,"internal_metadata":1143921595},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542207,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":28583260,"kb_used_data":26601032,"kb_used_omap":9,"kb_used_meta":1982070,"kb_avail":705419940,"statfs":{"total":751619276800,"available":722350018560,"internally_reserved":0,"allocated":27239456768,"data_stored":27239156002,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9980,"internal_metadata":2029639940},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,1],"upper_bound":4},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705730,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":32770212,"kb_used_data":31851920,"kb_used_omap":6,"kb_used_meta":918265,"kb_avail":701232988,"statfs":{"total":751619276800,"available":718062579712,"internally_reserved":0,"allocated":32616366080,"data_stored":32616057690,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6778,"internal_metadata":940303750},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":32217894912,"data_stored":32217890840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":27174633472,"data_stored":27174633472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":34751516672,"data_stored":34751512600,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":24794435584,"data_stored":24794431512,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-24T07:47:47.585 INFO:tasks.cephadm.ceph_manager.ceph:increase pool size by 10 2026-02-24T07:47:47.585 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph osd pool set unique_pool_1 pg_num 46 2026-02-24T07:47:47.719 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:47:47.840 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:47:47.838+0000 7ff3fbb17640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:47:47.840 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:47:47.838+0000 7ff3fbb17640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:47:47.978 INFO:teuthology.orchestra.run.trial035.stderr:Error EPERM: crimson_allow_pg_split is false; pg_num increase denied for crimson pool 2026-02-24T07:47:48.016 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-24T07:47:48.454 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:47:48 trial035 ceph-mon[39284]: from='client.14748 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2026-02-24T07:47:48.454 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:47:48 trial035 ceph-mon[39284]: pgmap v166: 17 pgs: 17 active+clean; 34 GiB data, 109 GiB used, 2.6 TiB / 2.7 TiB avail; 0 B/s wr, 9.19k op/s 2026-02-24T07:47:48.454 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:47:48 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/957681596' entity='client.admin' cmd={"prefix": "osd pool set", "pool": "unique_pool_1", "var": "pg_num", "val": "46"} : dispatch 2026-02-24T07:47:49.185 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2026-02-24T07:47:49.322 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:47:49.443 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:47:49.442+0000 7f7d38499640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:47:49.444 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:47:49.442+0000 7f7d38499640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:47:49.459 INFO:teuthology.orchestra.run.trial035.stdout:osd.0: {} 2026-02-24T07:47:49.459 INFO:teuthology.orchestra.run.trial035.stderr:osd.0: osd_enable_op_tracker = '' 2026-02-24T07:47:49.478 INFO:teuthology.orchestra.run.trial035.stdout:osd.1: {} 2026-02-24T07:47:49.478 INFO:teuthology.orchestra.run.trial035.stderr:osd.1: osd_enable_op_tracker = '' 2026-02-24T07:47:49.490 INFO:teuthology.orchestra.run.trial035.stdout:osd.2: {} 2026-02-24T07:47:49.490 INFO:teuthology.orchestra.run.trial035.stderr:osd.2: osd_enable_op_tracker = '' 2026-02-24T07:47:49.502 INFO:teuthology.orchestra.run.trial035.stdout:osd.3: {} 2026-02-24T07:47:49.502 INFO:teuthology.orchestra.run.trial035.stderr:osd.3: osd_enable_op_tracker = '' 2026-02-24T07:47:49.954 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:47:49 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/764639352' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2026-02-24T07:47:50.954 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:47:50 trial035 ceph-mon[39284]: pgmap v167: 17 pgs: 17 active+clean; 34 GiB data, 109 GiB used, 2.6 TiB / 2.7 TiB avail; 0 B/s wr, 9.19k op/s 2026-02-24T07:47:51.543 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2026-02-24T07:47:51.693 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:47:52.183 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:47:52.132+0000 7fab46c2a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:47:52.183 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:47:52.132+0000 7fab46c2a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:47:52.183 INFO:teuthology.orchestra.run.trial035.stderr:osd.0: osd_enable_op_tracker = '' 2026-02-24T07:47:52.183 INFO:teuthology.orchestra.run.trial035.stderr:osd.1: osd_enable_op_tracker = '' 2026-02-24T07:47:52.183 INFO:teuthology.orchestra.run.trial035.stdout:osd.0: {} 2026-02-24T07:47:52.183 INFO:teuthology.orchestra.run.trial035.stdout:osd.1: {} 2026-02-24T07:47:52.185 INFO:teuthology.orchestra.run.trial035.stdout:osd.2: {} 2026-02-24T07:47:52.185 INFO:teuthology.orchestra.run.trial035.stderr:osd.2: osd_enable_op_tracker = '' 2026-02-24T07:47:52.199 INFO:teuthology.orchestra.run.trial035.stdout:osd.3: {} 2026-02-24T07:47:52.199 INFO:teuthology.orchestra.run.trial035.stderr:osd.3: osd_enable_op_tracker = '' 2026-02-24T07:47:53.018 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2026-02-24T07:47:53.018 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2026-02-24T07:47:53.018 INFO:tasks.thrashosds.thrasher:Growing pool .mgr 2026-02-24T07:47:53.019 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph pg dump --format=json 2026-02-24T07:47:53.043 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:47:52 trial035 ceph-mon[39284]: pgmap v168: 17 pgs: 17 active+clean; 31 GiB data, 99 GiB used, 2.6 TiB / 2.7 TiB avail; 0 B/s wr, 9.33k op/s 2026-02-24T07:47:53.044 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:47:52 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/1147723989' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2026-02-24T07:47:53.161 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:47:53.284 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:47:53.282+0000 7f84369df640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:47:53.284 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:47:53.283+0000 7f84369df640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:47:53.431 INFO:teuthology.orchestra.run.trial035.stdout: 2026-02-24T07:47:53.432 INFO:teuthology.orchestra.run.trial035.stderr:dumped all 2026-02-24T07:47:53.474 INFO:teuthology.orchestra.run.trial035.stdout:{"pg_ready":true,"pg_map":{"version":169,"stamp":"2026-02-24T07:47:52.751119+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":33524810296,"num_objects":511542,"num_object_clones":0,"num_object_copies":1534626,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":511542,"num_whiteouts":0,"num_read":137,"num_read_kb":241,"num_write":815611,"num_write_kb":42466599,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":53,"ondisk_log_size":53,"up":51,"acting":51,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":51,"num_osds":4,"num_per_pool_osds":4,"num_per_pool_omap_osds":4,"kb":2936012800,"kb_used":104165944,"kb_used_data":99261224,"kb_used_omap":37,"kb_used_meta":4904474,"kb_avail":2831846856,"statfs":{"total":3006477107200,"available":2899811180544,"internally_reserved":0,"allocated":101643493376,"data_stored":101642236092,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":38060,"internal_metadata":5022182228},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1],"upper_bound":4},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-6116081664,"num_objects":-93324,"num_object_clones":0,"num_object_copies":-279972,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-93324,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":93324,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.055273"},"pg_stats":[{"pgid":"3.f","version":"48'50797","reported_seq":51165,"reported_epoch":48,"state":"active+clean","last_fresh":"2026-02-24T07:47:50.697978+0000","last_change":"2026-02-24T07:47:16.466592+0000","last_active":"2026-02-24T07:47:50.697978+0000","last_peered":"2026-02-24T07:47:50.697978+0000","last_clean":"2026-02-24T07:47:50.697978+0000","last_became_active":"2026-02-24T07:46:02.881625+0000","last_became_peered":"2026-02-24T07:46:02.881625+0000","last_unstale":"2026-02-24T07:47:50.697978+0000","last_undegraded":"2026-02-24T07:47:50.697978+0000","last_fullsized":"2026-02-24T07:47:50.697978+0000","mapping_epoch":42,"log_start":"48'50794","ondisk_log_start":"48'50794","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"45'34153","last_scrub_stamp":"2026-02-24T07:47:16.466530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_clean_scrub_stamp":"2026-02-24T07:47:16.466530+0000","objects_scrubbed":33463,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:48:23.487779+0000","scrub_duration":3074,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2084896768,"num_objects":31813,"num_object_clones":0,"num_object_copies":95439,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31813,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":50797,"num_write_kb":2643520,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.e","version":"48'51558","reported_seq":51998,"reported_epoch":48,"state":"active+clean","last_fresh":"2026-02-24T07:47:50.694741+0000","last_change":"2026-02-24T07:47:33.099762+0000","last_active":"2026-02-24T07:47:50.694741+0000","last_peered":"2026-02-24T07:47:50.694741+0000","last_clean":"2026-02-24T07:47:50.694741+0000","last_became_active":"2026-02-24T07:46:02.881083+0000","last_became_peered":"2026-02-24T07:46:02.881083+0000","last_unstale":"2026-02-24T07:47:50.694741+0000","last_undegraded":"2026-02-24T07:47:50.694741+0000","last_fullsized":"2026-02-24T07:47:50.694741+0000","mapping_epoch":42,"log_start":"48'51555","ondisk_log_start":"48'51555","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"46'41533","last_scrub_stamp":"2026-02-24T07:47:33.099675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_clean_scrub_stamp":"2026-02-24T07:47:33.099675+0000","objects_scrubbed":40754,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:48:53.832432+0000","scrub_duration":3969,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2112749568,"num_objects":32238,"num_object_clones":0,"num_object_copies":96714,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32238,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":51558,"num_write_kb":2681472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"48'50997","reported_seq":51327,"reported_epoch":48,"state":"active+clean","last_fresh":"2026-02-24T07:47:50.720300+0000","last_change":"2026-02-24T07:47:06.968813+0000","last_active":"2026-02-24T07:47:50.720300+0000","last_peered":"2026-02-24T07:47:50.720300+0000","last_clean":"2026-02-24T07:47:50.720300+0000","last_became_active":"2026-02-24T07:46:02.883799+0000","last_became_peered":"2026-02-24T07:46:02.883799+0000","last_unstale":"2026-02-24T07:47:50.720300+0000","last_undegraded":"2026-02-24T07:47:50.720300+0000","last_fullsized":"2026-02-24T07:47:50.720300+0000","mapping_epoch":42,"log_start":"48'50994","ondisk_log_start":"48'50994","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"45'30090","last_scrub_stamp":"2026-02-24T07:47:06.968732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_clean_scrub_stamp":"2026-02-24T07:47:06.968732+0000","objects_scrubbed":29542,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:48:25.198085+0000","scrub_duration":3070,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2102657048,"num_objects":32085,"num_object_clones":0,"num_object_copies":96255,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32085,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":50997,"num_write_kb":2658561,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"48'50109","reported_seq":50437,"reported_epoch":48,"state":"active+clean","last_fresh":"2026-02-24T07:47:50.718547+0000","last_change":"2026-02-24T07:47:07.546834+0000","last_active":"2026-02-24T07:47:50.718547+0000","last_peered":"2026-02-24T07:47:50.718547+0000","last_clean":"2026-02-24T07:47:50.718547+0000","last_became_active":"2026-02-24T07:46:02.880060+0000","last_became_peered":"2026-02-24T07:46:02.880060+0000","last_unstale":"2026-02-24T07:47:50.718547+0000","last_undegraded":"2026-02-24T07:47:50.718547+0000","last_fullsized":"2026-02-24T07:47:50.718547+0000","mapping_epoch":42,"log_start":"48'50106","ondisk_log_start":"48'50106","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"45'29910","last_scrub_stamp":"2026-02-24T07:47:07.546803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_clean_scrub_stamp":"2026-02-24T07:47:07.546803+0000","objects_scrubbed":29435,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:48:27.524515+0000","scrub_duration":2696,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2066546688,"num_objects":31533,"num_object_clones":0,"num_object_copies":94599,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31533,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":50109,"num_write_kb":2612544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"48'50994","reported_seq":51334,"reported_epoch":48,"state":"active+clean","last_fresh":"2026-02-24T07:47:50.772309+0000","last_change":"2026-02-24T07:47:10.376412+0000","last_active":"2026-02-24T07:47:50.772309+0000","last_peered":"2026-02-24T07:47:50.772309+0000","last_clean":"2026-02-24T07:47:50.772309+0000","last_became_active":"2026-02-24T07:46:02.880171+0000","last_became_peered":"2026-02-24T07:46:02.880171+0000","last_unstale":"2026-02-24T07:47:50.772309+0000","last_undegraded":"2026-02-24T07:47:50.772309+0000","last_fullsized":"2026-02-24T07:47:50.772309+0000","mapping_epoch":42,"log_start":"48'50991","ondisk_log_start":"48'50991","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"45'31519","last_scrub_stamp":"2026-02-24T07:47:10.376345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_clean_scrub_stamp":"2026-02-24T07:47:10.376345+0000","objects_scrubbed":30810,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:48:13.377288+0000","scrub_duration":4056,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2093088768,"num_objects":31938,"num_object_clones":0,"num_object_copies":95814,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31938,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":50994,"num_write_kb":2653824,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.a","version":"48'50665","reported_seq":51102,"reported_epoch":48,"state":"active+clean","last_fresh":"2026-02-24T07:47:50.720336+0000","last_change":"2026-02-24T07:47:35.439589+0000","last_active":"2026-02-24T07:47:50.720336+0000","last_peered":"2026-02-24T07:47:50.720336+0000","last_clean":"2026-02-24T07:47:50.720336+0000","last_became_active":"2026-02-24T07:46:02.882274+0000","last_became_peered":"2026-02-24T07:46:02.882274+0000","last_unstale":"2026-02-24T07:47:50.720336+0000","last_undegraded":"2026-02-24T07:47:50.720336+0000","last_fullsized":"2026-02-24T07:47:50.720336+0000","mapping_epoch":42,"log_start":"48'50661","ondisk_log_start":"48'50661","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"46'41859","last_scrub_stamp":"2026-02-24T07:47:35.439542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_clean_scrub_stamp":"2026-02-24T07:47:35.439542+0000","objects_scrubbed":40699,"log_size":4,"log_dups_size":3000,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:48:38.034606+0000","scrub_duration":3645,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2065760256,"num_objects":31521,"num_object_clones":0,"num_object_copies":94563,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31521,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":50665,"num_write_kb":2629952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"48'51437","reported_seq":51784,"reported_epoch":48,"state":"active+clean","last_fresh":"2026-02-24T07:47:50.718707+0000","last_change":"2026-02-24T07:47:10.197277+0000","last_active":"2026-02-24T07:47:50.718707+0000","last_peered":"2026-02-24T07:47:50.718707+0000","last_clean":"2026-02-24T07:47:50.718707+0000","last_became_active":"2026-02-24T07:46:02.881951+0000","last_became_peered":"2026-02-24T07:46:02.881951+0000","last_unstale":"2026-02-24T07:47:50.718707+0000","last_undegraded":"2026-02-24T07:47:50.718707+0000","last_fullsized":"2026-02-24T07:47:50.718707+0000","mapping_epoch":42,"log_start":"48'51434","ondisk_log_start":"48'51434","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"45'31847","last_scrub_stamp":"2026-02-24T07:47:10.197239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_clean_scrub_stamp":"2026-02-24T07:47:10.197239+0000","objects_scrubbed":31279,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:48:27.205650+0000","scrub_duration":3289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2125135872,"num_objects":32427,"num_object_clones":0,"num_object_copies":97281,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32427,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":51437,"num_write_kb":2683648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"48'51091","reported_seq":51484,"reported_epoch":48,"state":"active+clean","last_fresh":"2026-02-24T07:47:50.769115+0000","last_change":"2026-02-24T07:47:22.289650+0000","last_active":"2026-02-24T07:47:50.769115+0000","last_peered":"2026-02-24T07:47:50.769115+0000","last_clean":"2026-02-24T07:47:50.769115+0000","last_became_active":"2026-02-24T07:46:02.881345+0000","last_became_peered":"2026-02-24T07:46:02.881345+0000","last_unstale":"2026-02-24T07:47:50.769115+0000","last_undegraded":"2026-02-24T07:47:50.769115+0000","last_fullsized":"2026-02-24T07:47:50.769115+0000","mapping_epoch":42,"log_start":"48'51088","ondisk_log_start":"48'51088","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"45'36703","last_scrub_stamp":"2026-02-24T07:47:22.289617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_clean_scrub_stamp":"2026-02-24T07:47:22.289617+0000","objects_scrubbed":35960,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:48:31.332102+0000","scrub_duration":3891,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2105737216,"num_objects":32131,"num_object_clones":0,"num_object_copies":96393,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32131,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":51091,"num_write_kb":2663104,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"48'50683","reported_seq":51125,"reported_epoch":48,"state":"active+clean","last_fresh":"2026-02-24T07:47:50.768835+0000","last_change":"2026-02-24T07:47:35.925072+0000","last_active":"2026-02-24T07:47:50.768835+0000","last_peered":"2026-02-24T07:47:50.768835+0000","last_clean":"2026-02-24T07:47:50.768835+0000","last_became_active":"2026-02-24T07:46:02.880323+0000","last_became_peered":"2026-02-24T07:46:02.880323+0000","last_unstale":"2026-02-24T07:47:50.768835+0000","last_undegraded":"2026-02-24T07:47:50.768835+0000","last_fullsized":"2026-02-24T07:47:50.768835+0000","mapping_epoch":42,"log_start":"48'50680","ondisk_log_start":"48'50680","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"47'42248","last_scrub_stamp":"2026-02-24T07:47:35.925000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_clean_scrub_stamp":"2026-02-24T07:47:35.925000+0000","objects_scrubbed":40858,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:48:43.057825+0000","scrub_duration":3382,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2083586048,"num_objects":31793,"num_object_clones":0,"num_object_copies":95379,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31793,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":50683,"num_write_kb":2639232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.6","version":"48'51115","reported_seq":51513,"reported_epoch":48,"state":"active+clean","last_fresh":"2026-02-24T07:47:50.708937+0000","last_change":"2026-02-24T07:47:23.432795+0000","last_active":"2026-02-24T07:47:50.708937+0000","last_peered":"2026-02-24T07:47:50.708937+0000","last_clean":"2026-02-24T07:47:50.708937+0000","last_became_active":"2026-02-24T07:46:02.880005+0000","last_became_peered":"2026-02-24T07:46:02.880005+0000","last_unstale":"2026-02-24T07:47:50.708937+0000","last_undegraded":"2026-02-24T07:47:50.708937+0000","last_fullsized":"2026-02-24T07:47:50.708937+0000","mapping_epoch":42,"log_start":"48'51112","ondisk_log_start":"48'51112","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"45'37219","last_scrub_stamp":"2026-02-24T07:47:23.432744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_clean_scrub_stamp":"2026-02-24T07:47:23.432744+0000","objects_scrubbed":36510,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:48:51.822832+0000","scrub_duration":3412,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2091712512,"num_objects":31917,"num_object_clones":0,"num_object_copies":95751,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31917,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":51115,"num_write_kb":2657024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"48'51372","reported_seq":51747,"reported_epoch":48,"state":"active+clean","last_fresh":"2026-02-24T07:47:50.701631+0000","last_change":"2026-02-24T07:47:17.546625+0000","last_active":"2026-02-24T07:47:50.701631+0000","last_peered":"2026-02-24T07:47:50.701631+0000","last_clean":"2026-02-24T07:47:50.701631+0000","last_became_active":"2026-02-24T07:46:02.881420+0000","last_became_peered":"2026-02-24T07:46:02.881420+0000","last_unstale":"2026-02-24T07:47:50.701631+0000","last_undegraded":"2026-02-24T07:47:50.701631+0000","last_fullsized":"2026-02-24T07:47:50.701631+0000","mapping_epoch":42,"log_start":"48'51369","ondisk_log_start":"48'51369","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"45'34854","last_scrub_stamp":"2026-02-24T07:47:17.546590+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_clean_scrub_stamp":"2026-02-24T07:47:17.546590+0000","objects_scrubbed":34222,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:48:38.813192+0000","scrub_duration":3116,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2115633152,"num_objects":32282,"num_object_clones":0,"num_object_copies":96846,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32282,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":51372,"num_write_kb":2676928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.4","version":"48'50503","reported_seq":50937,"reported_epoch":48,"state":"active+clean","last_fresh":"2026-02-24T07:47:50.772676+0000","last_change":"2026-02-24T07:47:33.628245+0000","last_active":"2026-02-24T07:47:50.772676+0000","last_peered":"2026-02-24T07:47:50.772676+0000","last_clean":"2026-02-24T07:47:50.772676+0000","last_became_active":"2026-02-24T07:46:02.880345+0000","last_became_peered":"2026-02-24T07:46:02.880345+0000","last_unstale":"2026-02-24T07:47:50.772676+0000","last_undegraded":"2026-02-24T07:47:50.772676+0000","last_fullsized":"2026-02-24T07:47:50.772676+0000","mapping_epoch":42,"log_start":"48'50499","ondisk_log_start":"48'50499","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"46'41076","last_scrub_stamp":"2026-02-24T07:47:33.628218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_clean_scrub_stamp":"2026-02-24T07:47:33.628218+0000","objects_scrubbed":40364,"log_size":4,"log_dups_size":3000,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:48:34.035175+0000","scrub_duration":4080,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2091974656,"num_objects":31921,"num_object_clones":0,"num_object_copies":95763,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31921,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":50503,"num_write_kb":2637568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"48'51084","reported_seq":51444,"reported_epoch":48,"state":"active+clean","last_fresh":"2026-02-24T07:47:50.697209+0000","last_change":"2026-02-24T07:47:14.339634+0000","last_active":"2026-02-24T07:47:50.697209+0000","last_peered":"2026-02-24T07:47:50.697209+0000","last_clean":"2026-02-24T07:47:50.697209+0000","last_became_active":"2026-02-24T07:46:02.881035+0000","last_became_peered":"2026-02-24T07:46:02.881035+0000","last_unstale":"2026-02-24T07:47:50.697209+0000","last_undegraded":"2026-02-24T07:47:50.697209+0000","last_fullsized":"2026-02-24T07:47:50.697209+0000","mapping_epoch":42,"log_start":"48'51081","ondisk_log_start":"48'51081","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"45'33283","last_scrub_stamp":"2026-02-24T07:47:14.339597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_clean_scrub_stamp":"2026-02-24T07:47:14.339597+0000","objects_scrubbed":32630,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:48:22.078719+0000","scrub_duration":2951,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2102788096,"num_objects":32086,"num_object_clones":0,"num_object_copies":96258,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32086,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":51084,"num_write_kb":2661440,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"48'51064","reported_seq":51464,"reported_epoch":48,"state":"active+clean","last_fresh":"2026-02-24T07:47:50.711761+0000","last_change":"2026-02-24T07:47:24.807734+0000","last_active":"2026-02-24T07:47:50.711761+0000","last_peered":"2026-02-24T07:47:50.711761+0000","last_clean":"2026-02-24T07:47:50.711761+0000","last_became_active":"2026-02-24T07:46:02.879831+0000","last_became_peered":"2026-02-24T07:46:02.879831+0000","last_unstale":"2026-02-24T07:47:50.711761+0000","last_undegraded":"2026-02-24T07:47:50.711761+0000","last_fullsized":"2026-02-24T07:47:50.711761+0000","mapping_epoch":42,"log_start":"48'51061","ondisk_log_start":"48'51061","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"45'37558","last_scrub_stamp":"2026-02-24T07:47:24.807659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_clean_scrub_stamp":"2026-02-24T07:47:24.807659+0000","objects_scrubbed":36869,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:48:39.172465+0000","scrub_duration":3748,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2081292288,"num_objects":31758,"num_object_clones":0,"num_object_copies":95274,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31758,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":51064,"num_write_kb":2650304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.0","version":"48'50782","reported_seq":51115,"reported_epoch":48,"state":"active+clean","last_fresh":"2026-02-24T07:47:50.772392+0000","last_change":"2026-02-24T07:47:08.376360+0000","last_active":"2026-02-24T07:47:50.772392+0000","last_peered":"2026-02-24T07:47:50.772392+0000","last_clean":"2026-02-24T07:47:50.772392+0000","last_became_active":"2026-02-24T07:46:02.880333+0000","last_became_peered":"2026-02-24T07:46:02.880333+0000","last_unstale":"2026-02-24T07:47:50.772392+0000","last_undegraded":"2026-02-24T07:47:50.772392+0000","last_fullsized":"2026-02-24T07:47:50.772392+0000","mapping_epoch":42,"log_start":"48'50779","ondisk_log_start":"48'50779","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"45'30511","last_scrub_stamp":"2026-02-24T07:47:08.376304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_clean_scrub_stamp":"2026-02-24T07:47:08.376304+0000","objects_scrubbed":29912,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:48:24.552234+0000","scrub_duration":3049,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2093481984,"num_objects":31944,"num_object_clones":0,"num_object_copies":95832,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31944,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":50782,"num_write_kb":2647232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2","version":"48'51225","reported_seq":51640,"reported_epoch":48,"state":"active+clean","last_fresh":"2026-02-24T07:47:50.767420+0000","last_change":"2026-02-24T07:47:27.784936+0000","last_active":"2026-02-24T07:47:50.767420+0000","last_peered":"2026-02-24T07:47:50.767420+0000","last_clean":"2026-02-24T07:47:50.767420+0000","last_became_active":"2026-02-24T07:46:02.880185+0000","last_became_peered":"2026-02-24T07:46:02.880185+0000","last_unstale":"2026-02-24T07:47:50.767420+0000","last_undegraded":"2026-02-24T07:47:50.767420+0000","last_fullsized":"2026-02-24T07:47:50.767420+0000","mapping_epoch":42,"log_start":"48'51222","ondisk_log_start":"48'51222","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"45'39074","last_scrub_stamp":"2026-02-24T07:47:27.784888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_clean_scrub_stamp":"2026-02-24T07:47:27.784888+0000","objects_scrubbed":38200,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:48:47.063599+0000","scrub_duration":4266,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2107179008,"num_objects":32153,"num_object_clones":0,"num_object_copies":96459,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32153,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":51225,"num_write_kb":2668096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"29'99","reported_seq":243,"reported_epoch":48,"state":"active+clean","last_fresh":"2026-02-24T07:47:44.887442+0000","last_change":"2026-02-24T07:46:41.007499+0000","last_active":"2026-02-24T07:47:44.887442+0000","last_peered":"2026-02-24T07:47:44.887442+0000","last_clean":"2026-02-24T07:47:44.887442+0000","last_became_active":"2026-02-24T07:42:54.186719+0000","last_became_peered":"2026-02-24T07:42:54.186719+0000","last_unstale":"2026-02-24T07:47:44.887442+0000","last_undegraded":"2026-02-24T07:47:44.887442+0000","last_fullsized":"2026-02-24T07:47:44.887442+0000","mapping_epoch":27,"log_start":"29'96","ondisk_log_start":"29'96","created":20,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'99","last_scrub_stamp":"2026-02-24T07:46:41.007434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:42:43.640776+0000","last_clean_scrub_stamp":"2026-02-24T07:46:41.007434+0000","objects_scrubbed":2,"log_size":3,"log_dups_size":96,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:47:53.486511+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":240,"num_write":135,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":16,"stat_sum":{"num_bytes":33524219928,"num_objects":511540,"num_object_clones":0,"num_object_copies":1534620,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":511540,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":815476,"num_write_kb":42464449,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":100590628864,"data_stored":100590616648,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":50,"ondisk_log_size":50,"up":48,"acting":48,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":240,"num_write":135,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":3,"up_from":25,"seq":107374182462,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":22866352,"kb_used_data":20853980,"kb_used_omap":9,"kb_used_meta":2012342,"kb_avail":711136848,"statfs":{"total":751619276800,"available":728204132352,"internally_reserved":0,"allocated":21354475520,"data_stored":21354159293,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":10061,"internal_metadata":2060638387},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1],"upper_bound":4},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":19,"seq":81604378688,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":30222704,"kb_used_data":28961116,"kb_used_omap":9,"kb_used_meta":1261494,"kb_avail":703780496,"statfs":{"total":751619276800,"available":720671227904,"internally_reserved":0,"allocated":29656182784,"data_stored":29655866557,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9937,"internal_metadata":1291770159},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542209,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":23005800,"kb_used_data":22491924,"kb_used_omap":10,"kb_used_meta":513845,"kb_avail":710997400,"statfs":{"total":751619276800,"available":728061337600,"internally_reserved":0,"allocated":23031730176,"data_stored":23031421573,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":10632,"internal_metadata":526177912},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705732,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":28071088,"kb_used_data":26954204,"kb_used_omap":7,"kb_used_meta":1116792,"kb_avail":705932112,"statfs":{"total":751619276800,"available":722874482688,"internally_reserved":0,"allocated":27601104896,"data_stored":27600788669,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":7430,"internal_metadata":1143595770},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":27249348608,"data_stored":27249344536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":22982033408,"data_stored":22982033408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":29392179200,"data_stored":29392175128,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":20967067648,"data_stored":20967063576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-24T07:47:53.476 INFO:tasks.cephadm.ceph_manager.ceph:increase pool size by 10 2026-02-24T07:47:53.477 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph osd pool set .mgr pg_num 171 2026-02-24T07:47:53.669 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:47:53.803 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:47:53.802+0000 7f0540b20640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:47:53.803 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:47:53.802+0000 7f0540b20640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:47:53.954 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:47:53 trial035 ceph-mon[39284]: osdmap e49: 4 total, 4 up, 4 in 2026-02-24T07:47:53.954 INFO:teuthology.orchestra.run.trial035.stderr:Error EPERM: crimson_allow_pg_split is false; pg_num increase denied for crimson pool 2026-02-24T07:47:53.996 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-24T07:47:54.275 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2026-02-24T07:47:54.418 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:47:54.540 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:47:54.539+0000 7f657ad74640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:47:54.540 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:47:54.539+0000 7f657ad74640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:47:54.559 INFO:teuthology.orchestra.run.trial035.stdout:osd.0: {} 2026-02-24T07:47:54.559 INFO:teuthology.orchestra.run.trial035.stderr:osd.0: osd_enable_op_tracker = '' 2026-02-24T07:47:54.576 INFO:teuthology.orchestra.run.trial035.stdout:osd.1: {} 2026-02-24T07:47:54.576 INFO:teuthology.orchestra.run.trial035.stderr:osd.1: osd_enable_op_tracker = '' 2026-02-24T07:47:54.588 INFO:teuthology.orchestra.run.trial035.stdout:osd.2: {} 2026-02-24T07:47:54.589 INFO:teuthology.orchestra.run.trial035.stderr:osd.2: osd_enable_op_tracker = '' 2026-02-24T07:47:54.604 INFO:teuthology.orchestra.run.trial035.stdout:osd.3: {} 2026-02-24T07:47:54.605 INFO:teuthology.orchestra.run.trial035.stderr:osd.3: osd_enable_op_tracker = '' 2026-02-24T07:47:54.954 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:47:54 trial035 ceph-mon[39284]: purged_snaps scrub starts 2026-02-24T07:47:54.954 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:47:54 trial035 ceph-mon[39284]: purged_snaps scrub ok 2026-02-24T07:47:54.954 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:47:54 trial035 ceph-mon[39284]: from='client.14756 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2026-02-24T07:47:54.954 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:47:54 trial035 ceph-mon[39284]: pgmap v170: 17 pgs: 17 active+clean; 31 GiB data, 99 GiB used, 2.6 TiB / 2.7 TiB avail; 0 B/s wr, 10.59k op/s 2026-02-24T07:47:54.954 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:47:54 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/925176776' entity='client.admin' cmd={"prefix": "osd pool set", "pool": ".mgr", "var": "pg_num", "val": "171"} : dispatch 2026-02-24T07:47:54.954 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:47:54 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/728031976' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2026-02-24T07:47:56.646 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2026-02-24T07:47:56.790 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:47:56.915 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:47:56.914+0000 7f341fd27640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:47:56.916 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:47:56.914+0000 7f341fd27640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:47:56.932 INFO:teuthology.orchestra.run.trial035.stdout:osd.0: {} 2026-02-24T07:47:56.932 INFO:teuthology.orchestra.run.trial035.stderr:osd.0: osd_enable_op_tracker = '' 2026-02-24T07:47:56.932 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:47:56 trial035 ceph-mon[39284]: pgmap v171: 17 pgs: 17 active+clean; 31 GiB data, 99 GiB used, 2.6 TiB / 2.7 TiB avail; 0 B/s wr, 9.33k op/s 2026-02-24T07:47:56.946 INFO:teuthology.orchestra.run.trial035.stdout:osd.1: {} 2026-02-24T07:47:56.947 INFO:teuthology.orchestra.run.trial035.stderr:osd.1: osd_enable_op_tracker = '' 2026-02-24T07:47:56.960 INFO:teuthology.orchestra.run.trial035.stdout:osd.2: {} 2026-02-24T07:47:56.960 INFO:teuthology.orchestra.run.trial035.stderr:osd.2: osd_enable_op_tracker = '' 2026-02-24T07:47:56.973 INFO:teuthology.orchestra.run.trial035.stdout:osd.3: {} 2026-02-24T07:47:56.973 INFO:teuthology.orchestra.run.trial035.stderr:osd.3: osd_enable_op_tracker = '' 2026-02-24T07:47:57.204 INFO:journalctl@ceph.mgr.x.trial035.stdout:Feb 24 07:47:56 trial035 ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-mgr-x[39621]: ::ffff:10.20.193.91 - - [24/Feb/2026:07:47:56] "GET /metrics HTTP/1.1" 200 4240 "" "Prometheus/3.6.0" 2026-02-24T07:47:57.356 INFO:journalctl@ceph.prometheus.a.trial091.stdout:Feb 24 07:47:56 trial091 ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-prometheus-a[66007]: time=2026-02-24T07:47:56.973Z level=ERROR source=scrape.go:1630 msg="Failed to determine correct type of scrape target." component="scrape manager" scrape_pool=ceph target=http://trial035:9283/metrics content_type="text/html;charset=utf-8" fallback_media_type="" err="received unsupported Content-Type \"text/html;charset=utf-8\" and no fallback_scrape_protocol specified for target" 2026-02-24T07:47:58.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:47:57 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/3986642313' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2026-02-24T07:47:58.997 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2026-02-24T07:47:58.998 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2026-02-24T07:47:58.998 INFO:tasks.thrashosds.thrasher:Growing pool .mgr 2026-02-24T07:47:58.998 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph pg dump --format=json 2026-02-24T07:47:59.016 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2026-02-24T07:47:59.024 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:47:58 trial035 ceph-mon[39284]: pgmap v172: 17 pgs: 17 active+clean; 28 GiB data, 91 GiB used, 2.6 TiB / 2.7 TiB avail; 0 B/s wr, 9.22k op/s 2026-02-24T07:47:59.152 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:47:59.162 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:47:59.605 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:47:59.590+0000 7f8769c18640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:47:59.606 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:47:59.590+0000 7f8769c18640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:47:59.733 INFO:teuthology.orchestra.run.trial035.stdout: 2026-02-24T07:47:59.733 INFO:teuthology.orchestra.run.trial035.stderr:dumped all 2026-02-24T07:48:00.448 INFO:teuthology.orchestra.run.trial035.stdout:{"pg_ready":true,"pg_map":{"version":173,"stamp":"2026-02-24T07:47:59.698084+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":30476206648,"num_objects":465024,"num_object_clones":0,"num_object_copies":1395072,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":465024,"num_whiteouts":0,"num_read":137,"num_read_kb":241,"num_write":862129,"num_write_kb":42466599,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":52,"ondisk_log_size":52,"up":51,"acting":51,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":51,"num_osds":4,"num_per_pool_osds":4,"num_per_pool_omap_osds":4,"kb":2936012800,"kb_used":95097576,"kb_used_data":89958360,"kb_used_omap":40,"kb_used_meta":5139095,"kb_avail":2840915224,"statfs":{"total":3006477107200,"available":2909097189376,"internally_reserved":0,"allocated":92117360640,"data_stored":92116072008,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":41320,"internal_metadata":5262433944},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1],"upper_bound":4},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-6046154752,"num_objects":-92257,"num_object_clones":0,"num_object_copies":-276771,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-92257,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":92257,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.001412"},"pg_stats":[{"pgid":"3.f","version":"49'53700","reported_seq":54070,"reported_epoch":49,"state":"active+clean","last_fresh":"2026-02-24T07:47:55.698641+0000","last_change":"2026-02-24T07:47:16.466592+0000","last_active":"2026-02-24T07:47:55.698641+0000","last_peered":"2026-02-24T07:47:55.698641+0000","last_clean":"2026-02-24T07:47:55.698641+0000","last_became_active":"2026-02-24T07:46:02.881625+0000","last_became_peered":"2026-02-24T07:46:02.881625+0000","last_unstale":"2026-02-24T07:47:55.698641+0000","last_undegraded":"2026-02-24T07:47:55.698641+0000","last_fullsized":"2026-02-24T07:47:55.698641+0000","mapping_epoch":42,"log_start":"49'53697","ondisk_log_start":"49'53697","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"45'34153","last_scrub_stamp":"2026-02-24T07:47:16.466530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_clean_scrub_stamp":"2026-02-24T07:47:16.466530+0000","objects_scrubbed":33463,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:48:23.487779+0000","scrub_duration":3074,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1894645760,"num_objects":28910,"num_object_clones":0,"num_object_copies":86730,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28910,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":53700,"num_write_kb":2643520,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.e","version":"49'54503","reported_seq":54943,"reported_epoch":49,"state":"active+clean","last_fresh":"2026-02-24T07:47:55.702349+0000","last_change":"2026-02-24T07:47:33.099762+0000","last_active":"2026-02-24T07:47:55.702349+0000","last_peered":"2026-02-24T07:47:55.702349+0000","last_clean":"2026-02-24T07:47:55.702349+0000","last_became_active":"2026-02-24T07:46:02.881083+0000","last_became_peered":"2026-02-24T07:46:02.881083+0000","last_unstale":"2026-02-24T07:47:55.702349+0000","last_undegraded":"2026-02-24T07:47:55.702349+0000","last_fullsized":"2026-02-24T07:47:55.702349+0000","mapping_epoch":42,"log_start":"49'54500","ondisk_log_start":"49'54500","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"46'41533","last_scrub_stamp":"2026-02-24T07:47:33.099675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_clean_scrub_stamp":"2026-02-24T07:47:33.099675+0000","objects_scrubbed":40754,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:48:53.832432+0000","scrub_duration":3969,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1919746048,"num_objects":29293,"num_object_clones":0,"num_object_copies":87879,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29293,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":54503,"num_write_kb":2681472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"49'53975","reported_seq":54306,"reported_epoch":49,"state":"active+clean","last_fresh":"2026-02-24T07:47:55.720970+0000","last_change":"2026-02-24T07:47:06.968813+0000","last_active":"2026-02-24T07:47:55.720970+0000","last_peered":"2026-02-24T07:47:55.720970+0000","last_clean":"2026-02-24T07:47:55.720970+0000","last_became_active":"2026-02-24T07:46:02.883799+0000","last_became_peered":"2026-02-24T07:46:02.883799+0000","last_unstale":"2026-02-24T07:47:55.720970+0000","last_undegraded":"2026-02-24T07:47:55.720970+0000","last_fullsized":"2026-02-24T07:47:55.720970+0000","mapping_epoch":42,"log_start":"49'53972","ondisk_log_start":"49'53972","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"45'30090","last_scrub_stamp":"2026-02-24T07:47:06.968732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_clean_scrub_stamp":"2026-02-24T07:47:06.968732+0000","objects_scrubbed":29542,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:48:25.198085+0000","scrub_duration":3070,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1907490840,"num_objects":29107,"num_object_clones":0,"num_object_copies":87321,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29107,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":53975,"num_write_kb":2658561,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"49'52968","reported_seq":53298,"reported_epoch":49,"state":"active+clean","last_fresh":"2026-02-24T07:47:55.721010+0000","last_change":"2026-02-24T07:47:07.546834+0000","last_active":"2026-02-24T07:47:55.721010+0000","last_peered":"2026-02-24T07:47:55.721010+0000","last_clean":"2026-02-24T07:47:55.721010+0000","last_became_active":"2026-02-24T07:46:02.880060+0000","last_became_peered":"2026-02-24T07:46:02.880060+0000","last_unstale":"2026-02-24T07:47:55.721010+0000","last_undegraded":"2026-02-24T07:47:55.721010+0000","last_fullsized":"2026-02-24T07:47:55.721010+0000","mapping_epoch":42,"log_start":"49'52965","ondisk_log_start":"49'52965","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"45'29910","last_scrub_stamp":"2026-02-24T07:47:07.546803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_clean_scrub_stamp":"2026-02-24T07:47:07.546803+0000","objects_scrubbed":29435,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:48:27.524515+0000","scrub_duration":2696,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1879179264,"num_objects":28674,"num_object_clones":0,"num_object_copies":86022,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28674,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":52968,"num_write_kb":2612544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"49'53911","reported_seq":54252,"reported_epoch":49,"state":"active+clean","last_fresh":"2026-02-24T07:47:55.774292+0000","last_change":"2026-02-24T07:47:10.376412+0000","last_active":"2026-02-24T07:47:55.774292+0000","last_peered":"2026-02-24T07:47:55.774292+0000","last_clean":"2026-02-24T07:47:55.774292+0000","last_became_active":"2026-02-24T07:46:02.880171+0000","last_became_peered":"2026-02-24T07:46:02.880171+0000","last_unstale":"2026-02-24T07:47:55.774292+0000","last_undegraded":"2026-02-24T07:47:55.774292+0000","last_fullsized":"2026-02-24T07:47:55.774292+0000","mapping_epoch":42,"log_start":"49'53908","ondisk_log_start":"49'53908","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"45'31519","last_scrub_stamp":"2026-02-24T07:47:10.376345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_clean_scrub_stamp":"2026-02-24T07:47:10.376345+0000","objects_scrubbed":30810,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:48:13.377288+0000","scrub_duration":4056,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1901920256,"num_objects":29021,"num_object_clones":0,"num_object_copies":87063,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29021,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":53911,"num_write_kb":2653824,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.a","version":"49'53472","reported_seq":53914,"reported_epoch":49,"state":"active+clean","last_fresh":"2026-02-24T07:47:55.719023+0000","last_change":"2026-02-24T07:47:35.439589+0000","last_active":"2026-02-24T07:47:55.719023+0000","last_peered":"2026-02-24T07:47:55.719023+0000","last_clean":"2026-02-24T07:47:55.719023+0000","last_became_active":"2026-02-24T07:46:02.882274+0000","last_became_peered":"2026-02-24T07:46:02.882274+0000","last_unstale":"2026-02-24T07:47:55.719023+0000","last_undegraded":"2026-02-24T07:47:55.719023+0000","last_fullsized":"2026-02-24T07:47:55.719023+0000","mapping_epoch":42,"log_start":"49'53469","ondisk_log_start":"49'53469","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"46'41859","last_scrub_stamp":"2026-02-24T07:47:35.439542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_clean_scrub_stamp":"2026-02-24T07:47:35.439542+0000","objects_scrubbed":40699,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:48:38.034606+0000","scrub_duration":3645,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1881800704,"num_objects":28714,"num_object_clones":0,"num_object_copies":86142,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28714,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":53472,"num_write_kb":2629952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"49'54405","reported_seq":54754,"reported_epoch":49,"state":"active+clean","last_fresh":"2026-02-24T07:47:55.717889+0000","last_change":"2026-02-24T07:47:10.197277+0000","last_active":"2026-02-24T07:47:55.717889+0000","last_peered":"2026-02-24T07:47:55.717889+0000","last_clean":"2026-02-24T07:47:55.717889+0000","last_became_active":"2026-02-24T07:46:02.881951+0000","last_became_peered":"2026-02-24T07:46:02.881951+0000","last_unstale":"2026-02-24T07:47:55.717889+0000","last_undegraded":"2026-02-24T07:47:55.717889+0000","last_fullsized":"2026-02-24T07:47:55.717889+0000","mapping_epoch":42,"log_start":"49'54402","ondisk_log_start":"49'54402","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"45'31847","last_scrub_stamp":"2026-02-24T07:47:10.197239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_clean_scrub_stamp":"2026-02-24T07:47:10.197239+0000","objects_scrubbed":31279,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:48:27.205650+0000","scrub_duration":3289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1930625024,"num_objects":29459,"num_object_clones":0,"num_object_copies":88377,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29459,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":54405,"num_write_kb":2683648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"49'54014","reported_seq":54409,"reported_epoch":49,"state":"active+clean","last_fresh":"2026-02-24T07:47:55.772410+0000","last_change":"2026-02-24T07:47:22.289650+0000","last_active":"2026-02-24T07:47:55.772410+0000","last_peered":"2026-02-24T07:47:55.772410+0000","last_clean":"2026-02-24T07:47:55.772410+0000","last_became_active":"2026-02-24T07:46:02.881345+0000","last_became_peered":"2026-02-24T07:46:02.881345+0000","last_unstale":"2026-02-24T07:47:55.772410+0000","last_undegraded":"2026-02-24T07:47:55.772410+0000","last_fullsized":"2026-02-24T07:47:55.772410+0000","mapping_epoch":42,"log_start":"49'54011","ondisk_log_start":"49'54011","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"45'36703","last_scrub_stamp":"2026-02-24T07:47:22.289617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_clean_scrub_stamp":"2026-02-24T07:47:22.289617+0000","objects_scrubbed":35960,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:48:31.332102+0000","scrub_duration":3891,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1914175488,"num_objects":29208,"num_object_clones":0,"num_object_copies":87624,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29208,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":54014,"num_write_kb":2663104,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"49'53525","reported_seq":53969,"reported_epoch":49,"state":"active+clean","last_fresh":"2026-02-24T07:47:55.769172+0000","last_change":"2026-02-24T07:47:35.925072+0000","last_active":"2026-02-24T07:47:55.769172+0000","last_peered":"2026-02-24T07:47:55.769172+0000","last_clean":"2026-02-24T07:47:55.769172+0000","last_became_active":"2026-02-24T07:46:02.880323+0000","last_became_peered":"2026-02-24T07:46:02.880323+0000","last_unstale":"2026-02-24T07:47:55.769172+0000","last_undegraded":"2026-02-24T07:47:55.769172+0000","last_fullsized":"2026-02-24T07:47:55.769172+0000","mapping_epoch":42,"log_start":"49'53522","ondisk_log_start":"49'53522","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"47'42248","last_scrub_stamp":"2026-02-24T07:47:35.925000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_clean_scrub_stamp":"2026-02-24T07:47:35.925000+0000","objects_scrubbed":40858,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:48:43.057825+0000","scrub_duration":3382,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1897332736,"num_objects":28951,"num_object_clones":0,"num_object_copies":86853,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28951,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":53525,"num_write_kb":2639232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.6","version":"49'53965","reported_seq":54365,"reported_epoch":49,"state":"active+clean","last_fresh":"2026-02-24T07:47:55.712357+0000","last_change":"2026-02-24T07:47:23.432795+0000","last_active":"2026-02-24T07:47:55.712357+0000","last_peered":"2026-02-24T07:47:55.712357+0000","last_clean":"2026-02-24T07:47:55.712357+0000","last_became_active":"2026-02-24T07:46:02.880005+0000","last_became_peered":"2026-02-24T07:46:02.880005+0000","last_unstale":"2026-02-24T07:47:55.712357+0000","last_undegraded":"2026-02-24T07:47:55.712357+0000","last_fullsized":"2026-02-24T07:47:55.712357+0000","mapping_epoch":42,"log_start":"49'53962","ondisk_log_start":"49'53962","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"45'37219","last_scrub_stamp":"2026-02-24T07:47:23.432744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_clean_scrub_stamp":"2026-02-24T07:47:23.432744+0000","objects_scrubbed":36510,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:48:51.822832+0000","scrub_duration":3412,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1904934912,"num_objects":29067,"num_object_clones":0,"num_object_copies":87201,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29067,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":53965,"num_write_kb":2657024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"49'54251","reported_seq":54627,"reported_epoch":49,"state":"active+clean","last_fresh":"2026-02-24T07:47:55.692859+0000","last_change":"2026-02-24T07:47:17.546625+0000","last_active":"2026-02-24T07:47:55.692859+0000","last_peered":"2026-02-24T07:47:55.692859+0000","last_clean":"2026-02-24T07:47:55.692859+0000","last_became_active":"2026-02-24T07:46:02.881420+0000","last_became_peered":"2026-02-24T07:46:02.881420+0000","last_unstale":"2026-02-24T07:47:55.692859+0000","last_undegraded":"2026-02-24T07:47:55.692859+0000","last_fullsized":"2026-02-24T07:47:55.692859+0000","mapping_epoch":42,"log_start":"49'54248","ondisk_log_start":"49'54248","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"45'34854","last_scrub_stamp":"2026-02-24T07:47:17.546590+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_clean_scrub_stamp":"2026-02-24T07:47:17.546590+0000","objects_scrubbed":34222,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:48:38.813192+0000","scrub_duration":3116,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1926955008,"num_objects":29403,"num_object_clones":0,"num_object_copies":88209,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29403,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":54251,"num_write_kb":2676928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.4","version":"49'53393","reported_seq":53832,"reported_epoch":49,"state":"active+clean","last_fresh":"2026-02-24T07:47:55.773732+0000","last_change":"2026-02-24T07:47:33.628245+0000","last_active":"2026-02-24T07:47:55.773732+0000","last_peered":"2026-02-24T07:47:55.773732+0000","last_clean":"2026-02-24T07:47:55.773732+0000","last_became_active":"2026-02-24T07:46:02.880345+0000","last_became_peered":"2026-02-24T07:46:02.880345+0000","last_unstale":"2026-02-24T07:47:55.773732+0000","last_undegraded":"2026-02-24T07:47:55.773732+0000","last_fullsized":"2026-02-24T07:47:55.773732+0000","mapping_epoch":42,"log_start":"49'53390","ondisk_log_start":"49'53390","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"46'41076","last_scrub_stamp":"2026-02-24T07:47:33.628218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_clean_scrub_stamp":"2026-02-24T07:47:33.628218+0000","objects_scrubbed":40364,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:48:34.035175+0000","scrub_duration":4080,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1902575616,"num_objects":29031,"num_object_clones":0,"num_object_copies":87093,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29031,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":53393,"num_write_kb":2637568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"49'53999","reported_seq":54361,"reported_epoch":49,"state":"active+clean","last_fresh":"2026-02-24T07:47:55.700535+0000","last_change":"2026-02-24T07:47:14.339634+0000","last_active":"2026-02-24T07:47:55.700535+0000","last_peered":"2026-02-24T07:47:55.700535+0000","last_clean":"2026-02-24T07:47:55.700535+0000","last_became_active":"2026-02-24T07:46:02.881035+0000","last_became_peered":"2026-02-24T07:46:02.881035+0000","last_unstale":"2026-02-24T07:47:55.700535+0000","last_undegraded":"2026-02-24T07:47:55.700535+0000","last_fullsized":"2026-02-24T07:47:55.700535+0000","mapping_epoch":42,"log_start":"49'53995","ondisk_log_start":"49'53995","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"45'33283","last_scrub_stamp":"2026-02-24T07:47:14.339597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_clean_scrub_stamp":"2026-02-24T07:47:14.339597+0000","objects_scrubbed":32630,"log_size":4,"log_dups_size":3000,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:48:22.078719+0000","scrub_duration":2951,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1911750656,"num_objects":29171,"num_object_clones":0,"num_object_copies":87513,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29171,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":53999,"num_write_kb":2661440,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"49'53948","reported_seq":54351,"reported_epoch":49,"state":"active+clean","last_fresh":"2026-02-24T07:47:55.705025+0000","last_change":"2026-02-24T07:47:24.807734+0000","last_active":"2026-02-24T07:47:55.705025+0000","last_peered":"2026-02-24T07:47:55.705025+0000","last_clean":"2026-02-24T07:47:55.705025+0000","last_became_active":"2026-02-24T07:46:02.879831+0000","last_became_peered":"2026-02-24T07:46:02.879831+0000","last_unstale":"2026-02-24T07:47:55.705025+0000","last_undegraded":"2026-02-24T07:47:55.705025+0000","last_fullsized":"2026-02-24T07:47:55.705025+0000","mapping_epoch":42,"log_start":"49'53945","ondisk_log_start":"49'53945","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"45'37558","last_scrub_stamp":"2026-02-24T07:47:24.807659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_clean_scrub_stamp":"2026-02-24T07:47:24.807659+0000","objects_scrubbed":36869,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:48:39.172465+0000","scrub_duration":3748,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1892286464,"num_objects":28874,"num_object_clones":0,"num_object_copies":86622,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28874,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":53948,"num_write_kb":2650304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.0","version":"49'53760","reported_seq":54095,"reported_epoch":49,"state":"active+clean","last_fresh":"2026-02-24T07:47:55.770402+0000","last_change":"2026-02-24T07:47:08.376360+0000","last_active":"2026-02-24T07:47:55.770402+0000","last_peered":"2026-02-24T07:47:55.770402+0000","last_clean":"2026-02-24T07:47:55.770402+0000","last_became_active":"2026-02-24T07:46:02.880333+0000","last_became_peered":"2026-02-24T07:46:02.880333+0000","last_unstale":"2026-02-24T07:47:55.770402+0000","last_undegraded":"2026-02-24T07:47:55.770402+0000","last_fullsized":"2026-02-24T07:47:55.770402+0000","mapping_epoch":42,"log_start":"49'53757","ondisk_log_start":"49'53757","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"45'30511","last_scrub_stamp":"2026-02-24T07:47:08.376304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_clean_scrub_stamp":"2026-02-24T07:47:08.376304+0000","objects_scrubbed":29912,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:48:24.552234+0000","scrub_duration":3049,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1898315776,"num_objects":28966,"num_object_clones":0,"num_object_copies":86898,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28966,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":53760,"num_write_kb":2647232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2","version":"49'54205","reported_seq":54622,"reported_epoch":49,"state":"active+clean","last_fresh":"2026-02-24T07:47:55.771401+0000","last_change":"2026-02-24T07:47:27.784936+0000","last_active":"2026-02-24T07:47:55.771401+0000","last_peered":"2026-02-24T07:47:55.771401+0000","last_clean":"2026-02-24T07:47:55.771401+0000","last_became_active":"2026-02-24T07:46:02.880185+0000","last_became_peered":"2026-02-24T07:46:02.880185+0000","last_unstale":"2026-02-24T07:47:55.771401+0000","last_undegraded":"2026-02-24T07:47:55.771401+0000","last_fullsized":"2026-02-24T07:47:55.771401+0000","mapping_epoch":42,"log_start":"49'54202","ondisk_log_start":"49'54202","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"45'39074","last_scrub_stamp":"2026-02-24T07:47:27.784888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_clean_scrub_stamp":"2026-02-24T07:47:27.784888+0000","objects_scrubbed":38200,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:48:47.063599+0000","scrub_duration":4266,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1911881728,"num_objects":29173,"num_object_clones":0,"num_object_copies":87519,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29173,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":54205,"num_write_kb":2668096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"29'99","reported_seq":245,"reported_epoch":49,"state":"active+clean","last_fresh":"2026-02-24T07:47:52.751627+0000","last_change":"2026-02-24T07:46:41.007499+0000","last_active":"2026-02-24T07:47:52.751627+0000","last_peered":"2026-02-24T07:47:52.751627+0000","last_clean":"2026-02-24T07:47:52.751627+0000","last_became_active":"2026-02-24T07:42:54.186719+0000","last_became_peered":"2026-02-24T07:42:54.186719+0000","last_unstale":"2026-02-24T07:47:52.751627+0000","last_undegraded":"2026-02-24T07:47:52.751627+0000","last_fullsized":"2026-02-24T07:47:52.751627+0000","mapping_epoch":27,"log_start":"29'96","ondisk_log_start":"29'96","created":20,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'99","last_scrub_stamp":"2026-02-24T07:46:41.007434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:42:43.640776+0000","last_clean_scrub_stamp":"2026-02-24T07:46:41.007434+0000","objects_scrubbed":2,"log_size":3,"log_dups_size":96,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:47:53.486511+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":240,"num_write":135,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":16,"stat_sum":{"num_bytes":30475616280,"num_objects":465022,"num_object_clones":0,"num_object_copies":1395066,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":465022,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":861994,"num_write_kb":42464449,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":91444293632,"data_stored":91444281416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":49,"ondisk_log_size":49,"up":48,"acting":48,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":240,"num_write":135,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":3,"up_from":25,"seq":107374182463,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":20705916,"kb_used_data":18639400,"kb_used_omap":10,"kb_used_meta":2066485,"kb_avail":713297284,"statfs":{"total":751619276800,"available":730416418816,"internally_reserved":0,"allocated":19086745600,"data_stored":19086421536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":10713,"internal_metadata":2116081191},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1],"upper_bound":4},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":19,"seq":81604378689,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":27652732,"kb_used_data":26319016,"kb_used_omap":10,"kb_used_meta":1333685,"kb_avail":706350468,"statfs":{"total":751619276800,"available":723302879232,"internally_reserved":0,"allocated":26950672384,"data_stored":26950348320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":10589,"internal_metadata":1365694115},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542210,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":21025140,"kb_used_data":20457120,"kb_used_omap":11,"kb_used_meta":567988,"kb_avail":712978060,"statfs":{"total":751619276800,"available":730089533440,"internally_reserved":0,"allocated":20948090880,"data_stored":20947774440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":11936,"internal_metadata":581620064},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705733,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":25713788,"kb_used_data":24542824,"kb_used_omap":7,"kb_used_meta":1170936,"kb_avail":708289412,"statfs":{"total":751619276800,"available":725288357888,"internally_reserved":0,"allocated":25131851776,"data_stored":25131527712,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":8082,"internal_metadata":1199038574},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":24772874240,"data_stored":24772870168,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":20893204480,"data_stored":20893204480,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":26716278784,"data_stored":26716274712,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":19061936128,"data_stored":19061932056,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-24T07:48:00.451 INFO:tasks.cephadm.ceph_manager.ceph:increase pool size by 10 2026-02-24T07:48:00.452 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph osd pool set .mgr pg_num 181 2026-02-24T07:48:00.673 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:48:00.598+0000 7f33cfb8e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:48:00.674 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:48:00.599+0000 7f33cfb8e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:48:00.674 INFO:teuthology.orchestra.run.trial035.stderr:osd.0: osd_enable_op_tracker = '' 2026-02-24T07:48:00.674 INFO:teuthology.orchestra.run.trial035.stderr:osd.1: osd_enable_op_tracker = '' 2026-02-24T07:48:00.674 INFO:teuthology.orchestra.run.trial035.stdout:osd.0: {} 2026-02-24T07:48:00.675 INFO:teuthology.orchestra.run.trial035.stdout:osd.1: {} 2026-02-24T07:48:00.675 INFO:teuthology.orchestra.run.trial035.stdout:osd.2: {} 2026-02-24T07:48:00.675 INFO:teuthology.orchestra.run.trial035.stderr:osd.2: osd_enable_op_tracker = '' 2026-02-24T07:48:00.677 INFO:teuthology.orchestra.run.trial035.stdout:osd.3: {} 2026-02-24T07:48:00.677 INFO:teuthology.orchestra.run.trial035.stderr:osd.3: osd_enable_op_tracker = '' 2026-02-24T07:48:00.954 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:48:00 trial035 ceph-mon[39284]: pgmap v173: 17 pgs: 17 active+clean; 28 GiB data, 91 GiB used, 2.6 TiB / 2.7 TiB avail; 0 B/s wr, 9.22k op/s 2026-02-24T07:48:00.954 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:48:00 trial035 ceph-mon[39284]: from='client.14764 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2026-02-24T07:48:00.954 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:48:00 trial035 ceph-mon[39284]: from='mgr.14284 10.20.193.35:0/3681373222' entity='mgr.x' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2026-02-24T07:48:00.954 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:48:00 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/425806384' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2026-02-24T07:48:01.219 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:48:01.345 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:48:01.342+0000 7f099c913640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:48:01.345 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:48:01.343+0000 7f099c913640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:48:01.478 INFO:teuthology.orchestra.run.trial035.stderr:Error EPERM: crimson_allow_pg_split is false; pg_num increase denied for crimson pool 2026-02-24T07:48:01.519 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-24T07:48:02.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:48:01 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/1448573594' entity='client.admin' cmd={"prefix": "osd pool set", "pool": ".mgr", "var": "pg_num", "val": "181"} : dispatch 2026-02-24T07:48:03.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:48:02 trial035 ceph-mon[39284]: pgmap v174: 17 pgs: 17 active+clean; 26 GiB data, 82 GiB used, 2.7 TiB / 2.7 TiB avail; 0 B/s wr, 8.98k op/s 2026-02-24T07:48:03.214 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2026-02-24T07:48:03.353 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:48:03.475 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:48:03.473+0000 7f7126575640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:48:03.476 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:48:03.473+0000 7f7126575640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:48:03.489 INFO:teuthology.orchestra.run.trial035.stdout:osd.0: {} 2026-02-24T07:48:03.490 INFO:teuthology.orchestra.run.trial035.stderr:osd.0: osd_enable_op_tracker = '' 2026-02-24T07:48:03.504 INFO:teuthology.orchestra.run.trial035.stdout:osd.1: {} 2026-02-24T07:48:03.504 INFO:teuthology.orchestra.run.trial035.stderr:osd.1: osd_enable_op_tracker = '' 2026-02-24T07:48:03.519 INFO:teuthology.orchestra.run.trial035.stdout:osd.2: {} 2026-02-24T07:48:03.520 INFO:teuthology.orchestra.run.trial035.stderr:osd.2: osd_enable_op_tracker = '' 2026-02-24T07:48:03.532 INFO:teuthology.orchestra.run.trial035.stdout:osd.3: {} 2026-02-24T07:48:03.532 INFO:teuthology.orchestra.run.trial035.stderr:osd.3: osd_enable_op_tracker = '' 2026-02-24T07:48:04.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:48:03 trial035 ceph-mon[39284]: 1.0 scrub starts 2026-02-24T07:48:04.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:48:03 trial035 ceph-mon[39284]: 1.0 scrub ok 2026-02-24T07:48:04.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:48:03 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/4207898557' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2026-02-24T07:48:05.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:48:04 trial035 ceph-mon[39284]: pgmap v175: 17 pgs: 17 active+clean; 26 GiB data, 82 GiB used, 2.7 TiB / 2.7 TiB avail; 0 B/s wr, 8.20k op/s 2026-02-24T07:48:05.572 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2026-02-24T07:48:05.715 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:48:05.836 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:48:05.834+0000 7fbe52e43640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:48:05.836 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:48:05.834+0000 7fbe52e43640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:48:05.851 INFO:teuthology.orchestra.run.trial035.stdout:osd.0: {} 2026-02-24T07:48:05.851 INFO:teuthology.orchestra.run.trial035.stderr:osd.0: osd_enable_op_tracker = '' 2026-02-24T07:48:05.869 INFO:teuthology.orchestra.run.trial035.stdout:osd.1: {} 2026-02-24T07:48:05.869 INFO:teuthology.orchestra.run.trial035.stderr:osd.1: osd_enable_op_tracker = '' 2026-02-24T07:48:05.882 INFO:teuthology.orchestra.run.trial035.stdout:osd.2: {} 2026-02-24T07:48:05.882 INFO:teuthology.orchestra.run.trial035.stderr:osd.2: osd_enable_op_tracker = '' 2026-02-24T07:48:05.896 INFO:teuthology.orchestra.run.trial035.stdout:osd.3: {} 2026-02-24T07:48:05.896 INFO:teuthology.orchestra.run.trial035.stderr:osd.3: osd_enable_op_tracker = '' 2026-02-24T07:48:06.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:48:05 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/2670953575' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2026-02-24T07:48:06.520 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2026-02-24T07:48:06.521 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2026-02-24T07:48:06.521 INFO:tasks.thrashosds.thrasher:trim_stale_osdmaps 2026-02-24T07:48:06.521 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph tell osd.0 'trim stale osdmaps' 2026-02-24T07:48:06.660 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:48:06.778 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:48:06.776+0000 7f56ebf96640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:48:06.779 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:48:06.776+0000 7f56ebf96640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:48:06.800 INFO:teuthology.orchestra.run.trial035.stderr:no valid command found; 10 closest matches: 2026-02-24T07:48:06.800 INFO:teuthology.orchestra.run.trial035.stderr:0 2026-02-24T07:48:06.800 INFO:teuthology.orchestra.run.trial035.stderr:1 2026-02-24T07:48:06.800 INFO:teuthology.orchestra.run.trial035.stderr:2 2026-02-24T07:48:06.800 INFO:teuthology.orchestra.run.trial035.stderr:abort 2026-02-24T07:48:06.800 INFO:teuthology.orchestra.run.trial035.stderr:assert 2026-02-24T07:48:06.801 INFO:teuthology.orchestra.run.trial035.stderr:bench [] [] [] [] 2026-02-24T07:48:06.801 INFO:teuthology.orchestra.run.trial035.stderr:bluefs debug_inject_read_zeros 2026-02-24T07:48:06.801 INFO:teuthology.orchestra.run.trial035.stderr:bluefs files list 2026-02-24T07:48:06.801 INFO:teuthology.orchestra.run.trial035.stderr:bluefs stats 2026-02-24T07:48:06.801 INFO:teuthology.orchestra.run.trial035.stderr:bluestore allocator dump block 2026-02-24T07:48:06.801 INFO:teuthology.orchestra.run.trial035.stderr:Error EINVAL: invalid command 2026-02-24T07:48:06.835 DEBUG:teuthology.orchestra.run:got remote process result: 22 2026-02-24T07:48:06.836 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2026-02-24T07:48:06.837 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph tell osd.1 'trim stale osdmaps' 2026-02-24T07:48:06.969 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:48:07.086 INFO:journalctl@ceph.mgr.x.trial035.stdout:Feb 24 07:48:06 trial035 ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-mgr-x[39621]: ::ffff:10.20.193.91 - - [24/Feb/2026:07:48:06] "GET /metrics HTTP/1.1" 200 4235 "" "Prometheus/3.6.0" 2026-02-24T07:48:07.086 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:48:06 trial035 ceph-mon[39284]: pgmap v176: 17 pgs: 17 active+clean; 26 GiB data, 82 GiB used, 2.7 TiB / 2.7 TiB avail; 0 B/s wr, 7.48k op/s 2026-02-24T07:48:07.086 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:48:06 trial035 ceph-mon[39284]: from='mgr.14284 10.20.193.35:0/3681373222' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2026-02-24T07:48:07.096 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:48:07.094+0000 7fd768028640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:48:07.097 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:48:07.094+0000 7fd768028640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:48:07.116 INFO:teuthology.orchestra.run.trial035.stderr:no valid command found; 10 closest matches: 2026-02-24T07:48:07.116 INFO:teuthology.orchestra.run.trial035.stderr:0 2026-02-24T07:48:07.116 INFO:teuthology.orchestra.run.trial035.stderr:1 2026-02-24T07:48:07.116 INFO:teuthology.orchestra.run.trial035.stderr:2 2026-02-24T07:48:07.116 INFO:teuthology.orchestra.run.trial035.stderr:abort 2026-02-24T07:48:07.117 INFO:teuthology.orchestra.run.trial035.stderr:assert 2026-02-24T07:48:07.117 INFO:teuthology.orchestra.run.trial035.stderr:bench [] [] [] [] 2026-02-24T07:48:07.117 INFO:teuthology.orchestra.run.trial035.stderr:bluefs debug_inject_read_zeros 2026-02-24T07:48:07.117 INFO:teuthology.orchestra.run.trial035.stderr:bluefs files list 2026-02-24T07:48:07.117 INFO:teuthology.orchestra.run.trial035.stderr:bluefs stats 2026-02-24T07:48:07.117 INFO:teuthology.orchestra.run.trial035.stderr:bluestore allocator dump block 2026-02-24T07:48:07.117 INFO:teuthology.orchestra.run.trial035.stderr:Error EINVAL: invalid command 2026-02-24T07:48:07.150 DEBUG:teuthology.orchestra.run:got remote process result: 22 2026-02-24T07:48:07.151 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2026-02-24T07:48:07.151 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph tell osd.2 'trim stale osdmaps' 2026-02-24T07:48:07.288 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:48:07.356 INFO:journalctl@ceph.prometheus.a.trial091.stdout:Feb 24 07:48:06 trial091 ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-prometheus-a[66007]: time=2026-02-24T07:48:06.973Z level=ERROR source=scrape.go:1630 msg="Failed to determine correct type of scrape target." component="scrape manager" scrape_pool=ceph target=http://trial035:9283/metrics content_type="text/html;charset=utf-8" fallback_media_type="" err="received unsupported Content-Type \"text/html;charset=utf-8\" and no fallback_scrape_protocol specified for target" 2026-02-24T07:48:07.409 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:48:07.407+0000 7f2155c03640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:48:07.409 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:48:07.407+0000 7f2155c03640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:48:07.434 INFO:teuthology.orchestra.run.trial035.stderr:no valid command found; 10 closest matches: 2026-02-24T07:48:07.434 INFO:teuthology.orchestra.run.trial035.stderr:0 2026-02-24T07:48:07.434 INFO:teuthology.orchestra.run.trial035.stderr:1 2026-02-24T07:48:07.434 INFO:teuthology.orchestra.run.trial035.stderr:2 2026-02-24T07:48:07.434 INFO:teuthology.orchestra.run.trial035.stderr:abort 2026-02-24T07:48:07.434 INFO:teuthology.orchestra.run.trial035.stderr:assert 2026-02-24T07:48:07.434 INFO:teuthology.orchestra.run.trial035.stderr:bench [] [] [] [] 2026-02-24T07:48:07.434 INFO:teuthology.orchestra.run.trial035.stderr:bluefs debug_inject_read_zeros 2026-02-24T07:48:07.434 INFO:teuthology.orchestra.run.trial035.stderr:bluefs files list 2026-02-24T07:48:07.434 INFO:teuthology.orchestra.run.trial035.stderr:bluefs stats 2026-02-24T07:48:07.434 INFO:teuthology.orchestra.run.trial035.stderr:bluestore allocator dump block 2026-02-24T07:48:07.434 INFO:teuthology.orchestra.run.trial035.stderr:Error EINVAL: invalid command 2026-02-24T07:48:07.468 DEBUG:teuthology.orchestra.run:got remote process result: 22 2026-02-24T07:48:07.469 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2026-02-24T07:48:07.469 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph tell osd.3 'trim stale osdmaps' 2026-02-24T07:48:07.613 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:48:07.746 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:48:07.744+0000 7f864b043640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:48:07.746 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:48:07.745+0000 7f864b043640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:48:07.773 INFO:teuthology.orchestra.run.trial035.stderr:no valid command found; 10 closest matches: 2026-02-24T07:48:07.773 INFO:teuthology.orchestra.run.trial035.stderr:0 2026-02-24T07:48:07.773 INFO:teuthology.orchestra.run.trial035.stderr:1 2026-02-24T07:48:07.773 INFO:teuthology.orchestra.run.trial035.stderr:2 2026-02-24T07:48:07.773 INFO:teuthology.orchestra.run.trial035.stderr:abort 2026-02-24T07:48:07.773 INFO:teuthology.orchestra.run.trial035.stderr:assert 2026-02-24T07:48:07.773 INFO:teuthology.orchestra.run.trial035.stderr:bench [] [] [] [] 2026-02-24T07:48:07.773 INFO:teuthology.orchestra.run.trial035.stderr:bluefs debug_inject_read_zeros 2026-02-24T07:48:07.773 INFO:teuthology.orchestra.run.trial035.stderr:bluefs files list 2026-02-24T07:48:07.773 INFO:teuthology.orchestra.run.trial035.stderr:bluefs stats 2026-02-24T07:48:07.773 INFO:teuthology.orchestra.run.trial035.stderr:bluestore allocator dump block 2026-02-24T07:48:07.773 INFO:teuthology.orchestra.run.trial035.stderr:Error EINVAL: invalid command 2026-02-24T07:48:07.807 DEBUG:teuthology.orchestra.run:got remote process result: 22 2026-02-24T07:48:07.808 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2026-02-24T07:48:07.941 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2026-02-24T07:48:07.954 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:48:07 trial035 ceph-mon[39284]: from='mgr.14284 10.20.193.35:0/3681373222' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-24T07:48:07.954 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:48:07 trial035 ceph-mon[39284]: from='mgr.14284 10.20.193.35:0/3681373222' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2026-02-24T07:48:07.954 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:48:07 trial035 ceph-mon[39284]: from='mgr.14284 10.20.193.35:0/3681373222' entity='mgr.x' 2026-02-24T07:48:07.954 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:48:07 trial035 ceph-mon[39284]: from='mgr.14284 10.20.193.35:0/3681373222' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2026-02-24T07:48:08.079 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:48:08.207 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:48:08.204+0000 7f9e18658640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:48:08.207 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:48:08.204+0000 7f9e18658640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:48:08.221 INFO:teuthology.orchestra.run.trial035.stdout:osd.0: {} 2026-02-24T07:48:08.222 INFO:teuthology.orchestra.run.trial035.stderr:osd.0: osd_enable_op_tracker = '' 2026-02-24T07:48:08.236 INFO:teuthology.orchestra.run.trial035.stdout:osd.1: {} 2026-02-24T07:48:08.236 INFO:teuthology.orchestra.run.trial035.stderr:osd.1: osd_enable_op_tracker = '' 2026-02-24T07:48:08.250 INFO:teuthology.orchestra.run.trial035.stdout:osd.2: {} 2026-02-24T07:48:08.251 INFO:teuthology.orchestra.run.trial035.stderr:osd.2: osd_enable_op_tracker = '' 2026-02-24T07:48:08.264 INFO:teuthology.orchestra.run.trial035.stdout:osd.3: {} 2026-02-24T07:48:08.264 INFO:teuthology.orchestra.run.trial035.stderr:osd.3: osd_enable_op_tracker = '' 2026-02-24T07:48:09.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:48:08 trial035 ceph-mon[39284]: pgmap v177: 17 pgs: 17 active+clean; 23 GiB data, 74 GiB used, 2.7 TiB / 2.7 TiB avail; 0 B/s wr, 11.12k op/s 2026-02-24T07:48:09.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:48:08 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/2721592753' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2026-02-24T07:48:10.308 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2026-02-24T07:48:10.450 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:48:10.837 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:48:10.815+0000 7fa9d6887640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:48:10.837 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:48:10.816+0000 7fa9d6887640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:48:10.851 INFO:teuthology.orchestra.run.trial035.stdout:osd.0: {} 2026-02-24T07:48:10.852 INFO:teuthology.orchestra.run.trial035.stderr:osd.0: osd_enable_op_tracker = '' 2026-02-24T07:48:10.864 INFO:teuthology.orchestra.run.trial035.stdout:osd.1: {} 2026-02-24T07:48:10.864 INFO:teuthology.orchestra.run.trial035.stderr:osd.1: osd_enable_op_tracker = '' 2026-02-24T07:48:10.877 INFO:teuthology.orchestra.run.trial035.stdout:osd.2: {} 2026-02-24T07:48:10.877 INFO:teuthology.orchestra.run.trial035.stderr:osd.2: osd_enable_op_tracker = '' 2026-02-24T07:48:10.889 INFO:teuthology.orchestra.run.trial035.stdout:osd.3: {} 2026-02-24T07:48:10.890 INFO:teuthology.orchestra.run.trial035.stderr:osd.3: osd_enable_op_tracker = '' 2026-02-24T07:48:10.905 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:48:10 trial035 ceph-mon[39284]: pgmap v178: 17 pgs: 17 active+clean; 23 GiB data, 74 GiB used, 2.7 TiB / 2.7 TiB avail; 0 B/s wr, 7.25k op/s 2026-02-24T07:48:10.905 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:48:10 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/114136' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2026-02-24T07:48:12.808 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2026-02-24T07:48:12.808 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2026-02-24T07:48:12.808 INFO:tasks.thrashosds.thrasher:fixing pg num pool unique_pool_1 2026-02-24T07:48:12.808 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph pg dump --format=json 2026-02-24T07:48:12.931 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2026-02-24T07:48:12.948 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:48:13.116 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:48:12 trial035 ceph-mon[39284]: pgmap v179: 17 pgs: 17 active+clean; 20 GiB data, 67 GiB used, 2.7 TiB / 2.7 TiB avail; 0 B/s wr, 10.98k op/s 2026-02-24T07:48:13.195 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:48:13.396 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:48:13.374+0000 7fab524f8640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:48:13.397 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:48:13.375+0000 7fab524f8640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:48:13.516 INFO:teuthology.orchestra.run.trial035.stdout: 2026-02-24T07:48:13.516 INFO:teuthology.orchestra.run.trial035.stderr:dumped all 2026-02-24T07:48:13.938 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:48:13.885+0000 7f1575545640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:48:13.939 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:48:13.885+0000 7f1575545640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:48:13.939 INFO:teuthology.orchestra.run.trial035.stderr:osd.0: osd_enable_op_tracker = '' 2026-02-24T07:48:13.939 INFO:teuthology.orchestra.run.trial035.stderr:osd.1: osd_enable_op_tracker = '' 2026-02-24T07:48:13.939 INFO:teuthology.orchestra.run.trial035.stdout:osd.0: {} 2026-02-24T07:48:13.939 INFO:teuthology.orchestra.run.trial035.stdout:osd.1: {} 2026-02-24T07:48:13.940 INFO:teuthology.orchestra.run.trial035.stdout:osd.2: {} 2026-02-24T07:48:13.940 INFO:teuthology.orchestra.run.trial035.stderr:osd.2: osd_enable_op_tracker = '' 2026-02-24T07:48:13.948 INFO:teuthology.orchestra.run.trial035.stdout:osd.3: {} 2026-02-24T07:48:13.948 INFO:teuthology.orchestra.run.trial035.stderr:osd.3: osd_enable_op_tracker = '' 2026-02-24T07:48:14.065 INFO:teuthology.orchestra.run.trial035.stdout:{"pg_ready":true,"pg_map":{"version":179,"stamp":"2026-02-24T07:48:11.699610+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":21840265784,"num_objects":333250,"num_object_clones":0,"num_object_copies":999750,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":333250,"num_whiteouts":0,"num_read":137,"num_read_kb":241,"num_write":993903,"num_write_kb":42466599,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":51,"ondisk_log_size":51,"up":51,"acting":51,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":51,"num_osds":4,"num_per_pool_osds":4,"num_per_pool_omap_osds":4,"kb":2936012800,"kb_used":69772584,"kb_used_data":64560920,"kb_used_omap":40,"kb_used_meta":5211351,"kb_avail":2866240216,"statfs":{"total":3006477107200,"available":2935029981184,"internally_reserved":0,"allocated":66110382080,"data_stored":66109093448,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":41378,"internal_metadata":5336424030},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,1],"upper_bound":4},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-8635940864,"num_objects":-131774,"num_object_clones":0,"num_object_copies":-395322,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-131774,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":131774,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001525"},"pg_stats":[{"pgid":"3.f","version":"49'61972","reported_seq":62342,"reported_epoch":49,"state":"active+clean","last_fresh":"2026-02-24T07:48:10.701108+0000","last_change":"2026-02-24T07:47:16.466592+0000","last_active":"2026-02-24T07:48:10.701108+0000","last_peered":"2026-02-24T07:48:10.701108+0000","last_clean":"2026-02-24T07:48:10.701108+0000","last_became_active":"2026-02-24T07:46:02.881625+0000","last_became_peered":"2026-02-24T07:46:02.881625+0000","last_unstale":"2026-02-24T07:48:10.701108+0000","last_undegraded":"2026-02-24T07:48:10.701108+0000","last_fullsized":"2026-02-24T07:48:10.701108+0000","mapping_epoch":42,"log_start":"49'61969","ondisk_log_start":"49'61969","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"45'34153","last_scrub_stamp":"2026-02-24T07:47:16.466530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_clean_scrub_stamp":"2026-02-24T07:47:16.466530+0000","objects_scrubbed":33463,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:48:23.487779+0000","scrub_duration":3074,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1352531968,"num_objects":20638,"num_object_clones":0,"num_object_copies":61914,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20638,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":61972,"num_write_kb":2643520,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.e","version":"49'62789","reported_seq":63231,"reported_epoch":49,"state":"active+clean","last_fresh":"2026-02-24T07:48:10.701459+0000","last_change":"2026-02-24T07:47:33.099762+0000","last_active":"2026-02-24T07:48:10.701459+0000","last_peered":"2026-02-24T07:48:10.701459+0000","last_clean":"2026-02-24T07:48:10.701459+0000","last_became_active":"2026-02-24T07:46:02.881083+0000","last_became_peered":"2026-02-24T07:46:02.881083+0000","last_unstale":"2026-02-24T07:48:10.701459+0000","last_undegraded":"2026-02-24T07:48:10.701459+0000","last_fullsized":"2026-02-24T07:48:10.701459+0000","mapping_epoch":42,"log_start":"49'62786","ondisk_log_start":"49'62786","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"46'41533","last_scrub_stamp":"2026-02-24T07:47:33.099675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_clean_scrub_stamp":"2026-02-24T07:47:33.099675+0000","objects_scrubbed":40754,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:48:53.832432+0000","scrub_duration":3969,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1376714752,"num_objects":21007,"num_object_clones":0,"num_object_copies":63021,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21007,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":62789,"num_write_kb":2681472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"49'62227","reported_seq":62559,"reported_epoch":49,"state":"active+clean","last_fresh":"2026-02-24T07:48:10.700748+0000","last_change":"2026-02-24T07:47:06.968813+0000","last_active":"2026-02-24T07:48:10.700748+0000","last_peered":"2026-02-24T07:48:10.700748+0000","last_clean":"2026-02-24T07:48:10.700748+0000","last_became_active":"2026-02-24T07:46:02.883799+0000","last_became_peered":"2026-02-24T07:46:02.883799+0000","last_unstale":"2026-02-24T07:48:10.700748+0000","last_undegraded":"2026-02-24T07:48:10.700748+0000","last_fullsized":"2026-02-24T07:48:10.700748+0000","mapping_epoch":42,"log_start":"49'62224","ondisk_log_start":"49'62224","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"45'30090","last_scrub_stamp":"2026-02-24T07:47:06.968732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_clean_scrub_stamp":"2026-02-24T07:47:06.968732+0000","objects_scrubbed":29542,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:48:25.198085+0000","scrub_duration":3070,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1366687768,"num_objects":20855,"num_object_clones":0,"num_object_copies":62565,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20855,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":62227,"num_write_kb":2658561,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"49'61165","reported_seq":61495,"reported_epoch":49,"state":"active+clean","last_fresh":"2026-02-24T07:48:10.693225+0000","last_change":"2026-02-24T07:47:07.546834+0000","last_active":"2026-02-24T07:48:10.693225+0000","last_peered":"2026-02-24T07:48:10.693225+0000","last_clean":"2026-02-24T07:48:10.693225+0000","last_became_active":"2026-02-24T07:46:02.880060+0000","last_became_peered":"2026-02-24T07:46:02.880060+0000","last_unstale":"2026-02-24T07:48:10.693225+0000","last_undegraded":"2026-02-24T07:48:10.693225+0000","last_fullsized":"2026-02-24T07:48:10.693225+0000","mapping_epoch":42,"log_start":"49'61162","ondisk_log_start":"49'61162","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"45'29910","last_scrub_stamp":"2026-02-24T07:47:07.546803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_clean_scrub_stamp":"2026-02-24T07:47:07.546803+0000","objects_scrubbed":29435,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:48:27.524515+0000","scrub_duration":2696,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1341980672,"num_objects":20477,"num_object_clones":0,"num_object_copies":61431,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20477,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":61165,"num_write_kb":2612544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"49'62136","reported_seq":62477,"reported_epoch":49,"state":"active+clean","last_fresh":"2026-02-24T07:48:10.776081+0000","last_change":"2026-02-24T07:47:10.376412+0000","last_active":"2026-02-24T07:48:10.776081+0000","last_peered":"2026-02-24T07:48:10.776081+0000","last_clean":"2026-02-24T07:48:10.776081+0000","last_became_active":"2026-02-24T07:46:02.880171+0000","last_became_peered":"2026-02-24T07:46:02.880171+0000","last_unstale":"2026-02-24T07:48:10.776081+0000","last_undegraded":"2026-02-24T07:48:10.776081+0000","last_fullsized":"2026-02-24T07:48:10.776081+0000","mapping_epoch":42,"log_start":"49'62133","ondisk_log_start":"49'62133","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"45'31519","last_scrub_stamp":"2026-02-24T07:47:10.376345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_clean_scrub_stamp":"2026-02-24T07:47:10.376345+0000","objects_scrubbed":30810,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:48:13.377288+0000","scrub_duration":4056,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1362886656,"num_objects":20796,"num_object_clones":0,"num_object_copies":62388,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20796,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":62136,"num_write_kb":2653824,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.a","version":"49'61604","reported_seq":62046,"reported_epoch":49,"state":"active+clean","last_fresh":"2026-02-24T07:48:10.702870+0000","last_change":"2026-02-24T07:47:35.439589+0000","last_active":"2026-02-24T07:48:10.702870+0000","last_peered":"2026-02-24T07:48:10.702870+0000","last_clean":"2026-02-24T07:48:10.702870+0000","last_became_active":"2026-02-24T07:46:02.882274+0000","last_became_peered":"2026-02-24T07:46:02.882274+0000","last_unstale":"2026-02-24T07:48:10.702870+0000","last_undegraded":"2026-02-24T07:48:10.702870+0000","last_fullsized":"2026-02-24T07:48:10.702870+0000","mapping_epoch":42,"log_start":"49'61601","ondisk_log_start":"49'61601","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"46'41859","last_scrub_stamp":"2026-02-24T07:47:35.439542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_clean_scrub_stamp":"2026-02-24T07:47:35.439542+0000","objects_scrubbed":40699,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:48:38.034606+0000","scrub_duration":3645,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1348861952,"num_objects":20582,"num_object_clones":0,"num_object_copies":61746,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20582,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":61604,"num_write_kb":2629952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"49'62781","reported_seq":63130,"reported_epoch":49,"state":"active+clean","last_fresh":"2026-02-24T07:48:10.697870+0000","last_change":"2026-02-24T07:47:10.197277+0000","last_active":"2026-02-24T07:48:10.697870+0000","last_peered":"2026-02-24T07:48:10.697870+0000","last_clean":"2026-02-24T07:48:10.697870+0000","last_became_active":"2026-02-24T07:46:02.881951+0000","last_became_peered":"2026-02-24T07:46:02.881951+0000","last_unstale":"2026-02-24T07:48:10.697870+0000","last_undegraded":"2026-02-24T07:48:10.697870+0000","last_fullsized":"2026-02-24T07:48:10.697870+0000","mapping_epoch":42,"log_start":"49'62778","ondisk_log_start":"49'62778","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"45'31847","last_scrub_stamp":"2026-02-24T07:47:10.197239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_clean_scrub_stamp":"2026-02-24T07:47:10.197239+0000","objects_scrubbed":31279,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:48:27.205650+0000","scrub_duration":3289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1381695488,"num_objects":21083,"num_object_clones":0,"num_object_copies":63249,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21083,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":62781,"num_write_kb":2683648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"49'62271","reported_seq":62666,"reported_epoch":49,"state":"active+clean","last_fresh":"2026-02-24T07:48:10.774680+0000","last_change":"2026-02-24T07:47:22.289650+0000","last_active":"2026-02-24T07:48:10.774680+0000","last_peered":"2026-02-24T07:48:10.774680+0000","last_clean":"2026-02-24T07:48:10.774680+0000","last_became_active":"2026-02-24T07:46:02.881345+0000","last_became_peered":"2026-02-24T07:46:02.881345+0000","last_unstale":"2026-02-24T07:48:10.774680+0000","last_undegraded":"2026-02-24T07:48:10.774680+0000","last_fullsized":"2026-02-24T07:48:10.774680+0000","mapping_epoch":42,"log_start":"49'62268","ondisk_log_start":"49'62268","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"45'36703","last_scrub_stamp":"2026-02-24T07:47:22.289617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_clean_scrub_stamp":"2026-02-24T07:47:22.289617+0000","objects_scrubbed":35960,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:48:31.332102+0000","scrub_duration":3891,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1373044736,"num_objects":20951,"num_object_clones":0,"num_object_copies":62853,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20951,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":62271,"num_write_kb":2663104,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"49'61722","reported_seq":62166,"reported_epoch":49,"state":"active+clean","last_fresh":"2026-02-24T07:48:10.772054+0000","last_change":"2026-02-24T07:47:35.925072+0000","last_active":"2026-02-24T07:48:10.772054+0000","last_peered":"2026-02-24T07:48:10.772054+0000","last_clean":"2026-02-24T07:48:10.772054+0000","last_became_active":"2026-02-24T07:46:02.880323+0000","last_became_peered":"2026-02-24T07:46:02.880323+0000","last_unstale":"2026-02-24T07:48:10.772054+0000","last_undegraded":"2026-02-24T07:48:10.772054+0000","last_fullsized":"2026-02-24T07:48:10.772054+0000","mapping_epoch":42,"log_start":"49'61719","ondisk_log_start":"49'61719","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"47'42248","last_scrub_stamp":"2026-02-24T07:47:35.925000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_clean_scrub_stamp":"2026-02-24T07:47:35.925000+0000","objects_scrubbed":40858,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:48:43.057825+0000","scrub_duration":3382,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1360134144,"num_objects":20754,"num_object_clones":0,"num_object_copies":62262,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20754,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":61722,"num_write_kb":2639232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.6","version":"49'62297","reported_seq":62697,"reported_epoch":49,"state":"active+clean","last_fresh":"2026-02-24T07:48:10.693762+0000","last_change":"2026-02-24T07:47:23.432795+0000","last_active":"2026-02-24T07:48:10.693762+0000","last_peered":"2026-02-24T07:48:10.693762+0000","last_clean":"2026-02-24T07:48:10.693762+0000","last_became_active":"2026-02-24T07:46:02.880005+0000","last_became_peered":"2026-02-24T07:46:02.880005+0000","last_unstale":"2026-02-24T07:48:10.693762+0000","last_undegraded":"2026-02-24T07:48:10.693762+0000","last_fullsized":"2026-02-24T07:48:10.693762+0000","mapping_epoch":42,"log_start":"49'62294","ondisk_log_start":"49'62294","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"45'37219","last_scrub_stamp":"2026-02-24T07:47:23.432744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_clean_scrub_stamp":"2026-02-24T07:47:23.432744+0000","objects_scrubbed":36510,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:48:51.822832+0000","scrub_duration":3412,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1358888960,"num_objects":20735,"num_object_clones":0,"num_object_copies":62205,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20735,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":62297,"num_write_kb":2657024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"49'62585","reported_seq":62962,"reported_epoch":49,"state":"active+clean","last_fresh":"2026-02-24T07:48:10.692816+0000","last_change":"2026-02-24T07:47:17.546625+0000","last_active":"2026-02-24T07:48:10.692816+0000","last_peered":"2026-02-24T07:48:10.692816+0000","last_clean":"2026-02-24T07:48:10.692816+0000","last_became_active":"2026-02-24T07:46:02.881420+0000","last_became_peered":"2026-02-24T07:46:02.881420+0000","last_unstale":"2026-02-24T07:48:10.692816+0000","last_undegraded":"2026-02-24T07:48:10.692816+0000","last_fullsized":"2026-02-24T07:48:10.692816+0000","mapping_epoch":42,"log_start":"49'62582","ondisk_log_start":"49'62582","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"45'34854","last_scrub_stamp":"2026-02-24T07:47:17.546590+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_clean_scrub_stamp":"2026-02-24T07:47:17.546590+0000","objects_scrubbed":34222,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:48:38.813192+0000","scrub_duration":3116,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1380777984,"num_objects":21069,"num_object_clones":0,"num_object_copies":63207,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21069,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":62585,"num_write_kb":2676928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.4","version":"49'61556","reported_seq":61995,"reported_epoch":49,"state":"active+clean","last_fresh":"2026-02-24T07:48:10.774583+0000","last_change":"2026-02-24T07:47:33.628245+0000","last_active":"2026-02-24T07:48:10.774583+0000","last_peered":"2026-02-24T07:48:10.774583+0000","last_clean":"2026-02-24T07:48:10.774583+0000","last_became_active":"2026-02-24T07:46:02.880345+0000","last_became_peered":"2026-02-24T07:46:02.880345+0000","last_unstale":"2026-02-24T07:48:10.774583+0000","last_undegraded":"2026-02-24T07:48:10.774583+0000","last_fullsized":"2026-02-24T07:48:10.774583+0000","mapping_epoch":42,"log_start":"49'61553","ondisk_log_start":"49'61553","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"46'41076","last_scrub_stamp":"2026-02-24T07:47:33.628218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_clean_scrub_stamp":"2026-02-24T07:47:33.628218+0000","objects_scrubbed":40364,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:48:34.035175+0000","scrub_duration":4080,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1367605248,"num_objects":20868,"num_object_clones":0,"num_object_copies":62604,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20868,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":61556,"num_write_kb":2637568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"49'62164","reported_seq":62527,"reported_epoch":49,"state":"active+clean","last_fresh":"2026-02-24T07:48:10.701358+0000","last_change":"2026-02-24T07:47:14.339634+0000","last_active":"2026-02-24T07:48:10.701358+0000","last_peered":"2026-02-24T07:48:10.701358+0000","last_clean":"2026-02-24T07:48:10.701358+0000","last_became_active":"2026-02-24T07:46:02.881035+0000","last_became_peered":"2026-02-24T07:46:02.881035+0000","last_unstale":"2026-02-24T07:48:10.701358+0000","last_undegraded":"2026-02-24T07:48:10.701358+0000","last_fullsized":"2026-02-24T07:48:10.701358+0000","mapping_epoch":42,"log_start":"49'62161","ondisk_log_start":"49'62161","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"45'33283","last_scrub_stamp":"2026-02-24T07:47:14.339597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_clean_scrub_stamp":"2026-02-24T07:47:14.339597+0000","objects_scrubbed":32630,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:48:22.078719+0000","scrub_duration":2951,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1376649216,"num_objects":21006,"num_object_clones":0,"num_object_copies":63018,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21006,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":62164,"num_write_kb":2661440,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"49'62122","reported_seq":62523,"reported_epoch":49,"state":"active+clean","last_fresh":"2026-02-24T07:48:10.727550+0000","last_change":"2026-02-24T07:47:24.807734+0000","last_active":"2026-02-24T07:48:10.727550+0000","last_peered":"2026-02-24T07:48:10.727550+0000","last_clean":"2026-02-24T07:48:10.727550+0000","last_became_active":"2026-02-24T07:46:02.879831+0000","last_became_peered":"2026-02-24T07:46:02.879831+0000","last_unstale":"2026-02-24T07:48:10.727550+0000","last_undegraded":"2026-02-24T07:48:10.727550+0000","last_fullsized":"2026-02-24T07:48:10.727550+0000","mapping_epoch":42,"log_start":"49'62119","ondisk_log_start":"49'62119","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"45'37558","last_scrub_stamp":"2026-02-24T07:47:24.807659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_clean_scrub_stamp":"2026-02-24T07:47:24.807659+0000","objects_scrubbed":36869,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:48:39.172465+0000","scrub_duration":3748,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1356595200,"num_objects":20700,"num_object_clones":0,"num_object_copies":62100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20700,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":62122,"num_write_kb":2650304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.0","version":"49'61862","reported_seq":62197,"reported_epoch":49,"state":"active+clean","last_fresh":"2026-02-24T07:48:10.773455+0000","last_change":"2026-02-24T07:47:08.376360+0000","last_active":"2026-02-24T07:48:10.773455+0000","last_peered":"2026-02-24T07:48:10.773455+0000","last_clean":"2026-02-24T07:48:10.773455+0000","last_became_active":"2026-02-24T07:46:02.880333+0000","last_became_peered":"2026-02-24T07:46:02.880333+0000","last_unstale":"2026-02-24T07:48:10.773455+0000","last_undegraded":"2026-02-24T07:48:10.773455+0000","last_fullsized":"2026-02-24T07:48:10.773455+0000","mapping_epoch":42,"log_start":"49'61859","ondisk_log_start":"49'61859","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"45'30511","last_scrub_stamp":"2026-02-24T07:47:08.376304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_clean_scrub_stamp":"2026-02-24T07:47:08.376304+0000","objects_scrubbed":29912,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:48:24.552234+0000","scrub_duration":3049,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1367343104,"num_objects":20864,"num_object_clones":0,"num_object_copies":62592,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20864,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":61862,"num_write_kb":2647232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2","version":"49'62515","reported_seq":62932,"reported_epoch":49,"state":"active+clean","last_fresh":"2026-02-24T07:48:10.775820+0000","last_change":"2026-02-24T07:47:27.784936+0000","last_active":"2026-02-24T07:48:10.775820+0000","last_peered":"2026-02-24T07:48:10.775820+0000","last_clean":"2026-02-24T07:48:10.775820+0000","last_became_active":"2026-02-24T07:46:02.880185+0000","last_became_peered":"2026-02-24T07:46:02.880185+0000","last_unstale":"2026-02-24T07:48:10.775820+0000","last_undegraded":"2026-02-24T07:48:10.775820+0000","last_fullsized":"2026-02-24T07:48:10.775820+0000","mapping_epoch":42,"log_start":"49'62512","ondisk_log_start":"49'62512","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"45'39074","last_scrub_stamp":"2026-02-24T07:47:27.784888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_clean_scrub_stamp":"2026-02-24T07:47:27.784888+0000","objects_scrubbed":38200,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:48:47.063599+0000","scrub_duration":4266,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1367277568,"num_objects":20863,"num_object_clones":0,"num_object_copies":62589,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20863,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":62515,"num_write_kb":2668096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"29'99","reported_seq":253,"reported_epoch":49,"state":"active+clean","last_fresh":"2026-02-24T07:48:02.589664+0000","last_change":"2026-02-24T07:48:02.589664+0000","last_active":"2026-02-24T07:48:02.589664+0000","last_peered":"2026-02-24T07:48:02.589664+0000","last_clean":"2026-02-24T07:48:02.589664+0000","last_became_active":"2026-02-24T07:42:54.186719+0000","last_became_peered":"2026-02-24T07:42:54.186719+0000","last_unstale":"2026-02-24T07:48:02.589664+0000","last_undegraded":"2026-02-24T07:48:02.589664+0000","last_fullsized":"2026-02-24T07:48:02.589664+0000","mapping_epoch":27,"log_start":"29'96","ondisk_log_start":"29'96","created":20,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'99","last_scrub_stamp":"2026-02-24T07:48:02.589609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:42:43.640776+0000","last_clean_scrub_stamp":"2026-02-24T07:48:02.589609+0000","objects_scrubbed":2,"log_size":3,"log_dups_size":96,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:49:04.391555+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":240,"num_write":135,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":16,"stat_sum":{"num_bytes":21839675416,"num_objects":333248,"num_object_clones":0,"num_object_copies":999744,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":333248,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":993768,"num_write_kb":42464449,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65527558144,"data_stored":65527545928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":48,"ondisk_log_size":48,"up":48,"acting":48,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":240,"num_write":135,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1896,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":3,"up_from":25,"seq":107374182466,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":14931132,"kb_used_data":13351784,"kb_used_omap":10,"kb_used_meta":1579253,"kb_avail":719072068,"statfs":{"total":751619276800,"available":736329797632,"internally_reserved":0,"allocated":13672226816,"data_stored":13671902752,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":10771,"internal_metadata":1617155565},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,1],"upper_bound":4},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":19,"seq":81604378692,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":20449020,"kb_used_data":18916648,"kb_used_omap":10,"kb_used_meta":1532213,"kb_avail":713554180,"statfs":{"total":751619276800,"available":730679480320,"internally_reserved":0,"allocated":19370647552,"data_stored":19370323488,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":10589,"internal_metadata":1568986787},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542213,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":15402932,"kb_used_data":14672480,"kb_used_omap":11,"kb_used_meta":730420,"kb_avail":718600268,"statfs":{"total":751619276800,"available":735846674432,"internally_reserved":0,"allocated":15024619520,"data_stored":15024303080,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":11936,"internal_metadata":747950432},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705736,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":18989500,"kb_used_data":17620008,"kb_used_omap":7,"kb_used_meta":1369464,"kb_avail":715013700,"statfs":{"total":751619276800,"available":732174028800,"internally_reserved":0,"allocated":18042888192,"data_stored":18042564128,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":8082,"internal_metadata":1402331246},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1896,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":17755148288,"data_stored":17755144216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":14970978304,"data_stored":14970978304,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":19145822208,"data_stored":19145818136,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":13655609344,"data_stored":13655605272,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-24T07:48:14.069 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph osd pool set unique_pool_1 pgp_num 46 2026-02-24T07:48:14.219 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:48:14.349 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:48:14.346+0000 7f44ceaea640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:48:14.349 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:48:14.346+0000 7f44ceaea640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:48:14.494 INFO:teuthology.orchestra.run.trial035.stderr:Error EPERM: crimson_allow_pg_split is false; pgp_num increase denied 2026-02-24T07:48:14.531 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-24T07:48:15.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:48:14 trial035 ceph-mon[39284]: from='client.14786 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2026-02-24T07:48:15.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:48:14 trial035 ceph-mon[39284]: pgmap v180: 17 pgs: 17 active+clean; 20 GiB data, 67 GiB used, 2.7 TiB / 2.7 TiB avail; 0 B/s wr, 7.37k op/s 2026-02-24T07:48:15.205 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:48:14 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/628443930' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2026-02-24T07:48:15.205 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:48:14 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/2437431957' entity='client.admin' cmd={"prefix": "osd pool set", "pool": "unique_pool_1", "var": "pgp_num", "val": "46"} : dispatch 2026-02-24T07:48:16.094 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2026-02-24T07:48:16.119 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:48:15 trial035 ceph-mon[39284]: 3.b scrub starts 2026-02-24T07:48:16.119 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:48:15 trial035 ceph-mon[39284]: from='mgr.14284 10.20.193.35:0/3681373222' entity='mgr.x' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2026-02-24T07:48:16.240 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:48:16.371 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:48:16.368+0000 7f35f33cf640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:48:16.371 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:48:16.369+0000 7f35f33cf640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:48:16.386 INFO:teuthology.orchestra.run.trial035.stdout:osd.0: {} 2026-02-24T07:48:16.386 INFO:teuthology.orchestra.run.trial035.stderr:osd.0: osd_enable_op_tracker = '' 2026-02-24T07:48:16.400 INFO:teuthology.orchestra.run.trial035.stdout:osd.1: {} 2026-02-24T07:48:16.400 INFO:teuthology.orchestra.run.trial035.stderr:osd.1: osd_enable_op_tracker = '' 2026-02-24T07:48:16.415 INFO:teuthology.orchestra.run.trial035.stdout:osd.2: {} 2026-02-24T07:48:16.416 INFO:teuthology.orchestra.run.trial035.stderr:osd.2: osd_enable_op_tracker = '' 2026-02-24T07:48:16.428 INFO:teuthology.orchestra.run.trial035.stdout:osd.3: {} 2026-02-24T07:48:16.428 INFO:teuthology.orchestra.run.trial035.stderr:osd.3: osd_enable_op_tracker = '' 2026-02-24T07:48:17.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:48:16 trial035 ceph-mon[39284]: pgmap v181: 17 pgs: 17 active+clean; 20 GiB data, 67 GiB used, 2.7 TiB / 2.7 TiB avail; 0 B/s wr, 7.37k op/s 2026-02-24T07:48:17.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:48:16 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/4217537205' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2026-02-24T07:48:17.205 INFO:journalctl@ceph.mgr.x.trial035.stdout:Feb 24 07:48:16 trial035 ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-mgr-x[39621]: ::ffff:10.20.193.91 - - [24/Feb/2026:07:48:16] "GET /metrics HTTP/1.1" 200 4231 "" "Prometheus/3.6.0" 2026-02-24T07:48:17.266 INFO:journalctl@ceph.prometheus.a.trial091.stdout:Feb 24 07:48:16 trial091 ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-prometheus-a[66007]: time=2026-02-24T07:48:16.973Z level=ERROR source=scrape.go:1630 msg="Failed to determine correct type of scrape target." component="scrape manager" scrape_pool=ceph target=http://trial035:9283/metrics content_type="text/html;charset=utf-8" fallback_media_type="" err="received unsupported Content-Type \"text/html;charset=utf-8\" and no fallback_scrape_protocol specified for target" 2026-02-24T07:48:18.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:48:17 trial035 ceph-mon[39284]: 3.b scrub ok 2026-02-24T07:48:18.471 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2026-02-24T07:48:18.614 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:48:18.733 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:48:18.731+0000 7f6b06b64640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:48:18.734 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:48:18.731+0000 7f6b06b64640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:48:18.749 INFO:teuthology.orchestra.run.trial035.stdout:osd.0: {} 2026-02-24T07:48:18.749 INFO:teuthology.orchestra.run.trial035.stderr:osd.0: osd_enable_op_tracker = '' 2026-02-24T07:48:18.763 INFO:teuthology.orchestra.run.trial035.stdout:osd.1: {} 2026-02-24T07:48:18.764 INFO:teuthology.orchestra.run.trial035.stderr:osd.1: osd_enable_op_tracker = '' 2026-02-24T07:48:18.776 INFO:teuthology.orchestra.run.trial035.stdout:osd.2: {} 2026-02-24T07:48:18.777 INFO:teuthology.orchestra.run.trial035.stderr:osd.2: osd_enable_op_tracker = '' 2026-02-24T07:48:18.789 INFO:teuthology.orchestra.run.trial035.stdout:osd.3: {} 2026-02-24T07:48:18.789 INFO:teuthology.orchestra.run.trial035.stderr:osd.3: osd_enable_op_tracker = '' 2026-02-24T07:48:19.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:48:18 trial035 ceph-mon[39284]: pgmap v182: 17 pgs: 1 active+clean+scrubbing, 16 active+clean; 18 GiB data, 60 GiB used, 2.7 TiB / 2.7 TiB avail; 0 B/s wr, 10.84k op/s 2026-02-24T07:48:19.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:48:18 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/654644129' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2026-02-24T07:48:19.532 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2026-02-24T07:48:19.532 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2026-02-24T07:48:19.533 INFO:tasks.thrashosds.thrasher:Growing pool .mgr 2026-02-24T07:48:19.533 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph pg dump --format=json 2026-02-24T07:48:19.673 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:48:20.177 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:48:20.142+0000 7f11080c2640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:48:20.178 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:48:20.142+0000 7f11080c2640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:48:20.276 INFO:teuthology.orchestra.run.trial035.stdout: 2026-02-24T07:48:20.277 INFO:teuthology.orchestra.run.trial035.stderr:dumped all 2026-02-24T07:48:20.710 INFO:teuthology.orchestra.run.trial035.stdout:{"pg_ready":true,"pg_map":{"version":183,"stamp":"2026-02-24T07:48:19.700695+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19112723000,"num_objects":291631,"num_object_clones":0,"num_object_copies":874893,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":291631,"num_whiteouts":0,"num_read":137,"num_read_kb":241,"num_write":1035522,"num_write_kb":42466599,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":53,"ondisk_log_size":53,"up":51,"acting":51,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":51,"num_osds":4,"num_per_pool_osds":4,"num_per_pool_omap_osds":4,"kb":2936012800,"kb_used":62403560,"kb_used_data":56975768,"kb_used_omap":40,"kb_used_meta":5427543,"kb_avail":2873609240,"statfs":{"total":3006477107200,"available":2942575861760,"internally_reserved":0,"allocated":58343186432,"data_stored":58341897800,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":41471,"internal_metadata":5557804545},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[11,1],"upper_bound":4},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-5662113792,"num_objects":-86397,"num_object_clones":0,"num_object_copies":-259191,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-86397,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":86397,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001618"},"pg_stats":[{"pgid":"3.f","version":"49'64524","reported_seq":64894,"reported_epoch":49,"state":"active+clean","last_fresh":"2026-02-24T07:48:15.696772+0000","last_change":"2026-02-24T07:47:16.466592+0000","last_active":"2026-02-24T07:48:15.696772+0000","last_peered":"2026-02-24T07:48:15.696772+0000","last_clean":"2026-02-24T07:48:15.696772+0000","last_became_active":"2026-02-24T07:46:02.881625+0000","last_became_peered":"2026-02-24T07:46:02.881625+0000","last_unstale":"2026-02-24T07:48:15.696772+0000","last_undegraded":"2026-02-24T07:48:15.696772+0000","last_fullsized":"2026-02-24T07:48:15.696772+0000","mapping_epoch":42,"log_start":"49'64521","ondisk_log_start":"49'64521","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"45'34153","last_scrub_stamp":"2026-02-24T07:47:16.466530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_clean_scrub_stamp":"2026-02-24T07:47:16.466530+0000","objects_scrubbed":33463,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:48:23.487779+0000","scrub_duration":3074,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1185284096,"num_objects":18086,"num_object_clones":0,"num_object_copies":54258,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18086,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":64524,"num_write_kb":2643520,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.e","version":"49'65420","reported_seq":65862,"reported_epoch":49,"state":"active+clean","last_fresh":"2026-02-24T07:48:15.702066+0000","last_change":"2026-02-24T07:47:33.099762+0000","last_active":"2026-02-24T07:48:15.702066+0000","last_peered":"2026-02-24T07:48:15.702066+0000","last_clean":"2026-02-24T07:48:15.702066+0000","last_became_active":"2026-02-24T07:46:02.881083+0000","last_became_peered":"2026-02-24T07:46:02.881083+0000","last_unstale":"2026-02-24T07:48:15.702066+0000","last_undegraded":"2026-02-24T07:48:15.702066+0000","last_fullsized":"2026-02-24T07:48:15.702066+0000","mapping_epoch":42,"log_start":"49'65417","ondisk_log_start":"49'65417","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"46'41533","last_scrub_stamp":"2026-02-24T07:47:33.099675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_clean_scrub_stamp":"2026-02-24T07:47:33.099675+0000","objects_scrubbed":40754,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:48:53.832432+0000","scrub_duration":3969,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1204289536,"num_objects":18376,"num_object_clones":0,"num_object_copies":55128,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18376,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":65420,"num_write_kb":2681472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"49'64745","reported_seq":65077,"reported_epoch":49,"state":"active+clean","last_fresh":"2026-02-24T07:48:15.719818+0000","last_change":"2026-02-24T07:47:06.968813+0000","last_active":"2026-02-24T07:48:15.719818+0000","last_peered":"2026-02-24T07:48:15.719818+0000","last_clean":"2026-02-24T07:48:15.719818+0000","last_became_active":"2026-02-24T07:46:02.883799+0000","last_became_peered":"2026-02-24T07:46:02.883799+0000","last_unstale":"2026-02-24T07:48:15.719818+0000","last_undegraded":"2026-02-24T07:48:15.719818+0000","last_fullsized":"2026-02-24T07:48:15.719818+0000","mapping_epoch":42,"log_start":"49'64742","ondisk_log_start":"49'64742","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"45'30090","last_scrub_stamp":"2026-02-24T07:47:06.968732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_clean_scrub_stamp":"2026-02-24T07:47:06.968732+0000","objects_scrubbed":29542,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:48:25.198085+0000","scrub_duration":3070,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1201668120,"num_objects":18337,"num_object_clones":0,"num_object_copies":55011,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18337,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":64745,"num_write_kb":2658561,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"49'63730","reported_seq":64060,"reported_epoch":49,"state":"active+clean","last_fresh":"2026-02-24T07:48:15.721686+0000","last_change":"2026-02-24T07:47:07.546834+0000","last_active":"2026-02-24T07:48:15.721686+0000","last_peered":"2026-02-24T07:48:15.721686+0000","last_clean":"2026-02-24T07:48:15.721686+0000","last_became_active":"2026-02-24T07:46:02.880060+0000","last_became_peered":"2026-02-24T07:46:02.880060+0000","last_unstale":"2026-02-24T07:48:15.721686+0000","last_undegraded":"2026-02-24T07:48:15.721686+0000","last_fullsized":"2026-02-24T07:48:15.721686+0000","mapping_epoch":42,"log_start":"49'63727","ondisk_log_start":"49'63727","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"45'29910","last_scrub_stamp":"2026-02-24T07:47:07.546803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_clean_scrub_stamp":"2026-02-24T07:47:07.546803+0000","objects_scrubbed":29435,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:48:27.524515+0000","scrub_duration":2696,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1173880832,"num_objects":17912,"num_object_clones":0,"num_object_copies":53736,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17912,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":63730,"num_write_kb":2612544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"49'64696","reported_seq":65149,"reported_epoch":49,"state":"active+clean+scrubbing","last_fresh":"2026-02-24T07:48:15.776636+0000","last_change":"2026-02-24T07:48:14.695984+0000","last_active":"2026-02-24T07:48:15.776636+0000","last_peered":"2026-02-24T07:48:15.776636+0000","last_clean":"2026-02-24T07:48:15.776636+0000","last_became_active":"2026-02-24T07:46:02.880171+0000","last_became_peered":"2026-02-24T07:46:02.880171+0000","last_unstale":"2026-02-24T07:48:15.776636+0000","last_undegraded":"2026-02-24T07:48:15.776636+0000","last_fullsized":"2026-02-24T07:48:15.776636+0000","mapping_epoch":42,"log_start":"49'64691","ondisk_log_start":"49'64691","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"45'31519","last_scrub_stamp":"2026-02-24T07:47:10.376345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_clean_scrub_stamp":"2026-02-24T07:47:10.376345+0000","objects_scrubbed":11049,"log_size":5,"log_dups_size":3000,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"scrubbing for 2s","scrub_duration":4056,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1195114496,"num_objects":18236,"num_object_clones":0,"num_object_copies":54708,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18236,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":64696,"num_write_kb":2653824,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.a","version":"49'64205","reported_seq":64648,"reported_epoch":49,"state":"active+clean","last_fresh":"2026-02-24T07:48:15.724342+0000","last_change":"2026-02-24T07:47:35.439589+0000","last_active":"2026-02-24T07:48:15.724342+0000","last_peered":"2026-02-24T07:48:15.724342+0000","last_clean":"2026-02-24T07:48:15.724342+0000","last_became_active":"2026-02-24T07:46:02.882274+0000","last_became_peered":"2026-02-24T07:46:02.882274+0000","last_unstale":"2026-02-24T07:48:15.724342+0000","last_undegraded":"2026-02-24T07:48:15.724342+0000","last_fullsized":"2026-02-24T07:48:15.724342+0000","mapping_epoch":42,"log_start":"49'64202","ondisk_log_start":"49'64202","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"46'41859","last_scrub_stamp":"2026-02-24T07:47:35.439542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_clean_scrub_stamp":"2026-02-24T07:47:35.439542+0000","objects_scrubbed":40699,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:48:38.034606+0000","scrub_duration":3645,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1178402816,"num_objects":17981,"num_object_clones":0,"num_object_copies":53943,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17981,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":64205,"num_write_kb":2629952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"49'65488","reported_seq":65838,"reported_epoch":49,"state":"active+clean","last_fresh":"2026-02-24T07:48:15.721756+0000","last_change":"2026-02-24T07:47:10.197277+0000","last_active":"2026-02-24T07:48:15.721756+0000","last_peered":"2026-02-24T07:48:15.721756+0000","last_clean":"2026-02-24T07:48:15.721756+0000","last_became_active":"2026-02-24T07:46:02.881951+0000","last_became_peered":"2026-02-24T07:46:02.881951+0000","last_unstale":"2026-02-24T07:48:15.721756+0000","last_undegraded":"2026-02-24T07:48:15.721756+0000","last_fullsized":"2026-02-24T07:48:15.721756+0000","mapping_epoch":42,"log_start":"49'65485","ondisk_log_start":"49'65485","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"45'31847","last_scrub_stamp":"2026-02-24T07:47:10.197239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_clean_scrub_stamp":"2026-02-24T07:47:10.197239+0000","objects_scrubbed":31279,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:48:27.205650+0000","scrub_duration":3289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1204289536,"num_objects":18376,"num_object_clones":0,"num_object_copies":55128,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18376,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":65488,"num_write_kb":2683648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"49'64880","reported_seq":65275,"reported_epoch":49,"state":"active+clean","last_fresh":"2026-02-24T07:48:15.777104+0000","last_change":"2026-02-24T07:47:22.289650+0000","last_active":"2026-02-24T07:48:15.777104+0000","last_peered":"2026-02-24T07:48:15.777104+0000","last_clean":"2026-02-24T07:48:15.777104+0000","last_became_active":"2026-02-24T07:46:02.881345+0000","last_became_peered":"2026-02-24T07:46:02.881345+0000","last_unstale":"2026-02-24T07:48:15.777104+0000","last_undegraded":"2026-02-24T07:48:15.777104+0000","last_fullsized":"2026-02-24T07:48:15.777104+0000","mapping_epoch":42,"log_start":"49'64877","ondisk_log_start":"49'64877","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"45'36703","last_scrub_stamp":"2026-02-24T07:47:22.289617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_clean_scrub_stamp":"2026-02-24T07:47:22.289617+0000","objects_scrubbed":35960,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:48:31.332102+0000","scrub_duration":3891,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1202061312,"num_objects":18342,"num_object_clones":0,"num_object_copies":55026,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18342,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":64880,"num_write_kb":2663104,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"49'64347","reported_seq":64791,"reported_epoch":49,"state":"active+clean","last_fresh":"2026-02-24T07:48:15.777832+0000","last_change":"2026-02-24T07:47:35.925072+0000","last_active":"2026-02-24T07:48:15.777832+0000","last_peered":"2026-02-24T07:48:15.777832+0000","last_clean":"2026-02-24T07:48:15.777832+0000","last_became_active":"2026-02-24T07:46:02.880323+0000","last_became_peered":"2026-02-24T07:46:02.880323+0000","last_unstale":"2026-02-24T07:48:15.777832+0000","last_undegraded":"2026-02-24T07:48:15.777832+0000","last_fullsized":"2026-02-24T07:48:15.777832+0000","mapping_epoch":42,"log_start":"49'64344","ondisk_log_start":"49'64344","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"47'42248","last_scrub_stamp":"2026-02-24T07:47:35.925000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_clean_scrub_stamp":"2026-02-24T07:47:35.925000+0000","objects_scrubbed":40858,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:48:43.057825+0000","scrub_duration":3382,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1188102144,"num_objects":18129,"num_object_clones":0,"num_object_copies":54387,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18129,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":64347,"num_write_kb":2639232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.6","version":"49'64943","reported_seq":65343,"reported_epoch":49,"state":"active+clean","last_fresh":"2026-02-24T07:48:15.727657+0000","last_change":"2026-02-24T07:47:23.432795+0000","last_active":"2026-02-24T07:48:15.727657+0000","last_peered":"2026-02-24T07:48:15.727657+0000","last_clean":"2026-02-24T07:48:15.727657+0000","last_became_active":"2026-02-24T07:46:02.880005+0000","last_became_peered":"2026-02-24T07:46:02.880005+0000","last_unstale":"2026-02-24T07:48:15.727657+0000","last_undegraded":"2026-02-24T07:48:15.727657+0000","last_fullsized":"2026-02-24T07:48:15.727657+0000","mapping_epoch":42,"log_start":"49'64940","ondisk_log_start":"49'64940","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"45'37219","last_scrub_stamp":"2026-02-24T07:47:23.432744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_clean_scrub_stamp":"2026-02-24T07:47:23.432744+0000","objects_scrubbed":36510,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:48:51.822832+0000","scrub_duration":3412,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1185480704,"num_objects":18089,"num_object_clones":0,"num_object_copies":54267,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18089,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":64943,"num_write_kb":2657024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"49'65158","reported_seq":65535,"reported_epoch":49,"state":"active+clean","last_fresh":"2026-02-24T07:48:15.703181+0000","last_change":"2026-02-24T07:47:17.546625+0000","last_active":"2026-02-24T07:48:15.703181+0000","last_peered":"2026-02-24T07:48:15.703181+0000","last_clean":"2026-02-24T07:48:15.703181+0000","last_became_active":"2026-02-24T07:46:02.881420+0000","last_became_peered":"2026-02-24T07:46:02.881420+0000","last_unstale":"2026-02-24T07:48:15.703181+0000","last_undegraded":"2026-02-24T07:48:15.703181+0000","last_fullsized":"2026-02-24T07:48:15.703181+0000","mapping_epoch":42,"log_start":"49'65155","ondisk_log_start":"49'65155","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"45'34854","last_scrub_stamp":"2026-02-24T07:47:17.546590+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_clean_scrub_stamp":"2026-02-24T07:47:17.546590+0000","objects_scrubbed":34222,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:48:38.813192+0000","scrub_duration":3116,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1212153856,"num_objects":18496,"num_object_clones":0,"num_object_copies":55488,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18496,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":65158,"num_write_kb":2676928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.4","version":"49'64184","reported_seq":64623,"reported_epoch":49,"state":"active+clean","last_fresh":"2026-02-24T07:48:15.771093+0000","last_change":"2026-02-24T07:47:33.628245+0000","last_active":"2026-02-24T07:48:15.771093+0000","last_peered":"2026-02-24T07:48:15.771093+0000","last_clean":"2026-02-24T07:48:15.771093+0000","last_became_active":"2026-02-24T07:46:02.880345+0000","last_became_peered":"2026-02-24T07:46:02.880345+0000","last_unstale":"2026-02-24T07:48:15.771093+0000","last_undegraded":"2026-02-24T07:48:15.771093+0000","last_fullsized":"2026-02-24T07:48:15.771093+0000","mapping_epoch":42,"log_start":"49'64181","ondisk_log_start":"49'64181","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"46'41076","last_scrub_stamp":"2026-02-24T07:47:33.628218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_clean_scrub_stamp":"2026-02-24T07:47:33.628218+0000","objects_scrubbed":40364,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:48:34.035175+0000","scrub_duration":4080,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1195376640,"num_objects":18240,"num_object_clones":0,"num_object_copies":54720,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18240,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":64184,"num_write_kb":2637568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"49'64802","reported_seq":65165,"reported_epoch":49,"state":"active+clean","last_fresh":"2026-02-24T07:48:15.703103+0000","last_change":"2026-02-24T07:47:14.339634+0000","last_active":"2026-02-24T07:48:15.703103+0000","last_peered":"2026-02-24T07:48:15.703103+0000","last_clean":"2026-02-24T07:48:15.703103+0000","last_became_active":"2026-02-24T07:46:02.881035+0000","last_became_peered":"2026-02-24T07:46:02.881035+0000","last_unstale":"2026-02-24T07:48:15.703103+0000","last_undegraded":"2026-02-24T07:48:15.703103+0000","last_fullsized":"2026-02-24T07:48:15.703103+0000","mapping_epoch":42,"log_start":"49'64799","ondisk_log_start":"49'64799","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"45'33283","last_scrub_stamp":"2026-02-24T07:47:14.339597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_clean_scrub_stamp":"2026-02-24T07:47:14.339597+0000","objects_scrubbed":32630,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:48:22.078719+0000","scrub_duration":2951,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1203765248,"num_objects":18368,"num_object_clones":0,"num_object_copies":55104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18368,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":64802,"num_write_kb":2661440,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"49'64736","reported_seq":65140,"reported_epoch":49,"state":"active+clean","last_fresh":"2026-02-24T07:48:15.728539+0000","last_change":"2026-02-24T07:47:24.807734+0000","last_active":"2026-02-24T07:48:15.728539+0000","last_peered":"2026-02-24T07:48:15.728539+0000","last_clean":"2026-02-24T07:48:15.728539+0000","last_became_active":"2026-02-24T07:46:02.879831+0000","last_became_peered":"2026-02-24T07:46:02.879831+0000","last_unstale":"2026-02-24T07:48:15.728539+0000","last_undegraded":"2026-02-24T07:48:15.728539+0000","last_fullsized":"2026-02-24T07:48:15.728539+0000","mapping_epoch":42,"log_start":"49'64733","ondisk_log_start":"49'64733","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"45'37558","last_scrub_stamp":"2026-02-24T07:47:24.807659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_clean_scrub_stamp":"2026-02-24T07:47:24.807659+0000","objects_scrubbed":36869,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:48:39.172465+0000","scrub_duration":3748,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1185284096,"num_objects":18086,"num_object_clones":0,"num_object_copies":54258,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18086,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":64736,"num_write_kb":2650304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.0","version":"49'64445","reported_seq":64779,"reported_epoch":49,"state":"active+clean","last_fresh":"2026-02-24T07:48:15.776306+0000","last_change":"2026-02-24T07:47:08.376360+0000","last_active":"2026-02-24T07:48:15.776306+0000","last_peered":"2026-02-24T07:48:15.776306+0000","last_clean":"2026-02-24T07:48:15.776306+0000","last_became_active":"2026-02-24T07:46:02.880333+0000","last_became_peered":"2026-02-24T07:46:02.880333+0000","last_unstale":"2026-02-24T07:48:15.776306+0000","last_undegraded":"2026-02-24T07:48:15.776306+0000","last_fullsized":"2026-02-24T07:48:15.776306+0000","mapping_epoch":42,"log_start":"49'64442","ondisk_log_start":"49'64442","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"45'30511","last_scrub_stamp":"2026-02-24T07:47:08.376304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_clean_scrub_stamp":"2026-02-24T07:47:08.376304+0000","objects_scrubbed":29912,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:48:24.552234+0000","scrub_duration":3049,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1198063616,"num_objects":18281,"num_object_clones":0,"num_object_copies":54843,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18281,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":64445,"num_write_kb":2647232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2","version":"49'65084","reported_seq":65501,"reported_epoch":49,"state":"active+clean","last_fresh":"2026-02-24T07:48:15.772258+0000","last_change":"2026-02-24T07:47:27.784936+0000","last_active":"2026-02-24T07:48:15.772258+0000","last_peered":"2026-02-24T07:48:15.772258+0000","last_clean":"2026-02-24T07:48:15.772258+0000","last_became_active":"2026-02-24T07:46:02.880185+0000","last_became_peered":"2026-02-24T07:46:02.880185+0000","last_unstale":"2026-02-24T07:48:15.772258+0000","last_undegraded":"2026-02-24T07:48:15.772258+0000","last_fullsized":"2026-02-24T07:48:15.772258+0000","mapping_epoch":42,"log_start":"49'65081","ondisk_log_start":"49'65081","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"45'39074","last_scrub_stamp":"2026-02-24T07:47:27.784888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_clean_scrub_stamp":"2026-02-24T07:47:27.784888+0000","objects_scrubbed":38200,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:48:47.063599+0000","scrub_duration":4266,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1198915584,"num_objects":18294,"num_object_clones":0,"num_object_copies":54882,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18294,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":65084,"num_write_kb":2668096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"29'99","reported_seq":253,"reported_epoch":49,"state":"active+clean","last_fresh":"2026-02-24T07:48:02.589664+0000","last_change":"2026-02-24T07:48:02.589664+0000","last_active":"2026-02-24T07:48:02.589664+0000","last_peered":"2026-02-24T07:48:02.589664+0000","last_clean":"2026-02-24T07:48:02.589664+0000","last_became_active":"2026-02-24T07:42:54.186719+0000","last_became_peered":"2026-02-24T07:42:54.186719+0000","last_unstale":"2026-02-24T07:48:02.589664+0000","last_undegraded":"2026-02-24T07:48:02.589664+0000","last_fullsized":"2026-02-24T07:48:02.589664+0000","mapping_epoch":27,"log_start":"29'96","ondisk_log_start":"29'96","created":20,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'99","last_scrub_stamp":"2026-02-24T07:48:02.589609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:42:43.640776+0000","last_clean_scrub_stamp":"2026-02-24T07:48:02.589609+0000","objects_scrubbed":2,"log_size":3,"log_dups_size":96,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:49:04.391555+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":240,"num_write":135,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":16,"stat_sum":{"num_bytes":19112132632,"num_objects":291629,"num_object_clones":0,"num_object_copies":874887,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":291629,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1035387,"num_write_kb":42464449,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":57352335360,"data_stored":57352323144,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":93,"internal_metadata":0},"log_size":50,"ondisk_log_size":50,"up":48,"acting":48,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":240,"num_write":135,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1896,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":3,"up_from":25,"seq":107374182467,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":13657532,"kb_used_data":12042472,"kb_used_omap":10,"kb_used_meta":1614965,"kb_avail":720345668,"statfs":{"total":751619276800,"available":737633964032,"internally_reserved":0,"allocated":12331491328,"data_stored":12331167264,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":10771,"internal_metadata":1653724653},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":19,"seq":81604378693,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":18206652,"kb_used_data":16602216,"kb_used_omap":10,"kb_used_meta":1604405,"kb_avail":715796548,"statfs":{"total":751619276800,"available":732975665152,"internally_reserved":0,"allocated":17000669184,"data_stored":17000345120,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":10589,"internal_metadata":1642911395},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542214,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":13614708,"kb_used_data":12830048,"kb_used_omap":11,"kb_used_meta":784564,"kb_avail":720388492,"statfs":{"total":751619276800,"available":737677815808,"internally_reserved":0,"allocated":13137969152,"data_stored":13137652712,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12029,"internal_metadata":803393795},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[5,1],"upper_bound":4},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705737,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":16924668,"kb_used_data":15501032,"kb_used_omap":7,"kb_used_meta":1423608,"kb_avail":717078532,"statfs":{"total":751619276800,"available":734288416768,"internally_reserved":0,"allocated":15873056768,"data_stored":15872732704,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":8082,"internal_metadata":1457774702},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1896,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":15537868800,"data_stored":15537864728,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":13093502976,"data_stored":13093502976,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":93,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16769355776,"data_stored":16769351704,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":11951607808,"data_stored":11951603736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-24T07:48:20.712 INFO:tasks.cephadm.ceph_manager.ceph:increase pool size by 10 2026-02-24T07:48:20.712 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph osd pool set .mgr pg_num 191 2026-02-24T07:48:20.833 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2026-02-24T07:48:20.858 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:48:20.976 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:48:20 trial035 ceph-mon[39284]: pgmap v183: 17 pgs: 1 active+clean+scrubbing, 16 active+clean; 18 GiB data, 60 GiB used, 2.7 TiB / 2.7 TiB avail; 0 B/s wr, 7.20k op/s 2026-02-24T07:48:20.977 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:48:20.990 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:48:20.989+0000 7f8d05bbe640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:48:20.990 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:48:20.989+0000 7f8d05bbe640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:48:21.088 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:48:21.087+0000 7f8108904640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:48:21.088 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:48:21.087+0000 7f8108904640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:48:21.106 INFO:teuthology.orchestra.run.trial035.stdout:osd.0: {} 2026-02-24T07:48:21.106 INFO:teuthology.orchestra.run.trial035.stderr:osd.0: osd_enable_op_tracker = '' 2026-02-24T07:48:21.123 INFO:teuthology.orchestra.run.trial035.stdout:osd.1: {} 2026-02-24T07:48:21.123 INFO:teuthology.orchestra.run.trial035.stderr:osd.1: osd_enable_op_tracker = '' 2026-02-24T07:48:21.137 INFO:teuthology.orchestra.run.trial035.stdout:osd.2: {} 2026-02-24T07:48:21.137 INFO:teuthology.orchestra.run.trial035.stderr:osd.2: osd_enable_op_tracker = '' 2026-02-24T07:48:21.151 INFO:teuthology.orchestra.run.trial035.stderr:Error EPERM: crimson_allow_pg_split is false; pg_num increase denied for crimson pool 2026-02-24T07:48:21.151 INFO:teuthology.orchestra.run.trial035.stdout:osd.3: {} 2026-02-24T07:48:21.151 INFO:teuthology.orchestra.run.trial035.stderr:osd.3: osd_enable_op_tracker = '' 2026-02-24T07:48:21.470 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-24T07:48:22.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:48:21 trial035 ceph-mon[39284]: from='client.14796 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2026-02-24T07:48:22.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:48:21 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/1675005985' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2026-02-24T07:48:22.205 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:48:21 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/2697034288' entity='client.admin' cmd={"prefix": "osd pool set", "pool": ".mgr", "var": "pg_num", "val": "191"} : dispatch 2026-02-24T07:48:23.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:48:22 trial035 ceph-mon[39284]: pgmap v184: 17 pgs: 17 active+clean; 15 GiB data, 51 GiB used, 2.7 TiB / 2.7 TiB avail; 0 B/s wr, 10.86k op/s 2026-02-24T07:48:23.462 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2026-02-24T07:48:23.598 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:48:23.724 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:48:23.723+0000 7facbd438640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:48:23.724 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:48:23.723+0000 7facbd438640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:48:23.742 INFO:teuthology.orchestra.run.trial035.stdout:osd.0: {} 2026-02-24T07:48:23.742 INFO:teuthology.orchestra.run.trial035.stderr:osd.0: osd_enable_op_tracker = '' 2026-02-24T07:48:23.762 INFO:teuthology.orchestra.run.trial035.stdout:osd.1: {} 2026-02-24T07:48:23.762 INFO:teuthology.orchestra.run.trial035.stderr:osd.1: osd_enable_op_tracker = '' 2026-02-24T07:48:23.777 INFO:teuthology.orchestra.run.trial035.stdout:osd.2: {} 2026-02-24T07:48:23.777 INFO:teuthology.orchestra.run.trial035.stderr:osd.2: osd_enable_op_tracker = '' 2026-02-24T07:48:23.789 INFO:teuthology.orchestra.run.trial035.stdout:osd.3: {} 2026-02-24T07:48:23.790 INFO:teuthology.orchestra.run.trial035.stderr:osd.3: osd_enable_op_tracker = '' 2026-02-24T07:48:24.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:48:23 trial035 ceph-mon[39284]: 3.3 scrub starts 2026-02-24T07:48:24.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:48:23 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/3535472165' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2026-02-24T07:48:25.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:48:24 trial035 ceph-mon[39284]: pgmap v185: 17 pgs: 17 active+clean; 15 GiB data, 51 GiB used, 2.7 TiB / 2.7 TiB avail; 0 B/s wr, 7.13k op/s 2026-02-24T07:48:25.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:48:24 trial035 ceph-mon[39284]: 3.3 scrub ok 2026-02-24T07:48:25.831 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2026-02-24T07:48:25.974 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:48:26.109 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:48:26.107+0000 7fe640e1c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:48:26.109 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:48:26.107+0000 7fe640e1c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:48:26.109 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:48:25 trial035 ceph-mon[39284]: purged_snaps scrub starts 2026-02-24T07:48:26.109 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:48:25 trial035 ceph-mon[39284]: purged_snaps scrub ok 2026-02-24T07:48:26.109 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:48:25 trial035 ceph-mon[39284]: 3.f scrub starts 2026-02-24T07:48:26.122 INFO:teuthology.orchestra.run.trial035.stdout:osd.0: {} 2026-02-24T07:48:26.123 INFO:teuthology.orchestra.run.trial035.stderr:osd.0: osd_enable_op_tracker = '' 2026-02-24T07:48:26.138 INFO:teuthology.orchestra.run.trial035.stdout:osd.1: {} 2026-02-24T07:48:26.138 INFO:teuthology.orchestra.run.trial035.stderr:osd.1: osd_enable_op_tracker = '' 2026-02-24T07:48:26.153 INFO:teuthology.orchestra.run.trial035.stdout:osd.2: {} 2026-02-24T07:48:26.154 INFO:teuthology.orchestra.run.trial035.stderr:osd.2: osd_enable_op_tracker = '' 2026-02-24T07:48:26.166 INFO:teuthology.orchestra.run.trial035.stdout:osd.3: {} 2026-02-24T07:48:26.166 INFO:teuthology.orchestra.run.trial035.stderr:osd.3: osd_enable_op_tracker = '' 2026-02-24T07:48:26.471 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2026-02-24T07:48:26.472 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2026-02-24T07:48:26.472 INFO:tasks.thrashosds.thrasher:fixing pg num pool .mgr 2026-02-24T07:48:26.472 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph pg dump --format=json 2026-02-24T07:48:26.609 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:48:26.737 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:48:26.735+0000 7fcffdda2640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:48:26.737 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:48:26.735+0000 7fcffdda2640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:48:26.883 INFO:teuthology.orchestra.run.trial035.stdout: 2026-02-24T07:48:26.884 INFO:teuthology.orchestra.run.trial035.stderr:dumped all 2026-02-24T07:48:27.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:48:26 trial035 ceph-mon[39284]: pgmap v186: 17 pgs: 17 active+clean; 15 GiB data, 51 GiB used, 2.7 TiB / 2.7 TiB avail; 0 B/s wr, 7.13k op/s 2026-02-24T07:48:27.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:48:26 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/1882041254' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2026-02-24T07:48:27.205 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:48:26 trial035 ceph-mon[39284]: 3.f scrub ok 2026-02-24T07:48:27.205 INFO:journalctl@ceph.mgr.x.trial035.stdout:Feb 24 07:48:26 trial035 ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-mgr-x[39621]: ::ffff:10.20.193.91 - - [24/Feb/2026:07:48:26] "GET /metrics HTTP/1.1" 200 4231 "" "Prometheus/3.6.0" 2026-02-24T07:48:27.356 INFO:journalctl@ceph.prometheus.a.trial091.stdout:Feb 24 07:48:26 trial091 ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-prometheus-a[66007]: time=2026-02-24T07:48:26.973Z level=ERROR source=scrape.go:1630 msg="Failed to determine correct type of scrape target." component="scrape manager" scrape_pool=ceph target=http://trial035:9283/metrics content_type="text/html;charset=utf-8" fallback_media_type="" err="received unsupported Content-Type \"text/html;charset=utf-8\" and no fallback_scrape_protocol specified for target" 2026-02-24T07:48:27.605 INFO:teuthology.orchestra.run.trial035.stdout:{"pg_ready":true,"pg_map":{"version":186,"stamp":"2026-02-24T07:48:25.701478+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":16231825976,"num_objects":247672,"num_object_clones":0,"num_object_copies":743016,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":247672,"num_whiteouts":0,"num_read":137,"num_read_kb":241,"num_write":1079481,"num_write_kb":42466599,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":52,"ondisk_log_size":52,"up":51,"acting":51,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":51,"num_osds":4,"num_per_pool_osds":4,"num_per_pool_omap_osds":4,"kb":2936012800,"kb_used":53965224,"kb_used_data":48230616,"kb_used_omap":40,"kb_used_meta":5734359,"kb_avail":2882047576,"statfs":{"total":3006477107200,"available":2951216717824,"internally_reserved":0,"allocated":49388150784,"data_stored":49386862152,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":41471,"internal_metadata":5871984129},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-5608439808,"num_objects":-85578,"num_object_clones":0,"num_object_copies":-256734,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-85578,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":85578,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001558"},"pg_stats":[{"pgid":"3.f","version":"49'67217","reported_seq":67587,"reported_epoch":49,"state":"active+clean","last_fresh":"2026-02-24T07:48:20.699741+0000","last_change":"2026-02-24T07:47:16.466592+0000","last_active":"2026-02-24T07:48:20.699741+0000","last_peered":"2026-02-24T07:48:20.699741+0000","last_clean":"2026-02-24T07:48:20.699741+0000","last_became_active":"2026-02-24T07:46:02.881625+0000","last_became_peered":"2026-02-24T07:46:02.881625+0000","last_unstale":"2026-02-24T07:48:20.699741+0000","last_undegraded":"2026-02-24T07:48:20.699741+0000","last_fullsized":"2026-02-24T07:48:20.699741+0000","mapping_epoch":42,"log_start":"49'67214","ondisk_log_start":"49'67214","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"45'34153","last_scrub_stamp":"2026-02-24T07:47:16.466530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_clean_scrub_stamp":"2026-02-24T07:47:16.466530+0000","objects_scrubbed":33463,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:48:23.487779+0000","scrub_duration":3074,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1008795648,"num_objects":15393,"num_object_clones":0,"num_object_copies":46179,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15393,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":67217,"num_write_kb":2643520,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.e","version":"49'68212","reported_seq":68654,"reported_epoch":49,"state":"active+clean","last_fresh":"2026-02-24T07:48:20.704106+0000","last_change":"2026-02-24T07:47:33.099762+0000","last_active":"2026-02-24T07:48:20.704106+0000","last_peered":"2026-02-24T07:48:20.704106+0000","last_clean":"2026-02-24T07:48:20.704106+0000","last_became_active":"2026-02-24T07:46:02.881083+0000","last_became_peered":"2026-02-24T07:46:02.881083+0000","last_unstale":"2026-02-24T07:48:20.704106+0000","last_undegraded":"2026-02-24T07:48:20.704106+0000","last_fullsized":"2026-02-24T07:48:20.704106+0000","mapping_epoch":42,"log_start":"49'68209","ondisk_log_start":"49'68209","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"46'41533","last_scrub_stamp":"2026-02-24T07:47:33.099675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_clean_scrub_stamp":"2026-02-24T07:47:33.099675+0000","objects_scrubbed":40754,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:48:53.832432+0000","scrub_duration":3969,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1021313024,"num_objects":15584,"num_object_clones":0,"num_object_copies":46752,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15584,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":68212,"num_write_kb":2681472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"49'67436","reported_seq":67768,"reported_epoch":49,"state":"active+clean","last_fresh":"2026-02-24T07:48:20.722699+0000","last_change":"2026-02-24T07:47:06.968813+0000","last_active":"2026-02-24T07:48:20.722699+0000","last_peered":"2026-02-24T07:48:20.722699+0000","last_clean":"2026-02-24T07:48:20.722699+0000","last_became_active":"2026-02-24T07:46:02.883799+0000","last_became_peered":"2026-02-24T07:46:02.883799+0000","last_unstale":"2026-02-24T07:48:20.722699+0000","last_undegraded":"2026-02-24T07:48:20.722699+0000","last_fullsized":"2026-02-24T07:48:20.722699+0000","mapping_epoch":42,"log_start":"49'67433","ondisk_log_start":"49'67433","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"45'30090","last_scrub_stamp":"2026-02-24T07:47:06.968732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_clean_scrub_stamp":"2026-02-24T07:47:06.968732+0000","objects_scrubbed":29542,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:48:25.198085+0000","scrub_duration":3070,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1025310744,"num_objects":15646,"num_object_clones":0,"num_object_copies":46938,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15646,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":67436,"num_write_kb":2658561,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"49'66449","reported_seq":66779,"reported_epoch":49,"state":"active+clean","last_fresh":"2026-02-24T07:48:20.724021+0000","last_change":"2026-02-24T07:47:07.546834+0000","last_active":"2026-02-24T07:48:20.724021+0000","last_peered":"2026-02-24T07:48:20.724021+0000","last_clean":"2026-02-24T07:48:20.724021+0000","last_became_active":"2026-02-24T07:46:02.880060+0000","last_became_peered":"2026-02-24T07:46:02.880060+0000","last_unstale":"2026-02-24T07:48:20.724021+0000","last_undegraded":"2026-02-24T07:48:20.724021+0000","last_fullsized":"2026-02-24T07:48:20.724021+0000","mapping_epoch":42,"log_start":"49'66446","ondisk_log_start":"49'66446","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"45'29910","last_scrub_stamp":"2026-02-24T07:47:07.546803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_clean_scrub_stamp":"2026-02-24T07:47:07.546803+0000","objects_scrubbed":29435,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:48:27.524515+0000","scrub_duration":2696,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":995688448,"num_objects":15193,"num_object_clones":0,"num_object_copies":45579,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15193,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":66449,"num_write_kb":2612544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"49'67472","reported_seq":68008,"reported_epoch":49,"state":"active+clean","last_fresh":"2026-02-24T07:48:20.777903+0000","last_change":"2026-02-24T07:48:16.528595+0000","last_active":"2026-02-24T07:48:20.777903+0000","last_peered":"2026-02-24T07:48:20.777903+0000","last_clean":"2026-02-24T07:48:20.777903+0000","last_became_active":"2026-02-24T07:46:02.880171+0000","last_became_peered":"2026-02-24T07:46:02.880171+0000","last_unstale":"2026-02-24T07:48:20.777903+0000","last_undegraded":"2026-02-24T07:48:20.777903+0000","last_fullsized":"2026-02-24T07:48:20.777903+0000","mapping_epoch":42,"log_start":"49'67469","ondisk_log_start":"49'67469","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"49'65111","last_scrub_stamp":"2026-02-24T07:48:16.528547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_clean_scrub_stamp":"2026-02-24T07:48:16.528547+0000","objects_scrubbed":18357,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:49:45.487538+0000","scrub_duration":1833,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1013186560,"num_objects":15460,"num_object_clones":0,"num_object_copies":46380,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15460,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":67472,"num_write_kb":2653824,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.a","version":"49'66926","reported_seq":67369,"reported_epoch":49,"state":"active+clean","last_fresh":"2026-02-24T07:48:20.722575+0000","last_change":"2026-02-24T07:47:35.439589+0000","last_active":"2026-02-24T07:48:20.722575+0000","last_peered":"2026-02-24T07:48:20.722575+0000","last_clean":"2026-02-24T07:48:20.722575+0000","last_became_active":"2026-02-24T07:46:02.882274+0000","last_became_peered":"2026-02-24T07:46:02.882274+0000","last_unstale":"2026-02-24T07:48:20.722575+0000","last_undegraded":"2026-02-24T07:48:20.722575+0000","last_fullsized":"2026-02-24T07:48:20.722575+0000","mapping_epoch":42,"log_start":"49'66923","ondisk_log_start":"49'66923","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"46'41859","last_scrub_stamp":"2026-02-24T07:47:35.439542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_clean_scrub_stamp":"2026-02-24T07:47:35.439542+0000","objects_scrubbed":40699,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:48:38.034606+0000","scrub_duration":3645,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1000079360,"num_objects":15260,"num_object_clones":0,"num_object_copies":45780,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15260,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":66926,"num_write_kb":2629952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"49'68230","reported_seq":68580,"reported_epoch":49,"state":"active+clean","last_fresh":"2026-02-24T07:48:20.722662+0000","last_change":"2026-02-24T07:47:10.197277+0000","last_active":"2026-02-24T07:48:20.722662+0000","last_peered":"2026-02-24T07:48:20.722662+0000","last_clean":"2026-02-24T07:48:20.722662+0000","last_became_active":"2026-02-24T07:46:02.881951+0000","last_became_peered":"2026-02-24T07:46:02.881951+0000","last_unstale":"2026-02-24T07:48:20.722662+0000","last_undegraded":"2026-02-24T07:48:20.722662+0000","last_fullsized":"2026-02-24T07:48:20.722662+0000","mapping_epoch":42,"log_start":"49'68227","ondisk_log_start":"49'68227","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"45'31847","last_scrub_stamp":"2026-02-24T07:47:10.197239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_clean_scrub_stamp":"2026-02-24T07:47:10.197239+0000","objects_scrubbed":31279,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:48:27.205650+0000","scrub_duration":3289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1024589824,"num_objects":15634,"num_object_clones":0,"num_object_copies":46902,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15634,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":68230,"num_write_kb":2683648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"49'67692","reported_seq":68087,"reported_epoch":49,"state":"active+clean","last_fresh":"2026-02-24T07:48:20.774302+0000","last_change":"2026-02-24T07:47:22.289650+0000","last_active":"2026-02-24T07:48:20.774302+0000","last_peered":"2026-02-24T07:48:20.774302+0000","last_clean":"2026-02-24T07:48:20.774302+0000","last_became_active":"2026-02-24T07:46:02.881345+0000","last_became_peered":"2026-02-24T07:46:02.881345+0000","last_unstale":"2026-02-24T07:48:20.774302+0000","last_undegraded":"2026-02-24T07:48:20.774302+0000","last_fullsized":"2026-02-24T07:48:20.774302+0000","mapping_epoch":42,"log_start":"49'67689","ondisk_log_start":"49'67689","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"45'36703","last_scrub_stamp":"2026-02-24T07:47:22.289617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_clean_scrub_stamp":"2026-02-24T07:47:22.289617+0000","objects_scrubbed":35960,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:48:31.332102+0000","scrub_duration":3891,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1017774080,"num_objects":15530,"num_object_clones":0,"num_object_copies":46590,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15530,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":67692,"num_write_kb":2663104,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"49'67081","reported_seq":67525,"reported_epoch":49,"state":"active+clean","last_fresh":"2026-02-24T07:48:20.776439+0000","last_change":"2026-02-24T07:47:35.925072+0000","last_active":"2026-02-24T07:48:20.776439+0000","last_peered":"2026-02-24T07:48:20.776439+0000","last_clean":"2026-02-24T07:48:20.776439+0000","last_became_active":"2026-02-24T07:46:02.880323+0000","last_became_peered":"2026-02-24T07:46:02.880323+0000","last_unstale":"2026-02-24T07:48:20.776439+0000","last_undegraded":"2026-02-24T07:48:20.776439+0000","last_fullsized":"2026-02-24T07:48:20.776439+0000","mapping_epoch":42,"log_start":"49'67078","ondisk_log_start":"49'67078","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"47'42248","last_scrub_stamp":"2026-02-24T07:47:35.925000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_clean_scrub_stamp":"2026-02-24T07:47:35.925000+0000","objects_scrubbed":40858,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:48:43.057825+0000","scrub_duration":3382,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1008926720,"num_objects":15395,"num_object_clones":0,"num_object_copies":46185,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15395,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":67081,"num_write_kb":2639232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.6","version":"49'67676","reported_seq":68076,"reported_epoch":49,"state":"active+clean","last_fresh":"2026-02-24T07:48:20.726587+0000","last_change":"2026-02-24T07:47:23.432795+0000","last_active":"2026-02-24T07:48:20.726587+0000","last_peered":"2026-02-24T07:48:20.726587+0000","last_clean":"2026-02-24T07:48:20.726587+0000","last_became_active":"2026-02-24T07:46:02.880005+0000","last_became_peered":"2026-02-24T07:46:02.880005+0000","last_unstale":"2026-02-24T07:48:20.726587+0000","last_undegraded":"2026-02-24T07:48:20.726587+0000","last_fullsized":"2026-02-24T07:48:20.726587+0000","mapping_epoch":42,"log_start":"49'67673","ondisk_log_start":"49'67673","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"45'37219","last_scrub_stamp":"2026-02-24T07:47:23.432744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_clean_scrub_stamp":"2026-02-24T07:47:23.432744+0000","objects_scrubbed":36510,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:48:51.822832+0000","scrub_duration":3412,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1006370816,"num_objects":15356,"num_object_clones":0,"num_object_copies":46068,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15356,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":67676,"num_write_kb":2657024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"49'67971","reported_seq":68348,"reported_epoch":49,"state":"active+clean","last_fresh":"2026-02-24T07:48:20.696877+0000","last_change":"2026-02-24T07:47:17.546625+0000","last_active":"2026-02-24T07:48:20.696877+0000","last_peered":"2026-02-24T07:48:20.696877+0000","last_clean":"2026-02-24T07:48:20.696877+0000","last_became_active":"2026-02-24T07:46:02.881420+0000","last_became_peered":"2026-02-24T07:46:02.881420+0000","last_unstale":"2026-02-24T07:48:20.696877+0000","last_undegraded":"2026-02-24T07:48:20.696877+0000","last_fullsized":"2026-02-24T07:48:20.696877+0000","mapping_epoch":42,"log_start":"49'67968","ondisk_log_start":"49'67968","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"45'34854","last_scrub_stamp":"2026-02-24T07:47:17.546590+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_clean_scrub_stamp":"2026-02-24T07:47:17.546590+0000","objects_scrubbed":34222,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:48:38.813192+0000","scrub_duration":3116,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1027801088,"num_objects":15683,"num_object_clones":0,"num_object_copies":47049,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15683,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":67971,"num_write_kb":2676928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.4","version":"49'66874","reported_seq":67313,"reported_epoch":49,"state":"active+clean","last_fresh":"2026-02-24T07:48:20.778054+0000","last_change":"2026-02-24T07:47:33.628245+0000","last_active":"2026-02-24T07:48:20.778054+0000","last_peered":"2026-02-24T07:48:20.778054+0000","last_clean":"2026-02-24T07:48:20.778054+0000","last_became_active":"2026-02-24T07:46:02.880345+0000","last_became_peered":"2026-02-24T07:46:02.880345+0000","last_unstale":"2026-02-24T07:48:20.778054+0000","last_undegraded":"2026-02-24T07:48:20.778054+0000","last_fullsized":"2026-02-24T07:48:20.778054+0000","mapping_epoch":42,"log_start":"49'66871","ondisk_log_start":"49'66871","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"46'41076","last_scrub_stamp":"2026-02-24T07:47:33.628218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_clean_scrub_stamp":"2026-02-24T07:47:33.628218+0000","objects_scrubbed":40364,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:48:34.035175+0000","scrub_duration":4080,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1019084800,"num_objects":15550,"num_object_clones":0,"num_object_copies":46650,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15550,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":66874,"num_write_kb":2637568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"49'67570","reported_seq":67932,"reported_epoch":49,"state":"active+clean","last_fresh":"2026-02-24T07:48:20.702185+0000","last_change":"2026-02-24T07:47:14.339634+0000","last_active":"2026-02-24T07:48:20.702185+0000","last_peered":"2026-02-24T07:48:20.702185+0000","last_clean":"2026-02-24T07:48:20.702185+0000","last_became_active":"2026-02-24T07:46:02.881035+0000","last_became_peered":"2026-02-24T07:46:02.881035+0000","last_unstale":"2026-02-24T07:48:20.702185+0000","last_undegraded":"2026-02-24T07:48:20.702185+0000","last_fullsized":"2026-02-24T07:48:20.702185+0000","mapping_epoch":42,"log_start":"49'67567","ondisk_log_start":"49'67567","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"45'33283","last_scrub_stamp":"2026-02-24T07:47:14.339597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_clean_scrub_stamp":"2026-02-24T07:47:14.339597+0000","objects_scrubbed":32630,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:48:22.078719+0000","scrub_duration":2951,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1022361600,"num_objects":15600,"num_object_clones":0,"num_object_copies":46800,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15600,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":67570,"num_write_kb":2661440,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"49'67533","reported_seq":67937,"reported_epoch":49,"state":"active+clean","last_fresh":"2026-02-24T07:48:20.727883+0000","last_change":"2026-02-24T07:47:24.807734+0000","last_active":"2026-02-24T07:48:20.727883+0000","last_peered":"2026-02-24T07:48:20.727883+0000","last_clean":"2026-02-24T07:48:20.727883+0000","last_became_active":"2026-02-24T07:46:02.879831+0000","last_became_peered":"2026-02-24T07:46:02.879831+0000","last_unstale":"2026-02-24T07:48:20.727883+0000","last_undegraded":"2026-02-24T07:48:20.727883+0000","last_fullsized":"2026-02-24T07:48:20.727883+0000","mapping_epoch":42,"log_start":"49'67530","ondisk_log_start":"49'67530","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"45'37558","last_scrub_stamp":"2026-02-24T07:47:24.807659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_clean_scrub_stamp":"2026-02-24T07:47:24.807659+0000","objects_scrubbed":36869,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:48:39.172465+0000","scrub_duration":3748,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1001979904,"num_objects":15289,"num_object_clones":0,"num_object_copies":45867,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15289,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":67533,"num_write_kb":2650304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.0","version":"49'67174","reported_seq":67509,"reported_epoch":49,"state":"active+clean","last_fresh":"2026-02-24T07:48:20.775977+0000","last_change":"2026-02-24T07:47:08.376360+0000","last_active":"2026-02-24T07:48:20.775977+0000","last_peered":"2026-02-24T07:48:20.775977+0000","last_clean":"2026-02-24T07:48:20.775977+0000","last_became_active":"2026-02-24T07:46:02.880333+0000","last_became_peered":"2026-02-24T07:46:02.880333+0000","last_unstale":"2026-02-24T07:48:20.775977+0000","last_undegraded":"2026-02-24T07:48:20.775977+0000","last_fullsized":"2026-02-24T07:48:20.775977+0000","mapping_epoch":42,"log_start":"49'67171","ondisk_log_start":"49'67171","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"45'30511","last_scrub_stamp":"2026-02-24T07:47:08.376304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_clean_scrub_stamp":"2026-02-24T07:47:08.376304+0000","objects_scrubbed":29912,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:48:24.552234+0000","scrub_duration":3049,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1019215872,"num_objects":15552,"num_object_clones":0,"num_object_copies":46656,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15552,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":67174,"num_write_kb":2647232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2","version":"49'67833","reported_seq":68248,"reported_epoch":49,"state":"active+clean","last_fresh":"2026-02-24T07:48:20.779047+0000","last_change":"2026-02-24T07:47:27.784936+0000","last_active":"2026-02-24T07:48:20.779047+0000","last_peered":"2026-02-24T07:48:20.779047+0000","last_clean":"2026-02-24T07:48:20.779047+0000","last_became_active":"2026-02-24T07:46:02.880185+0000","last_became_peered":"2026-02-24T07:46:02.880185+0000","last_unstale":"2026-02-24T07:48:20.779047+0000","last_undegraded":"2026-02-24T07:48:20.779047+0000","last_fullsized":"2026-02-24T07:48:20.779047+0000","mapping_epoch":42,"log_start":"49'67829","ondisk_log_start":"49'67829","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"45'39074","last_scrub_stamp":"2026-02-24T07:47:27.784888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_clean_scrub_stamp":"2026-02-24T07:47:27.784888+0000","objects_scrubbed":38200,"log_size":4,"log_dups_size":3000,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:48:47.063599+0000","scrub_duration":4266,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1018757120,"num_objects":15545,"num_object_clones":0,"num_object_copies":46635,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15545,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":67833,"num_write_kb":2668096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"29'99","reported_seq":253,"reported_epoch":49,"state":"active+clean","last_fresh":"2026-02-24T07:48:02.589664+0000","last_change":"2026-02-24T07:48:02.589664+0000","last_active":"2026-02-24T07:48:02.589664+0000","last_peered":"2026-02-24T07:48:02.589664+0000","last_clean":"2026-02-24T07:48:02.589664+0000","last_became_active":"2026-02-24T07:42:54.186719+0000","last_became_peered":"2026-02-24T07:42:54.186719+0000","last_unstale":"2026-02-24T07:48:02.589664+0000","last_undegraded":"2026-02-24T07:48:02.589664+0000","last_fullsized":"2026-02-24T07:48:02.589664+0000","mapping_epoch":27,"log_start":"29'96","ondisk_log_start":"29'96","created":20,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'99","last_scrub_stamp":"2026-02-24T07:48:02.589609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:42:43.640776+0000","last_clean_scrub_stamp":"2026-02-24T07:48:02.589609+0000","objects_scrubbed":2,"log_size":3,"log_dups_size":96,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:49:04.391555+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":240,"num_write":135,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":16,"stat_sum":{"num_bytes":16231235608,"num_objects":247670,"num_object_clones":0,"num_object_copies":743010,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":247670,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1079346,"num_write_kb":42464449,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":48706367488,"data_stored":48706355272,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":93,"internal_metadata":0},"log_size":49,"ondisk_log_size":49,"up":48,"acting":48,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":240,"num_write":135,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1896,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":3,"up_from":25,"seq":107374182468,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":11856572,"kb_used_data":10187432,"kb_used_omap":10,"kb_used_meta":1669109,"kb_avail":722146628,"statfs":{"total":751619276800,"available":739478147072,"internally_reserved":0,"allocated":10431930368,"data_stored":10431606304,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":10771,"internal_metadata":1709168109},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":19,"seq":81604378694,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":15768764,"kb_used_data":14037992,"kb_used_omap":10,"kb_used_meta":1730741,"kb_avail":718234436,"statfs":{"total":751619276800,"available":735472062464,"internally_reserved":0,"allocated":14374903808,"data_stored":14374579744,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":10589,"internal_metadata":1772279459},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542215,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":11726068,"kb_used_data":10887264,"kb_used_omap":11,"kb_used_meta":838708,"kb_avail":722277132,"statfs":{"total":751619276800,"available":739611783168,"internally_reserved":0,"allocated":11148558336,"data_stored":11148241896,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12029,"internal_metadata":858837251},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705738,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":14613820,"kb_used_data":13117928,"kb_used_omap":7,"kb_used_meta":1495800,"kb_avail":719389380,"statfs":{"total":751619276800,"available":736654725120,"internally_reserved":0,"allocated":13432758272,"data_stored":13432434208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":8082,"internal_metadata":1531699310},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1896,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":13195415552,"data_stored":13195411480,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":11110842368,"data_stored":11110842368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":93,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":14244712448,"data_stored":14244708376,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":10155397120,"data_stored":10155393048,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-24T07:48:27.607 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph osd pool set .mgr pgp_num 191 2026-02-24T07:48:27.757 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:48:28.155 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:48:27 trial035 ceph-mon[39284]: 3.d scrub starts 2026-02-24T07:48:28.155 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:48:27 trial035 ceph-mon[39284]: from='client.14806 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2026-02-24T07:48:28.209 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2026-02-24T07:48:28.234 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:48:28.207+0000 7fa445750640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:48:28.235 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:48:28.208+0000 7fa445750640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:48:28.360 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:48:28.362 INFO:teuthology.orchestra.run.trial035.stderr:Error EPERM: crimson_allow_pg_split is false; pgp_num increase denied 2026-02-24T07:48:28.906 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:48:28 trial035 ceph-mon[39284]: pgmap v187: 17 pgs: 1 active+clean+scrubbing, 16 active+clean; 12 GiB data, 43 GiB used, 2.7 TiB / 2.7 TiB avail; 0 B/s wr, 10.92k op/s 2026-02-24T07:48:28.906 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:48:28 trial035 ceph-mon[39284]: 3.d scrub ok 2026-02-24T07:48:28.906 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:48:28 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/595477426' entity='client.admin' cmd={"prefix": "osd pool set", "pool": ".mgr", "var": "pgp_num", "val": "191"} : dispatch 2026-02-24T07:48:28.971 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-24T07:48:29.356 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:48:29.354+0000 7f1cfa623640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:48:29.356 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:48:29.355+0000 7f1cfa623640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:48:29.371 INFO:teuthology.orchestra.run.trial035.stdout:osd.0: {} 2026-02-24T07:48:29.372 INFO:teuthology.orchestra.run.trial035.stderr:osd.0: osd_enable_op_tracker = '' 2026-02-24T07:48:29.384 INFO:teuthology.orchestra.run.trial035.stdout:osd.1: {} 2026-02-24T07:48:29.385 INFO:teuthology.orchestra.run.trial035.stderr:osd.1: osd_enable_op_tracker = '' 2026-02-24T07:48:29.398 INFO:teuthology.orchestra.run.trial035.stdout:osd.2: {} 2026-02-24T07:48:29.398 INFO:teuthology.orchestra.run.trial035.stderr:osd.2: osd_enable_op_tracker = '' 2026-02-24T07:48:29.410 INFO:teuthology.orchestra.run.trial035.stdout:osd.3: {} 2026-02-24T07:48:29.410 INFO:teuthology.orchestra.run.trial035.stderr:osd.3: osd_enable_op_tracker = '' 2026-02-24T07:48:30.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:48:29 trial035 ceph-mon[39284]: 3.0 scrub starts 2026-02-24T07:48:30.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:48:29 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/2524145615' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2026-02-24T07:48:30.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:48:29 trial035 ceph-mon[39284]: from='mgr.14284 10.20.193.35:0/3681373222' entity='mgr.x' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2026-02-24T07:48:31.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:48:30 trial035 ceph-mon[39284]: pgmap v188: 17 pgs: 1 active+clean+scrubbing, 16 active+clean; 12 GiB data, 43 GiB used, 2.7 TiB / 2.7 TiB avail; 0 B/s wr, 7.45k op/s 2026-02-24T07:48:31.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:48:30 trial035 ceph-mon[39284]: 3.9 scrub starts 2026-02-24T07:48:31.451 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2026-02-24T07:48:31.599 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:48:31.725 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:48:31.722+0000 7f5bcaef1640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:48:31.725 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:48:31.723+0000 7f5bcaef1640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:48:31.740 INFO:teuthology.orchestra.run.trial035.stdout:osd.0: {} 2026-02-24T07:48:31.740 INFO:teuthology.orchestra.run.trial035.stderr:osd.0: osd_enable_op_tracker = '' 2026-02-24T07:48:31.756 INFO:teuthology.orchestra.run.trial035.stdout:osd.1: {} 2026-02-24T07:48:31.756 INFO:teuthology.orchestra.run.trial035.stderr:osd.1: osd_enable_op_tracker = '' 2026-02-24T07:48:31.769 INFO:teuthology.orchestra.run.trial035.stdout:osd.2: {} 2026-02-24T07:48:31.770 INFO:teuthology.orchestra.run.trial035.stderr:osd.2: osd_enable_op_tracker = '' 2026-02-24T07:48:31.782 INFO:teuthology.orchestra.run.trial035.stdout:osd.3: {} 2026-02-24T07:48:31.783 INFO:teuthology.orchestra.run.trial035.stderr:osd.3: osd_enable_op_tracker = '' 2026-02-24T07:48:32.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:48:31 trial035 ceph-mon[39284]: 3.0 scrub ok 2026-02-24T07:48:32.205 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:48:31 trial035 ceph-mon[39284]: 3.9 scrub ok 2026-02-24T07:48:32.205 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:48:31 trial035 ceph-mon[39284]: pgmap v189: 17 pgs: 17 active+clean; 10 GiB data, 36 GiB used, 2.7 TiB / 2.7 TiB avail; 0 B/s wr, 10.50k op/s 2026-02-24T07:48:32.205 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:48:31 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/935139195' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2026-02-24T07:48:33.826 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2026-02-24T07:48:33.970 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:48:33.973 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2026-02-24T07:48:33.973 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2026-02-24T07:48:33.974 INFO:tasks.thrashosds.thrasher:Growing pool .mgr 2026-02-24T07:48:33.974 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph pg dump --format=json 2026-02-24T07:48:34.083 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:48:34.080+0000 7f88e248d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:48:34.083 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:48:34.081+0000 7f88e248d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:48:34.099 INFO:teuthology.orchestra.run.trial035.stdout:osd.0: {} 2026-02-24T07:48:34.099 INFO:teuthology.orchestra.run.trial035.stderr:osd.0: osd_enable_op_tracker = '' 2026-02-24T07:48:34.115 INFO:teuthology.orchestra.run.trial035.stdout:osd.1: {} 2026-02-24T07:48:34.115 INFO:teuthology.orchestra.run.trial035.stderr:osd.1: osd_enable_op_tracker = '' 2026-02-24T07:48:34.122 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:48:34.131 INFO:teuthology.orchestra.run.trial035.stdout:osd.2: {} 2026-02-24T07:48:34.131 INFO:teuthology.orchestra.run.trial035.stderr:osd.2: osd_enable_op_tracker = '' 2026-02-24T07:48:34.146 INFO:teuthology.orchestra.run.trial035.stdout:osd.3: {} 2026-02-24T07:48:34.146 INFO:teuthology.orchestra.run.trial035.stderr:osd.3: osd_enable_op_tracker = '' 2026-02-24T07:48:34.238 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:48:34.236+0000 7fd1e1bab640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:48:34.239 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:48:34.236+0000 7fd1e1bab640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:48:34.377 INFO:teuthology.orchestra.run.trial035.stdout: 2026-02-24T07:48:34.378 INFO:teuthology.orchestra.run.trial035.stderr:dumped all 2026-02-24T07:48:34.420 INFO:teuthology.orchestra.run.trial035.stdout:{"pg_ready":true,"pg_map":{"version":190,"stamp":"2026-02-24T07:48:33.702510+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":10857153080,"num_objects":165661,"num_object_clones":0,"num_object_copies":496983,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":165661,"num_whiteouts":0,"num_read":137,"num_read_kb":241,"num_write":1161492,"num_write_kb":42466599,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":51,"ondisk_log_size":51,"up":51,"acting":51,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":51,"num_osds":4,"num_per_pool_osds":4,"num_per_pool_omap_osds":4,"kb":2936012800,"kb_used":38130344,"kb_used_data":32690072,"kb_used_omap":41,"kb_used_meta":5439958,"kb_avail":2897882456,"statfs":{"total":3006477107200,"available":2967431634944,"internally_reserved":0,"allocated":33474633728,"data_stored":33473345096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":42493,"internal_metadata":5570517507},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,1],"upper_bound":4},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-5374672896,"num_objects":-82011,"num_object_clones":0,"num_object_copies":-246033,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-82011,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":82011,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001579"},"pg_stats":[{"pgid":"3.f","version":"49'72323","reported_seq":72826,"reported_epoch":49,"state":"active+clean","last_fresh":"2026-02-24T07:48:30.703569+0000","last_change":"2026-02-24T07:48:26.639215+0000","last_active":"2026-02-24T07:48:30.703569+0000","last_peered":"2026-02-24T07:48:30.703569+0000","last_clean":"2026-02-24T07:48:30.703569+0000","last_became_active":"2026-02-24T07:46:02.881625+0000","last_became_peered":"2026-02-24T07:46:02.881625+0000","last_unstale":"2026-02-24T07:48:30.703569+0000","last_undegraded":"2026-02-24T07:48:30.703569+0000","last_fullsized":"2026-02-24T07:48:30.703569+0000","mapping_epoch":42,"log_start":"49'72320","ondisk_log_start":"49'72320","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"49'70570","last_scrub_stamp":"2026-02-24T07:48:26.639139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_clean_scrub_stamp":"2026-02-24T07:48:26.639139+0000","objects_scrubbed":12333,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:49:38.413618+0000","scrub_duration":1227,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":674168832,"num_objects":10287,"num_object_clones":0,"num_object_copies":30861,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10287,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":72323,"num_write_kb":2643520,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.e","version":"49'73357","reported_seq":73798,"reported_epoch":49,"state":"active+clean","last_fresh":"2026-02-24T07:48:30.709156+0000","last_change":"2026-02-24T07:47:33.099762+0000","last_active":"2026-02-24T07:48:30.709156+0000","last_peered":"2026-02-24T07:48:30.709156+0000","last_clean":"2026-02-24T07:48:30.709156+0000","last_became_active":"2026-02-24T07:46:02.881083+0000","last_became_peered":"2026-02-24T07:46:02.881083+0000","last_unstale":"2026-02-24T07:48:30.709156+0000","last_undegraded":"2026-02-24T07:48:30.709156+0000","last_fullsized":"2026-02-24T07:48:30.709156+0000","mapping_epoch":42,"log_start":"49'73354","ondisk_log_start":"49'73354","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"46'41533","last_scrub_stamp":"2026-02-24T07:47:33.099675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_clean_scrub_stamp":"2026-02-24T07:47:33.099675+0000","objects_scrubbed":40754,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:48:53.832432+0000","scrub_duration":3969,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":684130304,"num_objects":10439,"num_object_clones":0,"num_object_copies":31317,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10439,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":73357,"num_write_kb":2681472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"49'72590","reported_seq":73053,"reported_epoch":49,"state":"active+clean","last_fresh":"2026-02-24T07:48:30.722082+0000","last_change":"2026-02-24T07:48:28.181137+0000","last_active":"2026-02-24T07:48:30.722082+0000","last_peered":"2026-02-24T07:48:30.722082+0000","last_clean":"2026-02-24T07:48:30.722082+0000","last_became_active":"2026-02-24T07:46:02.883799+0000","last_became_peered":"2026-02-24T07:46:02.883799+0000","last_unstale":"2026-02-24T07:48:30.722082+0000","last_undegraded":"2026-02-24T07:48:30.722082+0000","last_fullsized":"2026-02-24T07:48:30.722082+0000","mapping_epoch":42,"log_start":"49'72587","ondisk_log_start":"49'72587","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"49'71407","last_scrub_stamp":"2026-02-24T07:48:28.181090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_clean_scrub_stamp":"2026-02-24T07:48:28.181090+0000","objects_scrubbed":11940,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:49:56.737033+0000","scrub_duration":1384,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":687538200,"num_objects":10492,"num_object_clones":0,"num_object_copies":31476,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10492,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":72590,"num_write_kb":2658561,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"49'71418","reported_seq":71747,"reported_epoch":49,"state":"active+clean","last_fresh":"2026-02-24T07:48:30.723386+0000","last_change":"2026-02-24T07:47:07.546834+0000","last_active":"2026-02-24T07:48:30.723386+0000","last_peered":"2026-02-24T07:48:30.723386+0000","last_clean":"2026-02-24T07:48:30.723386+0000","last_became_active":"2026-02-24T07:46:02.880060+0000","last_became_peered":"2026-02-24T07:46:02.880060+0000","last_unstale":"2026-02-24T07:48:30.723386+0000","last_undegraded":"2026-02-24T07:48:30.723386+0000","last_fullsized":"2026-02-24T07:48:30.723386+0000","mapping_epoch":42,"log_start":"49'71415","ondisk_log_start":"49'71415","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"45'29910","last_scrub_stamp":"2026-02-24T07:47:07.546803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_clean_scrub_stamp":"2026-02-24T07:47:07.546803+0000","objects_scrubbed":29435,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"queued for scrub","scrub_duration":2696,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":670040064,"num_objects":10224,"num_object_clones":0,"num_object_copies":30672,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10224,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":71418,"num_write_kb":2612544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"49'72582","reported_seq":73118,"reported_epoch":49,"state":"active+clean","last_fresh":"2026-02-24T07:48:30.779747+0000","last_change":"2026-02-24T07:48:16.528595+0000","last_active":"2026-02-24T07:48:30.779747+0000","last_peered":"2026-02-24T07:48:30.779747+0000","last_clean":"2026-02-24T07:48:30.779747+0000","last_became_active":"2026-02-24T07:46:02.880171+0000","last_became_peered":"2026-02-24T07:46:02.880171+0000","last_unstale":"2026-02-24T07:48:30.779747+0000","last_undegraded":"2026-02-24T07:48:30.779747+0000","last_fullsized":"2026-02-24T07:48:30.779747+0000","mapping_epoch":42,"log_start":"49'72579","ondisk_log_start":"49'72579","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"49'65111","last_scrub_stamp":"2026-02-24T07:48:16.528547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_clean_scrub_stamp":"2026-02-24T07:48:16.528547+0000","objects_scrubbed":18357,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:49:45.487538+0000","scrub_duration":1833,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":678297600,"num_objects":10350,"num_object_clones":0,"num_object_copies":31050,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10350,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":72582,"num_write_kb":2653824,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.a","version":"49'71986","reported_seq":72429,"reported_epoch":49,"state":"active+clean","last_fresh":"2026-02-24T07:48:30.723441+0000","last_change":"2026-02-24T07:47:35.439589+0000","last_active":"2026-02-24T07:48:30.723441+0000","last_peered":"2026-02-24T07:48:30.723441+0000","last_clean":"2026-02-24T07:48:30.723441+0000","last_became_active":"2026-02-24T07:46:02.882274+0000","last_became_peered":"2026-02-24T07:46:02.882274+0000","last_unstale":"2026-02-24T07:48:30.723441+0000","last_undegraded":"2026-02-24T07:48:30.723441+0000","last_fullsized":"2026-02-24T07:48:30.723441+0000","mapping_epoch":42,"log_start":"49'71983","ondisk_log_start":"49'71983","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"46'41859","last_scrub_stamp":"2026-02-24T07:47:35.439542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_clean_scrub_stamp":"2026-02-24T07:47:35.439542+0000","objects_scrubbed":40699,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:48:38.034606+0000","scrub_duration":3645,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":668467200,"num_objects":10200,"num_object_clones":0,"num_object_copies":30600,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10200,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":71986,"num_write_kb":2629952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"49'73414","reported_seq":73880,"reported_epoch":49,"state":"active+clean","last_fresh":"2026-02-24T07:48:30.724127+0000","last_change":"2026-02-24T07:48:30.710470+0000","last_active":"2026-02-24T07:48:30.724127+0000","last_peered":"2026-02-24T07:48:30.724127+0000","last_clean":"2026-02-24T07:48:30.724127+0000","last_became_active":"2026-02-24T07:46:02.881951+0000","last_became_peered":"2026-02-24T07:46:02.881951+0000","last_unstale":"2026-02-24T07:48:30.724127+0000","last_undegraded":"2026-02-24T07:48:30.724127+0000","last_fullsized":"2026-02-24T07:48:30.724127+0000","mapping_epoch":42,"log_start":"49'73411","ondisk_log_start":"49'73411","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"49'73406","last_scrub_stamp":"2026-02-24T07:48:30.710432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_clean_scrub_stamp":"2026-02-24T07:48:30.710432+0000","objects_scrubbed":10727,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:49:42.678619+0000","scrub_duration":946,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":684851200,"num_objects":10450,"num_object_clones":0,"num_object_copies":31350,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10450,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":73414,"num_write_kb":2683648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"49'72776","reported_seq":73171,"reported_epoch":49,"state":"active+clean","last_fresh":"2026-02-24T07:48:30.774948+0000","last_change":"2026-02-24T07:47:22.289650+0000","last_active":"2026-02-24T07:48:30.774948+0000","last_peered":"2026-02-24T07:48:30.774948+0000","last_clean":"2026-02-24T07:48:30.774948+0000","last_became_active":"2026-02-24T07:46:02.881345+0000","last_became_peered":"2026-02-24T07:46:02.881345+0000","last_unstale":"2026-02-24T07:48:30.774948+0000","last_undegraded":"2026-02-24T07:48:30.774948+0000","last_fullsized":"2026-02-24T07:48:30.774948+0000","mapping_epoch":42,"log_start":"49'72773","ondisk_log_start":"49'72773","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"45'36703","last_scrub_stamp":"2026-02-24T07:47:22.289617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_clean_scrub_stamp":"2026-02-24T07:47:22.289617+0000","objects_scrubbed":35960,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:48:31.332102+0000","scrub_duration":3891,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":684589056,"num_objects":10446,"num_object_clones":0,"num_object_copies":31338,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10446,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":72776,"num_write_kb":2663104,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"49'72172","reported_seq":72616,"reported_epoch":49,"state":"active+clean","last_fresh":"2026-02-24T07:48:30.774455+0000","last_change":"2026-02-24T07:47:35.925072+0000","last_active":"2026-02-24T07:48:30.774455+0000","last_peered":"2026-02-24T07:48:30.774455+0000","last_clean":"2026-02-24T07:48:30.774455+0000","last_became_active":"2026-02-24T07:46:02.880323+0000","last_became_peered":"2026-02-24T07:46:02.880323+0000","last_unstale":"2026-02-24T07:48:30.774455+0000","last_undegraded":"2026-02-24T07:48:30.774455+0000","last_fullsized":"2026-02-24T07:48:30.774455+0000","mapping_epoch":42,"log_start":"49'72169","ondisk_log_start":"49'72169","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"47'42248","last_scrub_stamp":"2026-02-24T07:47:35.925000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_clean_scrub_stamp":"2026-02-24T07:47:35.925000+0000","objects_scrubbed":40858,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:48:43.057825+0000","scrub_duration":3382,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":675282944,"num_objects":10304,"num_object_clones":0,"num_object_copies":30912,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10304,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":72172,"num_write_kb":2639232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.6","version":"49'72791","reported_seq":73190,"reported_epoch":49,"state":"active+clean","last_fresh":"2026-02-24T07:48:30.726950+0000","last_change":"2026-02-24T07:47:23.432795+0000","last_active":"2026-02-24T07:48:30.726950+0000","last_peered":"2026-02-24T07:48:30.726950+0000","last_clean":"2026-02-24T07:48:30.726950+0000","last_became_active":"2026-02-24T07:46:02.880005+0000","last_became_peered":"2026-02-24T07:46:02.880005+0000","last_unstale":"2026-02-24T07:48:30.726950+0000","last_undegraded":"2026-02-24T07:48:30.726950+0000","last_fullsized":"2026-02-24T07:48:30.726950+0000","mapping_epoch":42,"log_start":"49'72788","ondisk_log_start":"49'72788","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"45'37219","last_scrub_stamp":"2026-02-24T07:47:23.432744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_clean_scrub_stamp":"2026-02-24T07:47:23.432744+0000","objects_scrubbed":36510,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:48:51.822832+0000","scrub_duration":3412,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":671154176,"num_objects":10241,"num_object_clones":0,"num_object_copies":30723,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10241,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":72791,"num_write_kb":2657024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"49'73119","reported_seq":73496,"reported_epoch":49,"state":"active+clean","last_fresh":"2026-02-24T07:48:30.707616+0000","last_change":"2026-02-24T07:47:17.546625+0000","last_active":"2026-02-24T07:48:30.707616+0000","last_peered":"2026-02-24T07:48:30.707616+0000","last_clean":"2026-02-24T07:48:30.707616+0000","last_became_active":"2026-02-24T07:46:02.881420+0000","last_became_peered":"2026-02-24T07:46:02.881420+0000","last_unstale":"2026-02-24T07:48:30.707616+0000","last_undegraded":"2026-02-24T07:48:30.707616+0000","last_fullsized":"2026-02-24T07:48:30.707616+0000","mapping_epoch":42,"log_start":"49'73116","ondisk_log_start":"49'73116","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"45'34854","last_scrub_stamp":"2026-02-24T07:47:17.546590+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_clean_scrub_stamp":"2026-02-24T07:47:17.546590+0000","objects_scrubbed":34222,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:48:38.813192+0000","scrub_duration":3116,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":690421760,"num_objects":10535,"num_object_clones":0,"num_object_copies":31605,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10535,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":73119,"num_write_kb":2676928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.4","version":"49'72082","reported_seq":72521,"reported_epoch":49,"state":"active+clean","last_fresh":"2026-02-24T07:48:30.779257+0000","last_change":"2026-02-24T07:47:33.628245+0000","last_active":"2026-02-24T07:48:30.779257+0000","last_peered":"2026-02-24T07:48:30.779257+0000","last_clean":"2026-02-24T07:48:30.779257+0000","last_became_active":"2026-02-24T07:46:02.880345+0000","last_became_peered":"2026-02-24T07:46:02.880345+0000","last_unstale":"2026-02-24T07:48:30.779257+0000","last_undegraded":"2026-02-24T07:48:30.779257+0000","last_fullsized":"2026-02-24T07:48:30.779257+0000","mapping_epoch":42,"log_start":"49'72079","ondisk_log_start":"49'72079","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"46'41076","last_scrub_stamp":"2026-02-24T07:47:33.628218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_clean_scrub_stamp":"2026-02-24T07:47:33.628218+0000","objects_scrubbed":40364,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:48:34.035175+0000","scrub_duration":4080,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":677773312,"num_objects":10342,"num_object_clones":0,"num_object_copies":31026,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10342,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":72082,"num_write_kb":2637568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"49'72743","reported_seq":73254,"reported_epoch":49,"state":"active+clean","last_fresh":"2026-02-24T07:48:30.707443+0000","last_change":"2026-02-24T07:48:24.553666+0000","last_active":"2026-02-24T07:48:30.707443+0000","last_peered":"2026-02-24T07:48:30.707443+0000","last_clean":"2026-02-24T07:48:30.707443+0000","last_became_active":"2026-02-24T07:46:02.881035+0000","last_became_peered":"2026-02-24T07:46:02.881035+0000","last_unstale":"2026-02-24T07:48:30.707443+0000","last_undegraded":"2026-02-24T07:48:30.707443+0000","last_fullsized":"2026-02-24T07:48:30.707443+0000","mapping_epoch":42,"log_start":"49'72740","ondisk_log_start":"49'72740","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"49'69716","last_scrub_stamp":"2026-02-24T07:48:24.553517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_clean_scrub_stamp":"2026-02-24T07:48:24.553517+0000","objects_scrubbed":13748,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:49:36.493266+0000","scrub_duration":1167,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":683343872,"num_objects":10427,"num_object_clones":0,"num_object_copies":31281,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10427,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":72743,"num_write_kb":2661440,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"49'72684","reported_seq":73088,"reported_epoch":49,"state":"active+clean","last_fresh":"2026-02-24T07:48:30.726894+0000","last_change":"2026-02-24T07:47:24.807734+0000","last_active":"2026-02-24T07:48:30.726894+0000","last_peered":"2026-02-24T07:48:30.726894+0000","last_clean":"2026-02-24T07:48:30.726894+0000","last_became_active":"2026-02-24T07:46:02.879831+0000","last_became_peered":"2026-02-24T07:46:02.879831+0000","last_unstale":"2026-02-24T07:48:30.726894+0000","last_undegraded":"2026-02-24T07:48:30.726894+0000","last_fullsized":"2026-02-24T07:48:30.726894+0000","mapping_epoch":42,"log_start":"49'72681","ondisk_log_start":"49'72681","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"45'37558","last_scrub_stamp":"2026-02-24T07:47:24.807659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_clean_scrub_stamp":"2026-02-24T07:47:24.807659+0000","objects_scrubbed":36869,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:48:39.172465+0000","scrub_duration":3748,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":664403968,"num_objects":10138,"num_object_clones":0,"num_object_copies":30414,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10138,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":72684,"num_write_kb":2650304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.0","version":"49'72316","reported_seq":72771,"reported_epoch":49,"state":"active+clean","last_fresh":"2026-02-24T07:48:30.777685+0000","last_change":"2026-02-24T07:48:30.080353+0000","last_active":"2026-02-24T07:48:30.777685+0000","last_peered":"2026-02-24T07:48:30.777685+0000","last_clean":"2026-02-24T07:48:30.777685+0000","last_became_active":"2026-02-24T07:46:02.880333+0000","last_became_peered":"2026-02-24T07:46:02.880333+0000","last_unstale":"2026-02-24T07:48:30.777685+0000","last_undegraded":"2026-02-24T07:48:30.777685+0000","last_fullsized":"2026-02-24T07:48:30.777685+0000","mapping_epoch":42,"log_start":"49'72313","ondisk_log_start":"49'72313","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"49'71923","last_scrub_stamp":"2026-02-24T07:48:30.080305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_clean_scrub_stamp":"2026-02-24T07:48:30.080305+0000","objects_scrubbed":11094,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:49:53.942641+0000","scrub_duration":1405,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":682229760,"num_objects":10410,"num_object_clones":0,"num_object_copies":31230,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10410,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":72316,"num_write_kb":2647232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2","version":"49'73004","reported_seq":73421,"reported_epoch":49,"state":"active+clean","last_fresh":"2026-02-24T07:48:30.776064+0000","last_change":"2026-02-24T07:47:27.784936+0000","last_active":"2026-02-24T07:48:30.776064+0000","last_peered":"2026-02-24T07:48:30.776064+0000","last_clean":"2026-02-24T07:48:30.776064+0000","last_became_active":"2026-02-24T07:46:02.880185+0000","last_became_peered":"2026-02-24T07:46:02.880185+0000","last_unstale":"2026-02-24T07:48:30.776064+0000","last_undegraded":"2026-02-24T07:48:30.776064+0000","last_fullsized":"2026-02-24T07:48:30.776064+0000","mapping_epoch":42,"log_start":"49'73001","ondisk_log_start":"49'73001","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"45'39074","last_scrub_stamp":"2026-02-24T07:47:27.784888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_clean_scrub_stamp":"2026-02-24T07:47:27.784888+0000","objects_scrubbed":38200,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:48:47.063599+0000","scrub_duration":4266,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":679870464,"num_objects":10374,"num_object_clones":0,"num_object_copies":31122,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10374,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":73004,"num_write_kb":2668096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"29'99","reported_seq":253,"reported_epoch":49,"state":"active+clean","last_fresh":"2026-02-24T07:48:02.589664+0000","last_change":"2026-02-24T07:48:02.589664+0000","last_active":"2026-02-24T07:48:02.589664+0000","last_peered":"2026-02-24T07:48:02.589664+0000","last_clean":"2026-02-24T07:48:02.589664+0000","last_became_active":"2026-02-24T07:42:54.186719+0000","last_became_peered":"2026-02-24T07:42:54.186719+0000","last_unstale":"2026-02-24T07:48:02.589664+0000","last_undegraded":"2026-02-24T07:48:02.589664+0000","last_fullsized":"2026-02-24T07:48:02.589664+0000","mapping_epoch":27,"log_start":"29'96","ondisk_log_start":"29'96","created":20,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'99","last_scrub_stamp":"2026-02-24T07:48:02.589609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:42:43.640776+0000","last_clean_scrub_stamp":"2026-02-24T07:48:02.589609+0000","objects_scrubbed":2,"log_size":3,"log_dups_size":96,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:49:04.391555+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":240,"num_write":135,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":16,"stat_sum":{"num_bytes":10856562712,"num_objects":165659,"num_object_clones":0,"num_object_copies":496977,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":165659,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1161357,"num_write_kb":42464449,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32581758976,"data_stored":32581746760,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":3688,"internal_metadata":0},"log_size":48,"ondisk_log_size":48,"up":48,"acting":48,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":240,"num_write":135,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1076,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":3,"up_from":25,"seq":107374182470,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":8739004,"kb_used_data":6979496,"kb_used_omap":10,"kb_used_meta":1759349,"kb_avail":725264196,"statfs":{"total":751619276800,"available":742670536704,"internally_reserved":0,"allocated":7147003904,"data_stored":7146679840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":10863,"internal_metadata":1801573777},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":19,"seq":81604378696,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":10624508,"kb_used_data":9476840,"kb_used_omap":10,"kb_used_meta":1147573,"kb_avail":723378692,"statfs":{"total":751619276800,"available":740739780608,"internally_reserved":0,"allocated":9704284160,"data_stored":9703960096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":10774,"internal_metadata":1175115242},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542217,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":8255476,"kb_used_data":7326496,"kb_used_omap":11,"kb_used_meta":928948,"kb_avail":725747724,"statfs":{"total":751619276800,"available":743165669376,"internally_reserved":0,"allocated":7502331904,"data_stored":7502015464,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12122,"internal_metadata":951242918},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,1],"upper_bound":4},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705740,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":10511356,"kb_used_data":8907240,"kb_used_omap":8,"kb_used_meta":1604087,"kb_avail":723491844,"statfs":{"total":751619276800,"available":740855648256,"internally_reserved":0,"allocated":9121013760,"data_stored":9120689696,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":8734,"internal_metadata":1642585570},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1076,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8829865984,"data_stored":8829861912,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":7425490944,"data_stored":7425490944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":186,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":9526906880,"data_stored":9526902808,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1350,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":6799495168,"data_stored":6799491096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":2152,"internal_metadata":0}]}} 2026-02-24T07:48:34.422 INFO:tasks.cephadm.ceph_manager.ceph:increase pool size by 10 2026-02-24T07:48:34.422 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph osd pool set .mgr pg_num 201 2026-02-24T07:48:34.607 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:48:34.740 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:48:34.737+0000 7f05b599d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:48:34.740 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:48:34.738+0000 7f05b599d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:48:34.887 INFO:teuthology.orchestra.run.trial035.stderr:Error EPERM: crimson_allow_pg_split is false; pg_num increase denied for crimson pool 2026-02-24T07:48:34.888 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:48:34 trial035 ceph-mon[39284]: pgmap v190: 17 pgs: 17 active+clean; 10 GiB data, 36 GiB used, 2.7 TiB / 2.7 TiB avail; 0 B/s wr, 6.83k op/s 2026-02-24T07:48:34.888 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:48:34 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/3596305652' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2026-02-24T07:48:34.925 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-24T07:48:36.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:48:35 trial035 ceph-mon[39284]: from='client.14816 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2026-02-24T07:48:36.205 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:48:35 trial035 ceph-mon[39284]: 3.c scrub starts 2026-02-24T07:48:36.205 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:48:35 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/347981355' entity='client.admin' cmd={"prefix": "osd pool set", "pool": ".mgr", "var": "pg_num", "val": "201"} : dispatch 2026-02-24T07:48:36.205 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:48:35 trial035 ceph-mon[39284]: 3.c scrub ok 2026-02-24T07:48:36.214 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2026-02-24T07:48:36.363 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:48:36.488 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:48:36.485+0000 7f68d008c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:48:36.488 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:48:36.485+0000 7f68d008c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:48:36.502 INFO:teuthology.orchestra.run.trial035.stdout:osd.0: {} 2026-02-24T07:48:36.503 INFO:teuthology.orchestra.run.trial035.stderr:osd.0: osd_enable_op_tracker = '' 2026-02-24T07:48:36.518 INFO:teuthology.orchestra.run.trial035.stdout:osd.1: {} 2026-02-24T07:48:36.518 INFO:teuthology.orchestra.run.trial035.stderr:osd.1: osd_enable_op_tracker = '' 2026-02-24T07:48:36.531 INFO:teuthology.orchestra.run.trial035.stdout:osd.2: {} 2026-02-24T07:48:36.531 INFO:teuthology.orchestra.run.trial035.stderr:osd.2: osd_enable_op_tracker = '' 2026-02-24T07:48:36.544 INFO:teuthology.orchestra.run.trial035.stdout:osd.3: {} 2026-02-24T07:48:36.544 INFO:teuthology.orchestra.run.trial035.stderr:osd.3: osd_enable_op_tracker = '' 2026-02-24T07:48:37.205 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:48:36 trial035 ceph-mon[39284]: pgmap v191: 17 pgs: 17 active+clean; 10 GiB data, 36 GiB used, 2.7 TiB / 2.7 TiB avail; 0 B/s wr, 6.83k op/s 2026-02-24T07:48:37.205 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:48:36 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/2589263701' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2026-02-24T07:48:37.205 INFO:journalctl@ceph.mgr.x.trial035.stdout:Feb 24 07:48:36 trial035 ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-mgr-x[39621]: ::ffff:10.20.193.91 - - [24/Feb/2026:07:48:36] "GET /metrics HTTP/1.1" 200 4237 "" "Prometheus/3.6.0" 2026-02-24T07:48:37.356 INFO:journalctl@ceph.prometheus.a.trial091.stdout:Feb 24 07:48:36 trial091 ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-prometheus-a[66007]: time=2026-02-24T07:48:36.973Z level=ERROR source=scrape.go:1630 msg="Failed to determine correct type of scrape target." component="scrape manager" scrape_pool=ceph target=http://trial035:9283/metrics content_type="text/html;charset=utf-8" fallback_media_type="" err="received unsupported Content-Type \"text/html;charset=utf-8\" and no fallback_scrape_protocol specified for target" 2026-02-24T07:48:38.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:48:37 trial035 ceph-mon[39284]: 3.8 scrub starts 2026-02-24T07:48:39.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:48:38 trial035 ceph-mon[39284]: 3.8 scrub ok 2026-02-24T07:48:39.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:48:38 trial035 ceph-mon[39284]: pgmap v192: 17 pgs: 17 active+clean; 7.3 GiB data, 28 GiB used, 2.7 TiB / 2.7 TiB avail; 0 B/s wr, 10.67k op/s 2026-02-24T07:48:39.268 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2026-02-24T07:48:39.399 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:48:39.926 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2026-02-24T07:48:39.926 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2026-02-24T07:48:39.926 INFO:tasks.thrashosds.thrasher:Growing pool .mgr 2026-02-24T07:48:39.927 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph pg dump --format=json 2026-02-24T07:48:39.939 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:48:39.875+0000 7fad91684640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:48:39.939 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:48:39.875+0000 7fad91684640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:48:39.939 INFO:teuthology.orchestra.run.trial035.stdout:osd.0: {} 2026-02-24T07:48:39.940 INFO:teuthology.orchestra.run.trial035.stdout:osd.1: {} 2026-02-24T07:48:39.940 INFO:teuthology.orchestra.run.trial035.stdout:osd.2: {} 2026-02-24T07:48:39.940 INFO:teuthology.orchestra.run.trial035.stdout:osd.3: {} 2026-02-24T07:48:39.940 INFO:teuthology.orchestra.run.trial035.stderr:osd.0: osd_enable_op_tracker = '' 2026-02-24T07:48:39.940 INFO:teuthology.orchestra.run.trial035.stderr:osd.1: osd_enable_op_tracker = '' 2026-02-24T07:48:39.940 INFO:teuthology.orchestra.run.trial035.stderr:osd.2: osd_enable_op_tracker = '' 2026-02-24T07:48:39.940 INFO:teuthology.orchestra.run.trial035.stderr:osd.3: osd_enable_op_tracker = '' 2026-02-24T07:48:40.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:48:39 trial035 ceph-mon[39284]: 3.4 scrub starts 2026-02-24T07:48:40.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:48:39 trial035 ceph-mon[39284]: 3.a scrub starts 2026-02-24T07:48:40.205 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:48:39 trial035 ceph-mon[39284]: 3.a scrub ok 2026-02-24T07:48:40.205 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:48:39 trial035 ceph-mon[39284]: 3.5 scrub starts 2026-02-24T07:48:40.696 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:48:40.817 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:48:40.815+0000 7fb9d433b640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:48:40.817 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:48:40.815+0000 7fb9d433b640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:48:40.954 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:48:40 trial035 ceph-mon[39284]: 3.4 scrub ok 2026-02-24T07:48:40.954 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:48:40 trial035 ceph-mon[39284]: pgmap v193: 17 pgs: 17 active+clean; 7.3 GiB data, 28 GiB used, 2.7 TiB / 2.7 TiB avail; 0 B/s wr, 6.88k op/s 2026-02-24T07:48:40.955 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:48:40 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/3704835879' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2026-02-24T07:48:40.955 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:48:40 trial035 ceph-mon[39284]: 3.5 scrub ok 2026-02-24T07:48:40.962 INFO:teuthology.orchestra.run.trial035.stdout: 2026-02-24T07:48:40.962 INFO:teuthology.orchestra.run.trial035.stderr:dumped all 2026-02-24T07:48:41.007 INFO:teuthology.orchestra.run.trial035.stdout:{"pg_ready":true,"pg_map":{"version":193,"stamp":"2026-02-24T07:48:39.703250+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":7841776184,"num_objects":119650,"num_object_clones":0,"num_object_copies":358950,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":119650,"num_whiteouts":0,"num_read":137,"num_read_kb":241,"num_write":1207503,"num_write_kb":42466599,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":52,"ondisk_log_size":52,"up":51,"acting":51,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":51,"num_osds":4,"num_per_pool_osds":4,"num_per_pool_omap_osds":4,"kb":2936012800,"kb_used":29305256,"kb_used_data":23612504,"kb_used_omap":41,"kb_used_meta":5692630,"kb_avail":2906707544,"statfs":{"total":3006477107200,"available":2976468525056,"internally_reserved":0,"allocated":24179204096,"data_stored":24177915464,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":42683,"internal_metadata":5829253445},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,2],"upper_bound":4},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-5411766272,"num_objects":-82577,"num_object_clones":0,"num_object_copies":-247731,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-82577,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":82577,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001537"},"pg_stats":[{"pgid":"3.f","version":"49'75211","reported_seq":75714,"reported_epoch":49,"state":"active+clean","last_fresh":"2026-02-24T07:48:35.706724+0000","last_change":"2026-02-24T07:48:26.639215+0000","last_active":"2026-02-24T07:48:35.706724+0000","last_peered":"2026-02-24T07:48:35.706724+0000","last_clean":"2026-02-24T07:48:35.706724+0000","last_became_active":"2026-02-24T07:46:02.881625+0000","last_became_peered":"2026-02-24T07:46:02.881625+0000","last_unstale":"2026-02-24T07:48:35.706724+0000","last_undegraded":"2026-02-24T07:48:35.706724+0000","last_fullsized":"2026-02-24T07:48:35.706724+0000","mapping_epoch":42,"log_start":"49'75208","ondisk_log_start":"49'75208","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"49'70570","last_scrub_stamp":"2026-02-24T07:48:26.639139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_clean_scrub_stamp":"2026-02-24T07:48:26.639139+0000","objects_scrubbed":12333,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:49:38.413618+0000","scrub_duration":1227,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":484900864,"num_objects":7399,"num_object_clones":0,"num_object_copies":22197,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7399,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":75211,"num_write_kb":2643520,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.e","version":"49'76204","reported_seq":76646,"reported_epoch":49,"state":"active+clean","last_fresh":"2026-02-24T07:48:35.702631+0000","last_change":"2026-02-24T07:47:33.099762+0000","last_active":"2026-02-24T07:48:35.702631+0000","last_peered":"2026-02-24T07:48:35.702631+0000","last_clean":"2026-02-24T07:48:35.702631+0000","last_became_active":"2026-02-24T07:46:02.881083+0000","last_became_peered":"2026-02-24T07:46:02.881083+0000","last_unstale":"2026-02-24T07:48:35.702631+0000","last_undegraded":"2026-02-24T07:48:35.702631+0000","last_fullsized":"2026-02-24T07:48:35.702631+0000","mapping_epoch":42,"log_start":"49'76201","ondisk_log_start":"49'76201","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"46'41533","last_scrub_stamp":"2026-02-24T07:47:33.099675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_clean_scrub_stamp":"2026-02-24T07:47:33.099675+0000","objects_scrubbed":40754,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:48:53.832432+0000","scrub_duration":3969,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":497549312,"num_objects":7592,"num_object_clones":0,"num_object_copies":22776,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7592,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":76204,"num_write_kb":2681472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"49'75448","reported_seq":75911,"reported_epoch":49,"state":"active+clean","last_fresh":"2026-02-24T07:48:35.722743+0000","last_change":"2026-02-24T07:48:28.181137+0000","last_active":"2026-02-24T07:48:35.722743+0000","last_peered":"2026-02-24T07:48:35.722743+0000","last_clean":"2026-02-24T07:48:35.722743+0000","last_became_active":"2026-02-24T07:46:02.883799+0000","last_became_peered":"2026-02-24T07:46:02.883799+0000","last_unstale":"2026-02-24T07:48:35.722743+0000","last_undegraded":"2026-02-24T07:48:35.722743+0000","last_fullsized":"2026-02-24T07:48:35.722743+0000","mapping_epoch":42,"log_start":"49'75445","ondisk_log_start":"49'75445","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"49'71407","last_scrub_stamp":"2026-02-24T07:48:28.181090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_clean_scrub_stamp":"2026-02-24T07:48:28.181090+0000","objects_scrubbed":11940,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:49:56.737033+0000","scrub_duration":1384,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":500236312,"num_objects":7634,"num_object_clones":0,"num_object_copies":22902,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7634,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":75448,"num_write_kb":2658561,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"49'74242","reported_seq":74658,"reported_epoch":49,"state":"active+clean","last_fresh":"2026-02-24T07:48:35.724055+0000","last_change":"2026-02-24T07:48:35.306234+0000","last_active":"2026-02-24T07:48:35.724055+0000","last_peered":"2026-02-24T07:48:35.724055+0000","last_clean":"2026-02-24T07:48:35.724055+0000","last_became_active":"2026-02-24T07:46:02.880060+0000","last_became_peered":"2026-02-24T07:46:02.880060+0000","last_unstale":"2026-02-24T07:48:35.724055+0000","last_undegraded":"2026-02-24T07:48:35.724055+0000","last_fullsized":"2026-02-24T07:48:35.724055+0000","mapping_epoch":42,"log_start":"49'74239","ondisk_log_start":"49'74239","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"49'74008","last_scrub_stamp":"2026-02-24T07:48:35.306189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_clean_scrub_stamp":"2026-02-24T07:48:35.306189+0000","objects_scrubbed":7791,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:50:02.247501+0000","scrub_duration":624,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":484966400,"num_objects":7400,"num_object_clones":0,"num_object_copies":22200,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7400,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":74242,"num_write_kb":2612544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"49'75491","reported_seq":76027,"reported_epoch":49,"state":"active+clean","last_fresh":"2026-02-24T07:48:35.776674+0000","last_change":"2026-02-24T07:48:16.528595+0000","last_active":"2026-02-24T07:48:35.776674+0000","last_peered":"2026-02-24T07:48:35.776674+0000","last_clean":"2026-02-24T07:48:35.776674+0000","last_became_active":"2026-02-24T07:46:02.880171+0000","last_became_peered":"2026-02-24T07:46:02.880171+0000","last_unstale":"2026-02-24T07:48:35.776674+0000","last_undegraded":"2026-02-24T07:48:35.776674+0000","last_fullsized":"2026-02-24T07:48:35.776674+0000","mapping_epoch":42,"log_start":"49'75488","ondisk_log_start":"49'75488","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"49'65111","last_scrub_stamp":"2026-02-24T07:48:16.528547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_clean_scrub_stamp":"2026-02-24T07:48:16.528547+0000","objects_scrubbed":18357,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:49:45.487538+0000","scrub_duration":1833,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":487653376,"num_objects":7441,"num_object_clones":0,"num_object_copies":22323,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7441,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":75491,"num_write_kb":2653824,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.a","version":"49'74765","reported_seq":75208,"reported_epoch":49,"state":"active+clean","last_fresh":"2026-02-24T07:48:35.723949+0000","last_change":"2026-02-24T07:47:35.439589+0000","last_active":"2026-02-24T07:48:35.723949+0000","last_peered":"2026-02-24T07:48:35.723949+0000","last_clean":"2026-02-24T07:48:35.723949+0000","last_became_active":"2026-02-24T07:46:02.882274+0000","last_became_peered":"2026-02-24T07:46:02.882274+0000","last_unstale":"2026-02-24T07:48:35.723949+0000","last_undegraded":"2026-02-24T07:48:35.723949+0000","last_fullsized":"2026-02-24T07:48:35.723949+0000","mapping_epoch":42,"log_start":"49'74762","ondisk_log_start":"49'74762","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"46'41859","last_scrub_stamp":"2026-02-24T07:47:35.439542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_clean_scrub_stamp":"2026-02-24T07:47:35.439542+0000","objects_scrubbed":40699,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:48:38.034606+0000","scrub_duration":3645,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":486342656,"num_objects":7421,"num_object_clones":0,"num_object_copies":22263,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7421,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":74765,"num_write_kb":2629952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"49'76369","reported_seq":76835,"reported_epoch":49,"state":"active+clean","last_fresh":"2026-02-24T07:48:35.722906+0000","last_change":"2026-02-24T07:48:30.710470+0000","last_active":"2026-02-24T07:48:35.722906+0000","last_peered":"2026-02-24T07:48:35.722906+0000","last_clean":"2026-02-24T07:48:35.722906+0000","last_became_active":"2026-02-24T07:46:02.881951+0000","last_became_peered":"2026-02-24T07:46:02.881951+0000","last_unstale":"2026-02-24T07:48:35.722906+0000","last_undegraded":"2026-02-24T07:48:35.722906+0000","last_fullsized":"2026-02-24T07:48:35.722906+0000","mapping_epoch":42,"log_start":"49'76366","ondisk_log_start":"49'76366","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"49'73406","last_scrub_stamp":"2026-02-24T07:48:30.710432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_clean_scrub_stamp":"2026-02-24T07:48:30.710432+0000","objects_scrubbed":10727,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:49:42.678619+0000","scrub_duration":946,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":491192320,"num_objects":7495,"num_object_clones":0,"num_object_copies":22485,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7495,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":76369,"num_write_kb":2683648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"49'75665","reported_seq":76060,"reported_epoch":49,"state":"active+clean","last_fresh":"2026-02-24T07:48:35.770954+0000","last_change":"2026-02-24T07:47:22.289650+0000","last_active":"2026-02-24T07:48:35.770954+0000","last_peered":"2026-02-24T07:48:35.770954+0000","last_clean":"2026-02-24T07:48:35.770954+0000","last_became_active":"2026-02-24T07:46:02.881345+0000","last_became_peered":"2026-02-24T07:46:02.881345+0000","last_unstale":"2026-02-24T07:48:35.770954+0000","last_undegraded":"2026-02-24T07:48:35.770954+0000","last_fullsized":"2026-02-24T07:48:35.770954+0000","mapping_epoch":42,"log_start":"49'75662","ondisk_log_start":"49'75662","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"45'36703","last_scrub_stamp":"2026-02-24T07:47:22.289617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_clean_scrub_stamp":"2026-02-24T07:47:22.289617+0000","objects_scrubbed":35960,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"queued for scrub","scrub_duration":3891,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":495255552,"num_objects":7557,"num_object_clones":0,"num_object_copies":22671,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7557,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":75665,"num_write_kb":2663104,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"49'75084","reported_seq":75528,"reported_epoch":49,"state":"active+clean","last_fresh":"2026-02-24T07:48:35.772227+0000","last_change":"2026-02-24T07:47:35.925072+0000","last_active":"2026-02-24T07:48:35.772227+0000","last_peered":"2026-02-24T07:48:35.772227+0000","last_clean":"2026-02-24T07:48:35.772227+0000","last_became_active":"2026-02-24T07:46:02.880323+0000","last_became_peered":"2026-02-24T07:46:02.880323+0000","last_unstale":"2026-02-24T07:48:35.772227+0000","last_undegraded":"2026-02-24T07:48:35.772227+0000","last_fullsized":"2026-02-24T07:48:35.772227+0000","mapping_epoch":42,"log_start":"49'75081","ondisk_log_start":"49'75081","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"47'42248","last_scrub_stamp":"2026-02-24T07:47:35.925000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_clean_scrub_stamp":"2026-02-24T07:47:35.925000+0000","objects_scrubbed":40858,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:48:43.057825+0000","scrub_duration":3382,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":484442112,"num_objects":7392,"num_object_clones":0,"num_object_copies":22176,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7392,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":75084,"num_write_kb":2639232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.6","version":"49'75669","reported_seq":76069,"reported_epoch":49,"state":"active+clean","last_fresh":"2026-02-24T07:48:35.730070+0000","last_change":"2026-02-24T07:47:23.432795+0000","last_active":"2026-02-24T07:48:35.730070+0000","last_peered":"2026-02-24T07:48:35.730070+0000","last_clean":"2026-02-24T07:48:35.730070+0000","last_became_active":"2026-02-24T07:46:02.880005+0000","last_became_peered":"2026-02-24T07:46:02.880005+0000","last_unstale":"2026-02-24T07:48:35.730070+0000","last_undegraded":"2026-02-24T07:48:35.730070+0000","last_fullsized":"2026-02-24T07:48:35.730070+0000","mapping_epoch":42,"log_start":"49'75666","ondisk_log_start":"49'75666","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"45'37219","last_scrub_stamp":"2026-02-24T07:47:23.432744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_clean_scrub_stamp":"2026-02-24T07:47:23.432744+0000","objects_scrubbed":36510,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:48:51.822832+0000","scrub_duration":3412,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":482541568,"num_objects":7363,"num_object_clones":0,"num_object_copies":22089,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7363,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":75669,"num_write_kb":2657024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"49'76034","reported_seq":76411,"reported_epoch":49,"state":"active+clean","last_fresh":"2026-02-24T07:48:35.708041+0000","last_change":"2026-02-24T07:47:17.546625+0000","last_active":"2026-02-24T07:48:35.708041+0000","last_peered":"2026-02-24T07:48:35.708041+0000","last_clean":"2026-02-24T07:48:35.708041+0000","last_became_active":"2026-02-24T07:46:02.881420+0000","last_became_peered":"2026-02-24T07:46:02.881420+0000","last_unstale":"2026-02-24T07:48:35.708041+0000","last_undegraded":"2026-02-24T07:48:35.708041+0000","last_fullsized":"2026-02-24T07:48:35.708041+0000","mapping_epoch":42,"log_start":"49'76031","ondisk_log_start":"49'76031","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"45'34854","last_scrub_stamp":"2026-02-24T07:47:17.546590+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_clean_scrub_stamp":"2026-02-24T07:47:17.546590+0000","objects_scrubbed":34222,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:48:38.813192+0000","scrub_duration":3116,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":499384320,"num_objects":7620,"num_object_clones":0,"num_object_copies":22860,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7620,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":76034,"num_write_kb":2676928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.4","version":"49'74949","reported_seq":75388,"reported_epoch":49,"state":"active+clean","last_fresh":"2026-02-24T07:48:35.778799+0000","last_change":"2026-02-24T07:47:33.628245+0000","last_active":"2026-02-24T07:48:35.778799+0000","last_peered":"2026-02-24T07:48:35.778799+0000","last_clean":"2026-02-24T07:48:35.778799+0000","last_became_active":"2026-02-24T07:46:02.880345+0000","last_became_peered":"2026-02-24T07:46:02.880345+0000","last_unstale":"2026-02-24T07:48:35.778799+0000","last_undegraded":"2026-02-24T07:48:35.778799+0000","last_fullsized":"2026-02-24T07:48:35.778799+0000","mapping_epoch":42,"log_start":"49'74946","ondisk_log_start":"49'74946","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"46'41076","last_scrub_stamp":"2026-02-24T07:47:33.628218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_clean_scrub_stamp":"2026-02-24T07:47:33.628218+0000","objects_scrubbed":40364,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"queued for scrub","scrub_duration":4080,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":489881600,"num_objects":7475,"num_object_clones":0,"num_object_copies":22425,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7475,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":74949,"num_write_kb":2637568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"49'75566","reported_seq":76075,"reported_epoch":49,"state":"active+clean","last_fresh":"2026-02-24T07:48:35.709149+0000","last_change":"2026-02-24T07:48:24.553666+0000","last_active":"2026-02-24T07:48:35.709149+0000","last_peered":"2026-02-24T07:48:35.709149+0000","last_clean":"2026-02-24T07:48:35.709149+0000","last_became_active":"2026-02-24T07:46:02.881035+0000","last_became_peered":"2026-02-24T07:46:02.881035+0000","last_unstale":"2026-02-24T07:48:35.709149+0000","last_undegraded":"2026-02-24T07:48:35.709149+0000","last_fullsized":"2026-02-24T07:48:35.709149+0000","mapping_epoch":42,"log_start":"49'75563","ondisk_log_start":"49'75563","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"49'69716","last_scrub_stamp":"2026-02-24T07:48:24.553517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_clean_scrub_stamp":"2026-02-24T07:48:24.553517+0000","objects_scrubbed":13748,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:49:36.493266+0000","scrub_duration":1167,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":498335744,"num_objects":7604,"num_object_clones":0,"num_object_copies":22812,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7604,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":75566,"num_write_kb":2661440,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"49'75544","reported_seq":75948,"reported_epoch":49,"state":"active+clean","last_fresh":"2026-02-24T07:48:35.722943+0000","last_change":"2026-02-24T07:47:24.807734+0000","last_active":"2026-02-24T07:48:35.722943+0000","last_peered":"2026-02-24T07:48:35.722943+0000","last_clean":"2026-02-24T07:48:35.722943+0000","last_became_active":"2026-02-24T07:46:02.879831+0000","last_became_peered":"2026-02-24T07:46:02.879831+0000","last_unstale":"2026-02-24T07:48:35.722943+0000","last_undegraded":"2026-02-24T07:48:35.722943+0000","last_fullsized":"2026-02-24T07:48:35.722943+0000","mapping_epoch":42,"log_start":"49'75541","ondisk_log_start":"49'75541","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"45'37558","last_scrub_stamp":"2026-02-24T07:47:24.807659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_clean_scrub_stamp":"2026-02-24T07:47:24.807659+0000","objects_scrubbed":36869,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:48:39.172465+0000","scrub_duration":3748,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":476971008,"num_objects":7278,"num_object_clones":0,"num_object_copies":21834,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7278,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":75544,"num_write_kb":2650304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.0","version":"49'75157","reported_seq":75611,"reported_epoch":49,"state":"active+clean","last_fresh":"2026-02-24T07:48:35.781205+0000","last_change":"2026-02-24T07:48:30.080353+0000","last_active":"2026-02-24T07:48:35.781205+0000","last_peered":"2026-02-24T07:48:35.781205+0000","last_clean":"2026-02-24T07:48:35.781205+0000","last_became_active":"2026-02-24T07:46:02.880333+0000","last_became_peered":"2026-02-24T07:46:02.880333+0000","last_unstale":"2026-02-24T07:48:35.781205+0000","last_undegraded":"2026-02-24T07:48:35.781205+0000","last_fullsized":"2026-02-24T07:48:35.781205+0000","mapping_epoch":42,"log_start":"49'75153","ondisk_log_start":"49'75153","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"49'71923","last_scrub_stamp":"2026-02-24T07:48:30.080305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_clean_scrub_stamp":"2026-02-24T07:48:30.080305+0000","objects_scrubbed":11094,"log_size":4,"log_dups_size":3000,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:49:53.942641+0000","scrub_duration":1405,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":496041984,"num_objects":7569,"num_object_clones":0,"num_object_copies":22707,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7569,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":75157,"num_write_kb":2647232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2","version":"49'75970","reported_seq":76387,"reported_epoch":49,"state":"active+clean","last_fresh":"2026-02-24T07:48:35.778713+0000","last_change":"2026-02-24T07:47:27.784936+0000","last_active":"2026-02-24T07:48:35.778713+0000","last_peered":"2026-02-24T07:48:35.778713+0000","last_clean":"2026-02-24T07:48:35.778713+0000","last_became_active":"2026-02-24T07:46:02.880185+0000","last_became_peered":"2026-02-24T07:46:02.880185+0000","last_unstale":"2026-02-24T07:48:35.778713+0000","last_undegraded":"2026-02-24T07:48:35.778713+0000","last_fullsized":"2026-02-24T07:48:35.778713+0000","mapping_epoch":42,"log_start":"49'75967","ondisk_log_start":"49'75967","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"45'39074","last_scrub_stamp":"2026-02-24T07:47:27.784888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_clean_scrub_stamp":"2026-02-24T07:47:27.784888+0000","objects_scrubbed":38200,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:48:47.063599+0000","scrub_duration":4266,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":485490688,"num_objects":7408,"num_object_clones":0,"num_object_copies":22224,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7408,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":75970,"num_write_kb":2668096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"29'99","reported_seq":253,"reported_epoch":49,"state":"active+clean","last_fresh":"2026-02-24T07:48:02.589664+0000","last_change":"2026-02-24T07:48:02.589664+0000","last_active":"2026-02-24T07:48:02.589664+0000","last_peered":"2026-02-24T07:48:02.589664+0000","last_clean":"2026-02-24T07:48:02.589664+0000","last_became_active":"2026-02-24T07:42:54.186719+0000","last_became_peered":"2026-02-24T07:42:54.186719+0000","last_unstale":"2026-02-24T07:48:02.589664+0000","last_undegraded":"2026-02-24T07:48:02.589664+0000","last_fullsized":"2026-02-24T07:48:02.589664+0000","mapping_epoch":27,"log_start":"29'96","ondisk_log_start":"29'96","created":20,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'99","last_scrub_stamp":"2026-02-24T07:48:02.589609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:42:43.640776+0000","last_clean_scrub_stamp":"2026-02-24T07:48:02.589609+0000","objects_scrubbed":2,"log_size":3,"log_dups_size":96,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:49:04.391555+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":240,"num_write":135,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":16,"stat_sum":{"num_bytes":7841185816,"num_objects":119648,"num_object_clones":0,"num_object_copies":358944,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":119648,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1207368,"num_write_kb":42464449,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":23537528832,"data_stored":23537516616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":3336,"internal_metadata":0},"log_size":49,"ondisk_log_size":49,"up":48,"acting":48,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":240,"num_write":135,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":900,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":3,"up_from":25,"seq":107374182471,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":6637756,"kb_used_data":4824232,"kb_used_omap":10,"kb_used_meta":1813493,"kb_avail":727365444,"statfs":{"total":751619276800,"available":744822214656,"internally_reserved":0,"allocated":4940013568,"data_stored":4939689504,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":11053,"internal_metadata":1857017043},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":19,"seq":81604378697,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":8120508,"kb_used_data":6900712,"kb_used_omap":10,"kb_used_meta":1219765,"kb_avail":725882692,"statfs":{"total":751619276800,"available":743303876608,"internally_reserved":0,"allocated":7066329088,"data_stored":7066005024,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":10774,"internal_metadata":1249039850},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542218,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":6342836,"kb_used_data":5359712,"kb_used_omap":11,"kb_used_meta":983092,"kb_avail":727660364,"statfs":{"total":751619276800,"available":745124212736,"internally_reserved":0,"allocated":5488345088,"data_stored":5488028648,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12122,"internal_metadata":1006686374},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705741,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":8204156,"kb_used_data":6527848,"kb_used_omap":8,"kb_used_meta":1676279,"kb_avail":725799044,"statfs":{"total":751619276800,"available":743218221056,"internally_reserved":0,"allocated":6684516352,"data_stored":6684192288,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":8734,"internal_metadata":1716510178},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,2],"upper_bound":4},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":900,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":6378164224,"data_stored":6378160152,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":5356978176,"data_stored":5356978176,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":186,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":6887837696,"data_stored":6887833624,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1350,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4914548736,"data_stored":4914544664,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1800,"internal_metadata":0}]}} 2026-02-24T07:48:41.009 INFO:tasks.cephadm.ceph_manager.ceph:increase pool size by 10 2026-02-24T07:48:41.009 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph osd pool set .mgr pg_num 211 2026-02-24T07:48:41.151 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:48:41.277 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:48:41.274+0000 7f7499c26640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:48:41.277 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:48:41.275+0000 7f7499c26640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:48:41.415 INFO:teuthology.orchestra.run.trial035.stderr:Error EPERM: crimson_allow_pg_split is false; pg_num increase denied for crimson pool 2026-02-24T07:48:41.456 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-24T07:48:42.205 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:48:41 trial035 ceph-mon[39284]: 3.1 scrub starts 2026-02-24T07:48:42.205 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:48:41 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/2579944565' entity='client.admin' cmd={"prefix": "osd pool set", "pool": ".mgr", "var": "pg_num", "val": "211"} : dispatch 2026-02-24T07:48:42.641 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2026-02-24T07:48:42.782 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:48:42.906 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:48:42.904+0000 7f8364172640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:48:42.906 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:48:42.904+0000 7f8364172640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:48:42.923 INFO:teuthology.orchestra.run.trial035.stdout:osd.0: {} 2026-02-24T07:48:42.924 INFO:teuthology.orchestra.run.trial035.stderr:osd.0: osd_enable_op_tracker = '' 2026-02-24T07:48:42.924 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:48:42 trial035 ceph-mon[39284]: from='client.14824 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2026-02-24T07:48:42.924 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:48:42 trial035 ceph-mon[39284]: 3.1 scrub ok 2026-02-24T07:48:42.924 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:48:42 trial035 ceph-mon[39284]: pgmap v194: 17 pgs: 17 active+clean; 5.0 GiB data, 20 GiB used, 2.7 TiB / 2.7 TiB avail; 0 B/s wr, 10.04k op/s 2026-02-24T07:48:42.936 INFO:teuthology.orchestra.run.trial035.stdout:osd.1: {} 2026-02-24T07:48:42.936 INFO:teuthology.orchestra.run.trial035.stderr:osd.1: osd_enable_op_tracker = '' 2026-02-24T07:48:42.949 INFO:teuthology.orchestra.run.trial035.stdout:osd.2: {} 2026-02-24T07:48:42.950 INFO:teuthology.orchestra.run.trial035.stderr:osd.2: osd_enable_op_tracker = '' 2026-02-24T07:48:42.962 INFO:teuthology.orchestra.run.trial035.stdout:osd.3: {} 2026-02-24T07:48:42.962 INFO:teuthology.orchestra.run.trial035.stderr:osd.3: osd_enable_op_tracker = '' 2026-02-24T07:48:44.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:48:43 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/838759615' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2026-02-24T07:48:45.005 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2026-02-24T07:48:45.136 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:48:45.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:48:44 trial035 ceph-mon[39284]: pgmap v195: 17 pgs: 17 active+clean; 5.0 GiB data, 20 GiB used, 2.7 TiB / 2.7 TiB avail; 0 B/s wr, 6.99k op/s 2026-02-24T07:48:45.566 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:48:45.546+0000 7f26c9e39640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:48:45.566 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:48:45.546+0000 7f26c9e39640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:48:45.579 INFO:teuthology.orchestra.run.trial035.stdout:osd.0: {} 2026-02-24T07:48:45.579 INFO:teuthology.orchestra.run.trial035.stderr:osd.0: osd_enable_op_tracker = '' 2026-02-24T07:48:45.593 INFO:teuthology.orchestra.run.trial035.stdout:osd.1: {} 2026-02-24T07:48:45.593 INFO:teuthology.orchestra.run.trial035.stderr:osd.1: osd_enable_op_tracker = '' 2026-02-24T07:48:45.605 INFO:teuthology.orchestra.run.trial035.stdout:osd.2: {} 2026-02-24T07:48:45.606 INFO:teuthology.orchestra.run.trial035.stderr:osd.2: osd_enable_op_tracker = '' 2026-02-24T07:48:45.618 INFO:teuthology.orchestra.run.trial035.stdout:osd.3: {} 2026-02-24T07:48:45.618 INFO:teuthology.orchestra.run.trial035.stderr:osd.3: osd_enable_op_tracker = '' 2026-02-24T07:48:46.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:48:45 trial035 ceph-mon[39284]: 3.7 scrub starts 2026-02-24T07:48:46.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:48:45 trial035 ceph-mon[39284]: from='mgr.14284 10.20.193.35:0/3681373222' entity='mgr.x' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2026-02-24T07:48:46.205 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:48:45 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/917481373' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2026-02-24T07:48:46.457 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2026-02-24T07:48:46.458 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph osd reweight 0 1 2026-02-24T07:48:46.607 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:48:47.045 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:48:46 trial035 ceph-mon[39284]: 3.7 scrub ok 2026-02-24T07:48:47.045 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:48:46 trial035 ceph-mon[39284]: pgmap v196: 17 pgs: 17 active+clean; 5.0 GiB data, 20 GiB used, 2.7 TiB / 2.7 TiB avail; 0 B/s wr, 6.99k op/s 2026-02-24T07:48:47.046 INFO:journalctl@ceph.mgr.x.trial035.stdout:Feb 24 07:48:46 trial035 ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-mgr-x[39621]: ::ffff:10.20.193.91 - - [24/Feb/2026:07:48:46] "GET /metrics HTTP/1.1" 200 4237 "" "Prometheus/3.6.0" 2026-02-24T07:48:47.067 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:48:47.043+0000 7fc88ca18640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:48:47.067 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:48:47.043+0000 7fc88ca18640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:48:47.264 INFO:journalctl@ceph.prometheus.a.trial091.stdout:Feb 24 07:48:46 trial091 ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-prometheus-a[66007]: time=2026-02-24T07:48:46.973Z level=ERROR source=scrape.go:1630 msg="Failed to determine correct type of scrape target." component="scrape manager" scrape_pool=ceph target=http://trial035:9283/metrics content_type="text/html;charset=utf-8" fallback_media_type="" err="received unsupported Content-Type \"text/html;charset=utf-8\" and no fallback_scrape_protocol specified for target" 2026-02-24T07:48:47.871 INFO:teuthology.orchestra.run.trial035.stderr:reweighted osd.0 to 1 (10000) 2026-02-24T07:48:47.917 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph osd reweight 1 1 2026-02-24T07:48:48.062 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:48:48.124 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:48:47 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/3512846682' entity='client.admin' cmd={"prefix": "osd reweight", "id": 0, "weight": 1.0} : dispatch 2026-02-24T07:48:48.173 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:48:48.171+0000 7f5bc89c1640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:48:48.174 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:48:48.171+0000 7f5bc89c1640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:48:48.535 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2026-02-24T07:48:48.681 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:48:48.806 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:48:48.804+0000 7fe19bb7b640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:48:48.806 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:48:48.804+0000 7fe19bb7b640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:48:48.825 INFO:teuthology.orchestra.run.trial035.stdout:osd.0: {} 2026-02-24T07:48:48.825 INFO:teuthology.orchestra.run.trial035.stderr:osd.0: osd_enable_op_tracker = '' 2026-02-24T07:48:48.842 INFO:teuthology.orchestra.run.trial035.stdout:osd.1: {} 2026-02-24T07:48:48.843 INFO:teuthology.orchestra.run.trial035.stderr:osd.1: osd_enable_op_tracker = '' 2026-02-24T07:48:48.856 INFO:teuthology.orchestra.run.trial035.stdout:osd.2: {} 2026-02-24T07:48:48.856 INFO:teuthology.orchestra.run.trial035.stderr:osd.2: osd_enable_op_tracker = '' 2026-02-24T07:48:48.869 INFO:teuthology.orchestra.run.trial035.stdout:osd.3: {} 2026-02-24T07:48:48.869 INFO:teuthology.orchestra.run.trial035.stderr:osd.3: osd_enable_op_tracker = '' 2026-02-24T07:48:48.871 INFO:teuthology.orchestra.run.trial035.stderr:reweighted osd.1 to 1 (10000) 2026-02-24T07:48:49.183 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:48:48 trial035 ceph-mon[39284]: pgmap v197: 17 pgs: 17 active+clean; 2.3 GiB data, 11 GiB used, 2.7 TiB / 2.7 TiB avail; 0 B/s wr, 10.69k op/s 2026-02-24T07:48:49.184 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:48:48 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/3512846682' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 0, "weight": 1.0}]': finished 2026-02-24T07:48:49.184 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:48:48 trial035 ceph-mon[39284]: osdmap e50: 4 total, 4 up, 4 in 2026-02-24T07:48:49.184 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:48:48 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/829227549' entity='client.admin' cmd={"prefix": "osd reweight", "id": 1, "weight": 1.0} : dispatch 2026-02-24T07:48:49.184 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:48:48 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/48772364' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2026-02-24T07:48:49.447 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph osd reweight 2 1 2026-02-24T07:48:49.628 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:48:49.752 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:48:49.750+0000 7fde577e7640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:48:49.753 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:48:49.750+0000 7fde577e7640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:48:49.954 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:48:49 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/829227549' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 1, "weight": 1.0}]': finished 2026-02-24T07:48:49.954 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:48:49 trial035 ceph-mon[39284]: osdmap e51: 4 total, 4 up, 4 in 2026-02-24T07:48:50.149 INFO:tasks.radosbench.radosbench.0.trial035.stdout:Removed 663507 objects 2026-02-24T07:48:50.149 INFO:tasks.radosbench.radosbench.0.trial035.stdout:Clean up completed and total clean up time :76.1676 2026-02-24T07:48:50.878 INFO:teuthology.orchestra.run.trial035.stderr:reweighted osd.2 to 1 (10000) 2026-02-24T07:48:50.919 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph osd reweight 3 1 2026-02-24T07:48:51.036 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:48:51.132 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:48:50 trial035 ceph-mon[39284]: pgmap v200: 17 pgs: 17 active+clean; 2.3 GiB data, 11 GiB used, 2.7 TiB / 2.7 TiB avail; 0 B/s wr, 5.56k op/s 2026-02-24T07:48:51.132 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:48:50 trial035 ceph-mon[39284]: 3.2 scrub starts 2026-02-24T07:48:51.132 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:48:50 trial035 ceph-mon[39284]: 3.2 scrub ok 2026-02-24T07:48:51.132 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:48:50 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/2515542689' entity='client.admin' cmd={"prefix": "osd reweight", "id": 2, "weight": 1.0} : dispatch 2026-02-24T07:48:51.133 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:48:51.130+0000 7f837194e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:48:51.133 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:48:51.130+0000 7f837194e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:48:51.408 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2026-02-24T07:48:51.526 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:48:51.628 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:48:51.626+0000 7f597e1ca640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:48:51.629 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:48:51.626+0000 7f597e1ca640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:48:51.641 INFO:teuthology.orchestra.run.trial035.stdout:osd.0: {} 2026-02-24T07:48:51.641 INFO:teuthology.orchestra.run.trial035.stderr:osd.0: osd_enable_op_tracker = '' 2026-02-24T07:48:51.652 INFO:teuthology.orchestra.run.trial035.stdout:osd.1: {} 2026-02-24T07:48:51.653 INFO:teuthology.orchestra.run.trial035.stderr:osd.1: osd_enable_op_tracker = '' 2026-02-24T07:48:51.663 INFO:teuthology.orchestra.run.trial035.stdout:osd.2: {} 2026-02-24T07:48:51.663 INFO:teuthology.orchestra.run.trial035.stderr:osd.2: osd_enable_op_tracker = '' 2026-02-24T07:48:51.674 INFO:teuthology.orchestra.run.trial035.stdout:osd.3: {} 2026-02-24T07:48:51.674 INFO:teuthology.orchestra.run.trial035.stderr:osd.3: osd_enable_op_tracker = '' 2026-02-24T07:48:51.884 INFO:teuthology.orchestra.run.trial035.stderr:reweighted osd.3 to 1 (10000) 2026-02-24T07:48:51.922 INFO:tasks.cephadm.ceph_manager.ceph:waiting for recovery to complete 2026-02-24T07:48:51.922 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph pg dump --format=json 2026-02-24T07:48:51.946 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:48:51 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/2515542689' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 2, "weight": 1.0}]': finished 2026-02-24T07:48:51.946 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:48:51 trial035 ceph-mon[39284]: osdmap e52: 4 total, 4 up, 4 in 2026-02-24T07:48:51.946 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:48:51 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/874565567' entity='client.admin' cmd={"prefix": "osd reweight", "id": 3, "weight": 1.0} : dispatch 2026-02-24T07:48:51.946 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:48:51 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/3468192675' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2026-02-24T07:48:52.042 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:48:52.140 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:48:52.137+0000 7f0b7faf9640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:48:52.140 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:48:52.137+0000 7f0b7faf9640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:48:52.249 INFO:teuthology.orchestra.run.trial035.stdout: 2026-02-24T07:48:52.250 INFO:teuthology.orchestra.run.trial035.stderr:dumped all 2026-02-24T07:48:52.283 INFO:teuthology.orchestra.run.trial035.stdout:{"pg_ready":true,"pg_map":{"version":203,"stamp":"2026-02-24T07:48:51.883303+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":137,"num_read_kb":241,"num_write":1327151,"num_write_kb":42466599,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":51,"ondisk_log_size":51,"up":51,"acting":51,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":51,"num_osds":4,"num_per_pool_osds":4,"num_per_pool_omap_osds":4,"kb":2936012800,"kb_used":4759040,"kb_used_data":37040,"kb_used_omap":47,"kb_used_meta":4721808,"kb_avail":2931253760,"statfs":{"total":3006477107200,"available":3001603850240,"internally_reserved":0,"allocated":37928960,"data_stored":36585440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":48531,"internal_metadata":4835132013},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-2445148184,"num_objects":-37311,"num_object_clones":0,"num_object_copies":-111933,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-37311,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":37311,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.179056"},"pg_stats":[{"pgid":"3.f","version":"51'82610","reported_seq":83117,"reported_epoch":51,"state":"active+clean","last_fresh":"2026-02-24T07:48:50.147899+0000","last_change":"2026-02-24T07:48:26.639215+0000","last_active":"2026-02-24T07:48:50.147899+0000","last_peered":"2026-02-24T07:48:50.147899+0000","last_clean":"2026-02-24T07:48:50.147899+0000","last_became_active":"2026-02-24T07:46:02.881625+0000","last_became_peered":"2026-02-24T07:46:02.881625+0000","last_unstale":"2026-02-24T07:48:50.147899+0000","last_undegraded":"2026-02-24T07:48:50.147899+0000","last_fullsized":"2026-02-24T07:48:50.147899+0000","mapping_epoch":42,"log_start":"51'82607","ondisk_log_start":"51'82607","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"49'70570","last_scrub_stamp":"2026-02-24T07:48:26.639139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_clean_scrub_stamp":"2026-02-24T07:48:26.639139+0000","objects_scrubbed":12333,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:49:38.413618+0000","scrub_duration":1227,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":82610,"num_write_kb":2643520,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.e","version":"51'83796","reported_seq":84242,"reported_epoch":51,"state":"active+clean","last_fresh":"2026-02-24T07:48:50.144259+0000","last_change":"2026-02-24T07:47:33.099762+0000","last_active":"2026-02-24T07:48:50.144259+0000","last_peered":"2026-02-24T07:48:50.144259+0000","last_clean":"2026-02-24T07:48:50.144259+0000","last_became_active":"2026-02-24T07:46:02.881083+0000","last_became_peered":"2026-02-24T07:46:02.881083+0000","last_unstale":"2026-02-24T07:48:50.144259+0000","last_undegraded":"2026-02-24T07:48:50.144259+0000","last_fullsized":"2026-02-24T07:48:50.144259+0000","mapping_epoch":42,"log_start":"51'83793","ondisk_log_start":"51'83793","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"46'41533","last_scrub_stamp":"2026-02-24T07:47:33.099675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_clean_scrub_stamp":"2026-02-24T07:47:33.099675+0000","objects_scrubbed":40754,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:48:53.832432+0000","scrub_duration":3969,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":83796,"num_write_kb":2681472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"51'83082","reported_seq":83549,"reported_epoch":51,"state":"active+clean","last_fresh":"2026-02-24T07:48:50.147985+0000","last_change":"2026-02-24T07:48:28.181137+0000","last_active":"2026-02-24T07:48:50.147985+0000","last_peered":"2026-02-24T07:48:50.147985+0000","last_clean":"2026-02-24T07:48:50.147985+0000","last_became_active":"2026-02-24T07:46:02.883799+0000","last_became_peered":"2026-02-24T07:46:02.883799+0000","last_unstale":"2026-02-24T07:48:50.147985+0000","last_undegraded":"2026-02-24T07:48:50.147985+0000","last_fullsized":"2026-02-24T07:48:50.147985+0000","mapping_epoch":42,"log_start":"51'83079","ondisk_log_start":"51'83079","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"49'71407","last_scrub_stamp":"2026-02-24T07:48:28.181090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_clean_scrub_stamp":"2026-02-24T07:48:28.181090+0000","objects_scrubbed":11940,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:49:56.737033+0000","scrub_duration":1384,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":83082,"num_write_kb":2658561,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"51'81642","reported_seq":82062,"reported_epoch":51,"state":"active+clean","last_fresh":"2026-02-24T07:48:50.142582+0000","last_change":"2026-02-24T07:48:35.306234+0000","last_active":"2026-02-24T07:48:50.142582+0000","last_peered":"2026-02-24T07:48:50.142582+0000","last_clean":"2026-02-24T07:48:50.142582+0000","last_became_active":"2026-02-24T07:46:02.880060+0000","last_became_peered":"2026-02-24T07:46:02.880060+0000","last_unstale":"2026-02-24T07:48:50.142582+0000","last_undegraded":"2026-02-24T07:48:50.142582+0000","last_fullsized":"2026-02-24T07:48:50.142582+0000","mapping_epoch":42,"log_start":"51'81639","ondisk_log_start":"51'81639","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"49'74008","last_scrub_stamp":"2026-02-24T07:48:35.306189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_clean_scrub_stamp":"2026-02-24T07:48:35.306189+0000","objects_scrubbed":7791,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:50:02.247501+0000","scrub_duration":624,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":81642,"num_write_kb":2612544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"51'82932","reported_seq":83472,"reported_epoch":51,"state":"active+clean","last_fresh":"2026-02-24T07:48:50.141473+0000","last_change":"2026-02-24T07:48:16.528595+0000","last_active":"2026-02-24T07:48:50.141473+0000","last_peered":"2026-02-24T07:48:50.141473+0000","last_clean":"2026-02-24T07:48:50.141473+0000","last_became_active":"2026-02-24T07:46:02.880171+0000","last_became_peered":"2026-02-24T07:46:02.880171+0000","last_unstale":"2026-02-24T07:48:50.141473+0000","last_undegraded":"2026-02-24T07:48:50.141473+0000","last_fullsized":"2026-02-24T07:48:50.141473+0000","mapping_epoch":42,"log_start":"51'82929","ondisk_log_start":"51'82929","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"49'65111","last_scrub_stamp":"2026-02-24T07:48:16.528547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_clean_scrub_stamp":"2026-02-24T07:48:16.528547+0000","objects_scrubbed":18357,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:49:45.487538+0000","scrub_duration":1833,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":82932,"num_write_kb":2653824,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.a","version":"51'82186","reported_seq":82701,"reported_epoch":51,"state":"active+clean","last_fresh":"2026-02-24T07:48:50.145111+0000","last_change":"2026-02-24T07:48:39.224243+0000","last_active":"2026-02-24T07:48:50.145111+0000","last_peered":"2026-02-24T07:48:50.145111+0000","last_clean":"2026-02-24T07:48:50.145111+0000","last_became_active":"2026-02-24T07:46:02.882274+0000","last_became_peered":"2026-02-24T07:46:02.882274+0000","last_unstale":"2026-02-24T07:48:50.145111+0000","last_undegraded":"2026-02-24T07:48:50.145111+0000","last_fullsized":"2026-02-24T07:48:50.145111+0000","mapping_epoch":42,"log_start":"51'82183","ondisk_log_start":"51'82183","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"49'76461","last_scrub_stamp":"2026-02-24T07:48:39.224222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_clean_scrub_stamp":"2026-02-24T07:48:39.224222+0000","objects_scrubbed":5854,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:50:02.455067+0000","scrub_duration":543,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":82186,"num_write_kb":2629952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"51'83864","reported_seq":84334,"reported_epoch":51,"state":"active+clean","last_fresh":"2026-02-24T07:48:50.144442+0000","last_change":"2026-02-24T07:48:30.710470+0000","last_active":"2026-02-24T07:48:50.144442+0000","last_peered":"2026-02-24T07:48:50.144442+0000","last_clean":"2026-02-24T07:48:50.144442+0000","last_became_active":"2026-02-24T07:46:02.881951+0000","last_became_peered":"2026-02-24T07:46:02.881951+0000","last_unstale":"2026-02-24T07:48:50.144442+0000","last_undegraded":"2026-02-24T07:48:50.144442+0000","last_fullsized":"2026-02-24T07:48:50.144442+0000","mapping_epoch":42,"log_start":"51'83861","ondisk_log_start":"51'83861","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"49'73406","last_scrub_stamp":"2026-02-24T07:48:30.710432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_clean_scrub_stamp":"2026-02-24T07:48:30.710432+0000","objects_scrubbed":10727,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:49:42.678619+0000","scrub_duration":946,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":83864,"num_write_kb":2683648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"51'83222","reported_seq":83699,"reported_epoch":51,"state":"active+clean","last_fresh":"2026-02-24T07:48:50.147078+0000","last_change":"2026-02-24T07:48:37.650256+0000","last_active":"2026-02-24T07:48:50.147078+0000","last_peered":"2026-02-24T07:48:50.147078+0000","last_clean":"2026-02-24T07:48:50.147078+0000","last_became_active":"2026-02-24T07:46:02.881345+0000","last_became_peered":"2026-02-24T07:46:02.881345+0000","last_unstale":"2026-02-24T07:48:50.147078+0000","last_undegraded":"2026-02-24T07:48:50.147078+0000","last_fullsized":"2026-02-24T07:48:50.147078+0000","mapping_epoch":42,"log_start":"51'83219","ondisk_log_start":"51'83219","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"49'76565","last_scrub_stamp":"2026-02-24T07:48:37.650220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_clean_scrub_stamp":"2026-02-24T07:48:37.650220+0000","objects_scrubbed":6908,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:50:07.037617+0000","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":83222,"num_write_kb":2663104,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"51'82476","reported_seq":82960,"reported_epoch":51,"state":"active+clean","last_fresh":"2026-02-24T07:48:50.146836+0000","last_change":"2026-02-24T07:48:44.952653+0000","last_active":"2026-02-24T07:48:50.146836+0000","last_peered":"2026-02-24T07:48:50.146836+0000","last_clean":"2026-02-24T07:48:50.146836+0000","last_became_active":"2026-02-24T07:46:02.880323+0000","last_became_peered":"2026-02-24T07:46:02.880323+0000","last_unstale":"2026-02-24T07:48:50.146836+0000","last_undegraded":"2026-02-24T07:48:50.146836+0000","last_fullsized":"2026-02-24T07:48:50.146836+0000","mapping_epoch":42,"log_start":"51'82473","ondisk_log_start":"51'82473","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"49'79775","last_scrub_stamp":"2026-02-24T07:48:44.952583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_clean_scrub_stamp":"2026-02-24T07:48:44.952583+0000","objects_scrubbed":2777,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:50:13.253334+0000","scrub_duration":272,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":82476,"num_write_kb":2639232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.6","version":"51'83032","reported_seq":83436,"reported_epoch":51,"state":"active+clean","last_fresh":"2026-02-24T07:48:50.146007+0000","last_change":"2026-02-24T07:47:23.432795+0000","last_active":"2026-02-24T07:48:50.146007+0000","last_peered":"2026-02-24T07:48:50.146007+0000","last_clean":"2026-02-24T07:48:50.146007+0000","last_became_active":"2026-02-24T07:46:02.880005+0000","last_became_peered":"2026-02-24T07:46:02.880005+0000","last_unstale":"2026-02-24T07:48:50.146007+0000","last_undegraded":"2026-02-24T07:48:50.146007+0000","last_fullsized":"2026-02-24T07:48:50.146007+0000","mapping_epoch":42,"log_start":"51'83029","ondisk_log_start":"51'83029","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"45'37219","last_scrub_stamp":"2026-02-24T07:47:23.432744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_clean_scrub_stamp":"2026-02-24T07:47:23.432744+0000","objects_scrubbed":36510,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:48:51.822832+0000","scrub_duration":3412,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":83032,"num_write_kb":2657024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"51'83654","reported_seq":84100,"reported_epoch":51,"state":"active+clean","last_fresh":"2026-02-24T07:48:50.135582+0000","last_change":"2026-02-24T07:48:40.101800+0000","last_active":"2026-02-24T07:48:50.135582+0000","last_peered":"2026-02-24T07:48:50.135582+0000","last_clean":"2026-02-24T07:48:50.135582+0000","last_became_active":"2026-02-24T07:46:02.881420+0000","last_became_peered":"2026-02-24T07:46:02.881420+0000","last_unstale":"2026-02-24T07:48:50.135582+0000","last_undegraded":"2026-02-24T07:48:50.135582+0000","last_fullsized":"2026-02-24T07:48:50.135582+0000","mapping_epoch":42,"log_start":"51'83651","ondisk_log_start":"51'83651","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"49'78179","last_scrub_stamp":"2026-02-24T07:48:40.101746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_clean_scrub_stamp":"2026-02-24T07:48:40.101746+0000","objects_scrubbed":5622,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:50:04.402399+0000","scrub_duration":722,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":83654,"num_write_kb":2676928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.4","version":"51'82424","reported_seq":82936,"reported_epoch":51,"state":"active+clean","last_fresh":"2026-02-24T07:48:50.147772+0000","last_change":"2026-02-24T07:48:39.247670+0000","last_active":"2026-02-24T07:48:50.147772+0000","last_peered":"2026-02-24T07:48:50.147772+0000","last_clean":"2026-02-24T07:48:50.147772+0000","last_became_active":"2026-02-24T07:46:02.880345+0000","last_became_peered":"2026-02-24T07:46:02.880345+0000","last_unstale":"2026-02-24T07:48:50.147772+0000","last_undegraded":"2026-02-24T07:48:50.147772+0000","last_fullsized":"2026-02-24T07:48:50.147772+0000","mapping_epoch":42,"log_start":"51'82421","ondisk_log_start":"51'82421","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"49'76695","last_scrub_stamp":"2026-02-24T07:48:39.247623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_clean_scrub_stamp":"2026-02-24T07:48:39.247623+0000","objects_scrubbed":5906,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:49:50.405566+0000","scrub_duration":698,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":82424,"num_write_kb":2637568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"51'83170","reported_seq":83685,"reported_epoch":51,"state":"active+clean","last_fresh":"2026-02-24T07:48:50.146006+0000","last_change":"2026-02-24T07:48:24.553666+0000","last_active":"2026-02-24T07:48:50.146006+0000","last_peered":"2026-02-24T07:48:50.146006+0000","last_clean":"2026-02-24T07:48:50.146006+0000","last_became_active":"2026-02-24T07:46:02.881035+0000","last_became_peered":"2026-02-24T07:46:02.881035+0000","last_unstale":"2026-02-24T07:48:50.146006+0000","last_undegraded":"2026-02-24T07:48:50.146006+0000","last_fullsized":"2026-02-24T07:48:50.146006+0000","mapping_epoch":42,"log_start":"51'83167","ondisk_log_start":"51'83167","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"49'69716","last_scrub_stamp":"2026-02-24T07:48:24.553517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_clean_scrub_stamp":"2026-02-24T07:48:24.553517+0000","objects_scrubbed":13748,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:49:36.493266+0000","scrub_duration":1167,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":83170,"num_write_kb":2661440,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"51'82822","reported_seq":83286,"reported_epoch":51,"state":"active+clean","last_fresh":"2026-02-24T07:48:50.147792+0000","last_change":"2026-02-24T07:48:41.578732+0000","last_active":"2026-02-24T07:48:50.147792+0000","last_peered":"2026-02-24T07:48:50.147792+0000","last_clean":"2026-02-24T07:48:50.147792+0000","last_became_active":"2026-02-24T07:46:02.879831+0000","last_became_peered":"2026-02-24T07:46:02.879831+0000","last_unstale":"2026-02-24T07:48:50.147792+0000","last_undegraded":"2026-02-24T07:48:50.147792+0000","last_fullsized":"2026-02-24T07:48:50.147792+0000","mapping_epoch":42,"log_start":"51'82819","ondisk_log_start":"51'82819","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"49'78297","last_scrub_stamp":"2026-02-24T07:48:41.578688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_clean_scrub_stamp":"2026-02-24T07:48:41.578688+0000","objects_scrubbed":4645,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:50:03.512738+0000","scrub_duration":439,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":82822,"num_write_kb":2650304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.0","version":"51'82726","reported_seq":83185,"reported_epoch":51,"state":"active+clean","last_fresh":"2026-02-24T07:48:50.147000+0000","last_change":"2026-02-24T07:48:30.080353+0000","last_active":"2026-02-24T07:48:50.147000+0000","last_peered":"2026-02-24T07:48:50.147000+0000","last_clean":"2026-02-24T07:48:50.147000+0000","last_became_active":"2026-02-24T07:46:02.880333+0000","last_became_peered":"2026-02-24T07:46:02.880333+0000","last_unstale":"2026-02-24T07:48:50.147000+0000","last_undegraded":"2026-02-24T07:48:50.147000+0000","last_fullsized":"2026-02-24T07:48:50.147000+0000","mapping_epoch":42,"log_start":"51'82723","ondisk_log_start":"51'82723","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"49'71923","last_scrub_stamp":"2026-02-24T07:48:30.080305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_clean_scrub_stamp":"2026-02-24T07:48:30.080305+0000","objects_scrubbed":11094,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:49:53.942641+0000","scrub_duration":1405,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":82726,"num_write_kb":2647232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2","version":"51'83378","reported_seq":83810,"reported_epoch":51,"state":"active+clean","last_fresh":"2026-02-24T07:48:50.146306+0000","last_change":"2026-02-24T07:48:49.904480+0000","last_active":"2026-02-24T07:48:50.146306+0000","last_peered":"2026-02-24T07:48:50.146306+0000","last_clean":"2026-02-24T07:48:50.146306+0000","last_became_active":"2026-02-24T07:46:02.880185+0000","last_became_peered":"2026-02-24T07:46:02.880185+0000","last_unstale":"2026-02-24T07:48:50.146306+0000","last_undegraded":"2026-02-24T07:48:50.146306+0000","last_fullsized":"2026-02-24T07:48:50.146306+0000","mapping_epoch":42,"log_start":"51'83375","ondisk_log_start":"51'83375","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"51'83228","last_scrub_stamp":"2026-02-24T07:48:49.904420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_clean_scrub_stamp":"2026-02-24T07:48:49.904420+0000","objects_scrubbed":175,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:49:51.471610+0000","scrub_duration":185,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":83378,"num_write_kb":2668096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"29'99","reported_seq":257,"reported_epoch":51,"state":"active+clean","last_fresh":"2026-02-24T07:48:48.873925+0000","last_change":"2026-02-24T07:48:02.589664+0000","last_active":"2026-02-24T07:48:48.873925+0000","last_peered":"2026-02-24T07:48:48.873925+0000","last_clean":"2026-02-24T07:48:48.873925+0000","last_became_active":"2026-02-24T07:42:54.186719+0000","last_became_peered":"2026-02-24T07:42:54.186719+0000","last_unstale":"2026-02-24T07:48:48.873925+0000","last_undegraded":"2026-02-24T07:48:48.873925+0000","last_fullsized":"2026-02-24T07:48:48.873925+0000","mapping_epoch":27,"log_start":"29'96","ondisk_log_start":"29'96","created":20,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'99","last_scrub_stamp":"2026-02-24T07:48:02.589609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:42:43.640776+0000","last_clean_scrub_stamp":"2026-02-24T07:48:02.589609+0000","objects_scrubbed":2,"log_size":3,"log_dups_size":96,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:49:04.391555+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":240,"num_write":135,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":16,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1327016,"num_write_kb":42464449,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9060,"internal_metadata":0},"log_size":48,"ondisk_log_size":48,"up":48,"acting":48,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":240,"num_write":135,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1088,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":3,"up_from":25,"seq":107374182474,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":1007120,"kb_used_data":1084,"kb_used_omap":12,"kb_used_meta":1006003,"kb_avail":732996080,"statfs":{"total":751619276800,"available":750587985920,"internally_reserved":0,"allocated":1110016,"data_stored":774266,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12442,"internal_metadata":1030147942},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":19,"seq":81604378700,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":1419408,"kb_used_data":1084,"kb_used_omap":11,"kb_used_meta":1418292,"kb_avail":732583792,"statfs":{"total":751619276800,"available":750165803008,"internally_reserved":0,"allocated":1110016,"data_stored":774266,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12165,"internal_metadata":1452331131},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542221,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":1128012,"kb_used_data":504,"kb_used_omap":13,"kb_used_meta":1127474,"kb_avail":732875188,"statfs":{"total":751619276800,"available":750464192512,"internally_reserved":0,"allocated":516096,"data_stored":183898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13793,"internal_metadata":1154533919},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705744,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":1204500,"kb_used_data":34368,"kb_used_omap":9,"kb_used_meta":1170038,"kb_avail":732798700,"statfs":{"total":751619276800,"available":750385868800,"internally_reserved":0,"allocated":35192832,"data_stored":34853010,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":10131,"internal_metadata":1198119021},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1088,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":93,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":5117,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1581,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":2269,"internal_metadata":0}]}} 2026-02-24T07:48:52.285 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph pg dump --format=json 2026-02-24T07:48:52.439 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:48:52.541 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:48:52.539+0000 7f8769aeb640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:48:52.541 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:48:52.539+0000 7f8769aeb640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:48:52.650 INFO:teuthology.orchestra.run.trial035.stdout: 2026-02-24T07:48:52.652 INFO:teuthology.orchestra.run.trial035.stderr:dumped all 2026-02-24T07:48:52.685 INFO:teuthology.orchestra.run.trial035.stdout:{"pg_ready":true,"pg_map":{"version":203,"stamp":"2026-02-24T07:48:51.883303+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":137,"num_read_kb":241,"num_write":1327151,"num_write_kb":42466599,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":51,"ondisk_log_size":51,"up":51,"acting":51,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":51,"num_osds":4,"num_per_pool_osds":4,"num_per_pool_omap_osds":4,"kb":2936012800,"kb_used":4759040,"kb_used_data":37040,"kb_used_omap":47,"kb_used_meta":4721808,"kb_avail":2931253760,"statfs":{"total":3006477107200,"available":3001603850240,"internally_reserved":0,"allocated":37928960,"data_stored":36585440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":48531,"internal_metadata":4835132013},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-2445148184,"num_objects":-37311,"num_object_clones":0,"num_object_copies":-111933,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-37311,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":37311,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.179056"},"pg_stats":[{"pgid":"3.f","version":"51'82610","reported_seq":83117,"reported_epoch":51,"state":"active+clean","last_fresh":"2026-02-24T07:48:50.147899+0000","last_change":"2026-02-24T07:48:26.639215+0000","last_active":"2026-02-24T07:48:50.147899+0000","last_peered":"2026-02-24T07:48:50.147899+0000","last_clean":"2026-02-24T07:48:50.147899+0000","last_became_active":"2026-02-24T07:46:02.881625+0000","last_became_peered":"2026-02-24T07:46:02.881625+0000","last_unstale":"2026-02-24T07:48:50.147899+0000","last_undegraded":"2026-02-24T07:48:50.147899+0000","last_fullsized":"2026-02-24T07:48:50.147899+0000","mapping_epoch":42,"log_start":"51'82607","ondisk_log_start":"51'82607","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"49'70570","last_scrub_stamp":"2026-02-24T07:48:26.639139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_clean_scrub_stamp":"2026-02-24T07:48:26.639139+0000","objects_scrubbed":12333,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:49:38.413618+0000","scrub_duration":1227,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":82610,"num_write_kb":2643520,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.e","version":"51'83796","reported_seq":84242,"reported_epoch":51,"state":"active+clean","last_fresh":"2026-02-24T07:48:50.144259+0000","last_change":"2026-02-24T07:47:33.099762+0000","last_active":"2026-02-24T07:48:50.144259+0000","last_peered":"2026-02-24T07:48:50.144259+0000","last_clean":"2026-02-24T07:48:50.144259+0000","last_became_active":"2026-02-24T07:46:02.881083+0000","last_became_peered":"2026-02-24T07:46:02.881083+0000","last_unstale":"2026-02-24T07:48:50.144259+0000","last_undegraded":"2026-02-24T07:48:50.144259+0000","last_fullsized":"2026-02-24T07:48:50.144259+0000","mapping_epoch":42,"log_start":"51'83793","ondisk_log_start":"51'83793","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"46'41533","last_scrub_stamp":"2026-02-24T07:47:33.099675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_clean_scrub_stamp":"2026-02-24T07:47:33.099675+0000","objects_scrubbed":40754,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:48:53.832432+0000","scrub_duration":3969,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":83796,"num_write_kb":2681472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"51'83082","reported_seq":83549,"reported_epoch":51,"state":"active+clean","last_fresh":"2026-02-24T07:48:50.147985+0000","last_change":"2026-02-24T07:48:28.181137+0000","last_active":"2026-02-24T07:48:50.147985+0000","last_peered":"2026-02-24T07:48:50.147985+0000","last_clean":"2026-02-24T07:48:50.147985+0000","last_became_active":"2026-02-24T07:46:02.883799+0000","last_became_peered":"2026-02-24T07:46:02.883799+0000","last_unstale":"2026-02-24T07:48:50.147985+0000","last_undegraded":"2026-02-24T07:48:50.147985+0000","last_fullsized":"2026-02-24T07:48:50.147985+0000","mapping_epoch":42,"log_start":"51'83079","ondisk_log_start":"51'83079","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"49'71407","last_scrub_stamp":"2026-02-24T07:48:28.181090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_clean_scrub_stamp":"2026-02-24T07:48:28.181090+0000","objects_scrubbed":11940,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:49:56.737033+0000","scrub_duration":1384,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":83082,"num_write_kb":2658561,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"51'81642","reported_seq":82062,"reported_epoch":51,"state":"active+clean","last_fresh":"2026-02-24T07:48:50.142582+0000","last_change":"2026-02-24T07:48:35.306234+0000","last_active":"2026-02-24T07:48:50.142582+0000","last_peered":"2026-02-24T07:48:50.142582+0000","last_clean":"2026-02-24T07:48:50.142582+0000","last_became_active":"2026-02-24T07:46:02.880060+0000","last_became_peered":"2026-02-24T07:46:02.880060+0000","last_unstale":"2026-02-24T07:48:50.142582+0000","last_undegraded":"2026-02-24T07:48:50.142582+0000","last_fullsized":"2026-02-24T07:48:50.142582+0000","mapping_epoch":42,"log_start":"51'81639","ondisk_log_start":"51'81639","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"49'74008","last_scrub_stamp":"2026-02-24T07:48:35.306189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_clean_scrub_stamp":"2026-02-24T07:48:35.306189+0000","objects_scrubbed":7791,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:50:02.247501+0000","scrub_duration":624,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":81642,"num_write_kb":2612544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"51'82932","reported_seq":83472,"reported_epoch":51,"state":"active+clean","last_fresh":"2026-02-24T07:48:50.141473+0000","last_change":"2026-02-24T07:48:16.528595+0000","last_active":"2026-02-24T07:48:50.141473+0000","last_peered":"2026-02-24T07:48:50.141473+0000","last_clean":"2026-02-24T07:48:50.141473+0000","last_became_active":"2026-02-24T07:46:02.880171+0000","last_became_peered":"2026-02-24T07:46:02.880171+0000","last_unstale":"2026-02-24T07:48:50.141473+0000","last_undegraded":"2026-02-24T07:48:50.141473+0000","last_fullsized":"2026-02-24T07:48:50.141473+0000","mapping_epoch":42,"log_start":"51'82929","ondisk_log_start":"51'82929","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"49'65111","last_scrub_stamp":"2026-02-24T07:48:16.528547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_clean_scrub_stamp":"2026-02-24T07:48:16.528547+0000","objects_scrubbed":18357,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:49:45.487538+0000","scrub_duration":1833,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":82932,"num_write_kb":2653824,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.a","version":"51'82186","reported_seq":82701,"reported_epoch":51,"state":"active+clean","last_fresh":"2026-02-24T07:48:50.145111+0000","last_change":"2026-02-24T07:48:39.224243+0000","last_active":"2026-02-24T07:48:50.145111+0000","last_peered":"2026-02-24T07:48:50.145111+0000","last_clean":"2026-02-24T07:48:50.145111+0000","last_became_active":"2026-02-24T07:46:02.882274+0000","last_became_peered":"2026-02-24T07:46:02.882274+0000","last_unstale":"2026-02-24T07:48:50.145111+0000","last_undegraded":"2026-02-24T07:48:50.145111+0000","last_fullsized":"2026-02-24T07:48:50.145111+0000","mapping_epoch":42,"log_start":"51'82183","ondisk_log_start":"51'82183","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"49'76461","last_scrub_stamp":"2026-02-24T07:48:39.224222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_clean_scrub_stamp":"2026-02-24T07:48:39.224222+0000","objects_scrubbed":5854,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:50:02.455067+0000","scrub_duration":543,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":82186,"num_write_kb":2629952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"51'83864","reported_seq":84334,"reported_epoch":51,"state":"active+clean","last_fresh":"2026-02-24T07:48:50.144442+0000","last_change":"2026-02-24T07:48:30.710470+0000","last_active":"2026-02-24T07:48:50.144442+0000","last_peered":"2026-02-24T07:48:50.144442+0000","last_clean":"2026-02-24T07:48:50.144442+0000","last_became_active":"2026-02-24T07:46:02.881951+0000","last_became_peered":"2026-02-24T07:46:02.881951+0000","last_unstale":"2026-02-24T07:48:50.144442+0000","last_undegraded":"2026-02-24T07:48:50.144442+0000","last_fullsized":"2026-02-24T07:48:50.144442+0000","mapping_epoch":42,"log_start":"51'83861","ondisk_log_start":"51'83861","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"49'73406","last_scrub_stamp":"2026-02-24T07:48:30.710432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_clean_scrub_stamp":"2026-02-24T07:48:30.710432+0000","objects_scrubbed":10727,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:49:42.678619+0000","scrub_duration":946,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":83864,"num_write_kb":2683648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"51'83222","reported_seq":83699,"reported_epoch":51,"state":"active+clean","last_fresh":"2026-02-24T07:48:50.147078+0000","last_change":"2026-02-24T07:48:37.650256+0000","last_active":"2026-02-24T07:48:50.147078+0000","last_peered":"2026-02-24T07:48:50.147078+0000","last_clean":"2026-02-24T07:48:50.147078+0000","last_became_active":"2026-02-24T07:46:02.881345+0000","last_became_peered":"2026-02-24T07:46:02.881345+0000","last_unstale":"2026-02-24T07:48:50.147078+0000","last_undegraded":"2026-02-24T07:48:50.147078+0000","last_fullsized":"2026-02-24T07:48:50.147078+0000","mapping_epoch":42,"log_start":"51'83219","ondisk_log_start":"51'83219","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"49'76565","last_scrub_stamp":"2026-02-24T07:48:37.650220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_clean_scrub_stamp":"2026-02-24T07:48:37.650220+0000","objects_scrubbed":6908,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:50:07.037617+0000","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":83222,"num_write_kb":2663104,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"51'82476","reported_seq":82960,"reported_epoch":51,"state":"active+clean","last_fresh":"2026-02-24T07:48:50.146836+0000","last_change":"2026-02-24T07:48:44.952653+0000","last_active":"2026-02-24T07:48:50.146836+0000","last_peered":"2026-02-24T07:48:50.146836+0000","last_clean":"2026-02-24T07:48:50.146836+0000","last_became_active":"2026-02-24T07:46:02.880323+0000","last_became_peered":"2026-02-24T07:46:02.880323+0000","last_unstale":"2026-02-24T07:48:50.146836+0000","last_undegraded":"2026-02-24T07:48:50.146836+0000","last_fullsized":"2026-02-24T07:48:50.146836+0000","mapping_epoch":42,"log_start":"51'82473","ondisk_log_start":"51'82473","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"49'79775","last_scrub_stamp":"2026-02-24T07:48:44.952583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_clean_scrub_stamp":"2026-02-24T07:48:44.952583+0000","objects_scrubbed":2777,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:50:13.253334+0000","scrub_duration":272,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":82476,"num_write_kb":2639232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.6","version":"51'83032","reported_seq":83436,"reported_epoch":51,"state":"active+clean","last_fresh":"2026-02-24T07:48:50.146007+0000","last_change":"2026-02-24T07:47:23.432795+0000","last_active":"2026-02-24T07:48:50.146007+0000","last_peered":"2026-02-24T07:48:50.146007+0000","last_clean":"2026-02-24T07:48:50.146007+0000","last_became_active":"2026-02-24T07:46:02.880005+0000","last_became_peered":"2026-02-24T07:46:02.880005+0000","last_unstale":"2026-02-24T07:48:50.146007+0000","last_undegraded":"2026-02-24T07:48:50.146007+0000","last_fullsized":"2026-02-24T07:48:50.146007+0000","mapping_epoch":42,"log_start":"51'83029","ondisk_log_start":"51'83029","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"45'37219","last_scrub_stamp":"2026-02-24T07:47:23.432744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_clean_scrub_stamp":"2026-02-24T07:47:23.432744+0000","objects_scrubbed":36510,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:48:51.822832+0000","scrub_duration":3412,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":83032,"num_write_kb":2657024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"51'83654","reported_seq":84100,"reported_epoch":51,"state":"active+clean","last_fresh":"2026-02-24T07:48:50.135582+0000","last_change":"2026-02-24T07:48:40.101800+0000","last_active":"2026-02-24T07:48:50.135582+0000","last_peered":"2026-02-24T07:48:50.135582+0000","last_clean":"2026-02-24T07:48:50.135582+0000","last_became_active":"2026-02-24T07:46:02.881420+0000","last_became_peered":"2026-02-24T07:46:02.881420+0000","last_unstale":"2026-02-24T07:48:50.135582+0000","last_undegraded":"2026-02-24T07:48:50.135582+0000","last_fullsized":"2026-02-24T07:48:50.135582+0000","mapping_epoch":42,"log_start":"51'83651","ondisk_log_start":"51'83651","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"49'78179","last_scrub_stamp":"2026-02-24T07:48:40.101746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_clean_scrub_stamp":"2026-02-24T07:48:40.101746+0000","objects_scrubbed":5622,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:50:04.402399+0000","scrub_duration":722,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":83654,"num_write_kb":2676928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.4","version":"51'82424","reported_seq":82936,"reported_epoch":51,"state":"active+clean","last_fresh":"2026-02-24T07:48:50.147772+0000","last_change":"2026-02-24T07:48:39.247670+0000","last_active":"2026-02-24T07:48:50.147772+0000","last_peered":"2026-02-24T07:48:50.147772+0000","last_clean":"2026-02-24T07:48:50.147772+0000","last_became_active":"2026-02-24T07:46:02.880345+0000","last_became_peered":"2026-02-24T07:46:02.880345+0000","last_unstale":"2026-02-24T07:48:50.147772+0000","last_undegraded":"2026-02-24T07:48:50.147772+0000","last_fullsized":"2026-02-24T07:48:50.147772+0000","mapping_epoch":42,"log_start":"51'82421","ondisk_log_start":"51'82421","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"49'76695","last_scrub_stamp":"2026-02-24T07:48:39.247623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_clean_scrub_stamp":"2026-02-24T07:48:39.247623+0000","objects_scrubbed":5906,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:49:50.405566+0000","scrub_duration":698,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":82424,"num_write_kb":2637568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"51'83170","reported_seq":83685,"reported_epoch":51,"state":"active+clean","last_fresh":"2026-02-24T07:48:50.146006+0000","last_change":"2026-02-24T07:48:24.553666+0000","last_active":"2026-02-24T07:48:50.146006+0000","last_peered":"2026-02-24T07:48:50.146006+0000","last_clean":"2026-02-24T07:48:50.146006+0000","last_became_active":"2026-02-24T07:46:02.881035+0000","last_became_peered":"2026-02-24T07:46:02.881035+0000","last_unstale":"2026-02-24T07:48:50.146006+0000","last_undegraded":"2026-02-24T07:48:50.146006+0000","last_fullsized":"2026-02-24T07:48:50.146006+0000","mapping_epoch":42,"log_start":"51'83167","ondisk_log_start":"51'83167","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"49'69716","last_scrub_stamp":"2026-02-24T07:48:24.553517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_clean_scrub_stamp":"2026-02-24T07:48:24.553517+0000","objects_scrubbed":13748,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:49:36.493266+0000","scrub_duration":1167,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":83170,"num_write_kb":2661440,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"51'82822","reported_seq":83286,"reported_epoch":51,"state":"active+clean","last_fresh":"2026-02-24T07:48:50.147792+0000","last_change":"2026-02-24T07:48:41.578732+0000","last_active":"2026-02-24T07:48:50.147792+0000","last_peered":"2026-02-24T07:48:50.147792+0000","last_clean":"2026-02-24T07:48:50.147792+0000","last_became_active":"2026-02-24T07:46:02.879831+0000","last_became_peered":"2026-02-24T07:46:02.879831+0000","last_unstale":"2026-02-24T07:48:50.147792+0000","last_undegraded":"2026-02-24T07:48:50.147792+0000","last_fullsized":"2026-02-24T07:48:50.147792+0000","mapping_epoch":42,"log_start":"51'82819","ondisk_log_start":"51'82819","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"49'78297","last_scrub_stamp":"2026-02-24T07:48:41.578688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_clean_scrub_stamp":"2026-02-24T07:48:41.578688+0000","objects_scrubbed":4645,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:50:03.512738+0000","scrub_duration":439,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":82822,"num_write_kb":2650304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.0","version":"51'82726","reported_seq":83185,"reported_epoch":51,"state":"active+clean","last_fresh":"2026-02-24T07:48:50.147000+0000","last_change":"2026-02-24T07:48:30.080353+0000","last_active":"2026-02-24T07:48:50.147000+0000","last_peered":"2026-02-24T07:48:50.147000+0000","last_clean":"2026-02-24T07:48:50.147000+0000","last_became_active":"2026-02-24T07:46:02.880333+0000","last_became_peered":"2026-02-24T07:46:02.880333+0000","last_unstale":"2026-02-24T07:48:50.147000+0000","last_undegraded":"2026-02-24T07:48:50.147000+0000","last_fullsized":"2026-02-24T07:48:50.147000+0000","mapping_epoch":42,"log_start":"51'82723","ondisk_log_start":"51'82723","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"49'71923","last_scrub_stamp":"2026-02-24T07:48:30.080305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_clean_scrub_stamp":"2026-02-24T07:48:30.080305+0000","objects_scrubbed":11094,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:49:53.942641+0000","scrub_duration":1405,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":82726,"num_write_kb":2647232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2","version":"51'83378","reported_seq":83810,"reported_epoch":51,"state":"active+clean","last_fresh":"2026-02-24T07:48:50.146306+0000","last_change":"2026-02-24T07:48:49.904480+0000","last_active":"2026-02-24T07:48:50.146306+0000","last_peered":"2026-02-24T07:48:50.146306+0000","last_clean":"2026-02-24T07:48:50.146306+0000","last_became_active":"2026-02-24T07:46:02.880185+0000","last_became_peered":"2026-02-24T07:46:02.880185+0000","last_unstale":"2026-02-24T07:48:50.146306+0000","last_undegraded":"2026-02-24T07:48:50.146306+0000","last_fullsized":"2026-02-24T07:48:50.146306+0000","mapping_epoch":42,"log_start":"51'83375","ondisk_log_start":"51'83375","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"51'83228","last_scrub_stamp":"2026-02-24T07:48:49.904420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_clean_scrub_stamp":"2026-02-24T07:48:49.904420+0000","objects_scrubbed":175,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:49:51.471610+0000","scrub_duration":185,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":83378,"num_write_kb":2668096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"29'99","reported_seq":257,"reported_epoch":51,"state":"active+clean","last_fresh":"2026-02-24T07:48:48.873925+0000","last_change":"2026-02-24T07:48:02.589664+0000","last_active":"2026-02-24T07:48:48.873925+0000","last_peered":"2026-02-24T07:48:48.873925+0000","last_clean":"2026-02-24T07:48:48.873925+0000","last_became_active":"2026-02-24T07:42:54.186719+0000","last_became_peered":"2026-02-24T07:42:54.186719+0000","last_unstale":"2026-02-24T07:48:48.873925+0000","last_undegraded":"2026-02-24T07:48:48.873925+0000","last_fullsized":"2026-02-24T07:48:48.873925+0000","mapping_epoch":27,"log_start":"29'96","ondisk_log_start":"29'96","created":20,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'99","last_scrub_stamp":"2026-02-24T07:48:02.589609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:42:43.640776+0000","last_clean_scrub_stamp":"2026-02-24T07:48:02.589609+0000","objects_scrubbed":2,"log_size":3,"log_dups_size":96,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:49:04.391555+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":240,"num_write":135,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":16,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1327016,"num_write_kb":42464449,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9060,"internal_metadata":0},"log_size":48,"ondisk_log_size":48,"up":48,"acting":48,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":240,"num_write":135,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1088,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":3,"up_from":25,"seq":107374182474,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":1007120,"kb_used_data":1084,"kb_used_omap":12,"kb_used_meta":1006003,"kb_avail":732996080,"statfs":{"total":751619276800,"available":750587985920,"internally_reserved":0,"allocated":1110016,"data_stored":774266,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12442,"internal_metadata":1030147942},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":19,"seq":81604378700,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":1419408,"kb_used_data":1084,"kb_used_omap":11,"kb_used_meta":1418292,"kb_avail":732583792,"statfs":{"total":751619276800,"available":750165803008,"internally_reserved":0,"allocated":1110016,"data_stored":774266,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12165,"internal_metadata":1452331131},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542221,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":1128012,"kb_used_data":504,"kb_used_omap":13,"kb_used_meta":1127474,"kb_avail":732875188,"statfs":{"total":751619276800,"available":750464192512,"internally_reserved":0,"allocated":516096,"data_stored":183898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13793,"internal_metadata":1154533919},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705744,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":1204500,"kb_used_data":34368,"kb_used_omap":9,"kb_used_meta":1170038,"kb_avail":732798700,"statfs":{"total":751619276800,"available":750385868800,"internally_reserved":0,"allocated":35192832,"data_stored":34853010,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":10131,"internal_metadata":1198119021},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1088,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":93,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":5117,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1581,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":2269,"internal_metadata":0}]}} 2026-02-24T07:48:52.687 INFO:tasks.cephadm.ceph_manager.ceph:recovered! 2026-02-24T07:48:52.687 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2026-02-24T07:48:52.688 INFO:tasks.thrashosds.thrasher:Growing pool .mgr 2026-02-24T07:48:52.688 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph pg dump --format=json 2026-02-24T07:48:52.805 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:48:52.906 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:48:52.903+0000 7fc83f574640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:48:52.906 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:48:52.904+0000 7fc83f574640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:48:53.015 INFO:teuthology.orchestra.run.trial035.stdout: 2026-02-24T07:48:53.016 INFO:teuthology.orchestra.run.trial035.stderr:dumped all 2026-02-24T07:48:53.054 INFO:teuthology.orchestra.run.trial035.stdout:{"pg_ready":true,"pg_map":{"version":203,"stamp":"2026-02-24T07:48:51.883303+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":137,"num_read_kb":241,"num_write":1327151,"num_write_kb":42466599,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":51,"ondisk_log_size":51,"up":51,"acting":51,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":51,"num_osds":4,"num_per_pool_osds":4,"num_per_pool_omap_osds":4,"kb":2936012800,"kb_used":4759040,"kb_used_data":37040,"kb_used_omap":47,"kb_used_meta":4721808,"kb_avail":2931253760,"statfs":{"total":3006477107200,"available":3001603850240,"internally_reserved":0,"allocated":37928960,"data_stored":36585440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":48531,"internal_metadata":4835132013},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-2445148184,"num_objects":-37311,"num_object_clones":0,"num_object_copies":-111933,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-37311,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":37311,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.179056"},"pg_stats":[{"pgid":"3.f","version":"51'82610","reported_seq":83117,"reported_epoch":51,"state":"active+clean","last_fresh":"2026-02-24T07:48:50.147899+0000","last_change":"2026-02-24T07:48:26.639215+0000","last_active":"2026-02-24T07:48:50.147899+0000","last_peered":"2026-02-24T07:48:50.147899+0000","last_clean":"2026-02-24T07:48:50.147899+0000","last_became_active":"2026-02-24T07:46:02.881625+0000","last_became_peered":"2026-02-24T07:46:02.881625+0000","last_unstale":"2026-02-24T07:48:50.147899+0000","last_undegraded":"2026-02-24T07:48:50.147899+0000","last_fullsized":"2026-02-24T07:48:50.147899+0000","mapping_epoch":42,"log_start":"51'82607","ondisk_log_start":"51'82607","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"49'70570","last_scrub_stamp":"2026-02-24T07:48:26.639139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_clean_scrub_stamp":"2026-02-24T07:48:26.639139+0000","objects_scrubbed":12333,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:49:38.413618+0000","scrub_duration":1227,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":82610,"num_write_kb":2643520,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.e","version":"51'83796","reported_seq":84242,"reported_epoch":51,"state":"active+clean","last_fresh":"2026-02-24T07:48:50.144259+0000","last_change":"2026-02-24T07:47:33.099762+0000","last_active":"2026-02-24T07:48:50.144259+0000","last_peered":"2026-02-24T07:48:50.144259+0000","last_clean":"2026-02-24T07:48:50.144259+0000","last_became_active":"2026-02-24T07:46:02.881083+0000","last_became_peered":"2026-02-24T07:46:02.881083+0000","last_unstale":"2026-02-24T07:48:50.144259+0000","last_undegraded":"2026-02-24T07:48:50.144259+0000","last_fullsized":"2026-02-24T07:48:50.144259+0000","mapping_epoch":42,"log_start":"51'83793","ondisk_log_start":"51'83793","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"46'41533","last_scrub_stamp":"2026-02-24T07:47:33.099675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_clean_scrub_stamp":"2026-02-24T07:47:33.099675+0000","objects_scrubbed":40754,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:48:53.832432+0000","scrub_duration":3969,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":83796,"num_write_kb":2681472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"51'83082","reported_seq":83549,"reported_epoch":51,"state":"active+clean","last_fresh":"2026-02-24T07:48:50.147985+0000","last_change":"2026-02-24T07:48:28.181137+0000","last_active":"2026-02-24T07:48:50.147985+0000","last_peered":"2026-02-24T07:48:50.147985+0000","last_clean":"2026-02-24T07:48:50.147985+0000","last_became_active":"2026-02-24T07:46:02.883799+0000","last_became_peered":"2026-02-24T07:46:02.883799+0000","last_unstale":"2026-02-24T07:48:50.147985+0000","last_undegraded":"2026-02-24T07:48:50.147985+0000","last_fullsized":"2026-02-24T07:48:50.147985+0000","mapping_epoch":42,"log_start":"51'83079","ondisk_log_start":"51'83079","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"49'71407","last_scrub_stamp":"2026-02-24T07:48:28.181090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_clean_scrub_stamp":"2026-02-24T07:48:28.181090+0000","objects_scrubbed":11940,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:49:56.737033+0000","scrub_duration":1384,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":83082,"num_write_kb":2658561,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"51'81642","reported_seq":82062,"reported_epoch":51,"state":"active+clean","last_fresh":"2026-02-24T07:48:50.142582+0000","last_change":"2026-02-24T07:48:35.306234+0000","last_active":"2026-02-24T07:48:50.142582+0000","last_peered":"2026-02-24T07:48:50.142582+0000","last_clean":"2026-02-24T07:48:50.142582+0000","last_became_active":"2026-02-24T07:46:02.880060+0000","last_became_peered":"2026-02-24T07:46:02.880060+0000","last_unstale":"2026-02-24T07:48:50.142582+0000","last_undegraded":"2026-02-24T07:48:50.142582+0000","last_fullsized":"2026-02-24T07:48:50.142582+0000","mapping_epoch":42,"log_start":"51'81639","ondisk_log_start":"51'81639","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"49'74008","last_scrub_stamp":"2026-02-24T07:48:35.306189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_clean_scrub_stamp":"2026-02-24T07:48:35.306189+0000","objects_scrubbed":7791,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:50:02.247501+0000","scrub_duration":624,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":81642,"num_write_kb":2612544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"51'82932","reported_seq":83472,"reported_epoch":51,"state":"active+clean","last_fresh":"2026-02-24T07:48:50.141473+0000","last_change":"2026-02-24T07:48:16.528595+0000","last_active":"2026-02-24T07:48:50.141473+0000","last_peered":"2026-02-24T07:48:50.141473+0000","last_clean":"2026-02-24T07:48:50.141473+0000","last_became_active":"2026-02-24T07:46:02.880171+0000","last_became_peered":"2026-02-24T07:46:02.880171+0000","last_unstale":"2026-02-24T07:48:50.141473+0000","last_undegraded":"2026-02-24T07:48:50.141473+0000","last_fullsized":"2026-02-24T07:48:50.141473+0000","mapping_epoch":42,"log_start":"51'82929","ondisk_log_start":"51'82929","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"49'65111","last_scrub_stamp":"2026-02-24T07:48:16.528547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_clean_scrub_stamp":"2026-02-24T07:48:16.528547+0000","objects_scrubbed":18357,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:49:45.487538+0000","scrub_duration":1833,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":82932,"num_write_kb":2653824,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.a","version":"51'82186","reported_seq":82701,"reported_epoch":51,"state":"active+clean","last_fresh":"2026-02-24T07:48:50.145111+0000","last_change":"2026-02-24T07:48:39.224243+0000","last_active":"2026-02-24T07:48:50.145111+0000","last_peered":"2026-02-24T07:48:50.145111+0000","last_clean":"2026-02-24T07:48:50.145111+0000","last_became_active":"2026-02-24T07:46:02.882274+0000","last_became_peered":"2026-02-24T07:46:02.882274+0000","last_unstale":"2026-02-24T07:48:50.145111+0000","last_undegraded":"2026-02-24T07:48:50.145111+0000","last_fullsized":"2026-02-24T07:48:50.145111+0000","mapping_epoch":42,"log_start":"51'82183","ondisk_log_start":"51'82183","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"49'76461","last_scrub_stamp":"2026-02-24T07:48:39.224222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_clean_scrub_stamp":"2026-02-24T07:48:39.224222+0000","objects_scrubbed":5854,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:50:02.455067+0000","scrub_duration":543,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":82186,"num_write_kb":2629952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"51'83864","reported_seq":84334,"reported_epoch":51,"state":"active+clean","last_fresh":"2026-02-24T07:48:50.144442+0000","last_change":"2026-02-24T07:48:30.710470+0000","last_active":"2026-02-24T07:48:50.144442+0000","last_peered":"2026-02-24T07:48:50.144442+0000","last_clean":"2026-02-24T07:48:50.144442+0000","last_became_active":"2026-02-24T07:46:02.881951+0000","last_became_peered":"2026-02-24T07:46:02.881951+0000","last_unstale":"2026-02-24T07:48:50.144442+0000","last_undegraded":"2026-02-24T07:48:50.144442+0000","last_fullsized":"2026-02-24T07:48:50.144442+0000","mapping_epoch":42,"log_start":"51'83861","ondisk_log_start":"51'83861","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"49'73406","last_scrub_stamp":"2026-02-24T07:48:30.710432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_clean_scrub_stamp":"2026-02-24T07:48:30.710432+0000","objects_scrubbed":10727,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:49:42.678619+0000","scrub_duration":946,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":83864,"num_write_kb":2683648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"51'83222","reported_seq":83699,"reported_epoch":51,"state":"active+clean","last_fresh":"2026-02-24T07:48:50.147078+0000","last_change":"2026-02-24T07:48:37.650256+0000","last_active":"2026-02-24T07:48:50.147078+0000","last_peered":"2026-02-24T07:48:50.147078+0000","last_clean":"2026-02-24T07:48:50.147078+0000","last_became_active":"2026-02-24T07:46:02.881345+0000","last_became_peered":"2026-02-24T07:46:02.881345+0000","last_unstale":"2026-02-24T07:48:50.147078+0000","last_undegraded":"2026-02-24T07:48:50.147078+0000","last_fullsized":"2026-02-24T07:48:50.147078+0000","mapping_epoch":42,"log_start":"51'83219","ondisk_log_start":"51'83219","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"49'76565","last_scrub_stamp":"2026-02-24T07:48:37.650220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_clean_scrub_stamp":"2026-02-24T07:48:37.650220+0000","objects_scrubbed":6908,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:50:07.037617+0000","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":83222,"num_write_kb":2663104,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"51'82476","reported_seq":82960,"reported_epoch":51,"state":"active+clean","last_fresh":"2026-02-24T07:48:50.146836+0000","last_change":"2026-02-24T07:48:44.952653+0000","last_active":"2026-02-24T07:48:50.146836+0000","last_peered":"2026-02-24T07:48:50.146836+0000","last_clean":"2026-02-24T07:48:50.146836+0000","last_became_active":"2026-02-24T07:46:02.880323+0000","last_became_peered":"2026-02-24T07:46:02.880323+0000","last_unstale":"2026-02-24T07:48:50.146836+0000","last_undegraded":"2026-02-24T07:48:50.146836+0000","last_fullsized":"2026-02-24T07:48:50.146836+0000","mapping_epoch":42,"log_start":"51'82473","ondisk_log_start":"51'82473","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"49'79775","last_scrub_stamp":"2026-02-24T07:48:44.952583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_clean_scrub_stamp":"2026-02-24T07:48:44.952583+0000","objects_scrubbed":2777,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:50:13.253334+0000","scrub_duration":272,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":82476,"num_write_kb":2639232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.6","version":"51'83032","reported_seq":83436,"reported_epoch":51,"state":"active+clean","last_fresh":"2026-02-24T07:48:50.146007+0000","last_change":"2026-02-24T07:47:23.432795+0000","last_active":"2026-02-24T07:48:50.146007+0000","last_peered":"2026-02-24T07:48:50.146007+0000","last_clean":"2026-02-24T07:48:50.146007+0000","last_became_active":"2026-02-24T07:46:02.880005+0000","last_became_peered":"2026-02-24T07:46:02.880005+0000","last_unstale":"2026-02-24T07:48:50.146007+0000","last_undegraded":"2026-02-24T07:48:50.146007+0000","last_fullsized":"2026-02-24T07:48:50.146007+0000","mapping_epoch":42,"log_start":"51'83029","ondisk_log_start":"51'83029","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"45'37219","last_scrub_stamp":"2026-02-24T07:47:23.432744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_clean_scrub_stamp":"2026-02-24T07:47:23.432744+0000","objects_scrubbed":36510,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:48:51.822832+0000","scrub_duration":3412,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":83032,"num_write_kb":2657024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"51'83654","reported_seq":84100,"reported_epoch":51,"state":"active+clean","last_fresh":"2026-02-24T07:48:50.135582+0000","last_change":"2026-02-24T07:48:40.101800+0000","last_active":"2026-02-24T07:48:50.135582+0000","last_peered":"2026-02-24T07:48:50.135582+0000","last_clean":"2026-02-24T07:48:50.135582+0000","last_became_active":"2026-02-24T07:46:02.881420+0000","last_became_peered":"2026-02-24T07:46:02.881420+0000","last_unstale":"2026-02-24T07:48:50.135582+0000","last_undegraded":"2026-02-24T07:48:50.135582+0000","last_fullsized":"2026-02-24T07:48:50.135582+0000","mapping_epoch":42,"log_start":"51'83651","ondisk_log_start":"51'83651","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"49'78179","last_scrub_stamp":"2026-02-24T07:48:40.101746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_clean_scrub_stamp":"2026-02-24T07:48:40.101746+0000","objects_scrubbed":5622,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:50:04.402399+0000","scrub_duration":722,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":83654,"num_write_kb":2676928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.4","version":"51'82424","reported_seq":82936,"reported_epoch":51,"state":"active+clean","last_fresh":"2026-02-24T07:48:50.147772+0000","last_change":"2026-02-24T07:48:39.247670+0000","last_active":"2026-02-24T07:48:50.147772+0000","last_peered":"2026-02-24T07:48:50.147772+0000","last_clean":"2026-02-24T07:48:50.147772+0000","last_became_active":"2026-02-24T07:46:02.880345+0000","last_became_peered":"2026-02-24T07:46:02.880345+0000","last_unstale":"2026-02-24T07:48:50.147772+0000","last_undegraded":"2026-02-24T07:48:50.147772+0000","last_fullsized":"2026-02-24T07:48:50.147772+0000","mapping_epoch":42,"log_start":"51'82421","ondisk_log_start":"51'82421","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"49'76695","last_scrub_stamp":"2026-02-24T07:48:39.247623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_clean_scrub_stamp":"2026-02-24T07:48:39.247623+0000","objects_scrubbed":5906,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:49:50.405566+0000","scrub_duration":698,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":82424,"num_write_kb":2637568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"51'83170","reported_seq":83685,"reported_epoch":51,"state":"active+clean","last_fresh":"2026-02-24T07:48:50.146006+0000","last_change":"2026-02-24T07:48:24.553666+0000","last_active":"2026-02-24T07:48:50.146006+0000","last_peered":"2026-02-24T07:48:50.146006+0000","last_clean":"2026-02-24T07:48:50.146006+0000","last_became_active":"2026-02-24T07:46:02.881035+0000","last_became_peered":"2026-02-24T07:46:02.881035+0000","last_unstale":"2026-02-24T07:48:50.146006+0000","last_undegraded":"2026-02-24T07:48:50.146006+0000","last_fullsized":"2026-02-24T07:48:50.146006+0000","mapping_epoch":42,"log_start":"51'83167","ondisk_log_start":"51'83167","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"49'69716","last_scrub_stamp":"2026-02-24T07:48:24.553517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_clean_scrub_stamp":"2026-02-24T07:48:24.553517+0000","objects_scrubbed":13748,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:49:36.493266+0000","scrub_duration":1167,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":83170,"num_write_kb":2661440,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"51'82822","reported_seq":83286,"reported_epoch":51,"state":"active+clean","last_fresh":"2026-02-24T07:48:50.147792+0000","last_change":"2026-02-24T07:48:41.578732+0000","last_active":"2026-02-24T07:48:50.147792+0000","last_peered":"2026-02-24T07:48:50.147792+0000","last_clean":"2026-02-24T07:48:50.147792+0000","last_became_active":"2026-02-24T07:46:02.879831+0000","last_became_peered":"2026-02-24T07:46:02.879831+0000","last_unstale":"2026-02-24T07:48:50.147792+0000","last_undegraded":"2026-02-24T07:48:50.147792+0000","last_fullsized":"2026-02-24T07:48:50.147792+0000","mapping_epoch":42,"log_start":"51'82819","ondisk_log_start":"51'82819","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"49'78297","last_scrub_stamp":"2026-02-24T07:48:41.578688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_clean_scrub_stamp":"2026-02-24T07:48:41.578688+0000","objects_scrubbed":4645,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:50:03.512738+0000","scrub_duration":439,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":82822,"num_write_kb":2650304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.0","version":"51'82726","reported_seq":83185,"reported_epoch":51,"state":"active+clean","last_fresh":"2026-02-24T07:48:50.147000+0000","last_change":"2026-02-24T07:48:30.080353+0000","last_active":"2026-02-24T07:48:50.147000+0000","last_peered":"2026-02-24T07:48:50.147000+0000","last_clean":"2026-02-24T07:48:50.147000+0000","last_became_active":"2026-02-24T07:46:02.880333+0000","last_became_peered":"2026-02-24T07:46:02.880333+0000","last_unstale":"2026-02-24T07:48:50.147000+0000","last_undegraded":"2026-02-24T07:48:50.147000+0000","last_fullsized":"2026-02-24T07:48:50.147000+0000","mapping_epoch":42,"log_start":"51'82723","ondisk_log_start":"51'82723","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"49'71923","last_scrub_stamp":"2026-02-24T07:48:30.080305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_clean_scrub_stamp":"2026-02-24T07:48:30.080305+0000","objects_scrubbed":11094,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:49:53.942641+0000","scrub_duration":1405,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":82726,"num_write_kb":2647232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2","version":"51'83378","reported_seq":83810,"reported_epoch":51,"state":"active+clean","last_fresh":"2026-02-24T07:48:50.146306+0000","last_change":"2026-02-24T07:48:49.904480+0000","last_active":"2026-02-24T07:48:50.146306+0000","last_peered":"2026-02-24T07:48:50.146306+0000","last_clean":"2026-02-24T07:48:50.146306+0000","last_became_active":"2026-02-24T07:46:02.880185+0000","last_became_peered":"2026-02-24T07:46:02.880185+0000","last_unstale":"2026-02-24T07:48:50.146306+0000","last_undegraded":"2026-02-24T07:48:50.146306+0000","last_fullsized":"2026-02-24T07:48:50.146306+0000","mapping_epoch":42,"log_start":"51'83375","ondisk_log_start":"51'83375","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"51'83228","last_scrub_stamp":"2026-02-24T07:48:49.904420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:46:01.867949+0000","last_clean_scrub_stamp":"2026-02-24T07:48:49.904420+0000","objects_scrubbed":175,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:49:51.471610+0000","scrub_duration":185,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":83378,"num_write_kb":2668096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"29'99","reported_seq":257,"reported_epoch":51,"state":"active+clean","last_fresh":"2026-02-24T07:48:48.873925+0000","last_change":"2026-02-24T07:48:02.589664+0000","last_active":"2026-02-24T07:48:48.873925+0000","last_peered":"2026-02-24T07:48:48.873925+0000","last_clean":"2026-02-24T07:48:48.873925+0000","last_became_active":"2026-02-24T07:42:54.186719+0000","last_became_peered":"2026-02-24T07:42:54.186719+0000","last_unstale":"2026-02-24T07:48:48.873925+0000","last_undegraded":"2026-02-24T07:48:48.873925+0000","last_fullsized":"2026-02-24T07:48:48.873925+0000","mapping_epoch":27,"log_start":"29'96","ondisk_log_start":"29'96","created":20,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'99","last_scrub_stamp":"2026-02-24T07:48:02.589609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:42:43.640776+0000","last_clean_scrub_stamp":"2026-02-24T07:48:02.589609+0000","objects_scrubbed":2,"log_size":3,"log_dups_size":96,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:49:04.391555+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":240,"num_write":135,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":16,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1327016,"num_write_kb":42464449,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9060,"internal_metadata":0},"log_size":48,"ondisk_log_size":48,"up":48,"acting":48,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":240,"num_write":135,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1088,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":3,"up_from":25,"seq":107374182474,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":1007120,"kb_used_data":1084,"kb_used_omap":12,"kb_used_meta":1006003,"kb_avail":732996080,"statfs":{"total":751619276800,"available":750587985920,"internally_reserved":0,"allocated":1110016,"data_stored":774266,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12442,"internal_metadata":1030147942},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":19,"seq":81604378700,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":1419408,"kb_used_data":1084,"kb_used_omap":11,"kb_used_meta":1418292,"kb_avail":732583792,"statfs":{"total":751619276800,"available":750165803008,"internally_reserved":0,"allocated":1110016,"data_stored":774266,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12165,"internal_metadata":1452331131},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542221,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":1128012,"kb_used_data":504,"kb_used_omap":13,"kb_used_meta":1127474,"kb_avail":732875188,"statfs":{"total":751619276800,"available":750464192512,"internally_reserved":0,"allocated":516096,"data_stored":183898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13793,"internal_metadata":1154533919},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705744,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":1204500,"kb_used_data":34368,"kb_used_omap":9,"kb_used_meta":1170038,"kb_avail":732798700,"statfs":{"total":751619276800,"available":750385868800,"internally_reserved":0,"allocated":35192832,"data_stored":34853010,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":10131,"internal_metadata":1198119021},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1088,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":93,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":5117,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1581,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":2269,"internal_metadata":0}]}} 2026-02-24T07:48:53.056 INFO:tasks.cephadm.ceph_manager.ceph:increase pool size by 10 2026-02-24T07:48:53.056 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph osd pool set .mgr pg_num 221 2026-02-24T07:48:53.080 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:48:52 trial035 ceph-mon[39284]: pgmap v202: 17 pgs: 17 active+clean; 577 KiB data, 4.5 GiB used, 2.7 TiB / 2.7 TiB avail; 0 B/s wr, 13.63k op/s 2026-02-24T07:48:53.080 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:48:52 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/874565567' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 3, "weight": 1.0}]': finished 2026-02-24T07:48:53.080 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:48:52 trial035 ceph-mon[39284]: osdmap e53: 4 total, 4 up, 4 in 2026-02-24T07:48:53.173 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:48:53.280 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:48:53.278+0000 7ff0aed3b640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:48:53.280 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:48:53.278+0000 7ff0aed3b640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:48:53.391 INFO:teuthology.orchestra.run.trial035.stderr:Error EPERM: crimson_allow_pg_split is false; pg_num increase denied for crimson pool 2026-02-24T07:48:53.425 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-24T07:48:53.709 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2026-02-24T07:48:53.827 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:48:53.924 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:48:53.922+0000 7fd980186640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:48:53.925 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:48:53.922+0000 7fd980186640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:48:53.937 INFO:teuthology.orchestra.run.trial035.stdout:osd.0: {} 2026-02-24T07:48:53.937 INFO:teuthology.orchestra.run.trial035.stderr:osd.0: osd_enable_op_tracker = '' 2026-02-24T07:48:53.948 INFO:teuthology.orchestra.run.trial035.stdout:osd.1: {} 2026-02-24T07:48:53.948 INFO:teuthology.orchestra.run.trial035.stderr:osd.1: osd_enable_op_tracker = '' 2026-02-24T07:48:53.960 INFO:teuthology.orchestra.run.trial035.stdout:osd.2: {} 2026-02-24T07:48:53.960 INFO:teuthology.orchestra.run.trial035.stderr:osd.2: osd_enable_op_tracker = '' 2026-02-24T07:48:53.971 INFO:teuthology.orchestra.run.trial035.stdout:osd.3: {} 2026-02-24T07:48:53.971 INFO:teuthology.orchestra.run.trial035.stderr:osd.3: osd_enable_op_tracker = '' 2026-02-24T07:48:53.985 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:48:53 trial035 ceph-mon[39284]: from='client.14844 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2026-02-24T07:48:53.985 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:48:53 trial035 ceph-mon[39284]: from='client.14846 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2026-02-24T07:48:53.985 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:48:53 trial035 ceph-mon[39284]: purged_snaps scrub starts 2026-02-24T07:48:53.985 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:48:53 trial035 ceph-mon[39284]: purged_snaps scrub ok 2026-02-24T07:48:53.985 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:48:53 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/3714822986' entity='client.admin' cmd={"prefix": "osd pool set", "pool": ".mgr", "var": "pg_num", "val": "221"} : dispatch 2026-02-24T07:48:54.761 INFO:tasks.cephadm.ceph_manager.ceph:removing pool_name unique_pool_1 2026-02-24T07:48:54.761 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph osd pool rm unique_pool_1 unique_pool_1 --yes-i-really-really-mean-it 2026-02-24T07:48:54.876 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:48:55.153 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:48:54 trial035 ceph-mon[39284]: from='client.14848 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2026-02-24T07:48:55.153 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:48:54 trial035 ceph-mon[39284]: pgmap v204: 17 pgs: 17 active+clean; 577 KiB data, 4.5 GiB used, 2.7 TiB / 2.7 TiB avail; 0 B/s wr, 6.39k op/s 2026-02-24T07:48:55.153 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:48:54 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/2052075502' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2026-02-24T07:48:55.153 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:48:54 trial035 ceph-mon[39284]: 3.e scrub starts 2026-02-24T07:48:55.153 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:48:54 trial035 ceph-mon[39284]: 3.e scrub ok 2026-02-24T07:48:55.210 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:48:55.208+0000 7fe35ab90640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:48:55.210 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:48:55.208+0000 7fe35ab90640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:48:55.891 INFO:teuthology.orchestra.run.trial035.stderr:pool 'unique_pool_1' removed 2026-02-24T07:48:55.936 INFO:teuthology.task.full_sequential:In full_sequential, running task radosbench... 2026-02-24T07:48:55.936 INFO:tasks.radosbench:Beginning radosbench... 2026-02-24T07:48:55.937 INFO:tasks.cephadm.ceph_manager.ceph:creating pool_name unique_pool_2 2026-02-24T07:48:55.937 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph osd pool create unique_pool_2 16 2026-02-24T07:48:56.007 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2026-02-24T07:48:56.059 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:48:56.124 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:48:56.146 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:48:55 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/1074296496' entity='client.admin' cmd={"prefix": "osd pool rm", "pool": "unique_pool_1", "pool2": "unique_pool_1", "yes_i_really_really_mean_it": true} : dispatch 2026-02-24T07:48:56.146 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:48:55 trial035 ceph-mon[39284]: pgmap v205: 17 pgs: 17 active+clean; 577 KiB data, 4.5 GiB used, 2.7 TiB / 2.7 TiB avail; 0 B/s wr, 5.46k op/s 2026-02-24T07:48:56.173 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:48:56.171+0000 7f10c0907640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:48:56.174 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:48:56.172+0000 7f10c0907640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:48:56.235 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:48:56.233+0000 7f5c76a32640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:48:56.235 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:48:56.233+0000 7f5c76a32640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:48:56.249 INFO:teuthology.orchestra.run.trial035.stdout:osd.0: {} 2026-02-24T07:48:56.249 INFO:teuthology.orchestra.run.trial035.stderr:osd.0: osd_enable_op_tracker = '' 2026-02-24T07:48:56.263 INFO:teuthology.orchestra.run.trial035.stdout:osd.1: {} 2026-02-24T07:48:56.264 INFO:teuthology.orchestra.run.trial035.stderr:osd.1: osd_enable_op_tracker = '' 2026-02-24T07:48:56.276 INFO:teuthology.orchestra.run.trial035.stdout:osd.2: {} 2026-02-24T07:48:56.276 INFO:teuthology.orchestra.run.trial035.stderr:osd.2: osd_enable_op_tracker = '' 2026-02-24T07:48:56.287 INFO:teuthology.orchestra.run.trial035.stdout:osd.3: {} 2026-02-24T07:48:56.287 INFO:teuthology.orchestra.run.trial035.stderr:osd.3: osd_enable_op_tracker = '' 2026-02-24T07:48:56.896 INFO:teuthology.orchestra.run.trial035.stderr:pool 'unique_pool_2' created 2026-02-24T07:48:56.939 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph osd pool application enable unique_pool_2 rados --yes-i-really-mean-it || true 2026-02-24T07:48:57.057 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:48:57.169 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:48:57.168+0000 7f7fa8eda640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:48:57.170 INFO:journalctl@ceph.mgr.x.trial035.stdout:Feb 24 07:48:56 trial035 ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-mgr-x[39621]: ::ffff:10.20.193.91 - - [24/Feb/2026:07:48:56] "GET /metrics HTTP/1.1" 200 4237 "" "Prometheus/3.6.0" 2026-02-24T07:48:57.170 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:48:56 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/1074296496' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "unique_pool_1", "pool2": "unique_pool_1", "yes_i_really_really_mean_it": true}]': finished 2026-02-24T07:48:57.171 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:48:56 trial035 ceph-mon[39284]: osdmap e54: 4 total, 4 up, 4 in 2026-02-24T07:48:57.171 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:48:56 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/3613130250' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2026-02-24T07:48:57.171 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:48:56 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/1263474168' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "unique_pool_2", "pg_num": 16} : dispatch 2026-02-24T07:48:57.171 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:48:57.168+0000 7f7fa8eda640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:48:57.356 INFO:journalctl@ceph.prometheus.a.trial091.stdout:Feb 24 07:48:56 trial091 ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-prometheus-a[66007]: time=2026-02-24T07:48:56.973Z level=ERROR source=scrape.go:1630 msg="Failed to determine correct type of scrape target." component="scrape manager" scrape_pool=ceph target=http://trial035:9283/metrics content_type="text/html;charset=utf-8" fallback_media_type="" err="received unsupported Content-Type \"text/html;charset=utf-8\" and no fallback_scrape_protocol specified for target" 2026-02-24T07:48:57.897 INFO:teuthology.orchestra.run.trial035.stderr:enabled application 'rados' on pool 'unique_pool_2' 2026-02-24T07:48:58.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:48:57 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/1263474168' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "unique_pool_2", "pg_num": 16}]': finished 2026-02-24T07:48:58.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:48:57 trial035 ceph-mon[39284]: osdmap e55: 4 total, 4 up, 4 in 2026-02-24T07:48:58.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:48:57 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/3915674835' entity='client.admin' cmd={"prefix": "osd pool application enable", "pool": "unique_pool_2", "app": "rados", "yes_i_really_mean_it": true} : dispatch 2026-02-24T07:48:58.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:48:57 trial035 ceph-mon[39284]: pgmap v208: 17 pgs: 16 unknown, 1 active+clean; 577 KiB data, 4.5 GiB used, 2.7 TiB / 2.7 TiB avail 2026-02-24T07:48:58.328 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2026-02-24T07:48:58.426 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2026-02-24T07:48:58.426 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2026-02-24T07:48:58.445 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:48:58.540 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:48:58.539+0000 7fa8414ee640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:48:58.540 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:48:58.539+0000 7fa8414ee640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:48:58.552 INFO:teuthology.orchestra.run.trial035.stdout:osd.0: {} 2026-02-24T07:48:58.553 INFO:teuthology.orchestra.run.trial035.stderr:osd.0: osd_enable_op_tracker = '' 2026-02-24T07:48:58.564 INFO:teuthology.orchestra.run.trial035.stdout:osd.1: {} 2026-02-24T07:48:58.564 INFO:teuthology.orchestra.run.trial035.stderr:osd.1: osd_enable_op_tracker = '' 2026-02-24T07:48:58.575 INFO:teuthology.orchestra.run.trial035.stdout:osd.2: {} 2026-02-24T07:48:58.576 INFO:teuthology.orchestra.run.trial035.stderr:osd.2: osd_enable_op_tracker = '' 2026-02-24T07:48:58.586 INFO:teuthology.orchestra.run.trial035.stdout:osd.3: {} 2026-02-24T07:48:58.586 INFO:teuthology.orchestra.run.trial035.stderr:osd.3: osd_enable_op_tracker = '' 2026-02-24T07:48:58.940 DEBUG:teuthology.orchestra.run.trial035:> /bin/sh -c 'adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --no-log-to-stderr --name client.0 -t 16 -b 65536 --object-size 65536 -p unique_pool_2 bench 90 write' 2026-02-24T07:48:58.940 INFO:tasks.thrashosds.thrasher:fixing pg num pool .mgr 2026-02-24T07:48:58.941 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph pg dump --format=json 2026-02-24T07:48:58.944 INFO:tasks.radosbench:joining radosbench (timing out after 3000s) 2026-02-24T07:48:58.944 INFO:teuthology.orchestra.run:waiting for 3000 2026-02-24T07:48:58.993 INFO:tasks.radosbench.radosbench.0.trial035.stdout:hints = 1 2026-02-24T07:48:58.993 INFO:tasks.radosbench.radosbench.0.trial035.stdout:Maintaining 16 concurrent writes of 65536 bytes to objects of size 65536 for up to 90 seconds or 0 objects 2026-02-24T07:48:58.994 INFO:tasks.radosbench.radosbench.0.trial035.stdout:Object prefix: benchmark_data_trial035_105836 2026-02-24T07:48:58.994 INFO:tasks.radosbench.radosbench.0.trial035.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2026-02-24T07:48:58.994 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 0 0 0 0 0 0 - 0 2026-02-24T07:48:59.080 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:48:59.157 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:48:58 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/3915674835' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "unique_pool_2", "app": "rados", "yes_i_really_mean_it": true}]': finished 2026-02-24T07:48:59.157 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:48:58 trial035 ceph-mon[39284]: osdmap e56: 4 total, 4 up, 4 in 2026-02-24T07:48:59.157 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:48:58 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/4850921' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2026-02-24T07:48:59.218 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:48:59.216+0000 7fb02d3d8640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:48:59.218 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:48:59.217+0000 7fb02d3d8640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:48:59.355 INFO:teuthology.orchestra.run.trial035.stdout: 2026-02-24T07:48:59.356 INFO:teuthology.orchestra.run.trial035.stderr:dumped all 2026-02-24T07:48:59.403 INFO:teuthology.orchestra.run.trial035.stdout:{"pg_ready":true,"pg_map":{"version":210,"stamp":"2026-02-24T07:48:58.899937+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":240,"num_write":135,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":3,"acting":3,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":51,"num_osds":4,"num_per_pool_osds":4,"num_per_pool_omap_osds":4,"kb":2936012800,"kb_used":4725788,"kb_used_data":3852,"kb_used_omap":53,"kb_used_meta":4721802,"kb_avail":2931287012,"statfs":{"total":3006477107200,"available":3001637900288,"internally_reserved":0,"allocated":3944448,"data_stored":2541968,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":54480,"internal_metadata":4835126064},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":-1,"num_read_kb":-1,"num_write":-1327016,"num_write_kb":-42464449,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.194953"},"pg_stats":[{"pgid":"4.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2026-02-24T07:48:56.890768+0000","last_change":"2026-02-24T07:48:56.890768+0000","last_active":"2026-02-24T07:48:56.890768+0000","last_peered":"2026-02-24T07:48:56.890768+0000","last_clean":"2026-02-24T07:48:56.890768+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2026-02-24T07:48:56.890768+0000","last_undegraded":"2026-02-24T07:48:56.890768+0000","last_fullsized":"2026-02-24T07:48:56.890768+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:48:56.890768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2026-02-24T07:48:56.890768+0000","last_change":"2026-02-24T07:48:56.890768+0000","last_active":"2026-02-24T07:48:56.890768+0000","last_peered":"2026-02-24T07:48:56.890768+0000","last_clean":"2026-02-24T07:48:56.890768+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2026-02-24T07:48:56.890768+0000","last_undegraded":"2026-02-24T07:48:56.890768+0000","last_fullsized":"2026-02-24T07:48:56.890768+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:48:56.890768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2026-02-24T07:48:56.890768+0000","last_change":"2026-02-24T07:48:56.890768+0000","last_active":"2026-02-24T07:48:56.890768+0000","last_peered":"2026-02-24T07:48:56.890768+0000","last_clean":"2026-02-24T07:48:56.890768+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2026-02-24T07:48:56.890768+0000","last_undegraded":"2026-02-24T07:48:56.890768+0000","last_fullsized":"2026-02-24T07:48:56.890768+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:48:56.890768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2026-02-24T07:48:56.890768+0000","last_change":"2026-02-24T07:48:56.890768+0000","last_active":"2026-02-24T07:48:56.890768+0000","last_peered":"2026-02-24T07:48:56.890768+0000","last_clean":"2026-02-24T07:48:56.890768+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2026-02-24T07:48:56.890768+0000","last_undegraded":"2026-02-24T07:48:56.890768+0000","last_fullsized":"2026-02-24T07:48:56.890768+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:48:56.890768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2026-02-24T07:48:56.890768+0000","last_change":"2026-02-24T07:48:56.890768+0000","last_active":"2026-02-24T07:48:56.890768+0000","last_peered":"2026-02-24T07:48:56.890768+0000","last_clean":"2026-02-24T07:48:56.890768+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2026-02-24T07:48:56.890768+0000","last_undegraded":"2026-02-24T07:48:56.890768+0000","last_fullsized":"2026-02-24T07:48:56.890768+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:48:56.890768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2026-02-24T07:48:56.890768+0000","last_change":"2026-02-24T07:48:56.890768+0000","last_active":"2026-02-24T07:48:56.890768+0000","last_peered":"2026-02-24T07:48:56.890768+0000","last_clean":"2026-02-24T07:48:56.890768+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2026-02-24T07:48:56.890768+0000","last_undegraded":"2026-02-24T07:48:56.890768+0000","last_fullsized":"2026-02-24T07:48:56.890768+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:48:56.890768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2026-02-24T07:48:56.890768+0000","last_change":"2026-02-24T07:48:56.890768+0000","last_active":"2026-02-24T07:48:56.890768+0000","last_peered":"2026-02-24T07:48:56.890768+0000","last_clean":"2026-02-24T07:48:56.890768+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2026-02-24T07:48:56.890768+0000","last_undegraded":"2026-02-24T07:48:56.890768+0000","last_fullsized":"2026-02-24T07:48:56.890768+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:48:56.890768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2026-02-24T07:48:56.890768+0000","last_change":"2026-02-24T07:48:56.890768+0000","last_active":"2026-02-24T07:48:56.890768+0000","last_peered":"2026-02-24T07:48:56.890768+0000","last_clean":"2026-02-24T07:48:56.890768+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2026-02-24T07:48:56.890768+0000","last_undegraded":"2026-02-24T07:48:56.890768+0000","last_fullsized":"2026-02-24T07:48:56.890768+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:48:56.890768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2026-02-24T07:48:56.890768+0000","last_change":"2026-02-24T07:48:56.890768+0000","last_active":"2026-02-24T07:48:56.890768+0000","last_peered":"2026-02-24T07:48:56.890768+0000","last_clean":"2026-02-24T07:48:56.890768+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2026-02-24T07:48:56.890768+0000","last_undegraded":"2026-02-24T07:48:56.890768+0000","last_fullsized":"2026-02-24T07:48:56.890768+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:48:56.890768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2026-02-24T07:48:56.890768+0000","last_change":"2026-02-24T07:48:56.890768+0000","last_active":"2026-02-24T07:48:56.890768+0000","last_peered":"2026-02-24T07:48:56.890768+0000","last_clean":"2026-02-24T07:48:56.890768+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2026-02-24T07:48:56.890768+0000","last_undegraded":"2026-02-24T07:48:56.890768+0000","last_fullsized":"2026-02-24T07:48:56.890768+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:48:56.890768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2026-02-24T07:48:56.890768+0000","last_change":"2026-02-24T07:48:56.890768+0000","last_active":"2026-02-24T07:48:56.890768+0000","last_peered":"2026-02-24T07:48:56.890768+0000","last_clean":"2026-02-24T07:48:56.890768+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2026-02-24T07:48:56.890768+0000","last_undegraded":"2026-02-24T07:48:56.890768+0000","last_fullsized":"2026-02-24T07:48:56.890768+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:48:56.890768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2026-02-24T07:48:56.890768+0000","last_change":"2026-02-24T07:48:56.890768+0000","last_active":"2026-02-24T07:48:56.890768+0000","last_peered":"2026-02-24T07:48:56.890768+0000","last_clean":"2026-02-24T07:48:56.890768+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2026-02-24T07:48:56.890768+0000","last_undegraded":"2026-02-24T07:48:56.890768+0000","last_fullsized":"2026-02-24T07:48:56.890768+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:48:56.890768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2026-02-24T07:48:56.890768+0000","last_change":"2026-02-24T07:48:56.890768+0000","last_active":"2026-02-24T07:48:56.890768+0000","last_peered":"2026-02-24T07:48:56.890768+0000","last_clean":"2026-02-24T07:48:56.890768+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2026-02-24T07:48:56.890768+0000","last_undegraded":"2026-02-24T07:48:56.890768+0000","last_fullsized":"2026-02-24T07:48:56.890768+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:48:56.890768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2026-02-24T07:48:56.890768+0000","last_change":"2026-02-24T07:48:56.890768+0000","last_active":"2026-02-24T07:48:56.890768+0000","last_peered":"2026-02-24T07:48:56.890768+0000","last_clean":"2026-02-24T07:48:56.890768+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2026-02-24T07:48:56.890768+0000","last_undegraded":"2026-02-24T07:48:56.890768+0000","last_fullsized":"2026-02-24T07:48:56.890768+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:48:56.890768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2026-02-24T07:48:56.890768+0000","last_change":"2026-02-24T07:48:56.890768+0000","last_active":"2026-02-24T07:48:56.890768+0000","last_peered":"2026-02-24T07:48:56.890768+0000","last_clean":"2026-02-24T07:48:56.890768+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2026-02-24T07:48:56.890768+0000","last_undegraded":"2026-02-24T07:48:56.890768+0000","last_fullsized":"2026-02-24T07:48:56.890768+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:48:56.890768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2026-02-24T07:48:56.890768+0000","last_change":"2026-02-24T07:48:56.890768+0000","last_active":"2026-02-24T07:48:56.890768+0000","last_peered":"2026-02-24T07:48:56.890768+0000","last_clean":"2026-02-24T07:48:56.890768+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2026-02-24T07:48:56.890768+0000","last_undegraded":"2026-02-24T07:48:56.890768+0000","last_fullsized":"2026-02-24T07:48:56.890768+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:48:56.890768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"29'99","reported_seq":262,"reported_epoch":53,"state":"active+clean","last_fresh":"2026-02-24T07:48:53.046841+0000","last_change":"2026-02-24T07:48:02.589664+0000","last_active":"2026-02-24T07:48:53.046841+0000","last_peered":"2026-02-24T07:48:53.046841+0000","last_clean":"2026-02-24T07:48:53.046841+0000","last_became_active":"2026-02-24T07:42:54.186719+0000","last_became_peered":"2026-02-24T07:42:54.186719+0000","last_unstale":"2026-02-24T07:48:53.046841+0000","last_undegraded":"2026-02-24T07:48:53.046841+0000","last_fullsized":"2026-02-24T07:48:53.046841+0000","mapping_epoch":27,"log_start":"29'96","ondisk_log_start":"29'96","created":20,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'99","last_scrub_stamp":"2026-02-24T07:48:02.589609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:42:43.640776+0000","last_clean_scrub_stamp":"2026-02-24T07:48:02.589609+0000","objects_scrubbed":2,"log_size":3,"log_dups_size":96,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:49:04.391555+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":240,"num_write":135,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":16,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":240,"num_write":135,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1304,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":3,"up_from":25,"seq":107374182475,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":1007144,"kb_used_data":1108,"kb_used_omap":14,"kb_used_meta":1006001,"kb_avail":732996056,"statfs":{"total":751619276800,"available":750587961344,"internally_reserved":0,"allocated":1134592,"data_stored":783084,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":14392,"internal_metadata":1030145992},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":19,"seq":81604378701,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":1419432,"kb_used_data":1108,"kb_used_omap":13,"kb_used_meta":1418290,"kb_avail":732583768,"statfs":{"total":751619276800,"available":750165778432,"internally_reserved":0,"allocated":1134592,"data_stored":783084,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13560,"internal_metadata":1452329736},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542222,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":1128036,"kb_used_data":528,"kb_used_omap":14,"kb_used_meta":1127473,"kb_avail":732875164,"statfs":{"total":751619276800,"available":750464167936,"internally_reserved":0,"allocated":540672,"data_stored":192716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":15093,"internal_metadata":1154532619},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705745,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":1171176,"kb_used_data":1108,"kb_used_omap":11,"kb_used_meta":1170036,"kb_avail":732832024,"statfs":{"total":751619276800,"available":750419992576,"internally_reserved":0,"allocated":1134592,"data_stored":783084,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":11435,"internal_metadata":1198117717},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1304,"internal_metadata":0}]}} 2026-02-24T07:48:59.405 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph osd pool set .mgr pgp_num 221 2026-02-24T07:48:59.566 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:48:59.696 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:48:59.695+0000 7fd7d61a9640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:48:59.696 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:48:59.696+0000 7fd7d61a9640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:48:59.846 INFO:teuthology.orchestra.run.trial035.stderr:Error EPERM: crimson_allow_pg_split is false; pgp_num increase denied 2026-02-24T07:48:59.892 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-24T07:48:59.954 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:48:59 trial035 ceph-mon[39284]: osdmap e57: 4 total, 4 up, 4 in 2026-02-24T07:48:59.954 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:48:59 trial035 ceph-mon[39284]: from='client.14867 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2026-02-24T07:48:59.954 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:48:59 trial035 ceph-mon[39284]: pgmap v211: 17 pgs: 16 unknown, 1 active+clean; 577 KiB data, 4.5 GiB used, 2.7 TiB / 2.7 TiB avail 2026-02-24T07:48:59.955 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:48:59 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/3951359795' entity='client.admin' cmd={"prefix": "osd pool set", "pool": ".mgr", "var": "pgp_num", "val": "221"} : dispatch 2026-02-24T07:48:59.955 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:48:59 trial035 ceph-mon[39284]: from='mgr.14284 10.20.193.35:0/3681373222' entity='mgr.x' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2026-02-24T07:48:59.993 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 1 16 7944 7928 495.467 495.5 0.00242748 0.00201477 2026-02-24T07:49:00.622 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2026-02-24T07:49:00.767 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:49:00.887 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:49:00.886+0000 7f236491b640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:49:00.888 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:49:00.886+0000 7f236491b640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:49:00.906 INFO:teuthology.orchestra.run.trial035.stdout:osd.0: {} 2026-02-24T07:49:00.906 INFO:teuthology.orchestra.run.trial035.stderr:osd.0: osd_enable_op_tracker = '' 2026-02-24T07:49:00.920 INFO:teuthology.orchestra.run.trial035.stdout:osd.1: {} 2026-02-24T07:49:00.920 INFO:teuthology.orchestra.run.trial035.stderr:osd.1: osd_enable_op_tracker = '' 2026-02-24T07:49:00.932 INFO:teuthology.orchestra.run.trial035.stdout:osd.2: {} 2026-02-24T07:49:00.933 INFO:teuthology.orchestra.run.trial035.stderr:osd.2: osd_enable_op_tracker = '' 2026-02-24T07:49:00.946 INFO:teuthology.orchestra.run.trial035.stdout:osd.3: {} 2026-02-24T07:49:00.946 INFO:teuthology.orchestra.run.trial035.stderr:osd.3: osd_enable_op_tracker = '' 2026-02-24T07:49:00.993 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 2 16 16208 16192 505.964 516.5 0.00148013 0.00197385 2026-02-24T07:49:01.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:49:00 trial035 ceph-mon[39284]: purged_snaps scrub starts 2026-02-24T07:49:01.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:49:00 trial035 ceph-mon[39284]: purged_snaps scrub ok 2026-02-24T07:49:01.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:49:00 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/759349636' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2026-02-24T07:49:01.994 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 3 16 24455 24439 509.109 515.438 0.00203658 0.00196265 2026-02-24T07:49:02.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:49:01 trial035 ceph-mon[39284]: pgmap v212: 17 pgs: 17 active+clean; 886 MiB data, 6.6 GiB used, 2.7 TiB / 2.7 TiB avail; 152 MiB/s wr, 2.44k op/s 2026-02-24T07:49:02.991 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2026-02-24T07:49:02.995 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 4 16 32658 32642 509.995 512.688 0.00256365 0.00195939 2026-02-24T07:49:03.134 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:49:03.253 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:49:03.251+0000 7fbddea07640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:49:03.253 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:49:03.252+0000 7fbddea07640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:49:03.269 INFO:teuthology.orchestra.run.trial035.stdout:osd.0: {} 2026-02-24T07:49:03.269 INFO:teuthology.orchestra.run.trial035.stderr:osd.0: osd_enable_op_tracker = '' 2026-02-24T07:49:03.282 INFO:teuthology.orchestra.run.trial035.stdout:osd.1: {} 2026-02-24T07:49:03.282 INFO:teuthology.orchestra.run.trial035.stderr:osd.1: osd_enable_op_tracker = '' 2026-02-24T07:49:03.295 INFO:teuthology.orchestra.run.trial035.stdout:osd.2: {} 2026-02-24T07:49:03.295 INFO:teuthology.orchestra.run.trial035.stderr:osd.2: osd_enable_op_tracker = '' 2026-02-24T07:49:03.308 INFO:teuthology.orchestra.run.trial035.stdout:osd.3: {} 2026-02-24T07:49:03.308 INFO:teuthology.orchestra.run.trial035.stderr:osd.3: osd_enable_op_tracker = '' 2026-02-24T07:49:03.704 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:49:03 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/1363984059' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2026-02-24T07:49:03.994 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 5 16 41100 41084 513.513 527.625 0.0022155 0.00194577 2026-02-24T07:49:04.704 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:49:04 trial035 ceph-mon[39284]: pgmap v213: 17 pgs: 17 active+clean; 886 MiB data, 6.6 GiB used, 2.7 TiB / 2.7 TiB avail; 130 MiB/s wr, 2.08k op/s 2026-02-24T07:49:04.893 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2026-02-24T07:49:04.893 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2026-02-24T07:49:04.894 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph pg dump --format=json 2026-02-24T07:49:04.994 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 6 16 49282 49266 513.15 511.375 0.00315344 0.00194752 2026-02-24T07:49:05.039 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:49:05.169 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:49:05.167+0000 7fb6600a8640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:49:05.169 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:49:05.168+0000 7fb6600a8640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:49:05.311 INFO:teuthology.orchestra.run.trial035.stdout: 2026-02-24T07:49:05.312 INFO:teuthology.orchestra.run.trial035.stderr:dumped all 2026-02-24T07:49:05.353 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2026-02-24T07:49:05.357 INFO:teuthology.orchestra.run.trial035.stdout:{"pg_ready":true,"pg_map":{"version":213,"stamp":"2026-02-24T07:49:03.706511+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":929301024,"num_objects":14173,"num_object_clones":0,"num_object_copies":42519,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14173,"num_whiteouts":0,"num_read":136,"num_read_kb":240,"num_write":14306,"num_write_kb":909094,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":51,"ondisk_log_size":51,"up":51,"acting":51,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":51,"num_osds":4,"num_per_pool_osds":4,"num_per_pool_omap_osds":4,"kb":2936012800,"kb_used":6908508,"kb_used_data":2114188,"kb_used_omap":63,"kb_used_meta":4794112,"kb_avail":2929104292,"statfs":{"total":3006477107200,"available":2999402795008,"internally_reserved":0,"allocated":2164928512,"data_stored":2163404192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":65401,"internal_metadata":4909170823},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[18,2],"upper_bound":4},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":928710656,"num_objects":14171,"num_object_clones":0,"num_object_copies":42513,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14171,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14171,"num_write_kb":906944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.810955"},"pg_stats":[{"pgid":"4.f","version":"57'899","reported_seq":913,"reported_epoch":57,"state":"active+clean","last_fresh":"2026-02-24T07:49:00.804736+0000","last_change":"2026-02-24T07:48:57.902569+0000","last_active":"2026-02-24T07:49:00.804736+0000","last_peered":"2026-02-24T07:49:00.804736+0000","last_clean":"2026-02-24T07:49:00.804736+0000","last_became_active":"2026-02-24T07:48:57.902313+0000","last_became_peered":"2026-02-24T07:48:57.902313+0000","last_unstale":"2026-02-24T07:49:00.804736+0000","last_undegraded":"2026-02-24T07:49:00.804736+0000","last_fullsized":"2026-02-24T07:49:00.804736+0000","mapping_epoch":55,"log_start":"57'896","ondisk_log_start":"57'896","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:48:56.890768+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":896,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:49:59.578048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":58916864,"num_objects":899,"num_object_clones":0,"num_object_copies":2697,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":899,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":899,"num_write_kb":57536,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"57'888","reported_seq":902,"reported_epoch":57,"state":"active+clean","last_fresh":"2026-02-24T07:49:00.712852+0000","last_change":"2026-02-24T07:48:57.903920+0000","last_active":"2026-02-24T07:49:00.712852+0000","last_peered":"2026-02-24T07:49:00.712852+0000","last_clean":"2026-02-24T07:49:00.712852+0000","last_became_active":"2026-02-24T07:48:57.903584+0000","last_became_peered":"2026-02-24T07:48:57.903584+0000","last_unstale":"2026-02-24T07:49:00.712852+0000","last_undegraded":"2026-02-24T07:49:00.712852+0000","last_fullsized":"2026-02-24T07:49:00.712852+0000","mapping_epoch":55,"log_start":"57'885","ondisk_log_start":"57'885","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:48:56.890768+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":885,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:50:17.159571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":58195968,"num_objects":888,"num_object_clones":0,"num_object_copies":2664,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":888,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":888,"num_write_kb":56832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.d","version":"57'899","reported_seq":911,"reported_epoch":57,"state":"active+clean","last_fresh":"2026-02-24T07:49:00.728234+0000","last_change":"2026-02-24T07:48:57.905520+0000","last_active":"2026-02-24T07:49:00.728234+0000","last_peered":"2026-02-24T07:49:00.728234+0000","last_clean":"2026-02-24T07:49:00.728234+0000","last_became_active":"2026-02-24T07:48:57.905031+0000","last_became_peered":"2026-02-24T07:48:57.905031+0000","last_unstale":"2026-02-24T07:49:00.728234+0000","last_undegraded":"2026-02-24T07:49:00.728234+0000","last_fullsized":"2026-02-24T07:49:00.728234+0000","mapping_epoch":55,"log_start":"57'896","ondisk_log_start":"57'896","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:48:56.890768+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":896,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:50:02.301170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":58916864,"num_objects":899,"num_object_clones":0,"num_object_copies":2697,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":899,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":899,"num_write_kb":57536,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"57'870","reported_seq":884,"reported_epoch":57,"state":"active+clean","last_fresh":"2026-02-24T07:49:00.728012+0000","last_change":"2026-02-24T07:48:57.905290+0000","last_active":"2026-02-24T07:49:00.728012+0000","last_peered":"2026-02-24T07:49:00.728012+0000","last_clean":"2026-02-24T07:49:00.728012+0000","last_became_active":"2026-02-24T07:48:57.904683+0000","last_became_peered":"2026-02-24T07:48:57.904683+0000","last_unstale":"2026-02-24T07:49:00.728012+0000","last_undegraded":"2026-02-24T07:49:00.728012+0000","last_fullsized":"2026-02-24T07:49:00.728012+0000","mapping_epoch":55,"log_start":"57'867","ondisk_log_start":"57'867","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:48:56.890768+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":867,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:50:16.643522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":57016320,"num_objects":870,"num_object_clones":0,"num_object_copies":2610,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":870,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":870,"num_write_kb":55680,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.b","version":"57'893","reported_seq":905,"reported_epoch":57,"state":"active+clean","last_fresh":"2026-02-24T07:49:00.732407+0000","last_change":"2026-02-24T07:48:57.902025+0000","last_active":"2026-02-24T07:49:00.732407+0000","last_peered":"2026-02-24T07:49:00.732407+0000","last_clean":"2026-02-24T07:49:00.732407+0000","last_became_active":"2026-02-24T07:48:57.901700+0000","last_became_peered":"2026-02-24T07:48:57.901700+0000","last_unstale":"2026-02-24T07:49:00.732407+0000","last_undegraded":"2026-02-24T07:49:00.732407+0000","last_fullsized":"2026-02-24T07:49:00.732407+0000","mapping_epoch":55,"log_start":"57'890","ondisk_log_start":"57'890","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:48:56.890768+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":890,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:50:23.370319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":58523648,"num_objects":893,"num_object_clones":0,"num_object_copies":2679,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":893,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":893,"num_write_kb":57152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.a","version":"57'893","reported_seq":907,"reported_epoch":57,"state":"active+clean","last_fresh":"2026-02-24T07:49:00.708614+0000","last_change":"2026-02-24T07:48:57.903698+0000","last_active":"2026-02-24T07:49:00.708614+0000","last_peered":"2026-02-24T07:49:00.708614+0000","last_clean":"2026-02-24T07:49:00.708614+0000","last_became_active":"2026-02-24T07:48:57.903228+0000","last_became_peered":"2026-02-24T07:48:57.903228+0000","last_unstale":"2026-02-24T07:49:00.708614+0000","last_undegraded":"2026-02-24T07:49:00.708614+0000","last_fullsized":"2026-02-24T07:49:00.708614+0000","mapping_epoch":55,"log_start":"57'890","ondisk_log_start":"57'890","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:48:56.890768+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":890,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:50:26.780637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":58523648,"num_objects":893,"num_object_clones":0,"num_object_copies":2679,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":893,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":893,"num_write_kb":57152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"57'870","reported_seq":884,"reported_epoch":57,"state":"active+clean","last_fresh":"2026-02-24T07:49:00.712748+0000","last_change":"2026-02-24T07:48:57.902011+0000","last_active":"2026-02-24T07:49:00.712748+0000","last_peered":"2026-02-24T07:49:00.712748+0000","last_clean":"2026-02-24T07:49:00.712748+0000","last_became_active":"2026-02-24T07:48:57.901730+0000","last_became_peered":"2026-02-24T07:48:57.901730+0000","last_unstale":"2026-02-24T07:49:00.712748+0000","last_undegraded":"2026-02-24T07:49:00.712748+0000","last_fullsized":"2026-02-24T07:49:00.712748+0000","mapping_epoch":55,"log_start":"57'867","ondisk_log_start":"57'867","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:48:56.890768+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":867,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:50:05.273318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":57016320,"num_objects":870,"num_object_clones":0,"num_object_copies":2610,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":870,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":870,"num_write_kb":55680,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"57'874","reported_seq":887,"reported_epoch":57,"state":"active+clean","last_fresh":"2026-02-24T07:49:00.727189+0000","last_change":"2026-02-24T07:48:57.904924+0000","last_active":"2026-02-24T07:49:00.727189+0000","last_peered":"2026-02-24T07:49:00.727189+0000","last_clean":"2026-02-24T07:49:00.727189+0000","last_became_active":"2026-02-24T07:48:57.904376+0000","last_became_peered":"2026-02-24T07:48:57.904376+0000","last_unstale":"2026-02-24T07:49:00.727189+0000","last_undegraded":"2026-02-24T07:49:00.727189+0000","last_fullsized":"2026-02-24T07:49:00.727189+0000","mapping_epoch":55,"log_start":"57'871","ondisk_log_start":"57'871","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:48:56.890768+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":871,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:49:57.806612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":57278464,"num_objects":874,"num_object_clones":0,"num_object_copies":2622,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":874,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":874,"num_write_kb":55936,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.7","version":"57'921","reported_seq":934,"reported_epoch":57,"state":"active+clean","last_fresh":"2026-02-24T07:49:00.800882+0000","last_change":"2026-02-24T07:48:57.902698+0000","last_active":"2026-02-24T07:49:00.800882+0000","last_peered":"2026-02-24T07:49:00.800882+0000","last_clean":"2026-02-24T07:49:00.800882+0000","last_became_active":"2026-02-24T07:48:57.902484+0000","last_became_peered":"2026-02-24T07:48:57.902484+0000","last_unstale":"2026-02-24T07:49:00.800882+0000","last_undegraded":"2026-02-24T07:49:00.800882+0000","last_fullsized":"2026-02-24T07:49:00.800882+0000","mapping_epoch":55,"log_start":"57'918","ondisk_log_start":"57'918","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:48:56.890768+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":918,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:50:03.263698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":60358656,"num_objects":921,"num_object_clones":0,"num_object_copies":2763,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":921,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":921,"num_write_kb":58944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.6","version":"57'839","reported_seq":852,"reported_epoch":57,"state":"active+clean","last_fresh":"2026-02-24T07:49:00.733723+0000","last_change":"2026-02-24T07:48:57.901839+0000","last_active":"2026-02-24T07:49:00.733723+0000","last_peered":"2026-02-24T07:49:00.733723+0000","last_clean":"2026-02-24T07:49:00.733723+0000","last_became_active":"2026-02-24T07:48:57.901565+0000","last_became_peered":"2026-02-24T07:48:57.901565+0000","last_unstale":"2026-02-24T07:49:00.733723+0000","last_undegraded":"2026-02-24T07:49:00.733723+0000","last_fullsized":"2026-02-24T07:49:00.733723+0000","mapping_epoch":55,"log_start":"57'836","ondisk_log_start":"57'836","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:48:56.890768+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":836,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:50:09.631523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":54984704,"num_objects":839,"num_object_clones":0,"num_object_copies":2517,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":839,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":839,"num_write_kb":53696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"57'952","reported_seq":966,"reported_epoch":57,"state":"active+clean","last_fresh":"2026-02-24T07:49:00.802856+0000","last_change":"2026-02-24T07:48:57.902165+0000","last_active":"2026-02-24T07:49:00.802856+0000","last_peered":"2026-02-24T07:49:00.802856+0000","last_clean":"2026-02-24T07:49:00.802856+0000","last_became_active":"2026-02-24T07:48:57.901824+0000","last_became_peered":"2026-02-24T07:48:57.901824+0000","last_unstale":"2026-02-24T07:49:00.802856+0000","last_undegraded":"2026-02-24T07:49:00.802856+0000","last_fullsized":"2026-02-24T07:49:00.802856+0000","mapping_epoch":55,"log_start":"57'949","ondisk_log_start":"57'949","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:48:56.890768+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":949,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:50:12.091202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":62390272,"num_objects":952,"num_object_clones":0,"num_object_copies":2856,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":952,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":952,"num_write_kb":60928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[1,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"57'867","reported_seq":881,"reported_epoch":57,"state":"active+clean","last_fresh":"2026-02-24T07:49:00.729074+0000","last_change":"2026-02-24T07:48:57.902537+0000","last_active":"2026-02-24T07:49:00.729074+0000","last_peered":"2026-02-24T07:49:00.729074+0000","last_clean":"2026-02-24T07:49:00.729074+0000","last_became_active":"2026-02-24T07:48:57.902224+0000","last_became_peered":"2026-02-24T07:48:57.902224+0000","last_unstale":"2026-02-24T07:49:00.729074+0000","last_undegraded":"2026-02-24T07:49:00.729074+0000","last_fullsized":"2026-02-24T07:49:00.729074+0000","mapping_epoch":55,"log_start":"57'864","ondisk_log_start":"57'864","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:48:56.890768+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":864,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:50:06.186992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":56819712,"num_objects":867,"num_object_clones":0,"num_object_copies":2601,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":867,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":867,"num_write_kb":55488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"57'886","reported_seq":900,"reported_epoch":57,"state":"active+clean","last_fresh":"2026-02-24T07:49:00.803140+0000","last_change":"2026-02-24T07:48:57.902384+0000","last_active":"2026-02-24T07:49:00.803140+0000","last_peered":"2026-02-24T07:49:00.803140+0000","last_clean":"2026-02-24T07:49:00.803140+0000","last_became_active":"2026-02-24T07:48:57.902125+0000","last_became_peered":"2026-02-24T07:48:57.902125+0000","last_unstale":"2026-02-24T07:49:00.803140+0000","last_undegraded":"2026-02-24T07:49:00.803140+0000","last_fullsized":"2026-02-24T07:49:00.803140+0000","mapping_epoch":55,"log_start":"57'883","ondisk_log_start":"57'883","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:48:56.890768+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":883,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:49:57.995484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":58064896,"num_objects":886,"num_object_clones":0,"num_object_copies":2658,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":886,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":886,"num_write_kb":56704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1","version":"57'873","reported_seq":886,"reported_epoch":57,"state":"active+clean","last_fresh":"2026-02-24T07:49:00.712343+0000","last_change":"2026-02-24T07:48:57.903745+0000","last_active":"2026-02-24T07:49:00.712343+0000","last_peered":"2026-02-24T07:49:00.712343+0000","last_clean":"2026-02-24T07:49:00.712343+0000","last_became_active":"2026-02-24T07:48:57.903439+0000","last_became_peered":"2026-02-24T07:48:57.903439+0000","last_unstale":"2026-02-24T07:49:00.712343+0000","last_undegraded":"2026-02-24T07:49:00.712343+0000","last_fullsized":"2026-02-24T07:49:00.712343+0000","mapping_epoch":55,"log_start":"57'870","ondisk_log_start":"57'870","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:48:56.890768+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":870,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:50:24.058110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":57212928,"num_objects":873,"num_object_clones":0,"num_object_copies":2619,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":873,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":873,"num_write_kb":55872,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.0","version":"57'893","reported_seq":906,"reported_epoch":57,"state":"active+clean","last_fresh":"2026-02-24T07:49:00.732986+0000","last_change":"2026-02-24T07:48:57.902838+0000","last_active":"2026-02-24T07:49:00.732986+0000","last_peered":"2026-02-24T07:49:00.732986+0000","last_clean":"2026-02-24T07:49:00.732986+0000","last_became_active":"2026-02-24T07:48:57.902565+0000","last_became_peered":"2026-02-24T07:48:57.902565+0000","last_unstale":"2026-02-24T07:49:00.732986+0000","last_undegraded":"2026-02-24T07:49:00.732986+0000","last_fullsized":"2026-02-24T07:49:00.732986+0000","mapping_epoch":55,"log_start":"57'890","ondisk_log_start":"57'890","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:48:56.890768+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":890,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:50:25.079534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":58523648,"num_objects":893,"num_object_clones":0,"num_object_copies":2679,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":893,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":893,"num_write_kb":57152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,3],"acting":[0,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.5","version":"57'854","reported_seq":867,"reported_epoch":57,"state":"active+clean","last_fresh":"2026-02-24T07:49:00.728201+0000","last_change":"2026-02-24T07:48:57.905643+0000","last_active":"2026-02-24T07:49:00.728201+0000","last_peered":"2026-02-24T07:49:00.728201+0000","last_clean":"2026-02-24T07:49:00.728201+0000","last_became_active":"2026-02-24T07:48:57.905127+0000","last_became_peered":"2026-02-24T07:48:57.905127+0000","last_unstale":"2026-02-24T07:49:00.728201+0000","last_undegraded":"2026-02-24T07:49:00.728201+0000","last_fullsized":"2026-02-24T07:49:00.728201+0000","mapping_epoch":55,"log_start":"57'851","ondisk_log_start":"57'851","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:48:56.890768+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":851,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:49:59.684568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":55967744,"num_objects":854,"num_object_clones":0,"num_object_copies":2562,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":854,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":854,"num_write_kb":54656,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"29'99","reported_seq":269,"reported_epoch":57,"state":"active+clean","last_fresh":"2026-02-24T07:48:58.900622+0000","last_change":"2026-02-24T07:48:02.589664+0000","last_active":"2026-02-24T07:48:58.900622+0000","last_peered":"2026-02-24T07:48:58.900622+0000","last_clean":"2026-02-24T07:48:58.900622+0000","last_became_active":"2026-02-24T07:42:54.186719+0000","last_became_peered":"2026-02-24T07:42:54.186719+0000","last_unstale":"2026-02-24T07:48:58.900622+0000","last_undegraded":"2026-02-24T07:48:58.900622+0000","last_fullsized":"2026-02-24T07:48:58.900622+0000","mapping_epoch":27,"log_start":"29'96","ondisk_log_start":"29'96","created":20,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'99","last_scrub_stamp":"2026-02-24T07:48:02.589609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:42:43.640776+0000","last_clean_scrub_stamp":"2026-02-24T07:48:02.589609+0000","objects_scrubbed":2,"log_size":3,"log_dups_size":96,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:49:04.391555+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":240,"num_write":135,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":16,"stat_sum":{"num_bytes":928710656,"num_objects":14171,"num_object_clones":0,"num_object_copies":42513,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14171,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14171,"num_write_kb":906944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2780495872,"data_stored":2780495872,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":48,"ondisk_log_size":48,"up":48,"acting":48,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":240,"num_write":135,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1320,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":3,"up_from":25,"seq":107374182476,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":1601240,"kb_used_data":559108,"kb_used_omap":16,"kb_used_meta":1042095,"kb_avail":732401960,"statfs":{"total":751619276800,"available":749979607040,"internally_reserved":0,"allocated":572526592,"data_stored":572144624,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16487,"internal_metadata":1067106201},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":19,"seq":81604378702,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":2055768,"kb_used_data":619332,"kb_used_omap":16,"kb_used_meta":1436335,"kb_avail":731947432,"statfs":{"total":751619276800,"available":749514170368,"internally_reserved":0,"allocated":634195968,"data_stored":633814000,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16953,"internal_metadata":1470807495},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[8],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542223,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":1672532,"kb_used_data":526976,"kb_used_omap":17,"kb_used_meta":1145518,"kb_avail":732330668,"statfs":{"total":751619276800,"available":749906604032,"internally_reserved":0,"allocated":539623424,"data_stored":539245008,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":17887,"internal_metadata":1173010977},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[4,2],"upper_bound":4},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705746,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":1578968,"kb_used_data":408772,"kb_used_omap":13,"kb_used_meta":1170162,"kb_avail":732424232,"statfs":{"total":751619276800,"available":750002413568,"internally_reserved":0,"allocated":418582528,"data_stored":418200560,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":14074,"internal_metadata":1198246150},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1320,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":691011584,"data_stored":691011584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":600047616,"data_stored":600047616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":796852224,"data_stored":796852224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":692584448,"data_stored":692584448,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-24T07:49:05.494 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:49:05.613 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:49:05.611+0000 7f8c5a09b640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:49:05.613 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:49:05.611+0000 7f8c5a09b640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:49:05.627 INFO:teuthology.orchestra.run.trial035.stdout:osd.0: {} 2026-02-24T07:49:05.628 INFO:teuthology.orchestra.run.trial035.stderr:osd.0: osd_enable_op_tracker = '' 2026-02-24T07:49:05.640 INFO:teuthology.orchestra.run.trial035.stdout:osd.1: {} 2026-02-24T07:49:05.640 INFO:teuthology.orchestra.run.trial035.stderr:osd.1: osd_enable_op_tracker = '' 2026-02-24T07:49:05.652 INFO:teuthology.orchestra.run.trial035.stdout:osd.2: {} 2026-02-24T07:49:05.653 INFO:teuthology.orchestra.run.trial035.stderr:osd.2: osd_enable_op_tracker = '' 2026-02-24T07:49:05.665 INFO:teuthology.orchestra.run.trial035.stdout:osd.3: {} 2026-02-24T07:49:05.665 INFO:teuthology.orchestra.run.trial035.stderr:osd.3: osd_enable_op_tracker = '' 2026-02-24T07:49:05.954 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:49:05 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/3422030108' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2026-02-24T07:49:05.994 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 7 16 57373 57357 512.079 505.688 0.00146727 0.00195156 2026-02-24T07:49:06.954 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:49:06 trial035 ceph-mon[39284]: from='client.14875 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2026-02-24T07:49:06.954 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:49:06 trial035 ceph-mon[39284]: pgmap v214: 17 pgs: 17 active+clean; 886 MiB data, 6.6 GiB used, 2.7 TiB / 2.7 TiB avail; 111 MiB/s wr, 1.77k op/s 2026-02-24T07:49:06.954 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:49:06 trial035 ceph-mon[39284]: purged_snaps scrub starts 2026-02-24T07:49:06.955 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:49:06 trial035 ceph-mon[39284]: purged_snaps scrub ok 2026-02-24T07:49:06.994 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 8 15 65971 65956 515.244 537.438 0.00265744 0.00193947 2026-02-24T07:49:07.356 INFO:journalctl@ceph.prometheus.a.trial091.stdout:Feb 24 07:49:06 trial091 ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-prometheus-a[66007]: time=2026-02-24T07:49:06.973Z level=ERROR source=scrape.go:1630 msg="Failed to determine correct type of scrape target." component="scrape manager" scrape_pool=ceph target=http://trial035:9283/metrics content_type="text/html;charset=utf-8" fallback_media_type="" err="received unsupported Content-Type \"text/html;charset=utf-8\" and no fallback_scrape_protocol specified for target" 2026-02-24T07:49:07.454 INFO:journalctl@ceph.mgr.x.trial035.stdout:Feb 24 07:49:06 trial035 ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-mgr-x[39621]: ::ffff:10.20.193.91 - - [24/Feb/2026:07:49:06] "GET /metrics HTTP/1.1" 200 4158 "" "Prometheus/3.6.0" 2026-02-24T07:49:07.709 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2026-02-24T07:49:07.850 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:49:07.918 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:49:07 trial035 ceph-mon[39284]: 1.0 scrub starts 2026-02-24T07:49:07.919 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:49:07 trial035 ceph-mon[39284]: 1.0 scrub ok 2026-02-24T07:49:07.919 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:49:07 trial035 ceph-mon[39284]: from='mgr.14284 10.20.193.35:0/3681373222' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2026-02-24T07:49:07.919 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:49:07 trial035 ceph-mon[39284]: from='mgr.14284 10.20.193.35:0/3681373222' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-24T07:49:07.919 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:49:07 trial035 ceph-mon[39284]: from='mgr.14284 10.20.193.35:0/3681373222' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2026-02-24T07:49:07.919 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:49:07 trial035 ceph-mon[39284]: from='mgr.14284 10.20.193.35:0/3681373222' entity='mgr.x' 2026-02-24T07:49:07.919 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:49:07 trial035 ceph-mon[39284]: from='mgr.14284 10.20.193.35:0/3681373222' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2026-02-24T07:49:07.979 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:49:07.977+0000 7f3118afe640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:49:07.979 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:49:07.978+0000 7f3118afe640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:49:07.994 INFO:teuthology.orchestra.run.trial035.stdout:osd.0: {} 2026-02-24T07:49:07.995 INFO:teuthology.orchestra.run.trial035.stderr:osd.0: osd_enable_op_tracker = '' 2026-02-24T07:49:07.995 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 9 15 74012 73997 513.83 502.562 0.00159283 0.0019449 2026-02-24T07:49:08.008 INFO:teuthology.orchestra.run.trial035.stdout:osd.1: {} 2026-02-24T07:49:08.009 INFO:teuthology.orchestra.run.trial035.stderr:osd.1: osd_enable_op_tracker = '' 2026-02-24T07:49:08.023 INFO:teuthology.orchestra.run.trial035.stdout:osd.2: {} 2026-02-24T07:49:08.023 INFO:teuthology.orchestra.run.trial035.stderr:osd.2: osd_enable_op_tracker = '' 2026-02-24T07:49:08.036 INFO:teuthology.orchestra.run.trial035.stdout:osd.3: {} 2026-02-24T07:49:08.036 INFO:teuthology.orchestra.run.trial035.stderr:osd.3: osd_enable_op_tracker = '' 2026-02-24T07:49:08.954 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:49:08 trial035 ceph-mon[39284]: pgmap v215: 17 pgs: 17 active+clean; 3.4 GiB data, 14 GiB used, 2.7 TiB / 2.7 TiB avail; 353 MiB/s wr, 5.64k op/s 2026-02-24T07:49:08.954 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:49:08 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/2242122084' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2026-02-24T07:49:08.994 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 10 16 82169 82153 513.418 509.75 0.00185607 0.00194646 2026-02-24T07:49:09.994 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 11 15 90207 90192 512.417 502.438 0.00171388 0.00195024 2026-02-24T07:49:10.076 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2026-02-24T07:49:10.221 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:49:10.360 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2026-02-24T07:49:10.360 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2026-02-24T07:49:10.360 INFO:tasks.thrashosds.thrasher:Growing pool unique_pool_2 2026-02-24T07:49:10.361 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph pg dump --format=json 2026-02-24T07:49:10.392 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:49:10.378+0000 7f57a21de640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:49:10.392 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:49:10.378+0000 7f57a21de640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:49:10.399 INFO:teuthology.orchestra.run.trial035.stdout:osd.0: {} 2026-02-24T07:49:10.399 INFO:teuthology.orchestra.run.trial035.stderr:osd.0: osd_enable_op_tracker = '' 2026-02-24T07:49:10.416 INFO:teuthology.orchestra.run.trial035.stdout:osd.1: {} 2026-02-24T07:49:10.417 INFO:teuthology.orchestra.run.trial035.stderr:osd.1: osd_enable_op_tracker = '' 2026-02-24T07:49:10.430 INFO:teuthology.orchestra.run.trial035.stdout:osd.2: {} 2026-02-24T07:49:10.430 INFO:teuthology.orchestra.run.trial035.stderr:osd.2: osd_enable_op_tracker = '' 2026-02-24T07:49:10.443 INFO:teuthology.orchestra.run.trial035.stdout:osd.3: {} 2026-02-24T07:49:10.444 INFO:teuthology.orchestra.run.trial035.stderr:osd.3: osd_enable_op_tracker = '' 2026-02-24T07:49:10.511 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:49:10.635 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:49:10.632+0000 7f0685b0d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:49:10.635 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:49:10.632+0000 7f0685b0d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:49:10.768 INFO:teuthology.orchestra.run.trial035.stdout: 2026-02-24T07:49:10.769 INFO:teuthology.orchestra.run.trial035.stderr:dumped all 2026-02-24T07:49:10.816 INFO:teuthology.orchestra.run.trial035.stdout:{"pg_ready":true,"pg_map":{"version":216,"stamp":"2026-02-24T07:49:09.707325+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":3627614752,"num_objects":55346,"num_object_clones":0,"num_object_copies":166038,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":55346,"num_whiteouts":0,"num_read":136,"num_read_kb":240,"num_write":55479,"num_write_kb":3544166,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":51,"ondisk_log_size":51,"up":51,"acting":51,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":51,"num_osds":4,"num_per_pool_osds":4,"num_per_pool_omap_osds":4,"kb":2936012800,"kb_used":14340700,"kb_used_data":10073932,"kb_used_omap":56,"kb_used_meta":4266567,"kb_avail":2921672100,"statfs":{"total":3006477107200,"available":2991792230400,"internally_reserved":0,"allocated":10315706368,"data_stored":10314182048,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":57844,"internal_metadata":4368965132},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[9],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":3627024384,"num_objects":55344,"num_object_clones":0,"num_object_copies":166032,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":55344,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":55344,"num_write_kb":3542016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.807388"},"pg_stats":[{"pgid":"4.f","version":"57'3510","reported_seq":3524,"reported_epoch":57,"state":"active+clean","last_fresh":"2026-02-24T07:49:05.801160+0000","last_change":"2026-02-24T07:48:57.902569+0000","last_active":"2026-02-24T07:49:05.801160+0000","last_peered":"2026-02-24T07:49:05.801160+0000","last_clean":"2026-02-24T07:49:05.801160+0000","last_became_active":"2026-02-24T07:48:57.902313+0000","last_became_peered":"2026-02-24T07:48:57.902313+0000","last_unstale":"2026-02-24T07:49:05.801160+0000","last_undegraded":"2026-02-24T07:49:05.801160+0000","last_fullsized":"2026-02-24T07:49:05.801160+0000","mapping_epoch":55,"log_start":"57'3507","ondisk_log_start":"57'3507","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:48:56.890768+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:49:59.578048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":230031360,"num_objects":3510,"num_object_clones":0,"num_object_copies":10530,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3510,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3510,"num_write_kb":224640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"57'3479","reported_seq":3493,"reported_epoch":57,"state":"active+clean","last_fresh":"2026-02-24T07:49:05.710986+0000","last_change":"2026-02-24T07:48:57.903920+0000","last_active":"2026-02-24T07:49:05.710986+0000","last_peered":"2026-02-24T07:49:05.710986+0000","last_clean":"2026-02-24T07:49:05.710986+0000","last_became_active":"2026-02-24T07:48:57.903584+0000","last_became_peered":"2026-02-24T07:48:57.903584+0000","last_unstale":"2026-02-24T07:49:05.710986+0000","last_undegraded":"2026-02-24T07:49:05.710986+0000","last_fullsized":"2026-02-24T07:49:05.710986+0000","mapping_epoch":55,"log_start":"57'3476","ondisk_log_start":"57'3476","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:48:56.890768+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:50:17.159571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":227999744,"num_objects":3479,"num_object_clones":0,"num_object_copies":10437,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3479,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3479,"num_write_kb":222656,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.d","version":"57'3429","reported_seq":3443,"reported_epoch":57,"state":"active+clean","last_fresh":"2026-02-24T07:49:05.726588+0000","last_change":"2026-02-24T07:48:57.905520+0000","last_active":"2026-02-24T07:49:05.726588+0000","last_peered":"2026-02-24T07:49:05.726588+0000","last_clean":"2026-02-24T07:49:05.726588+0000","last_became_active":"2026-02-24T07:48:57.905031+0000","last_became_peered":"2026-02-24T07:48:57.905031+0000","last_unstale":"2026-02-24T07:49:05.726588+0000","last_undegraded":"2026-02-24T07:49:05.726588+0000","last_fullsized":"2026-02-24T07:49:05.726588+0000","mapping_epoch":55,"log_start":"57'3426","ondisk_log_start":"57'3426","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:48:56.890768+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:50:02.301170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":224722944,"num_objects":3429,"num_object_clones":0,"num_object_copies":10287,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3429,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3429,"num_write_kb":219456,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"57'3457","reported_seq":3471,"reported_epoch":57,"state":"active+clean","last_fresh":"2026-02-24T07:49:05.728984+0000","last_change":"2026-02-24T07:48:57.905290+0000","last_active":"2026-02-24T07:49:05.728984+0000","last_peered":"2026-02-24T07:49:05.728984+0000","last_clean":"2026-02-24T07:49:05.728984+0000","last_became_active":"2026-02-24T07:48:57.904683+0000","last_became_peered":"2026-02-24T07:48:57.904683+0000","last_unstale":"2026-02-24T07:49:05.728984+0000","last_undegraded":"2026-02-24T07:49:05.728984+0000","last_fullsized":"2026-02-24T07:49:05.728984+0000","mapping_epoch":55,"log_start":"57'3454","ondisk_log_start":"57'3454","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:48:56.890768+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:50:16.643522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":226557952,"num_objects":3457,"num_object_clones":0,"num_object_copies":10371,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3457,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3457,"num_write_kb":221248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.b","version":"57'3473","reported_seq":3487,"reported_epoch":57,"state":"active+clean","last_fresh":"2026-02-24T07:49:05.727998+0000","last_change":"2026-02-24T07:48:57.902025+0000","last_active":"2026-02-24T07:49:05.727998+0000","last_peered":"2026-02-24T07:49:05.727998+0000","last_clean":"2026-02-24T07:49:05.727998+0000","last_became_active":"2026-02-24T07:48:57.901700+0000","last_became_peered":"2026-02-24T07:48:57.901700+0000","last_unstale":"2026-02-24T07:49:05.727998+0000","last_undegraded":"2026-02-24T07:49:05.727998+0000","last_fullsized":"2026-02-24T07:49:05.727998+0000","mapping_epoch":55,"log_start":"57'3470","ondisk_log_start":"57'3470","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:48:56.890768+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:50:23.370319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":227606528,"num_objects":3473,"num_object_clones":0,"num_object_copies":10419,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3473,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3473,"num_write_kb":222272,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.a","version":"57'3483","reported_seq":3497,"reported_epoch":57,"state":"active+clean","last_fresh":"2026-02-24T07:49:05.712173+0000","last_change":"2026-02-24T07:48:57.903698+0000","last_active":"2026-02-24T07:49:05.712173+0000","last_peered":"2026-02-24T07:49:05.712173+0000","last_clean":"2026-02-24T07:49:05.712173+0000","last_became_active":"2026-02-24T07:48:57.903228+0000","last_became_peered":"2026-02-24T07:48:57.903228+0000","last_unstale":"2026-02-24T07:49:05.712173+0000","last_undegraded":"2026-02-24T07:49:05.712173+0000","last_fullsized":"2026-02-24T07:49:05.712173+0000","mapping_epoch":55,"log_start":"57'3480","ondisk_log_start":"57'3480","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:48:56.890768+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:50:26.780637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":228261888,"num_objects":3483,"num_object_clones":0,"num_object_copies":10449,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3483,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3483,"num_write_kb":222912,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"57'3396","reported_seq":3410,"reported_epoch":57,"state":"active+clean","last_fresh":"2026-02-24T07:49:05.703915+0000","last_change":"2026-02-24T07:48:57.902011+0000","last_active":"2026-02-24T07:49:05.703915+0000","last_peered":"2026-02-24T07:49:05.703915+0000","last_clean":"2026-02-24T07:49:05.703915+0000","last_became_active":"2026-02-24T07:48:57.901730+0000","last_became_peered":"2026-02-24T07:48:57.901730+0000","last_unstale":"2026-02-24T07:49:05.703915+0000","last_undegraded":"2026-02-24T07:49:05.703915+0000","last_fullsized":"2026-02-24T07:49:05.703915+0000","mapping_epoch":55,"log_start":"57'3393","ondisk_log_start":"57'3393","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:48:56.890768+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:50:05.273318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":222560256,"num_objects":3396,"num_object_clones":0,"num_object_copies":10188,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3396,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3396,"num_write_kb":217344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"57'3413","reported_seq":3427,"reported_epoch":57,"state":"active+clean","last_fresh":"2026-02-24T07:49:05.728697+0000","last_change":"2026-02-24T07:48:57.904924+0000","last_active":"2026-02-24T07:49:05.728697+0000","last_peered":"2026-02-24T07:49:05.728697+0000","last_clean":"2026-02-24T07:49:05.728697+0000","last_became_active":"2026-02-24T07:48:57.904376+0000","last_became_peered":"2026-02-24T07:48:57.904376+0000","last_unstale":"2026-02-24T07:49:05.728697+0000","last_undegraded":"2026-02-24T07:49:05.728697+0000","last_fullsized":"2026-02-24T07:49:05.728697+0000","mapping_epoch":55,"log_start":"57'3410","ondisk_log_start":"57'3410","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:48:56.890768+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:49:57.806612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":223674368,"num_objects":3413,"num_object_clones":0,"num_object_copies":10239,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3413,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3413,"num_write_kb":218432,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.7","version":"57'3530","reported_seq":3544,"reported_epoch":57,"state":"active+clean","last_fresh":"2026-02-24T07:49:05.803366+0000","last_change":"2026-02-24T07:48:57.902698+0000","last_active":"2026-02-24T07:49:05.803366+0000","last_peered":"2026-02-24T07:49:05.803366+0000","last_clean":"2026-02-24T07:49:05.803366+0000","last_became_active":"2026-02-24T07:48:57.902484+0000","last_became_peered":"2026-02-24T07:48:57.902484+0000","last_unstale":"2026-02-24T07:49:05.803366+0000","last_undegraded":"2026-02-24T07:49:05.803366+0000","last_fullsized":"2026-02-24T07:49:05.803366+0000","mapping_epoch":55,"log_start":"57'3527","ondisk_log_start":"57'3527","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:48:56.890768+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:50:03.263698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":231342080,"num_objects":3530,"num_object_clones":0,"num_object_copies":10590,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3530,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3530,"num_write_kb":225920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.6","version":"57'3463","reported_seq":3477,"reported_epoch":57,"state":"active+clean","last_fresh":"2026-02-24T07:49:05.730461+0000","last_change":"2026-02-24T07:48:57.901839+0000","last_active":"2026-02-24T07:49:05.730461+0000","last_peered":"2026-02-24T07:49:05.730461+0000","last_clean":"2026-02-24T07:49:05.730461+0000","last_became_active":"2026-02-24T07:48:57.901565+0000","last_became_peered":"2026-02-24T07:48:57.901565+0000","last_unstale":"2026-02-24T07:49:05.730461+0000","last_undegraded":"2026-02-24T07:49:05.730461+0000","last_fullsized":"2026-02-24T07:49:05.730461+0000","mapping_epoch":55,"log_start":"57'3460","ondisk_log_start":"57'3460","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:48:56.890768+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:50:09.631523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":226951168,"num_objects":3463,"num_object_clones":0,"num_object_copies":10389,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3463,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3463,"num_write_kb":221632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"57'3585","reported_seq":3599,"reported_epoch":57,"state":"active+clean","last_fresh":"2026-02-24T07:49:05.800681+0000","last_change":"2026-02-24T07:48:57.902165+0000","last_active":"2026-02-24T07:49:05.800681+0000","last_peered":"2026-02-24T07:49:05.800681+0000","last_clean":"2026-02-24T07:49:05.800681+0000","last_became_active":"2026-02-24T07:48:57.901824+0000","last_became_peered":"2026-02-24T07:48:57.901824+0000","last_unstale":"2026-02-24T07:49:05.800681+0000","last_undegraded":"2026-02-24T07:49:05.800681+0000","last_fullsized":"2026-02-24T07:49:05.800681+0000","mapping_epoch":55,"log_start":"57'3582","ondisk_log_start":"57'3582","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:48:56.890768+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:50:12.091202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":234946560,"num_objects":3585,"num_object_clones":0,"num_object_copies":10755,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3585,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3585,"num_write_kb":229440,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[1,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"57'3377","reported_seq":3389,"reported_epoch":57,"state":"active+clean","last_fresh":"2026-02-24T07:49:05.732150+0000","last_change":"2026-02-24T07:48:57.902537+0000","last_active":"2026-02-24T07:49:05.732150+0000","last_peered":"2026-02-24T07:49:05.732150+0000","last_clean":"2026-02-24T07:49:05.732150+0000","last_became_active":"2026-02-24T07:48:57.902224+0000","last_became_peered":"2026-02-24T07:48:57.902224+0000","last_unstale":"2026-02-24T07:49:05.732150+0000","last_undegraded":"2026-02-24T07:49:05.732150+0000","last_fullsized":"2026-02-24T07:49:05.732150+0000","mapping_epoch":55,"log_start":"57'3374","ondisk_log_start":"57'3374","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:48:56.890768+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:50:06.186992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":221315072,"num_objects":3377,"num_object_clones":0,"num_object_copies":10131,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3377,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3377,"num_write_kb":216128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"57'3482","reported_seq":3496,"reported_epoch":57,"state":"active+clean","last_fresh":"2026-02-24T07:49:05.798865+0000","last_change":"2026-02-24T07:48:57.902384+0000","last_active":"2026-02-24T07:49:05.798865+0000","last_peered":"2026-02-24T07:49:05.798865+0000","last_clean":"2026-02-24T07:49:05.798865+0000","last_became_active":"2026-02-24T07:48:57.902125+0000","last_became_peered":"2026-02-24T07:48:57.902125+0000","last_unstale":"2026-02-24T07:49:05.798865+0000","last_undegraded":"2026-02-24T07:49:05.798865+0000","last_fullsized":"2026-02-24T07:49:05.798865+0000","mapping_epoch":55,"log_start":"57'3479","ondisk_log_start":"57'3479","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:48:56.890768+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:49:57.995484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":228196352,"num_objects":3482,"num_object_clones":0,"num_object_copies":10446,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3482,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3482,"num_write_kb":222848,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1","version":"57'3451","reported_seq":3465,"reported_epoch":57,"state":"active+clean","last_fresh":"2026-02-24T07:49:05.706524+0000","last_change":"2026-02-24T07:48:57.903745+0000","last_active":"2026-02-24T07:49:05.706524+0000","last_peered":"2026-02-24T07:49:05.706524+0000","last_clean":"2026-02-24T07:49:05.706524+0000","last_became_active":"2026-02-24T07:48:57.903439+0000","last_became_peered":"2026-02-24T07:48:57.903439+0000","last_unstale":"2026-02-24T07:49:05.706524+0000","last_undegraded":"2026-02-24T07:49:05.706524+0000","last_fullsized":"2026-02-24T07:49:05.706524+0000","mapping_epoch":55,"log_start":"57'3448","ondisk_log_start":"57'3448","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:48:56.890768+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:50:24.058110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":226164736,"num_objects":3451,"num_object_clones":0,"num_object_copies":10353,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3451,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3451,"num_write_kb":220864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.0","version":"57'3426","reported_seq":3440,"reported_epoch":57,"state":"active+clean","last_fresh":"2026-02-24T07:49:05.731468+0000","last_change":"2026-02-24T07:48:57.902838+0000","last_active":"2026-02-24T07:49:05.731468+0000","last_peered":"2026-02-24T07:49:05.731468+0000","last_clean":"2026-02-24T07:49:05.731468+0000","last_became_active":"2026-02-24T07:48:57.902565+0000","last_became_peered":"2026-02-24T07:48:57.902565+0000","last_unstale":"2026-02-24T07:49:05.731468+0000","last_undegraded":"2026-02-24T07:49:05.731468+0000","last_fullsized":"2026-02-24T07:49:05.731468+0000","mapping_epoch":55,"log_start":"57'3423","ondisk_log_start":"57'3423","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:48:56.890768+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:50:25.079534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":224526336,"num_objects":3426,"num_object_clones":0,"num_object_copies":10278,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3426,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3426,"num_write_kb":219264,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,3],"acting":[0,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.5","version":"57'3390","reported_seq":3404,"reported_epoch":57,"state":"active+clean","last_fresh":"2026-02-24T07:49:05.719399+0000","last_change":"2026-02-24T07:48:57.905643+0000","last_active":"2026-02-24T07:49:05.719399+0000","last_peered":"2026-02-24T07:49:05.719399+0000","last_clean":"2026-02-24T07:49:05.719399+0000","last_became_active":"2026-02-24T07:48:57.905127+0000","last_became_peered":"2026-02-24T07:48:57.905127+0000","last_unstale":"2026-02-24T07:49:05.719399+0000","last_undegraded":"2026-02-24T07:49:05.719399+0000","last_fullsized":"2026-02-24T07:49:05.719399+0000","mapping_epoch":55,"log_start":"57'3387","ondisk_log_start":"57'3387","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:48:56.890768+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:49:59.684568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":222167040,"num_objects":3390,"num_object_clones":0,"num_object_copies":10170,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3390,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3390,"num_write_kb":216960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"29'99","reported_seq":269,"reported_epoch":57,"state":"active+clean","last_fresh":"2026-02-24T07:48:58.900622+0000","last_change":"2026-02-24T07:48:02.589664+0000","last_active":"2026-02-24T07:48:58.900622+0000","last_peered":"2026-02-24T07:48:58.900622+0000","last_clean":"2026-02-24T07:48:58.900622+0000","last_became_active":"2026-02-24T07:42:54.186719+0000","last_became_peered":"2026-02-24T07:42:54.186719+0000","last_unstale":"2026-02-24T07:48:58.900622+0000","last_undegraded":"2026-02-24T07:48:58.900622+0000","last_fullsized":"2026-02-24T07:48:58.900622+0000","mapping_epoch":27,"log_start":"29'96","ondisk_log_start":"29'96","created":20,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'99","last_scrub_stamp":"2026-02-24T07:48:02.589609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:42:43.640776+0000","last_clean_scrub_stamp":"2026-02-24T07:48:02.589609+0000","objects_scrubbed":2,"log_size":3,"log_dups_size":96,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:49:04.391555+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":240,"num_write":135,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":16,"stat_sum":{"num_bytes":3627024384,"num_objects":55344,"num_object_clones":0,"num_object_copies":166032,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":55344,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":55344,"num_write_kb":3542016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":10874519552,"data_stored":10874519552,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":48,"ondisk_log_size":48,"up":48,"acting":48,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":240,"num_write":135,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1320,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":3,"up_from":25,"seq":107374182477,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":3696088,"kb_used_data":2563716,"kb_used_omap":16,"kb_used_meta":1132335,"kb_avail":730307112,"statfs":{"total":751619276800,"available":747834482688,"internally_reserved":0,"allocated":2625245184,"data_stored":2624863216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16487,"internal_metadata":1159511961},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":19,"seq":81604378703,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":3439128,"kb_used_data":2891140,"kb_used_omap":9,"kb_used_meta":547958,"kb_avail":730564072,"statfs":{"total":751619276800,"available":748097609728,"internally_reserved":0,"allocated":2960527360,"data_stored":2960145392,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9396,"internal_metadata":561109836},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[4],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542224,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":3488980,"kb_used_data":2199040,"kb_used_omap":17,"kb_used_meta":1289902,"kb_avail":730514220,"statfs":{"total":751619276800,"available":748046561280,"internally_reserved":0,"allocated":2251816960,"data_stored":2251438544,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":17887,"internal_metadata":1320860193},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705747,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":3716504,"kb_used_data":2420036,"kb_used_omap":13,"kb_used_meta":1296370,"kb_avail":730286696,"statfs":{"total":751619276800,"available":747813576704,"internally_reserved":0,"allocated":2478116864,"data_stored":2477734896,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":14074,"internal_metadata":1327483142},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[5],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1320,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":2708865024,"data_stored":2708865024,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":2299133952,"data_stored":2299133952,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":3153526784,"data_stored":3153526784,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":2712993792,"data_stored":2712993792,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-24T07:49:10.817 INFO:tasks.cephadm.ceph_manager.ceph:increase pool size by 10 2026-02-24T07:49:10.817 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph osd pool set unique_pool_2 pg_num 26 2026-02-24T07:49:10.954 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:49:10 trial035 ceph-mon[39284]: pgmap v216: 17 pgs: 17 active+clean; 3.4 GiB data, 14 GiB used, 2.7 TiB / 2.7 TiB avail; 320 MiB/s wr, 5.12k op/s 2026-02-24T07:49:10.954 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:49:10 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/3954005205' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2026-02-24T07:49:10.994 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 12 16 97945 97929 510.009 483.562 0.00151594 0.00195936 2026-02-24T07:49:11.000 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:49:11.135 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:49:11.133+0000 7efe24c73640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:49:11.136 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:49:11.133+0000 7efe24c73640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:49:11.272 INFO:teuthology.orchestra.run.trial035.stderr:Error EPERM: crimson_allow_pg_split is false; pg_num increase denied for crimson pool 2026-02-24T07:49:11.313 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-24T07:49:11.994 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 13 16 106017 106001 509.583 504.5 0.00263386 0.00196111 2026-02-24T07:49:12.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:49:11 trial035 ceph-mon[39284]: from='client.14883 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2026-02-24T07:49:12.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:49:11 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/323511403' entity='client.admin' cmd={"prefix": "osd pool set", "pool": "unique_pool_2", "var": "pg_num", "val": "26"} : dispatch 2026-02-24T07:49:12.492 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2026-02-24T07:49:12.639 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:49:12.765 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:49:12.763+0000 7fb78ae17640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:49:12.766 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:49:12.763+0000 7fb78ae17640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:49:12.780 INFO:teuthology.orchestra.run.trial035.stdout:osd.0: {} 2026-02-24T07:49:12.780 INFO:teuthology.orchestra.run.trial035.stderr:osd.0: osd_enable_op_tracker = '' 2026-02-24T07:49:12.781 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:49:12 trial035 ceph-mon[39284]: pgmap v217: 17 pgs: 17 active+clean; 5.9 GiB data, 22 GiB used, 2.7 TiB / 2.7 TiB avail; 500 MiB/s wr, 8.00k op/s 2026-02-24T07:49:12.793 INFO:teuthology.orchestra.run.trial035.stdout:osd.1: {} 2026-02-24T07:49:12.794 INFO:teuthology.orchestra.run.trial035.stderr:osd.1: osd_enable_op_tracker = '' 2026-02-24T07:49:12.806 INFO:teuthology.orchestra.run.trial035.stdout:osd.2: {} 2026-02-24T07:49:12.806 INFO:teuthology.orchestra.run.trial035.stderr:osd.2: osd_enable_op_tracker = '' 2026-02-24T07:49:12.819 INFO:teuthology.orchestra.run.trial035.stdout:osd.3: {} 2026-02-24T07:49:12.820 INFO:teuthology.orchestra.run.trial035.stderr:osd.3: osd_enable_op_tracker = '' 2026-02-24T07:49:12.994 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 14 16 113920 113904 508.462 493.938 0.00233796 0.00196545 2026-02-24T07:49:13.995 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 15 16 122040 122024 508.395 507.5 0.00111731 0.0019657 2026-02-24T07:49:14.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:49:13 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/3818833846' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2026-02-24T07:49:14.868 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2026-02-24T07:49:14.995 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 16 15 129956 129941 507.544 494.812 0.00244691 0.00196902 2026-02-24T07:49:15.008 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:49:15.034 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:49:14 trial035 ceph-mon[39284]: pgmap v218: 17 pgs: 17 active+clean; 5.9 GiB data, 22 GiB used, 2.7 TiB / 2.7 TiB avail; 426 MiB/s wr, 6.81k op/s 2026-02-24T07:49:15.128 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:49:15.125+0000 7f3fe6ddb640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:49:15.128 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:49:15.126+0000 7f3fe6ddb640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:49:15.144 INFO:teuthology.orchestra.run.trial035.stdout:osd.0: {} 2026-02-24T07:49:15.144 INFO:teuthology.orchestra.run.trial035.stderr:osd.0: osd_enable_op_tracker = '' 2026-02-24T07:49:15.159 INFO:teuthology.orchestra.run.trial035.stdout:osd.1: {} 2026-02-24T07:49:15.159 INFO:teuthology.orchestra.run.trial035.stderr:osd.1: osd_enable_op_tracker = '' 2026-02-24T07:49:15.173 INFO:teuthology.orchestra.run.trial035.stdout:osd.2: {} 2026-02-24T07:49:15.173 INFO:teuthology.orchestra.run.trial035.stderr:osd.2: osd_enable_op_tracker = '' 2026-02-24T07:49:15.185 INFO:teuthology.orchestra.run.trial035.stdout:osd.3: {} 2026-02-24T07:49:15.185 INFO:teuthology.orchestra.run.trial035.stderr:osd.3: osd_enable_op_tracker = '' 2026-02-24T07:49:15.995 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 17 15 137713 137698 506.205 484.812 0.000991734 0.00197421 2026-02-24T07:49:16.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:49:15 trial035 ceph-mon[39284]: from='mgr.14284 10.20.193.35:0/3681373222' entity='mgr.x' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2026-02-24T07:49:16.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:49:15 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/4044992189' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2026-02-24T07:49:16.315 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2026-02-24T07:49:16.315 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2026-02-24T07:49:16.315 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph pg dump --format=json 2026-02-24T07:49:16.454 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:49:16.875 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:49:16 trial035 ceph-mon[39284]: pgmap v219: 17 pgs: 17 active+clean; 5.9 GiB data, 22 GiB used, 2.7 TiB / 2.7 TiB avail; 426 MiB/s wr, 6.81k op/s 2026-02-24T07:49:16.950 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:49:16.873+0000 7ff3f6d2e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:49:16.950 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:49:16.873+0000 7ff3f6d2e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:49:16.995 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 18 16 144662 144646 502.206 434.25 0.000954465 0.00198993 2026-02-24T07:49:17.016 INFO:teuthology.orchestra.run.trial035.stdout: 2026-02-24T07:49:17.016 INFO:teuthology.orchestra.run.trial035.stderr:dumped all 2026-02-24T07:49:17.204 INFO:journalctl@ceph.mgr.x.trial035.stdout:Feb 24 07:49:16 trial035 ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-mgr-x[39621]: ::ffff:10.20.193.91 - - [24/Feb/2026:07:49:16] "GET /metrics HTTP/1.1" 200 4210 "" "Prometheus/3.6.0" 2026-02-24T07:49:17.232 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2026-02-24T07:49:17.264 INFO:journalctl@ceph.prometheus.a.trial091.stdout:Feb 24 07:49:16 trial091 ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-prometheus-a[66007]: time=2026-02-24T07:49:16.973Z level=ERROR source=scrape.go:1630 msg="Failed to determine correct type of scrape target." component="scrape manager" scrape_pool=ceph target=http://trial035:9283/metrics content_type="text/html;charset=utf-8" fallback_media_type="" err="received unsupported Content-Type \"text/html;charset=utf-8\" and no fallback_scrape_protocol specified for target" 2026-02-24T07:49:17.551 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:49:17.644 INFO:teuthology.orchestra.run.trial035.stdout:{"pg_ready":true,"pg_map":{"version":219,"stamp":"2026-02-24T07:49:15.708126+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":6289293856,"num_objects":95960,"num_object_clones":0,"num_object_copies":287880,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":95960,"num_whiteouts":0,"num_read":136,"num_read_kb":240,"num_write":96093,"num_write_kb":6143462,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":55,"ondisk_log_size":55,"up":51,"acting":51,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":51,"num_osds":4,"num_per_pool_osds":4,"num_per_pool_omap_osds":4,"kb":2936012800,"kb_used":22613596,"kb_used_data":17913740,"kb_used_omap":57,"kb_used_meta":4699718,"kb_avail":2913399204,"statfs":{"total":3006477107200,"available":2983320784896,"internally_reserved":0,"allocated":18343669760,"data_stored":18342145440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":58552,"internal_metadata":4812512072},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[4,6],"upper_bound":4},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":5359992832,"num_objects":81787,"num_object_clones":0,"num_object_copies":245361,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":81787,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":81787,"num_write_kb":5234368,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001615"},"pg_stats":[{"pgid":"4.f","version":"57'6044","reported_seq":6058,"reported_epoch":57,"state":"active+clean","last_fresh":"2026-02-24T07:49:10.804031+0000","last_change":"2026-02-24T07:48:57.902569+0000","last_active":"2026-02-24T07:49:10.804031+0000","last_peered":"2026-02-24T07:49:10.804031+0000","last_clean":"2026-02-24T07:49:10.804031+0000","last_became_active":"2026-02-24T07:48:57.902313+0000","last_became_peered":"2026-02-24T07:48:57.902313+0000","last_unstale":"2026-02-24T07:49:10.804031+0000","last_undegraded":"2026-02-24T07:49:10.804031+0000","last_fullsized":"2026-02-24T07:49:10.804031+0000","mapping_epoch":55,"log_start":"57'6041","ondisk_log_start":"57'6041","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:48:56.890768+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:49:59.578048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":396099584,"num_objects":6044,"num_object_clones":0,"num_object_copies":18132,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6044,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6044,"num_write_kb":386816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"57'6033","reported_seq":6047,"reported_epoch":57,"state":"active+clean","last_fresh":"2026-02-24T07:49:10.707984+0000","last_change":"2026-02-24T07:48:57.903920+0000","last_active":"2026-02-24T07:49:10.707984+0000","last_peered":"2026-02-24T07:49:10.707984+0000","last_clean":"2026-02-24T07:49:10.707984+0000","last_became_active":"2026-02-24T07:48:57.903584+0000","last_became_peered":"2026-02-24T07:48:57.903584+0000","last_unstale":"2026-02-24T07:49:10.707984+0000","last_undegraded":"2026-02-24T07:49:10.707984+0000","last_fullsized":"2026-02-24T07:49:10.707984+0000","mapping_epoch":55,"log_start":"57'6030","ondisk_log_start":"57'6030","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:48:56.890768+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:50:17.159571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":395378688,"num_objects":6033,"num_object_clones":0,"num_object_copies":18099,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6033,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6033,"num_write_kb":386112,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.d","version":"57'5989","reported_seq":6003,"reported_epoch":57,"state":"active+clean","last_fresh":"2026-02-24T07:49:10.727590+0000","last_change":"2026-02-24T07:48:57.905520+0000","last_active":"2026-02-24T07:49:10.727590+0000","last_peered":"2026-02-24T07:49:10.727590+0000","last_clean":"2026-02-24T07:49:10.727590+0000","last_became_active":"2026-02-24T07:48:57.905031+0000","last_became_peered":"2026-02-24T07:48:57.905031+0000","last_unstale":"2026-02-24T07:49:10.727590+0000","last_undegraded":"2026-02-24T07:49:10.727590+0000","last_fullsized":"2026-02-24T07:49:10.727590+0000","mapping_epoch":55,"log_start":"57'5986","ondisk_log_start":"57'5986","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:48:56.890768+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:50:02.301170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":392495104,"num_objects":5989,"num_object_clones":0,"num_object_copies":17967,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5989,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5989,"num_write_kb":383296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"57'5947","reported_seq":5959,"reported_epoch":57,"state":"active+clean","last_fresh":"2026-02-24T07:49:10.730162+0000","last_change":"2026-02-24T07:48:57.905290+0000","last_active":"2026-02-24T07:49:10.730162+0000","last_peered":"2026-02-24T07:49:10.730162+0000","last_clean":"2026-02-24T07:49:10.730162+0000","last_became_active":"2026-02-24T07:48:57.904683+0000","last_became_peered":"2026-02-24T07:48:57.904683+0000","last_unstale":"2026-02-24T07:49:10.730162+0000","last_undegraded":"2026-02-24T07:49:10.730162+0000","last_fullsized":"2026-02-24T07:49:10.730162+0000","mapping_epoch":55,"log_start":"57'5943","ondisk_log_start":"57'5943","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:48:56.890768+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":3000,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:50:16.643522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":389742592,"num_objects":5947,"num_object_clones":0,"num_object_copies":17841,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5947,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5947,"num_write_kb":380608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.b","version":"57'6025","reported_seq":6039,"reported_epoch":57,"state":"active+clean","last_fresh":"2026-02-24T07:49:10.729774+0000","last_change":"2026-02-24T07:48:57.902025+0000","last_active":"2026-02-24T07:49:10.729774+0000","last_peered":"2026-02-24T07:49:10.729774+0000","last_clean":"2026-02-24T07:49:10.729774+0000","last_became_active":"2026-02-24T07:48:57.901700+0000","last_became_peered":"2026-02-24T07:48:57.901700+0000","last_unstale":"2026-02-24T07:49:10.729774+0000","last_undegraded":"2026-02-24T07:49:10.729774+0000","last_fullsized":"2026-02-24T07:49:10.729774+0000","mapping_epoch":55,"log_start":"57'6022","ondisk_log_start":"57'6022","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:48:56.890768+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:50:23.370319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":394854400,"num_objects":6025,"num_object_clones":0,"num_object_copies":18075,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6025,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6025,"num_write_kb":385600,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.a","version":"57'6013","reported_seq":6027,"reported_epoch":57,"state":"active+clean","last_fresh":"2026-02-24T07:49:10.708670+0000","last_change":"2026-02-24T07:48:57.903698+0000","last_active":"2026-02-24T07:49:10.708670+0000","last_peered":"2026-02-24T07:49:10.708670+0000","last_clean":"2026-02-24T07:49:10.708670+0000","last_became_active":"2026-02-24T07:48:57.903228+0000","last_became_peered":"2026-02-24T07:48:57.903228+0000","last_unstale":"2026-02-24T07:49:10.708670+0000","last_undegraded":"2026-02-24T07:49:10.708670+0000","last_fullsized":"2026-02-24T07:49:10.708670+0000","mapping_epoch":55,"log_start":"57'6010","ondisk_log_start":"57'6010","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:48:56.890768+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:50:26.780637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":394067968,"num_objects":6013,"num_object_clones":0,"num_object_copies":18039,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6013,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6013,"num_write_kb":384832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"57'5962","reported_seq":5976,"reported_epoch":57,"state":"active+clean","last_fresh":"2026-02-24T07:49:10.709766+0000","last_change":"2026-02-24T07:48:57.902011+0000","last_active":"2026-02-24T07:49:10.709766+0000","last_peered":"2026-02-24T07:49:10.709766+0000","last_clean":"2026-02-24T07:49:10.709766+0000","last_became_active":"2026-02-24T07:48:57.901730+0000","last_became_peered":"2026-02-24T07:48:57.901730+0000","last_unstale":"2026-02-24T07:49:10.709766+0000","last_undegraded":"2026-02-24T07:49:10.709766+0000","last_fullsized":"2026-02-24T07:49:10.709766+0000","mapping_epoch":55,"log_start":"57'5959","ondisk_log_start":"57'5959","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:48:56.890768+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:50:05.273318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":390725632,"num_objects":5962,"num_object_clones":0,"num_object_copies":17886,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5962,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5962,"num_write_kb":381568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"57'6014","reported_seq":6028,"reported_epoch":57,"state":"active+clean","last_fresh":"2026-02-24T07:49:10.724201+0000","last_change":"2026-02-24T07:48:57.904924+0000","last_active":"2026-02-24T07:49:10.724201+0000","last_peered":"2026-02-24T07:49:10.724201+0000","last_clean":"2026-02-24T07:49:10.724201+0000","last_became_active":"2026-02-24T07:48:57.904376+0000","last_became_peered":"2026-02-24T07:48:57.904376+0000","last_unstale":"2026-02-24T07:49:10.724201+0000","last_undegraded":"2026-02-24T07:49:10.724201+0000","last_fullsized":"2026-02-24T07:49:10.724201+0000","mapping_epoch":55,"log_start":"57'6009","ondisk_log_start":"57'6009","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:48:56.890768+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":3000,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:49:57.806612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":394133504,"num_objects":6014,"num_object_clones":0,"num_object_copies":18042,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6014,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6014,"num_write_kb":384896,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.7","version":"57'6114","reported_seq":6128,"reported_epoch":57,"state":"active+clean","last_fresh":"2026-02-24T07:49:10.801724+0000","last_change":"2026-02-24T07:48:57.902698+0000","last_active":"2026-02-24T07:49:10.801724+0000","last_peered":"2026-02-24T07:49:10.801724+0000","last_clean":"2026-02-24T07:49:10.801724+0000","last_became_active":"2026-02-24T07:48:57.902484+0000","last_became_peered":"2026-02-24T07:48:57.902484+0000","last_unstale":"2026-02-24T07:49:10.801724+0000","last_undegraded":"2026-02-24T07:49:10.801724+0000","last_fullsized":"2026-02-24T07:49:10.801724+0000","mapping_epoch":55,"log_start":"57'6111","ondisk_log_start":"57'6111","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:48:56.890768+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:50:03.263698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":400687104,"num_objects":6114,"num_object_clones":0,"num_object_copies":18342,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6114,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6114,"num_write_kb":391296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.6","version":"57'5943","reported_seq":5957,"reported_epoch":57,"state":"active+clean","last_fresh":"2026-02-24T07:49:10.733279+0000","last_change":"2026-02-24T07:48:57.901839+0000","last_active":"2026-02-24T07:49:10.733279+0000","last_peered":"2026-02-24T07:49:10.733279+0000","last_clean":"2026-02-24T07:49:10.733279+0000","last_became_active":"2026-02-24T07:48:57.901565+0000","last_became_peered":"2026-02-24T07:48:57.901565+0000","last_unstale":"2026-02-24T07:49:10.733279+0000","last_undegraded":"2026-02-24T07:49:10.733279+0000","last_fullsized":"2026-02-24T07:49:10.733279+0000","mapping_epoch":55,"log_start":"57'5940","ondisk_log_start":"57'5940","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:48:56.890768+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:50:09.631523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":389480448,"num_objects":5943,"num_object_clones":0,"num_object_copies":17829,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5943,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5943,"num_write_kb":380352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"57'6155","reported_seq":6169,"reported_epoch":57,"state":"active+clean","last_fresh":"2026-02-24T07:49:10.805955+0000","last_change":"2026-02-24T07:48:57.902165+0000","last_active":"2026-02-24T07:49:10.805955+0000","last_peered":"2026-02-24T07:49:10.805955+0000","last_clean":"2026-02-24T07:49:10.805955+0000","last_became_active":"2026-02-24T07:48:57.901824+0000","last_became_peered":"2026-02-24T07:48:57.901824+0000","last_unstale":"2026-02-24T07:49:10.805955+0000","last_undegraded":"2026-02-24T07:49:10.805955+0000","last_fullsized":"2026-02-24T07:49:10.805955+0000","mapping_epoch":55,"log_start":"57'6152","ondisk_log_start":"57'6152","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:48:56.890768+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:50:12.091202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":403374080,"num_objects":6155,"num_object_clones":0,"num_object_copies":18465,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6155,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6155,"num_write_kb":393920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[1,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"57'5891","reported_seq":5904,"reported_epoch":57,"state":"active+clean","last_fresh":"2026-02-24T07:49:10.732222+0000","last_change":"2026-02-24T07:48:57.902537+0000","last_active":"2026-02-24T07:49:10.732222+0000","last_peered":"2026-02-24T07:49:10.732222+0000","last_clean":"2026-02-24T07:49:10.732222+0000","last_became_active":"2026-02-24T07:48:57.902224+0000","last_became_peered":"2026-02-24T07:48:57.902224+0000","last_unstale":"2026-02-24T07:49:10.732222+0000","last_undegraded":"2026-02-24T07:49:10.732222+0000","last_fullsized":"2026-02-24T07:49:10.732222+0000","mapping_epoch":55,"log_start":"57'5888","ondisk_log_start":"57'5888","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:48:56.890768+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:50:06.186992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":386072576,"num_objects":5891,"num_object_clones":0,"num_object_copies":17673,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5891,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5891,"num_write_kb":377024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"57'6071","reported_seq":6085,"reported_epoch":57,"state":"active+clean","last_fresh":"2026-02-24T07:49:10.803228+0000","last_change":"2026-02-24T07:48:57.902384+0000","last_active":"2026-02-24T07:49:10.803228+0000","last_peered":"2026-02-24T07:49:10.803228+0000","last_clean":"2026-02-24T07:49:10.803228+0000","last_became_active":"2026-02-24T07:48:57.902125+0000","last_became_peered":"2026-02-24T07:48:57.902125+0000","last_unstale":"2026-02-24T07:49:10.803228+0000","last_undegraded":"2026-02-24T07:49:10.803228+0000","last_fullsized":"2026-02-24T07:49:10.803228+0000","mapping_epoch":55,"log_start":"57'6068","ondisk_log_start":"57'6068","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:48:56.890768+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:49:57.995484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":397869056,"num_objects":6071,"num_object_clones":0,"num_object_copies":18213,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6071,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6071,"num_write_kb":388544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1","version":"57'6007","reported_seq":6021,"reported_epoch":57,"state":"active+clean","last_fresh":"2026-02-24T07:49:10.708028+0000","last_change":"2026-02-24T07:48:57.903745+0000","last_active":"2026-02-24T07:49:10.708028+0000","last_peered":"2026-02-24T07:49:10.708028+0000","last_clean":"2026-02-24T07:49:10.708028+0000","last_became_active":"2026-02-24T07:48:57.903439+0000","last_became_peered":"2026-02-24T07:48:57.903439+0000","last_unstale":"2026-02-24T07:49:10.708028+0000","last_undegraded":"2026-02-24T07:49:10.708028+0000","last_fullsized":"2026-02-24T07:49:10.708028+0000","mapping_epoch":55,"log_start":"57'6004","ondisk_log_start":"57'6004","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:48:56.890768+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:50:24.058110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393674752,"num_objects":6007,"num_object_clones":0,"num_object_copies":18021,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6007,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6007,"num_write_kb":384448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.0","version":"57'5926","reported_seq":5940,"reported_epoch":57,"state":"active+clean","last_fresh":"2026-02-24T07:49:10.733400+0000","last_change":"2026-02-24T07:48:57.902838+0000","last_active":"2026-02-24T07:49:10.733400+0000","last_peered":"2026-02-24T07:49:10.733400+0000","last_clean":"2026-02-24T07:49:10.733400+0000","last_became_active":"2026-02-24T07:48:57.902565+0000","last_became_peered":"2026-02-24T07:48:57.902565+0000","last_unstale":"2026-02-24T07:49:10.733400+0000","last_undegraded":"2026-02-24T07:49:10.733400+0000","last_fullsized":"2026-02-24T07:49:10.733400+0000","mapping_epoch":55,"log_start":"57'5923","ondisk_log_start":"57'5923","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:48:56.890768+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:50:25.079534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":388366336,"num_objects":5926,"num_object_clones":0,"num_object_copies":17778,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5926,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5926,"num_write_kb":379264,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,3],"acting":[0,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.5","version":"57'5824","reported_seq":5838,"reported_epoch":57,"state":"active+clean","last_fresh":"2026-02-24T07:49:10.726714+0000","last_change":"2026-02-24T07:48:57.905643+0000","last_active":"2026-02-24T07:49:10.726714+0000","last_peered":"2026-02-24T07:49:10.726714+0000","last_clean":"2026-02-24T07:49:10.726714+0000","last_became_active":"2026-02-24T07:48:57.905127+0000","last_became_peered":"2026-02-24T07:48:57.905127+0000","last_unstale":"2026-02-24T07:49:10.726714+0000","last_undegraded":"2026-02-24T07:49:10.726714+0000","last_fullsized":"2026-02-24T07:49:10.726714+0000","mapping_epoch":55,"log_start":"57'5820","ondisk_log_start":"57'5820","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:48:56.890768+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":3000,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:49:59.684568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":381681664,"num_objects":5824,"num_object_clones":0,"num_object_copies":17472,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5824,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5824,"num_write_kb":372736,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"29'99","reported_seq":277,"reported_epoch":57,"state":"active+clean","last_fresh":"2026-02-24T07:49:06.478572+0000","last_change":"2026-02-24T07:49:06.478572+0000","last_active":"2026-02-24T07:49:06.478572+0000","last_peered":"2026-02-24T07:49:06.478572+0000","last_clean":"2026-02-24T07:49:06.478572+0000","last_became_active":"2026-02-24T07:42:54.186719+0000","last_became_peered":"2026-02-24T07:42:54.186719+0000","last_unstale":"2026-02-24T07:49:06.478572+0000","last_undegraded":"2026-02-24T07:49:06.478572+0000","last_fullsized":"2026-02-24T07:49:06.478572+0000","mapping_epoch":27,"log_start":"29'96","ondisk_log_start":"29'96","created":20,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'99","last_scrub_stamp":"2026-02-24T07:49:06.478534+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:42:43.640776+0000","last_clean_scrub_stamp":"2026-02-24T07:49:06.478534+0000","objects_scrubbed":2,"log_size":3,"log_dups_size":96,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:50:26.893586+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":240,"num_write":135,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":16,"stat_sum":{"num_bytes":6288703488,"num_objects":95958,"num_object_clones":0,"num_object_copies":287874,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":95958,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":95958,"num_write_kb":6141312,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":18860212224,"data_stored":18860212224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":52,"ondisk_log_size":52,"up":48,"acting":48,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":240,"num_write":135,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1236,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":3,"up_from":25,"seq":107374182478,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":5725720,"kb_used_data":4485060,"kb_used_omap":16,"kb_used_meta":1240623,"kb_avail":728277480,"statfs":{"total":751619276800,"available":745756139520,"internally_reserved":0,"allocated":4592701440,"data_stored":4592319472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16538,"internal_metadata":1270398822},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":19,"seq":81604378704,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":5894616,"kb_used_data":5202244,"kb_used_omap":9,"kb_used_meta":692342,"kb_avail":728108584,"statfs":{"total":751619276800,"available":745583190016,"internally_reserved":0,"allocated":5327097856,"data_stored":5326715888,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9396,"internal_metadata":708959052},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2],"upper_bound":4},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542225,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":5180052,"kb_used_data":3817920,"kb_used_omap":18,"kb_used_meta":1362093,"kb_avail":728823148,"statfs":{"total":751619276800,"available":746314903552,"internally_reserved":0,"allocated":3909550080,"data_stored":3909171664,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":18544,"internal_metadata":1394784144},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705748,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":5813208,"kb_used_data":4408516,"kb_used_omap":13,"kb_used_meta":1404658,"kb_avail":728189992,"statfs":{"total":751619276800,"available":745666551808,"internally_reserved":0,"allocated":4514320384,"data_stored":4513938416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":14074,"internal_metadata":1438370054},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,4],"upper_bound":4},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1236,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4707713024,"data_stored":4707713024,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":3968991232,"data_stored":3968991232,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":5477367808,"data_stored":5477367808,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4706140160,"data_stored":4706140160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-24T07:49:17.995 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 19 16 149858 149842 492.865 324.75 0.00171485 0.00202772 2026-02-24T07:49:18.075 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:49:18.009+0000 7f90bba09640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:49:18.075 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:49:18.009+0000 7f90bba09640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:49:18.076 INFO:teuthology.orchestra.run.trial035.stderr:osd.0: osd_enable_op_tracker = '' 2026-02-24T07:49:18.076 INFO:teuthology.orchestra.run.trial035.stderr:osd.1: osd_enable_op_tracker = '' 2026-02-24T07:49:18.076 INFO:teuthology.orchestra.run.trial035.stdout:osd.0: {} 2026-02-24T07:49:18.076 INFO:teuthology.orchestra.run.trial035.stdout:osd.1: {} 2026-02-24T07:49:18.076 INFO:teuthology.orchestra.run.trial035.stdout:osd.2: {} 2026-02-24T07:49:18.077 INFO:teuthology.orchestra.run.trial035.stderr:osd.2: osd_enable_op_tracker = '' 2026-02-24T07:49:18.088 INFO:teuthology.orchestra.run.trial035.stdout:osd.3: {} 2026-02-24T07:49:18.088 INFO:teuthology.orchestra.run.trial035.stderr:osd.3: osd_enable_op_tracker = '' 2026-02-24T07:49:18.995 INFO:tasks.radosbench.radosbench.0.trial035.stdout:2026-02-24T07:49:18.994806+0000 min lat: 0.000682058 max lat: 0.0391224 avg lat: 0.00205295 2026-02-24T07:49:18.995 INFO:tasks.radosbench.radosbench.0.trial035.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2026-02-24T07:49:18.995 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 20 16 155800 155784 486.789 371.375 0.00137644 0.00205295 2026-02-24T07:49:19.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:49:18 trial035 ceph-mon[39284]: from='client.14891 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2026-02-24T07:49:19.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:49:18 trial035 ceph-mon[39284]: pgmap v220: 17 pgs: 17 active+clean; 8.3 GiB data, 29 GiB used, 2.7 TiB / 2.7 TiB avail; 633 MiB/s wr, 10.13k op/s 2026-02-24T07:49:19.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:49:18 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/4174468971' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2026-02-24T07:49:19.995 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 21 16 163801 163785 487.418 500.062 0.002073 0.00205038 2026-02-24T07:49:20.742 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2026-02-24T07:49:20.886 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:49:20.995 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 22 16 171463 171447 487.028 478.875 0.0110751 0.00205087 2026-02-24T07:49:21.056 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:49:20 trial035 ceph-mon[39284]: pgmap v221: 17 pgs: 17 active+clean; 8.3 GiB data, 29 GiB used, 2.7 TiB / 2.7 TiB avail; 419 MiB/s wr, 6.70k op/s 2026-02-24T07:49:21.351 INFO:teuthology.orchestra.run.trial035.stdout:osd.0: {} 2026-02-24T07:49:21.351 INFO:teuthology.orchestra.run.trial035.stdout:osd.1: {} 2026-02-24T07:49:21.351 INFO:teuthology.orchestra.run.trial035.stdout:osd.2: {} 2026-02-24T07:49:21.351 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:49:21.286+0000 7f854b2d0640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:49:21.351 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:49:21.286+0000 7f854b2d0640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:49:21.351 INFO:teuthology.orchestra.run.trial035.stderr:osd.0: osd_enable_op_tracker = '' 2026-02-24T07:49:21.351 INFO:teuthology.orchestra.run.trial035.stderr:osd.1: osd_enable_op_tracker = '' 2026-02-24T07:49:21.351 INFO:teuthology.orchestra.run.trial035.stderr:osd.2: osd_enable_op_tracker = '' 2026-02-24T07:49:21.362 INFO:teuthology.orchestra.run.trial035.stdout:osd.3: {} 2026-02-24T07:49:21.362 INFO:teuthology.orchestra.run.trial035.stderr:osd.3: osd_enable_op_tracker = '' 2026-02-24T07:49:21.954 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:49:21 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/1510709807' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2026-02-24T07:49:21.995 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 23 15 176969 176954 480.817 344.188 0.00224581 0.00207857 2026-02-24T07:49:22.648 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2026-02-24T07:49:22.648 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2026-02-24T07:49:22.648 INFO:tasks.thrashosds.thrasher:Growing pool .mgr 2026-02-24T07:49:22.648 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph pg dump --format=json 2026-02-24T07:49:22.790 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:49:22.922 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:49:22.920+0000 7f5ff23b8640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:49:22.922 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:49:22.920+0000 7f5ff23b8640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:49:22.954 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:49:22 trial035 ceph-mon[39284]: pgmap v222: 17 pgs: 17 active+clean; 10 GiB data, 35 GiB used, 2.7 TiB / 2.7 TiB avail; 596 MiB/s wr, 9.53k op/s 2026-02-24T07:49:22.995 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 24 15 184024 184009 479.154 440.938 0.00229927 0.00208578 2026-02-24T07:49:23.075 INFO:teuthology.orchestra.run.trial035.stdout: 2026-02-24T07:49:23.076 INFO:teuthology.orchestra.run.trial035.stderr:dumped all 2026-02-24T07:49:23.121 INFO:teuthology.orchestra.run.trial035.stdout:{"pg_ready":true,"pg_map":{"version":222,"stamp":"2026-02-24T07:49:21.708916+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":11122573856,"num_objects":169710,"num_object_clones":0,"num_object_copies":509130,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":169710,"num_whiteouts":0,"num_read":136,"num_read_kb":240,"num_write":169843,"num_write_kb":10863462,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":51,"ondisk_log_size":51,"up":51,"acting":51,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":51,"num_osds":4,"num_per_pool_osds":4,"num_per_pool_omap_osds":4,"kb":2936012800,"kb_used":36753820,"kb_used_data":32103052,"kb_used_omap":51,"kb_used_meta":4650636,"kb_avail":2899258980,"statfs":{"total":3006477107200,"available":2968841195520,"internally_reserved":0,"allocated":32873525248,"data_stored":32872000928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":52404,"internal_metadata":4762252108},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[5,1],"upper_bound":4},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":7494959104,"num_objects":114364,"num_object_clones":0,"num_object_copies":343092,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":114364,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":114364,"num_write_kb":7319296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001590"},"pg_stats":[{"pgid":"4.f","version":"57'10616","reported_seq":10629,"reported_epoch":57,"state":"active+clean","last_fresh":"2026-02-24T07:49:20.804995+0000","last_change":"2026-02-24T07:48:57.902569+0000","last_active":"2026-02-24T07:49:20.804995+0000","last_peered":"2026-02-24T07:49:20.804995+0000","last_clean":"2026-02-24T07:49:20.804995+0000","last_became_active":"2026-02-24T07:48:57.902313+0000","last_became_peered":"2026-02-24T07:48:57.902313+0000","last_unstale":"2026-02-24T07:49:20.804995+0000","last_undegraded":"2026-02-24T07:49:20.804995+0000","last_fullsized":"2026-02-24T07:49:20.804995+0000","mapping_epoch":55,"log_start":"57'10613","ondisk_log_start":"57'10613","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:48:56.890768+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:49:59.578048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":695730176,"num_objects":10616,"num_object_clones":0,"num_object_copies":31848,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10616,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10616,"num_write_kb":679424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"57'10649","reported_seq":10661,"reported_epoch":57,"state":"active+clean","last_fresh":"2026-02-24T07:49:20.713599+0000","last_change":"2026-02-24T07:48:57.903920+0000","last_active":"2026-02-24T07:49:20.713599+0000","last_peered":"2026-02-24T07:49:20.713599+0000","last_clean":"2026-02-24T07:49:20.713599+0000","last_became_active":"2026-02-24T07:48:57.903584+0000","last_became_peered":"2026-02-24T07:48:57.903584+0000","last_unstale":"2026-02-24T07:49:20.713599+0000","last_undegraded":"2026-02-24T07:49:20.713599+0000","last_fullsized":"2026-02-24T07:49:20.713599+0000","mapping_epoch":55,"log_start":"57'10646","ondisk_log_start":"57'10646","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:48:56.890768+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:50:17.159571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":697892864,"num_objects":10649,"num_object_clones":0,"num_object_copies":31947,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10649,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10649,"num_write_kb":681536,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.d","version":"57'10564","reported_seq":10578,"reported_epoch":57,"state":"active+clean","last_fresh":"2026-02-24T07:49:20.727416+0000","last_change":"2026-02-24T07:48:57.905520+0000","last_active":"2026-02-24T07:49:20.727416+0000","last_peered":"2026-02-24T07:49:20.727416+0000","last_clean":"2026-02-24T07:49:20.727416+0000","last_became_active":"2026-02-24T07:48:57.905031+0000","last_became_peered":"2026-02-24T07:48:57.905031+0000","last_unstale":"2026-02-24T07:49:20.727416+0000","last_undegraded":"2026-02-24T07:49:20.727416+0000","last_fullsized":"2026-02-24T07:49:20.727416+0000","mapping_epoch":55,"log_start":"57'10561","ondisk_log_start":"57'10561","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:48:56.890768+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:50:02.301170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":692322304,"num_objects":10564,"num_object_clones":0,"num_object_copies":31692,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10564,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10564,"num_write_kb":676096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"57'10532","reported_seq":10546,"reported_epoch":57,"state":"active+clean","last_fresh":"2026-02-24T07:49:20.728495+0000","last_change":"2026-02-24T07:48:57.905290+0000","last_active":"2026-02-24T07:49:20.728495+0000","last_peered":"2026-02-24T07:49:20.728495+0000","last_clean":"2026-02-24T07:49:20.728495+0000","last_became_active":"2026-02-24T07:48:57.904683+0000","last_became_peered":"2026-02-24T07:48:57.904683+0000","last_unstale":"2026-02-24T07:49:20.728495+0000","last_undegraded":"2026-02-24T07:49:20.728495+0000","last_fullsized":"2026-02-24T07:49:20.728495+0000","mapping_epoch":55,"log_start":"57'10529","ondisk_log_start":"57'10529","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:48:56.890768+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:50:16.643522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":690225152,"num_objects":10532,"num_object_clones":0,"num_object_copies":31596,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10532,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10532,"num_write_kb":674048,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.b","version":"57'10658","reported_seq":10672,"reported_epoch":57,"state":"active+clean","last_fresh":"2026-02-24T07:49:20.729763+0000","last_change":"2026-02-24T07:48:57.902025+0000","last_active":"2026-02-24T07:49:20.729763+0000","last_peered":"2026-02-24T07:49:20.729763+0000","last_clean":"2026-02-24T07:49:20.729763+0000","last_became_active":"2026-02-24T07:48:57.901700+0000","last_became_peered":"2026-02-24T07:48:57.901700+0000","last_unstale":"2026-02-24T07:49:20.729763+0000","last_undegraded":"2026-02-24T07:49:20.729763+0000","last_fullsized":"2026-02-24T07:49:20.729763+0000","mapping_epoch":55,"log_start":"57'10655","ondisk_log_start":"57'10655","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:48:56.890768+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:50:23.370319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":698482688,"num_objects":10658,"num_object_clones":0,"num_object_copies":31974,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10658,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10658,"num_write_kb":682112,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.a","version":"57'10665","reported_seq":10679,"reported_epoch":57,"state":"active+clean","last_fresh":"2026-02-24T07:49:20.706488+0000","last_change":"2026-02-24T07:48:57.903698+0000","last_active":"2026-02-24T07:49:20.706488+0000","last_peered":"2026-02-24T07:49:20.706488+0000","last_clean":"2026-02-24T07:49:20.706488+0000","last_became_active":"2026-02-24T07:48:57.903228+0000","last_became_peered":"2026-02-24T07:48:57.903228+0000","last_unstale":"2026-02-24T07:49:20.706488+0000","last_undegraded":"2026-02-24T07:49:20.706488+0000","last_fullsized":"2026-02-24T07:49:20.706488+0000","mapping_epoch":55,"log_start":"57'10662","ondisk_log_start":"57'10662","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:48:56.890768+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:50:26.780637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":698941440,"num_objects":10665,"num_object_clones":0,"num_object_copies":31995,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10665,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10665,"num_write_kb":682560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"57'10592","reported_seq":10606,"reported_epoch":57,"state":"active+clean","last_fresh":"2026-02-24T07:49:20.706065+0000","last_change":"2026-02-24T07:48:57.902011+0000","last_active":"2026-02-24T07:49:20.706065+0000","last_peered":"2026-02-24T07:49:20.706065+0000","last_clean":"2026-02-24T07:49:20.706065+0000","last_became_active":"2026-02-24T07:48:57.901730+0000","last_became_peered":"2026-02-24T07:48:57.901730+0000","last_unstale":"2026-02-24T07:49:20.706065+0000","last_undegraded":"2026-02-24T07:49:20.706065+0000","last_fullsized":"2026-02-24T07:49:20.706065+0000","mapping_epoch":55,"log_start":"57'10589","ondisk_log_start":"57'10589","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:48:56.890768+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:50:05.273318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":694157312,"num_objects":10592,"num_object_clones":0,"num_object_copies":31776,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10592,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10592,"num_write_kb":677888,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"57'10522","reported_seq":10536,"reported_epoch":57,"state":"active+clean","last_fresh":"2026-02-24T07:49:20.726423+0000","last_change":"2026-02-24T07:48:57.904924+0000","last_active":"2026-02-24T07:49:20.726423+0000","last_peered":"2026-02-24T07:49:20.726423+0000","last_clean":"2026-02-24T07:49:20.726423+0000","last_became_active":"2026-02-24T07:48:57.904376+0000","last_became_peered":"2026-02-24T07:48:57.904376+0000","last_unstale":"2026-02-24T07:49:20.726423+0000","last_undegraded":"2026-02-24T07:49:20.726423+0000","last_fullsized":"2026-02-24T07:49:20.726423+0000","mapping_epoch":55,"log_start":"57'10519","ondisk_log_start":"57'10519","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:48:56.890768+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:49:57.806612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":689569792,"num_objects":10522,"num_object_clones":0,"num_object_copies":31566,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10522,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10522,"num_write_kb":673408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.7","version":"57'10772","reported_seq":10785,"reported_epoch":57,"state":"active+clean","last_fresh":"2026-02-24T07:49:20.807524+0000","last_change":"2026-02-24T07:48:57.902698+0000","last_active":"2026-02-24T07:49:20.807524+0000","last_peered":"2026-02-24T07:49:20.807524+0000","last_clean":"2026-02-24T07:49:20.807524+0000","last_became_active":"2026-02-24T07:48:57.902484+0000","last_became_peered":"2026-02-24T07:48:57.902484+0000","last_unstale":"2026-02-24T07:49:20.807524+0000","last_undegraded":"2026-02-24T07:49:20.807524+0000","last_fullsized":"2026-02-24T07:49:20.807524+0000","mapping_epoch":55,"log_start":"57'10769","ondisk_log_start":"57'10769","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:48:56.890768+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:50:03.263698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":705953792,"num_objects":10772,"num_object_clones":0,"num_object_copies":32316,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10772,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10772,"num_write_kb":689408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.6","version":"57'10449","reported_seq":10463,"reported_epoch":57,"state":"active+clean","last_fresh":"2026-02-24T07:49:20.733978+0000","last_change":"2026-02-24T07:48:57.901839+0000","last_active":"2026-02-24T07:49:20.733978+0000","last_peered":"2026-02-24T07:49:20.733978+0000","last_clean":"2026-02-24T07:49:20.733978+0000","last_became_active":"2026-02-24T07:48:57.901565+0000","last_became_peered":"2026-02-24T07:48:57.901565+0000","last_unstale":"2026-02-24T07:49:20.733978+0000","last_undegraded":"2026-02-24T07:49:20.733978+0000","last_fullsized":"2026-02-24T07:49:20.733978+0000","mapping_epoch":55,"log_start":"57'10446","ondisk_log_start":"57'10446","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:48:56.890768+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:50:09.631523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":684785664,"num_objects":10449,"num_object_clones":0,"num_object_copies":31347,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10449,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10449,"num_write_kb":668736,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"57'10808","reported_seq":10821,"reported_epoch":57,"state":"active+clean","last_fresh":"2026-02-24T07:49:20.806474+0000","last_change":"2026-02-24T07:48:57.902165+0000","last_active":"2026-02-24T07:49:20.806474+0000","last_peered":"2026-02-24T07:49:20.806474+0000","last_clean":"2026-02-24T07:49:20.806474+0000","last_became_active":"2026-02-24T07:48:57.901824+0000","last_became_peered":"2026-02-24T07:48:57.901824+0000","last_unstale":"2026-02-24T07:49:20.806474+0000","last_undegraded":"2026-02-24T07:49:20.806474+0000","last_fullsized":"2026-02-24T07:49:20.806474+0000","mapping_epoch":55,"log_start":"57'10805","ondisk_log_start":"57'10805","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:48:56.890768+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:50:12.091202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":708313088,"num_objects":10808,"num_object_clones":0,"num_object_copies":32424,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10808,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10808,"num_write_kb":691712,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[1,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"57'10608","reported_seq":10622,"reported_epoch":57,"state":"active+clean","last_fresh":"2026-02-24T07:49:20.726250+0000","last_change":"2026-02-24T07:48:57.902537+0000","last_active":"2026-02-24T07:49:20.726250+0000","last_peered":"2026-02-24T07:49:20.726250+0000","last_clean":"2026-02-24T07:49:20.726250+0000","last_became_active":"2026-02-24T07:48:57.902224+0000","last_became_peered":"2026-02-24T07:48:57.902224+0000","last_unstale":"2026-02-24T07:49:20.726250+0000","last_undegraded":"2026-02-24T07:49:20.726250+0000","last_fullsized":"2026-02-24T07:49:20.726250+0000","mapping_epoch":55,"log_start":"57'10605","ondisk_log_start":"57'10605","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:48:56.890768+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:50:06.186992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":695205888,"num_objects":10608,"num_object_clones":0,"num_object_copies":31824,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10608,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10608,"num_write_kb":678912,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"57'10650","reported_seq":10664,"reported_epoch":57,"state":"active+clean","last_fresh":"2026-02-24T07:49:20.805525+0000","last_change":"2026-02-24T07:48:57.902384+0000","last_active":"2026-02-24T07:49:20.805525+0000","last_peered":"2026-02-24T07:49:20.805525+0000","last_clean":"2026-02-24T07:49:20.805525+0000","last_became_active":"2026-02-24T07:48:57.902125+0000","last_became_peered":"2026-02-24T07:48:57.902125+0000","last_unstale":"2026-02-24T07:49:20.805525+0000","last_undegraded":"2026-02-24T07:49:20.805525+0000","last_fullsized":"2026-02-24T07:49:20.805525+0000","mapping_epoch":55,"log_start":"57'10647","ondisk_log_start":"57'10647","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:48:56.890768+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:49:57.995484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":697958400,"num_objects":10650,"num_object_clones":0,"num_object_copies":31950,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10650,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10650,"num_write_kb":681600,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1","version":"57'10478","reported_seq":10492,"reported_epoch":57,"state":"active+clean","last_fresh":"2026-02-24T07:49:20.711917+0000","last_change":"2026-02-24T07:48:57.903745+0000","last_active":"2026-02-24T07:49:20.711917+0000","last_peered":"2026-02-24T07:49:20.711917+0000","last_clean":"2026-02-24T07:49:20.711917+0000","last_became_active":"2026-02-24T07:48:57.903439+0000","last_became_peered":"2026-02-24T07:48:57.903439+0000","last_unstale":"2026-02-24T07:49:20.711917+0000","last_undegraded":"2026-02-24T07:49:20.711917+0000","last_fullsized":"2026-02-24T07:49:20.711917+0000","mapping_epoch":55,"log_start":"57'10475","ondisk_log_start":"57'10475","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:48:56.890768+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:50:24.058110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":686686208,"num_objects":10478,"num_object_clones":0,"num_object_copies":31434,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10478,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10478,"num_write_kb":670592,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.0","version":"57'10617","reported_seq":10631,"reported_epoch":57,"state":"active+clean","last_fresh":"2026-02-24T07:49:20.732327+0000","last_change":"2026-02-24T07:48:57.902838+0000","last_active":"2026-02-24T07:49:20.732327+0000","last_peered":"2026-02-24T07:49:20.732327+0000","last_clean":"2026-02-24T07:49:20.732327+0000","last_became_active":"2026-02-24T07:48:57.902565+0000","last_became_peered":"2026-02-24T07:48:57.902565+0000","last_unstale":"2026-02-24T07:49:20.732327+0000","last_undegraded":"2026-02-24T07:49:20.732327+0000","last_fullsized":"2026-02-24T07:49:20.732327+0000","mapping_epoch":55,"log_start":"57'10614","ondisk_log_start":"57'10614","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:48:56.890768+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:50:25.079534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":695795712,"num_objects":10617,"num_object_clones":0,"num_object_copies":31851,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10617,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10617,"num_write_kb":679488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,3],"acting":[0,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.5","version":"57'10528","reported_seq":10541,"reported_epoch":57,"state":"active+clean","last_fresh":"2026-02-24T07:49:20.730497+0000","last_change":"2026-02-24T07:48:57.905643+0000","last_active":"2026-02-24T07:49:20.730497+0000","last_peered":"2026-02-24T07:49:20.730497+0000","last_clean":"2026-02-24T07:49:20.730497+0000","last_became_active":"2026-02-24T07:48:57.905127+0000","last_became_peered":"2026-02-24T07:48:57.905127+0000","last_unstale":"2026-02-24T07:49:20.730497+0000","last_undegraded":"2026-02-24T07:49:20.730497+0000","last_fullsized":"2026-02-24T07:49:20.730497+0000","mapping_epoch":55,"log_start":"57'10525","ondisk_log_start":"57'10525","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:48:56.890768+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:49:59.684568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":689963008,"num_objects":10528,"num_object_clones":0,"num_object_copies":31584,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10528,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10528,"num_write_kb":673792,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"29'99","reported_seq":277,"reported_epoch":57,"state":"active+clean","last_fresh":"2026-02-24T07:49:06.478572+0000","last_change":"2026-02-24T07:49:06.478572+0000","last_active":"2026-02-24T07:49:06.478572+0000","last_peered":"2026-02-24T07:49:06.478572+0000","last_clean":"2026-02-24T07:49:06.478572+0000","last_became_active":"2026-02-24T07:42:54.186719+0000","last_became_peered":"2026-02-24T07:42:54.186719+0000","last_unstale":"2026-02-24T07:49:06.478572+0000","last_undegraded":"2026-02-24T07:49:06.478572+0000","last_fullsized":"2026-02-24T07:49:06.478572+0000","mapping_epoch":27,"log_start":"29'96","ondisk_log_start":"29'96","created":20,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'99","last_scrub_stamp":"2026-02-24T07:49:06.478534+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:42:43.640776+0000","last_clean_scrub_stamp":"2026-02-24T07:49:06.478534+0000","objects_scrubbed":2,"log_size":3,"log_dups_size":96,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:50:26.893586+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":240,"num_write":135,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":16,"stat_sum":{"num_bytes":11121983488,"num_objects":169708,"num_object_clones":0,"num_object_copies":509124,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":169708,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":169708,"num_write_kb":10861312,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":33357627392,"data_stored":33357627392,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":48,"ondisk_log_size":48,"up":48,"acting":48,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":240,"num_write":135,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1236,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":3,"up_from":25,"seq":107374182480,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":9499032,"kb_used_data":8023748,"kb_used_omap":16,"kb_used_meta":1475247,"kb_avail":724504168,"statfs":{"total":751619276800,"available":741892268032,"internally_reserved":0,"allocated":8216317952,"data_stored":8215935984,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16538,"internal_metadata":1510653798},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":19,"seq":81604378706,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":10288088,"kb_used_data":9343044,"kb_used_omap":9,"kb_used_meta":945014,"kb_avail":723715112,"statfs":{"total":751619276800,"available":741084274688,"internally_reserved":0,"allocated":9567277056,"data_stored":9566895088,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9396,"internal_metadata":967695180},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,1],"upper_bound":4},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542227,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":8314708,"kb_used_data":6772096,"kb_used_omap":18,"kb_used_meta":1542573,"kb_avail":725688492,"statfs":{"total":751619276800,"available":743105015808,"internally_reserved":0,"allocated":6934626304,"data_stored":6934247888,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":18544,"internal_metadata":1579595664},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705750,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":8651992,"kb_used_data":7964164,"kb_used_omap":7,"kb_used_meta":687800,"kb_avail":725351208,"statfs":{"total":751619276800,"available":742759636992,"internally_reserved":0,"allocated":8155303936,"data_stored":8154921968,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":7926,"internal_metadata":704307466},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1236,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8333754368,"data_stored":8333754368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":6987579392,"data_stored":6987579392,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":9704833024,"data_stored":9704833024,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8331460608,"data_stored":8331460608,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-24T07:49:23.123 INFO:tasks.cephadm.ceph_manager.ceph:increase pool size by 10 2026-02-24T07:49:23.123 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph osd pool set .mgr pg_num 231 2026-02-24T07:49:23.263 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:49:23.707 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:49:23.704+0000 7f75e4bc6640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:49:23.707 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:49:23.705+0000 7f75e4bc6640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:49:23.856 INFO:teuthology.orchestra.run.trial035.stderr:Error EPERM: crimson_allow_pg_split is false; pg_num increase denied for crimson pool 2026-02-24T07:49:23.896 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-24T07:49:23.954 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:49:23 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/2087021363' entity='client.admin' cmd={"prefix": "osd pool set", "pool": ".mgr", "var": "pg_num", "val": "231"} : dispatch 2026-02-24T07:49:23.996 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 25 16 191310 191294 478.199 455.312 0.00131093 0.00208998 2026-02-24T07:49:24.137 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2026-02-24T07:49:24.278 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:49:24.407 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:49:24.404+0000 7fc4e86b3640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:49:24.407 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:49:24.405+0000 7fc4e86b3640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:49:24.422 INFO:teuthology.orchestra.run.trial035.stdout:osd.0: {} 2026-02-24T07:49:24.423 INFO:teuthology.orchestra.run.trial035.stderr:osd.0: osd_enable_op_tracker = '' 2026-02-24T07:49:24.437 INFO:teuthology.orchestra.run.trial035.stdout:osd.1: {} 2026-02-24T07:49:24.437 INFO:teuthology.orchestra.run.trial035.stderr:osd.1: osd_enable_op_tracker = '' 2026-02-24T07:49:24.450 INFO:teuthology.orchestra.run.trial035.stdout:osd.2: {} 2026-02-24T07:49:24.450 INFO:teuthology.orchestra.run.trial035.stderr:osd.2: osd_enable_op_tracker = '' 2026-02-24T07:49:24.461 INFO:teuthology.orchestra.run.trial035.stdout:osd.3: {} 2026-02-24T07:49:24.462 INFO:teuthology.orchestra.run.trial035.stderr:osd.3: osd_enable_op_tracker = '' 2026-02-24T07:49:24.996 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 26 16 199486 199470 479.459 511 0.00128739 0.00208447 2026-02-24T07:49:25.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:49:24 trial035 ceph-mon[39284]: from='client.14897 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2026-02-24T07:49:25.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:49:24 trial035 ceph-mon[39284]: pgmap v223: 17 pgs: 17 active+clean; 10 GiB data, 35 GiB used, 2.7 TiB / 2.7 TiB avail; 384 MiB/s wr, 6.14k op/s 2026-02-24T07:49:25.205 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:49:24 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/553633369' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2026-02-24T07:49:25.995 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 27 15 207569 207554 480.413 505.25 0.00147161 0.00208033 2026-02-24T07:49:26.509 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2026-02-24T07:49:26.655 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:49:26.777 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:49:26.775+0000 7fbb790c8640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:49:26.778 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:49:26.775+0000 7fbb790c8640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:49:26.793 INFO:teuthology.orchestra.run.trial035.stdout:osd.0: {} 2026-02-24T07:49:26.794 INFO:teuthology.orchestra.run.trial035.stderr:osd.0: osd_enable_op_tracker = '' 2026-02-24T07:49:26.807 INFO:teuthology.orchestra.run.trial035.stdout:osd.1: {} 2026-02-24T07:49:26.807 INFO:teuthology.orchestra.run.trial035.stderr:osd.1: osd_enable_op_tracker = '' 2026-02-24T07:49:26.821 INFO:teuthology.orchestra.run.trial035.stdout:osd.2: {} 2026-02-24T07:49:26.822 INFO:teuthology.orchestra.run.trial035.stderr:osd.2: osd_enable_op_tracker = '' 2026-02-24T07:49:26.835 INFO:teuthology.orchestra.run.trial035.stdout:osd.3: {} 2026-02-24T07:49:26.835 INFO:teuthology.orchestra.run.trial035.stderr:osd.3: osd_enable_op_tracker = '' 2026-02-24T07:49:26.996 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 28 15 215561 215546 481.093 499.5 0.00120356 0.00207742 2026-02-24T07:49:27.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:49:26 trial035 ceph-mon[39284]: pgmap v224: 17 pgs: 17 active+clean; 10 GiB data, 35 GiB used, 2.7 TiB / 2.7 TiB avail; 384 MiB/s wr, 6.14k op/s 2026-02-24T07:49:27.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:49:26 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/3414950149' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2026-02-24T07:49:27.205 INFO:journalctl@ceph.mgr.x.trial035.stdout:Feb 24 07:49:26 trial035 ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-mgr-x[39621]: ::ffff:10.20.193.91 - - [24/Feb/2026:07:49:26] "GET /metrics HTTP/1.1" 200 4210 "" "Prometheus/3.6.0" 2026-02-24T07:49:27.356 INFO:journalctl@ceph.prometheus.a.trial091.stdout:Feb 24 07:49:26 trial091 ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-prometheus-a[66007]: time=2026-02-24T07:49:26.973Z level=ERROR source=scrape.go:1630 msg="Failed to determine correct type of scrape target." component="scrape manager" scrape_pool=ceph target=http://trial035:9283/metrics content_type="text/html;charset=utf-8" fallback_media_type="" err="received unsupported Content-Type \"text/html;charset=utf-8\" and no fallback_scrape_protocol specified for target" 2026-02-24T07:49:27.995 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 29 16 223530 223514 481.675 498 0.00132658 0.00207488 2026-02-24T07:49:28.877 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2026-02-24T07:49:28.897 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2026-02-24T07:49:28.898 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2026-02-24T07:49:28.898 INFO:tasks.thrashosds.thrasher:trim_stale_osdmaps 2026-02-24T07:49:28.898 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph tell osd.0 'trim stale osdmaps' 2026-02-24T07:49:28.996 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 30 16 231441 231425 482.099 494.438 0.00181628 0.00207305 2026-02-24T07:49:29.032 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:49:29.091 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:49:29.150 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:49:28 trial035 ceph-mon[39284]: pgmap v225: 17 pgs: 17 active+clean; 13 GiB data, 42 GiB used, 2.7 TiB / 2.7 TiB avail; 571 MiB/s wr, 9.13k op/s 2026-02-24T07:49:29.541 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:49:29.459+0000 7f3f471f0640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:49:29.541 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:49:29.459+0000 7f3f471f0640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:49:29.542 INFO:teuthology.orchestra.run.trial035.stderr:osd.0: osd_enable_op_tracker = '' 2026-02-24T07:49:29.542 INFO:teuthology.orchestra.run.trial035.stderr:osd.1: osd_enable_op_tracker = '' 2026-02-24T07:49:29.542 INFO:teuthology.orchestra.run.trial035.stderr:osd.2: osd_enable_op_tracker = '' 2026-02-24T07:49:29.542 INFO:teuthology.orchestra.run.trial035.stdout:osd.0: {} 2026-02-24T07:49:29.542 INFO:teuthology.orchestra.run.trial035.stdout:osd.1: {} 2026-02-24T07:49:29.542 INFO:teuthology.orchestra.run.trial035.stdout:osd.2: {} 2026-02-24T07:49:29.543 INFO:teuthology.orchestra.run.trial035.stdout:osd.3: {} 2026-02-24T07:49:29.543 INFO:teuthology.orchestra.run.trial035.stderr:osd.3: osd_enable_op_tracker = '' 2026-02-24T07:49:29.996 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 31 16 237037 237021 477.829 349.75 0.00128959 0.00209041 2026-02-24T07:49:30.161 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:49:29 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/1095137737' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2026-02-24T07:49:30.161 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:49:29 trial035 ceph-mon[39284]: from='mgr.14284 10.20.193.35:0/3681373222' entity='mgr.x' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2026-02-24T07:49:30.568 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:49:30.550+0000 7f7d862a9640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:49:30.568 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:49:30.550+0000 7f7d862a9640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:49:30.587 INFO:teuthology.orchestra.run.trial035.stderr:no valid command found; 10 closest matches: 2026-02-24T07:49:30.587 INFO:teuthology.orchestra.run.trial035.stderr:0 2026-02-24T07:49:30.587 INFO:teuthology.orchestra.run.trial035.stderr:1 2026-02-24T07:49:30.587 INFO:teuthology.orchestra.run.trial035.stderr:2 2026-02-24T07:49:30.587 INFO:teuthology.orchestra.run.trial035.stderr:abort 2026-02-24T07:49:30.587 INFO:teuthology.orchestra.run.trial035.stderr:assert 2026-02-24T07:49:30.587 INFO:teuthology.orchestra.run.trial035.stderr:bench [] [] [] [] 2026-02-24T07:49:30.587 INFO:teuthology.orchestra.run.trial035.stderr:bluefs debug_inject_read_zeros 2026-02-24T07:49:30.587 INFO:teuthology.orchestra.run.trial035.stderr:bluefs files list 2026-02-24T07:49:30.587 INFO:teuthology.orchestra.run.trial035.stderr:bluefs stats 2026-02-24T07:49:30.587 INFO:teuthology.orchestra.run.trial035.stderr:bluestore allocator dump block 2026-02-24T07:49:30.588 INFO:teuthology.orchestra.run.trial035.stderr:Error EINVAL: invalid command 2026-02-24T07:49:30.996 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 32 16 242549 242533 473.661 344.5 0.00170246 0.00210799 2026-02-24T07:49:31.048 DEBUG:teuthology.orchestra.run:got remote process result: 22 2026-02-24T07:49:31.048 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2026-02-24T07:49:31.049 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph tell osd.1 'trim stale osdmaps' 2026-02-24T07:49:31.194 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:49:31.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:49:30 trial035 ceph-mon[39284]: pgmap v226: 17 pgs: 17 active+clean; 13 GiB data, 42 GiB used, 2.7 TiB / 2.7 TiB avail; 363 MiB/s wr, 5.82k op/s 2026-02-24T07:49:31.309 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:49:31.308+0000 7f8a5e42e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:49:31.310 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:49:31.308+0000 7f8a5e42e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:49:31.331 INFO:teuthology.orchestra.run.trial035.stderr:no valid command found; 10 closest matches: 2026-02-24T07:49:31.331 INFO:teuthology.orchestra.run.trial035.stderr:0 2026-02-24T07:49:31.332 INFO:teuthology.orchestra.run.trial035.stderr:1 2026-02-24T07:49:31.332 INFO:teuthology.orchestra.run.trial035.stderr:2 2026-02-24T07:49:31.332 INFO:teuthology.orchestra.run.trial035.stderr:abort 2026-02-24T07:49:31.332 INFO:teuthology.orchestra.run.trial035.stderr:assert 2026-02-24T07:49:31.332 INFO:teuthology.orchestra.run.trial035.stderr:bench [] [] [] [] 2026-02-24T07:49:31.332 INFO:teuthology.orchestra.run.trial035.stderr:bluefs debug_inject_read_zeros 2026-02-24T07:49:31.332 INFO:teuthology.orchestra.run.trial035.stderr:bluefs files list 2026-02-24T07:49:31.332 INFO:teuthology.orchestra.run.trial035.stderr:bluefs stats 2026-02-24T07:49:31.332 INFO:teuthology.orchestra.run.trial035.stderr:bluestore allocator dump block 2026-02-24T07:49:31.332 INFO:teuthology.orchestra.run.trial035.stderr:Error EINVAL: invalid command 2026-02-24T07:49:31.367 DEBUG:teuthology.orchestra.run:got remote process result: 22 2026-02-24T07:49:31.368 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2026-02-24T07:49:31.368 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph tell osd.2 'trim stale osdmaps' 2026-02-24T07:49:31.515 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:49:31.647 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:49:31.645+0000 7efec91d9640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:49:31.647 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:49:31.645+0000 7efec91d9640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:49:31.672 INFO:teuthology.orchestra.run.trial035.stderr:no valid command found; 10 closest matches: 2026-02-24T07:49:31.672 INFO:teuthology.orchestra.run.trial035.stderr:0 2026-02-24T07:49:31.672 INFO:teuthology.orchestra.run.trial035.stderr:1 2026-02-24T07:49:31.672 INFO:teuthology.orchestra.run.trial035.stderr:2 2026-02-24T07:49:31.672 INFO:teuthology.orchestra.run.trial035.stderr:abort 2026-02-24T07:49:31.672 INFO:teuthology.orchestra.run.trial035.stderr:assert 2026-02-24T07:49:31.672 INFO:teuthology.orchestra.run.trial035.stderr:bench [] [] [] [] 2026-02-24T07:49:31.673 INFO:teuthology.orchestra.run.trial035.stderr:bluefs debug_inject_read_zeros 2026-02-24T07:49:31.673 INFO:teuthology.orchestra.run.trial035.stderr:bluefs files list 2026-02-24T07:49:31.673 INFO:teuthology.orchestra.run.trial035.stderr:bluefs stats 2026-02-24T07:49:31.673 INFO:teuthology.orchestra.run.trial035.stderr:bluestore allocator dump block 2026-02-24T07:49:31.673 INFO:teuthology.orchestra.run.trial035.stderr:Error EINVAL: invalid command 2026-02-24T07:49:31.707 DEBUG:teuthology.orchestra.run:got remote process result: 22 2026-02-24T07:49:31.708 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2026-02-24T07:49:31.708 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph tell osd.3 'trim stale osdmaps' 2026-02-24T07:49:31.855 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:49:31.996 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 33 16 250311 250295 474.008 485.125 0.00130706 0.00210846 2026-02-24T07:49:32.161 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:49:32.159+0000 7f80e4f66640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:49:32.161 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:49:32.160+0000 7f80e4f66640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:49:32.182 INFO:teuthology.orchestra.run.trial035.stderr:no valid command found; 10 closest matches: 2026-02-24T07:49:32.182 INFO:teuthology.orchestra.run.trial035.stderr:0 2026-02-24T07:49:32.182 INFO:teuthology.orchestra.run.trial035.stderr:1 2026-02-24T07:49:32.182 INFO:teuthology.orchestra.run.trial035.stderr:2 2026-02-24T07:49:32.182 INFO:teuthology.orchestra.run.trial035.stderr:abort 2026-02-24T07:49:32.182 INFO:teuthology.orchestra.run.trial035.stderr:assert 2026-02-24T07:49:32.183 INFO:teuthology.orchestra.run.trial035.stderr:bench [] [] [] [] 2026-02-24T07:49:32.183 INFO:teuthology.orchestra.run.trial035.stderr:bluefs debug_inject_read_zeros 2026-02-24T07:49:32.183 INFO:teuthology.orchestra.run.trial035.stderr:bluefs files list 2026-02-24T07:49:32.183 INFO:teuthology.orchestra.run.trial035.stderr:bluefs stats 2026-02-24T07:49:32.183 INFO:teuthology.orchestra.run.trial035.stderr:bluestore allocator dump block 2026-02-24T07:49:32.183 INFO:teuthology.orchestra.run.trial035.stderr:Error EINVAL: invalid command 2026-02-24T07:49:32.183 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2026-02-24T07:49:32.218 DEBUG:teuthology.orchestra.run:got remote process result: 22 2026-02-24T07:49:32.219 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2026-02-24T07:49:32.323 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:49:32.444 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:49:32.443+0000 7fb93a9bd640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:49:32.444 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:49:32.443+0000 7fb93a9bd640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:49:32.459 INFO:teuthology.orchestra.run.trial035.stdout:osd.0: {} 2026-02-24T07:49:32.459 INFO:teuthology.orchestra.run.trial035.stderr:osd.0: osd_enable_op_tracker = '' 2026-02-24T07:49:32.473 INFO:teuthology.orchestra.run.trial035.stdout:osd.1: {} 2026-02-24T07:49:32.473 INFO:teuthology.orchestra.run.trial035.stderr:osd.1: osd_enable_op_tracker = '' 2026-02-24T07:49:32.488 INFO:teuthology.orchestra.run.trial035.stdout:osd.2: {} 2026-02-24T07:49:32.488 INFO:teuthology.orchestra.run.trial035.stderr:osd.2: osd_enable_op_tracker = '' 2026-02-24T07:49:32.501 INFO:teuthology.orchestra.run.trial035.stdout:osd.3: {} 2026-02-24T07:49:32.501 INFO:teuthology.orchestra.run.trial035.stderr:osd.3: osd_enable_op_tracker = '' 2026-02-24T07:49:32.996 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 34 16 257997 257981 474.194 480.375 0.00138768 0.00210762 2026-02-24T07:49:33.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:49:32 trial035 ceph-mon[39284]: pgmap v227: 17 pgs: 17 active+clean; 15 GiB data, 48 GiB used, 2.7 TiB / 2.7 TiB avail; 549 MiB/s wr, 8.79k op/s 2026-02-24T07:49:33.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:49:32 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/1756228781' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2026-02-24T07:49:33.996 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 35 16 266238 266222 475.36 515.062 0.00173472 0.00210247 2026-02-24T07:49:34.546 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2026-02-24T07:49:34.686 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:49:34.817 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:49:34.815+0000 7fbbbe47c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:49:34.817 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:49:34.816+0000 7fbbbe47c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:49:34.833 INFO:teuthology.orchestra.run.trial035.stdout:osd.0: {} 2026-02-24T07:49:34.833 INFO:teuthology.orchestra.run.trial035.stderr:osd.0: osd_enable_op_tracker = '' 2026-02-24T07:49:34.846 INFO:teuthology.orchestra.run.trial035.stdout:osd.1: {} 2026-02-24T07:49:34.846 INFO:teuthology.orchestra.run.trial035.stderr:osd.1: osd_enable_op_tracker = '' 2026-02-24T07:49:34.862 INFO:teuthology.orchestra.run.trial035.stdout:osd.2: {} 2026-02-24T07:49:34.862 INFO:teuthology.orchestra.run.trial035.stderr:osd.2: osd_enable_op_tracker = '' 2026-02-24T07:49:34.874 INFO:teuthology.orchestra.run.trial035.stdout:osd.3: {} 2026-02-24T07:49:34.875 INFO:teuthology.orchestra.run.trial035.stderr:osd.3: osd_enable_op_tracker = '' 2026-02-24T07:49:34.996 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 36 15 274403 274388 476.332 510.375 0.00130095 0.00209817 2026-02-24T07:49:35.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:49:34 trial035 ceph-mon[39284]: pgmap v228: 17 pgs: 17 active+clean; 15 GiB data, 48 GiB used, 2.7 TiB / 2.7 TiB avail; 372 MiB/s wr, 5.96k op/s 2026-02-24T07:49:35.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:49:34 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/1213934099' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2026-02-24T07:49:35.996 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 37 16 282765 282749 477.58 522.562 0.00231411 0.00209268 2026-02-24T07:49:36.918 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2026-02-24T07:49:36.996 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 38 16 290685 290669 478.038 495 0.00139995 0.00209069 2026-02-24T07:49:37.059 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:49:37.127 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:49:36 trial035 ceph-mon[39284]: pgmap v229: 17 pgs: 17 active+clean; 15 GiB data, 48 GiB used, 2.7 TiB / 2.7 TiB avail; 372 MiB/s wr, 5.96k op/s 2026-02-24T07:49:37.128 INFO:journalctl@ceph.mgr.x.trial035.stdout:Feb 24 07:49:36 trial035 ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-mgr-x[39621]: ::ffff:10.20.193.91 - - [24/Feb/2026:07:49:36] "GET /metrics HTTP/1.1" 200 4213 "" "Prometheus/3.6.0" 2026-02-24T07:49:37.187 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:49:37.186+0000 7fd0cb210640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:49:37.187 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:49:37.186+0000 7fd0cb210640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:49:37.203 INFO:teuthology.orchestra.run.trial035.stdout:osd.0: {} 2026-02-24T07:49:37.204 INFO:teuthology.orchestra.run.trial035.stderr:osd.0: osd_enable_op_tracker = '' 2026-02-24T07:49:37.218 INFO:teuthology.orchestra.run.trial035.stdout:osd.1: {} 2026-02-24T07:49:37.219 INFO:teuthology.orchestra.run.trial035.stderr:osd.1: osd_enable_op_tracker = '' 2026-02-24T07:49:37.219 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2026-02-24T07:49:37.219 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph osd reweight 0 1 2026-02-24T07:49:37.233 INFO:teuthology.orchestra.run.trial035.stdout:osd.2: {} 2026-02-24T07:49:37.233 INFO:teuthology.orchestra.run.trial035.stderr:osd.2: osd_enable_op_tracker = '' 2026-02-24T07:49:37.250 INFO:teuthology.orchestra.run.trial035.stdout:osd.3: {} 2026-02-24T07:49:37.250 INFO:teuthology.orchestra.run.trial035.stderr:osd.3: osd_enable_op_tracker = '' 2026-02-24T07:49:37.356 INFO:journalctl@ceph.prometheus.a.trial091.stdout:Feb 24 07:49:36 trial091 ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-prometheus-a[66007]: time=2026-02-24T07:49:36.973Z level=ERROR source=scrape.go:1630 msg="Failed to determine correct type of scrape target." component="scrape manager" scrape_pool=ceph target=http://trial035:9283/metrics content_type="text/html;charset=utf-8" fallback_media_type="" err="received unsupported Content-Type \"text/html;charset=utf-8\" and no fallback_scrape_protocol specified for target" 2026-02-24T07:49:37.359 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:49:37.482 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:49:37.480+0000 7f3513331640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:49:37.482 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:49:37.481+0000 7f3513331640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:49:37.879 INFO:teuthology.orchestra.run.trial035.stderr:reweighted osd.0 to 1 (10000) 2026-02-24T07:49:37.996 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 39 16 298325 298309 478.023 477.5 0.00213424 0.00209074 2026-02-24T07:49:38.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:49:37 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/517962816' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2026-02-24T07:49:38.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:49:37 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/2150982056' entity='client.admin' cmd={"prefix": "osd reweight", "id": 0, "weight": 1.0} : dispatch 2026-02-24T07:49:38.690 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph osd reweight 1 1 2026-02-24T07:49:38.831 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:49:38.947 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:49:38 trial035 ceph-mon[39284]: pgmap v230: 17 pgs: 17 active+clean; 17 GiB data, 56 GiB used, 2.7 TiB / 2.7 TiB avail; 578 MiB/s wr, 9.24k op/s 2026-02-24T07:49:38.947 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:49:38 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/2150982056' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 0, "weight": 1.0}]': finished 2026-02-24T07:49:38.947 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:49:38 trial035 ceph-mon[39284]: osdmap e58: 4 total, 4 up, 4 in 2026-02-24T07:49:38.996 INFO:tasks.radosbench.radosbench.0.trial035.stdout:2026-02-24T07:49:38.996345+0000 min lat: 0.000682058 max lat: 0.0426186 avg lat: 0.00210055 2026-02-24T07:49:38.997 INFO:tasks.radosbench.radosbench.0.trial035.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2026-02-24T07:49:38.997 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 40 15 304546 304531 475.794 388.875 0.00273096 0.00210055 2026-02-24T07:49:39.295 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2026-02-24T07:49:39.299 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:49:39.280+0000 7f569e653640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:49:39.299 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:49:39.280+0000 7f569e653640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:49:39.440 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:49:39.883 INFO:teuthology.orchestra.run.trial035.stderr:reweighted osd.1 to 1 (10000) 2026-02-24T07:49:39.941 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:49:39.925+0000 7f5c96a29640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:49:39.942 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:49:39.926+0000 7f5c96a29640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:49:39.955 INFO:teuthology.orchestra.run.trial035.stdout:osd.0: {} 2026-02-24T07:49:39.955 INFO:teuthology.orchestra.run.trial035.stderr:osd.0: osd_enable_op_tracker = '' 2026-02-24T07:49:39.971 INFO:teuthology.orchestra.run.trial035.stdout:osd.1: {} 2026-02-24T07:49:39.972 INFO:teuthology.orchestra.run.trial035.stderr:osd.1: osd_enable_op_tracker = '' 2026-02-24T07:49:39.984 INFO:teuthology.orchestra.run.trial035.stdout:osd.2: {} 2026-02-24T07:49:39.985 INFO:teuthology.orchestra.run.trial035.stderr:osd.2: osd_enable_op_tracker = '' 2026-02-24T07:49:39.995 INFO:teuthology.orchestra.run.trial035.stdout:osd.3: {} 2026-02-24T07:49:39.995 INFO:teuthology.orchestra.run.trial035.stderr:osd.3: osd_enable_op_tracker = '' 2026-02-24T07:49:39.996 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 41 16 310417 310401 473.136 366.875 0.000909521 0.00211161 2026-02-24T07:49:40.071 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph osd reweight 2 1 2026-02-24T07:49:40.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:49:39 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/3546813606' entity='client.admin' cmd={"prefix": "osd reweight", "id": 1, "weight": 1.0} : dispatch 2026-02-24T07:49:40.209 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:49:40.324 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:49:40.323+0000 7f2c6af82640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:49:40.325 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:49:40.323+0000 7f2c6af82640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:49:40.884 INFO:teuthology.orchestra.run.trial035.stderr:reweighted osd.2 to 1 (10000) 2026-02-24T07:49:40.929 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph osd reweight 3 1 2026-02-24T07:49:40.997 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 42 16 318063 318047 473.248 477.875 0.00222094 0.00211187 2026-02-24T07:49:41.079 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:49:41.138 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:49:40 trial035 ceph-mon[39284]: pgmap v232: 17 pgs: 17 active+clean; 17 GiB data, 56 GiB used, 2.7 TiB / 2.7 TiB avail; 469 MiB/s wr, 7.51k op/s 2026-02-24T07:49:41.138 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:49:40 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/3546813606' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 1, "weight": 1.0}]': finished 2026-02-24T07:49:41.139 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:49:40 trial035 ceph-mon[39284]: osdmap e59: 4 total, 4 up, 4 in 2026-02-24T07:49:41.139 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:49:40 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/722215799' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2026-02-24T07:49:41.139 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:49:40 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/3648720224' entity='client.admin' cmd={"prefix": "osd reweight", "id": 2, "weight": 1.0} : dispatch 2026-02-24T07:49:41.212 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:49:41.209+0000 7f9579e49640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:49:41.212 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:49:41.210+0000 7f9579e49640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:49:41.888 INFO:teuthology.orchestra.run.trial035.stderr:reweighted osd.3 to 1 (10000) 2026-02-24T07:49:41.940 INFO:tasks.cephadm.ceph_manager.ceph:waiting for recovery to complete 2026-02-24T07:49:41.940 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph pg dump --format=json 2026-02-24T07:49:41.997 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 43 16 325755 325739 473.422 480.75 0.00178141 0.00211109 2026-02-24T07:49:42.085 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:49:42.091 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2026-02-24T07:49:42.144 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:49:41 trial035 ceph-mon[39284]: purged_snaps scrub starts 2026-02-24T07:49:42.145 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:49:41 trial035 ceph-mon[39284]: purged_snaps scrub ok 2026-02-24T07:49:42.145 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:49:41 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/3648720224' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 2, "weight": 1.0}]': finished 2026-02-24T07:49:42.145 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:49:41 trial035 ceph-mon[39284]: osdmap e60: 4 total, 4 up, 4 in 2026-02-24T07:49:42.145 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:49:41 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/2878273769' entity='client.admin' cmd={"prefix": "osd reweight", "id": 3, "weight": 1.0} : dispatch 2026-02-24T07:49:42.212 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:49:42.210+0000 7f0110d37640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:49:42.213 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:49:42.210+0000 7f0110d37640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:49:42.232 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:49:42.364 INFO:teuthology.orchestra.run.trial035.stdout: 2026-02-24T07:49:42.364 INFO:teuthology.orchestra.run.trial035.stderr:dumped all 2026-02-24T07:49:42.365 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:49:42.362+0000 7f9d4dac7640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:49:42.365 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:49:42.362+0000 7f9d4dac7640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:49:42.382 INFO:teuthology.orchestra.run.trial035.stdout:osd.0: {} 2026-02-24T07:49:42.383 INFO:teuthology.orchestra.run.trial035.stderr:osd.0: osd_enable_op_tracker = '' 2026-02-24T07:49:42.398 INFO:teuthology.orchestra.run.trial035.stdout:osd.1: {} 2026-02-24T07:49:42.398 INFO:teuthology.orchestra.run.trial035.stderr:osd.1: osd_enable_op_tracker = '' 2026-02-24T07:49:42.411 INFO:teuthology.orchestra.run.trial035.stdout:{"pg_ready":true,"pg_map":{"version":236,"stamp":"2026-02-24T07:49:41.888072+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":20715471392,"num_objects":316086,"num_object_clones":0,"num_object_copies":948258,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":316086,"num_whiteouts":0,"num_read":136,"num_read_kb":240,"num_write":316219,"num_write_kb":20231526,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":52,"ondisk_log_size":52,"up":51,"acting":51,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":51,"num_osds":4,"num_per_pool_osds":4,"num_per_pool_omap_osds":4,"kb":2936012800,"kb_used":65623292,"kb_used_data":60358764,"kb_used_omap":41,"kb_used_meta":5264406,"kb_avail":2870389508,"statfs":{"total":3006477107200,"available":2939278856192,"internally_reserved":0,"allocated":61807374336,"data_stored":61805786984,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":42778,"internal_metadata":5390751974},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[7,2],"upper_bound":4},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2322923520,"num_objects":35445,"num_object_clones":0,"num_object_copies":106335,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35445,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":35445,"num_write_kb":2268480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.176978"},"pg_stats":[{"pgid":"4.f","version":"59'19867","reported_seq":19883,"reported_epoch":59,"state":"active+clean","last_fresh":"2026-02-24T07:49:40.810230+0000","last_change":"2026-02-24T07:48:57.902569+0000","last_active":"2026-02-24T07:49:40.810230+0000","last_peered":"2026-02-24T07:49:40.810230+0000","last_clean":"2026-02-24T07:49:40.810230+0000","last_became_active":"2026-02-24T07:48:57.902313+0000","last_became_peered":"2026-02-24T07:48:57.902313+0000","last_unstale":"2026-02-24T07:49:40.810230+0000","last_undegraded":"2026-02-24T07:49:40.810230+0000","last_fullsized":"2026-02-24T07:49:40.810230+0000","mapping_epoch":55,"log_start":"59'19864","ondisk_log_start":"59'19864","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:48:56.890768+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:49:59.578048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1302003712,"num_objects":19867,"num_object_clones":0,"num_object_copies":59601,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19867,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19867,"num_write_kb":1271488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"59'19854","reported_seq":19872,"reported_epoch":59,"state":"active+clean","last_fresh":"2026-02-24T07:49:40.706494+0000","last_change":"2026-02-24T07:48:57.903920+0000","last_active":"2026-02-24T07:49:40.706494+0000","last_peered":"2026-02-24T07:49:40.706494+0000","last_clean":"2026-02-24T07:49:40.706494+0000","last_became_active":"2026-02-24T07:48:57.903584+0000","last_became_peered":"2026-02-24T07:48:57.903584+0000","last_unstale":"2026-02-24T07:49:40.706494+0000","last_undegraded":"2026-02-24T07:49:40.706494+0000","last_fullsized":"2026-02-24T07:49:40.706494+0000","mapping_epoch":55,"log_start":"59'19851","ondisk_log_start":"59'19851","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:48:56.890768+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:50:17.159571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1301151744,"num_objects":19854,"num_object_clones":0,"num_object_copies":59562,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19854,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19854,"num_write_kb":1270656,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.d","version":"59'19592","reported_seq":19610,"reported_epoch":59,"state":"active+clean","last_fresh":"2026-02-24T07:49:40.729965+0000","last_change":"2026-02-24T07:48:57.905520+0000","last_active":"2026-02-24T07:49:40.729965+0000","last_peered":"2026-02-24T07:49:40.729965+0000","last_clean":"2026-02-24T07:49:40.729965+0000","last_became_active":"2026-02-24T07:48:57.905031+0000","last_became_peered":"2026-02-24T07:48:57.905031+0000","last_unstale":"2026-02-24T07:49:40.729965+0000","last_undegraded":"2026-02-24T07:49:40.729965+0000","last_fullsized":"2026-02-24T07:49:40.729965+0000","mapping_epoch":55,"log_start":"59'19589","ondisk_log_start":"59'19589","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:48:56.890768+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:50:02.301170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1283981312,"num_objects":19592,"num_object_clones":0,"num_object_copies":58776,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19592,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19592,"num_write_kb":1253888,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"59'19606","reported_seq":19624,"reported_epoch":59,"state":"active+clean","last_fresh":"2026-02-24T07:49:40.731932+0000","last_change":"2026-02-24T07:48:57.905290+0000","last_active":"2026-02-24T07:49:40.731932+0000","last_peered":"2026-02-24T07:49:40.731932+0000","last_clean":"2026-02-24T07:49:40.731932+0000","last_became_active":"2026-02-24T07:48:57.904683+0000","last_became_peered":"2026-02-24T07:48:57.904683+0000","last_unstale":"2026-02-24T07:49:40.731932+0000","last_undegraded":"2026-02-24T07:49:40.731932+0000","last_fullsized":"2026-02-24T07:49:40.731932+0000","mapping_epoch":55,"log_start":"59'19603","ondisk_log_start":"59'19603","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:48:56.890768+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:50:16.643522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1284898816,"num_objects":19606,"num_object_clones":0,"num_object_copies":58818,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19606,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19606,"num_write_kb":1254784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.b","version":"59'19631","reported_seq":19649,"reported_epoch":59,"state":"active+clean","last_fresh":"2026-02-24T07:49:40.737697+0000","last_change":"2026-02-24T07:48:57.902025+0000","last_active":"2026-02-24T07:49:40.737697+0000","last_peered":"2026-02-24T07:49:40.737697+0000","last_clean":"2026-02-24T07:49:40.737697+0000","last_became_active":"2026-02-24T07:48:57.901700+0000","last_became_peered":"2026-02-24T07:48:57.901700+0000","last_unstale":"2026-02-24T07:49:40.737697+0000","last_undegraded":"2026-02-24T07:49:40.737697+0000","last_fullsized":"2026-02-24T07:49:40.737697+0000","mapping_epoch":55,"log_start":"59'19628","ondisk_log_start":"59'19628","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:48:56.890768+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:50:23.370319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1286537216,"num_objects":19631,"num_object_clones":0,"num_object_copies":58893,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19631,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19631,"num_write_kb":1256384,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.a","version":"59'19925","reported_seq":19941,"reported_epoch":59,"state":"active+clean","last_fresh":"2026-02-24T07:49:40.716206+0000","last_change":"2026-02-24T07:48:57.903698+0000","last_active":"2026-02-24T07:49:40.716206+0000","last_peered":"2026-02-24T07:49:40.716206+0000","last_clean":"2026-02-24T07:49:40.716206+0000","last_became_active":"2026-02-24T07:48:57.903228+0000","last_became_peered":"2026-02-24T07:48:57.903228+0000","last_unstale":"2026-02-24T07:49:40.716206+0000","last_undegraded":"2026-02-24T07:49:40.716206+0000","last_fullsized":"2026-02-24T07:49:40.716206+0000","mapping_epoch":55,"log_start":"59'19921","ondisk_log_start":"59'19921","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:48:56.890768+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":3000,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:50:26.780637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1305804800,"num_objects":19925,"num_object_clones":0,"num_object_copies":59775,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19925,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19925,"num_write_kb":1275200,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"59'19790","reported_seq":19808,"reported_epoch":59,"state":"active+clean","last_fresh":"2026-02-24T07:49:40.713440+0000","last_change":"2026-02-24T07:48:57.902011+0000","last_active":"2026-02-24T07:49:40.713440+0000","last_peered":"2026-02-24T07:49:40.713440+0000","last_clean":"2026-02-24T07:49:40.713440+0000","last_became_active":"2026-02-24T07:48:57.901730+0000","last_became_peered":"2026-02-24T07:48:57.901730+0000","last_unstale":"2026-02-24T07:49:40.713440+0000","last_undegraded":"2026-02-24T07:49:40.713440+0000","last_fullsized":"2026-02-24T07:49:40.713440+0000","mapping_epoch":55,"log_start":"59'19787","ondisk_log_start":"59'19787","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:48:56.890768+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:50:05.273318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1296957440,"num_objects":19790,"num_object_clones":0,"num_object_copies":59370,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19790,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19790,"num_write_kb":1266560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"59'19745","reported_seq":19763,"reported_epoch":59,"state":"active+clean","last_fresh":"2026-02-24T07:49:40.731372+0000","last_change":"2026-02-24T07:48:57.904924+0000","last_active":"2026-02-24T07:49:40.731372+0000","last_peered":"2026-02-24T07:49:40.731372+0000","last_clean":"2026-02-24T07:49:40.731372+0000","last_became_active":"2026-02-24T07:48:57.904376+0000","last_became_peered":"2026-02-24T07:48:57.904376+0000","last_unstale":"2026-02-24T07:49:40.731372+0000","last_undegraded":"2026-02-24T07:49:40.731372+0000","last_fullsized":"2026-02-24T07:49:40.731372+0000","mapping_epoch":55,"log_start":"59'19742","ondisk_log_start":"59'19742","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:48:56.890768+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:49:57.806612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1294008320,"num_objects":19745,"num_object_clones":0,"num_object_copies":59235,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19745,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19745,"num_write_kb":1263680,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.7","version":"59'19840","reported_seq":19858,"reported_epoch":59,"state":"active+clean","last_fresh":"2026-02-24T07:49:40.801382+0000","last_change":"2026-02-24T07:48:57.902698+0000","last_active":"2026-02-24T07:49:40.801382+0000","last_peered":"2026-02-24T07:49:40.801382+0000","last_clean":"2026-02-24T07:49:40.801382+0000","last_became_active":"2026-02-24T07:48:57.902484+0000","last_became_peered":"2026-02-24T07:48:57.902484+0000","last_unstale":"2026-02-24T07:49:40.801382+0000","last_undegraded":"2026-02-24T07:49:40.801382+0000","last_fullsized":"2026-02-24T07:49:40.801382+0000","mapping_epoch":55,"log_start":"59'19837","ondisk_log_start":"59'19837","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:48:56.890768+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:50:03.263698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1300234240,"num_objects":19840,"num_object_clones":0,"num_object_copies":59520,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19840,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19840,"num_write_kb":1269760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.6","version":"59'19481","reported_seq":19498,"reported_epoch":59,"state":"active+clean","last_fresh":"2026-02-24T07:49:40.731562+0000","last_change":"2026-02-24T07:48:57.901839+0000","last_active":"2026-02-24T07:49:40.731562+0000","last_peered":"2026-02-24T07:49:40.731562+0000","last_clean":"2026-02-24T07:49:40.731562+0000","last_became_active":"2026-02-24T07:48:57.901565+0000","last_became_peered":"2026-02-24T07:48:57.901565+0000","last_unstale":"2026-02-24T07:49:40.731562+0000","last_undegraded":"2026-02-24T07:49:40.731562+0000","last_fullsized":"2026-02-24T07:49:40.731562+0000","mapping_epoch":55,"log_start":"59'19478","ondisk_log_start":"59'19478","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:48:56.890768+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:50:09.631523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1276706816,"num_objects":19481,"num_object_clones":0,"num_object_copies":58443,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19481,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19481,"num_write_kb":1246784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"59'19941","reported_seq":19959,"reported_epoch":59,"state":"active+clean","last_fresh":"2026-02-24T07:49:40.807618+0000","last_change":"2026-02-24T07:48:57.902165+0000","last_active":"2026-02-24T07:49:40.807618+0000","last_peered":"2026-02-24T07:49:40.807618+0000","last_clean":"2026-02-24T07:49:40.807618+0000","last_became_active":"2026-02-24T07:48:57.901824+0000","last_became_peered":"2026-02-24T07:48:57.901824+0000","last_unstale":"2026-02-24T07:49:40.807618+0000","last_undegraded":"2026-02-24T07:49:40.807618+0000","last_fullsized":"2026-02-24T07:49:40.807618+0000","mapping_epoch":55,"log_start":"59'19938","ondisk_log_start":"59'19938","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:48:56.890768+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:50:12.091202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1306853376,"num_objects":19941,"num_object_clones":0,"num_object_copies":59823,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19941,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19941,"num_write_kb":1276224,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[1,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"59'19720","reported_seq":19738,"reported_epoch":59,"state":"active+clean","last_fresh":"2026-02-24T07:49:40.735441+0000","last_change":"2026-02-24T07:48:57.902537+0000","last_active":"2026-02-24T07:49:40.735441+0000","last_peered":"2026-02-24T07:49:40.735441+0000","last_clean":"2026-02-24T07:49:40.735441+0000","last_became_active":"2026-02-24T07:48:57.902224+0000","last_became_peered":"2026-02-24T07:48:57.902224+0000","last_unstale":"2026-02-24T07:49:40.735441+0000","last_undegraded":"2026-02-24T07:49:40.735441+0000","last_fullsized":"2026-02-24T07:49:40.735441+0000","mapping_epoch":55,"log_start":"59'19717","ondisk_log_start":"59'19717","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:48:56.890768+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:50:06.186992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1292369920,"num_objects":19720,"num_object_clones":0,"num_object_copies":59160,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19720,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19720,"num_write_kb":1262080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"59'19785","reported_seq":19803,"reported_epoch":59,"state":"active+clean","last_fresh":"2026-02-24T07:49:40.808510+0000","last_change":"2026-02-24T07:48:57.902384+0000","last_active":"2026-02-24T07:49:40.808510+0000","last_peered":"2026-02-24T07:49:40.808510+0000","last_clean":"2026-02-24T07:49:40.808510+0000","last_became_active":"2026-02-24T07:48:57.902125+0000","last_became_peered":"2026-02-24T07:48:57.902125+0000","last_unstale":"2026-02-24T07:49:40.808510+0000","last_undegraded":"2026-02-24T07:49:40.808510+0000","last_fullsized":"2026-02-24T07:49:40.808510+0000","mapping_epoch":55,"log_start":"59'19782","ondisk_log_start":"59'19782","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:48:56.890768+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:49:57.995484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1296629760,"num_objects":19785,"num_object_clones":0,"num_object_copies":59355,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19785,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19785,"num_write_kb":1266240,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1","version":"59'19802","reported_seq":19820,"reported_epoch":59,"state":"active+clean","last_fresh":"2026-02-24T07:49:40.709453+0000","last_change":"2026-02-24T07:48:57.903745+0000","last_active":"2026-02-24T07:49:40.709453+0000","last_peered":"2026-02-24T07:49:40.709453+0000","last_clean":"2026-02-24T07:49:40.709453+0000","last_became_active":"2026-02-24T07:48:57.903439+0000","last_became_peered":"2026-02-24T07:48:57.903439+0000","last_unstale":"2026-02-24T07:49:40.709453+0000","last_undegraded":"2026-02-24T07:49:40.709453+0000","last_fullsized":"2026-02-24T07:49:40.709453+0000","mapping_epoch":55,"log_start":"59'19799","ondisk_log_start":"59'19799","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:48:56.890768+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:50:24.058110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1297743872,"num_objects":19802,"num_object_clones":0,"num_object_copies":59406,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19802,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19802,"num_write_kb":1267328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.0","version":"59'19752","reported_seq":19768,"reported_epoch":59,"state":"active+clean","last_fresh":"2026-02-24T07:49:40.736692+0000","last_change":"2026-02-24T07:48:57.902838+0000","last_active":"2026-02-24T07:49:40.736692+0000","last_peered":"2026-02-24T07:49:40.736692+0000","last_clean":"2026-02-24T07:49:40.736692+0000","last_became_active":"2026-02-24T07:48:57.902565+0000","last_became_peered":"2026-02-24T07:48:57.902565+0000","last_unstale":"2026-02-24T07:49:40.736692+0000","last_undegraded":"2026-02-24T07:49:40.736692+0000","last_fullsized":"2026-02-24T07:49:40.736692+0000","mapping_epoch":55,"log_start":"59'19749","ondisk_log_start":"59'19749","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:48:56.890768+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:50:25.079534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1294467072,"num_objects":19752,"num_object_clones":0,"num_object_copies":59256,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19752,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19752,"num_write_kb":1264128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,3],"acting":[0,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.5","version":"59'19753","reported_seq":19771,"reported_epoch":59,"state":"active+clean","last_fresh":"2026-02-24T07:49:40.720547+0000","last_change":"2026-02-24T07:48:57.905643+0000","last_active":"2026-02-24T07:49:40.720547+0000","last_peered":"2026-02-24T07:49:40.720547+0000","last_clean":"2026-02-24T07:49:40.720547+0000","last_became_active":"2026-02-24T07:48:57.905127+0000","last_became_peered":"2026-02-24T07:48:57.905127+0000","last_unstale":"2026-02-24T07:49:40.720547+0000","last_undegraded":"2026-02-24T07:49:40.720547+0000","last_fullsized":"2026-02-24T07:49:40.720547+0000","mapping_epoch":55,"log_start":"59'19750","ondisk_log_start":"59'19750","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:48:56.890768+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:49:59.684568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1294532608,"num_objects":19753,"num_object_clones":0,"num_object_copies":59259,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19753,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19753,"num_write_kb":1264192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"29'99","reported_seq":281,"reported_epoch":59,"state":"active+clean","last_fresh":"2026-02-24T07:49:39.884150+0000","last_change":"2026-02-24T07:49:06.478572+0000","last_active":"2026-02-24T07:49:39.884150+0000","last_peered":"2026-02-24T07:49:39.884150+0000","last_clean":"2026-02-24T07:49:39.884150+0000","last_became_active":"2026-02-24T07:42:54.186719+0000","last_became_peered":"2026-02-24T07:42:54.186719+0000","last_unstale":"2026-02-24T07:49:39.884150+0000","last_undegraded":"2026-02-24T07:49:39.884150+0000","last_fullsized":"2026-02-24T07:49:39.884150+0000","mapping_epoch":27,"log_start":"29'96","ondisk_log_start":"29'96","created":20,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'99","last_scrub_stamp":"2026-02-24T07:49:06.478534+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:42:43.640776+0000","last_clean_scrub_stamp":"2026-02-24T07:49:06.478534+0000","objects_scrubbed":2,"log_size":3,"log_dups_size":96,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:50:26.893586+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":240,"num_write":135,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":16,"stat_sum":{"num_bytes":20714881024,"num_objects":316084,"num_object_clones":0,"num_object_copies":948252,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":316084,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":316084,"num_write_kb":20229376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":62140121088,"data_stored":62140121088,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":49,"ondisk_log_size":49,"up":48,"acting":48,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":240,"num_write":135,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":3,"up_from":25,"seq":107374182484,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":16191792,"kb_used_data":15059228,"kb_used_omap":10,"kb_used_meta":1132533,"kb_avail":717811408,"statfs":{"total":751619276800,"available":735038881792,"internally_reserved":0,"allocated":15420649472,"data_stored":15420251746,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":10649,"internal_metadata":1159714407},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[7,2],"upper_bound":4},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":19,"seq":81604378710,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":19052080,"kb_used_data":17601692,"kb_used_omap":10,"kb_used_meta":1450357,"kb_avail":714951120,"statfs":{"total":751619276800,"available":732109946880,"internally_reserved":0,"allocated":18024132608,"data_stored":18023734882,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":10700,"internal_metadata":1485166132},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542231,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":14197356,"kb_used_data":12636760,"kb_used_omap":11,"kb_used_meta":1560564,"kb_avail":719805844,"statfs":{"total":751619276800,"available":737081184256,"internally_reserved":0,"allocated":12940042240,"data_stored":12939648066,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12199,"internal_metadata":1598017625},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705754,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":16182064,"kb_used_data":15061084,"kb_used_omap":9,"kb_used_meta":1120950,"kb_avail":717821136,"statfs":{"total":751619276800,"available":735048843264,"internally_reserved":0,"allocated":15422550016,"data_stored":15422152290,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9230,"internal_metadata":1147853810},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":15516434432,"data_stored":15516434432,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12976324608,"data_stored":12976324608,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":18099798016,"data_stored":18099798016,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":15547564032,"data_stored":15547564032,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-24T07:49:42.411 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph pg dump --format=json 2026-02-24T07:49:42.413 INFO:teuthology.orchestra.run.trial035.stdout:osd.2: {} 2026-02-24T07:49:42.413 INFO:teuthology.orchestra.run.trial035.stderr:osd.2: osd_enable_op_tracker = '' 2026-02-24T07:49:42.426 INFO:teuthology.orchestra.run.trial035.stdout:osd.3: {} 2026-02-24T07:49:42.426 INFO:teuthology.orchestra.run.trial035.stderr:osd.3: osd_enable_op_tracker = '' 2026-02-24T07:49:42.562 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:49:42.703 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:49:42.700+0000 7f053e1ae640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:49:42.703 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:49:42.701+0000 7f053e1ae640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:49:42.848 INFO:teuthology.orchestra.run.trial035.stdout: 2026-02-24T07:49:42.848 INFO:teuthology.orchestra.run.trial035.stderr:dumped all 2026-02-24T07:49:42.894 INFO:teuthology.orchestra.run.trial035.stdout:{"pg_ready":true,"pg_map":{"version":236,"stamp":"2026-02-24T07:49:41.888072+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":20715471392,"num_objects":316086,"num_object_clones":0,"num_object_copies":948258,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":316086,"num_whiteouts":0,"num_read":136,"num_read_kb":240,"num_write":316219,"num_write_kb":20231526,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":52,"ondisk_log_size":52,"up":51,"acting":51,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":51,"num_osds":4,"num_per_pool_osds":4,"num_per_pool_omap_osds":4,"kb":2936012800,"kb_used":65623292,"kb_used_data":60358764,"kb_used_omap":41,"kb_used_meta":5264406,"kb_avail":2870389508,"statfs":{"total":3006477107200,"available":2939278856192,"internally_reserved":0,"allocated":61807374336,"data_stored":61805786984,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":42778,"internal_metadata":5390751974},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[7,2],"upper_bound":4},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2322923520,"num_objects":35445,"num_object_clones":0,"num_object_copies":106335,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35445,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":35445,"num_write_kb":2268480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.176978"},"pg_stats":[{"pgid":"4.f","version":"59'19867","reported_seq":19883,"reported_epoch":59,"state":"active+clean","last_fresh":"2026-02-24T07:49:40.810230+0000","last_change":"2026-02-24T07:48:57.902569+0000","last_active":"2026-02-24T07:49:40.810230+0000","last_peered":"2026-02-24T07:49:40.810230+0000","last_clean":"2026-02-24T07:49:40.810230+0000","last_became_active":"2026-02-24T07:48:57.902313+0000","last_became_peered":"2026-02-24T07:48:57.902313+0000","last_unstale":"2026-02-24T07:49:40.810230+0000","last_undegraded":"2026-02-24T07:49:40.810230+0000","last_fullsized":"2026-02-24T07:49:40.810230+0000","mapping_epoch":55,"log_start":"59'19864","ondisk_log_start":"59'19864","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:48:56.890768+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:49:59.578048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1302003712,"num_objects":19867,"num_object_clones":0,"num_object_copies":59601,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19867,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19867,"num_write_kb":1271488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"59'19854","reported_seq":19872,"reported_epoch":59,"state":"active+clean","last_fresh":"2026-02-24T07:49:40.706494+0000","last_change":"2026-02-24T07:48:57.903920+0000","last_active":"2026-02-24T07:49:40.706494+0000","last_peered":"2026-02-24T07:49:40.706494+0000","last_clean":"2026-02-24T07:49:40.706494+0000","last_became_active":"2026-02-24T07:48:57.903584+0000","last_became_peered":"2026-02-24T07:48:57.903584+0000","last_unstale":"2026-02-24T07:49:40.706494+0000","last_undegraded":"2026-02-24T07:49:40.706494+0000","last_fullsized":"2026-02-24T07:49:40.706494+0000","mapping_epoch":55,"log_start":"59'19851","ondisk_log_start":"59'19851","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:48:56.890768+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:50:17.159571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1301151744,"num_objects":19854,"num_object_clones":0,"num_object_copies":59562,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19854,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19854,"num_write_kb":1270656,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.d","version":"59'19592","reported_seq":19610,"reported_epoch":59,"state":"active+clean","last_fresh":"2026-02-24T07:49:40.729965+0000","last_change":"2026-02-24T07:48:57.905520+0000","last_active":"2026-02-24T07:49:40.729965+0000","last_peered":"2026-02-24T07:49:40.729965+0000","last_clean":"2026-02-24T07:49:40.729965+0000","last_became_active":"2026-02-24T07:48:57.905031+0000","last_became_peered":"2026-02-24T07:48:57.905031+0000","last_unstale":"2026-02-24T07:49:40.729965+0000","last_undegraded":"2026-02-24T07:49:40.729965+0000","last_fullsized":"2026-02-24T07:49:40.729965+0000","mapping_epoch":55,"log_start":"59'19589","ondisk_log_start":"59'19589","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:48:56.890768+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:50:02.301170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1283981312,"num_objects":19592,"num_object_clones":0,"num_object_copies":58776,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19592,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19592,"num_write_kb":1253888,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"59'19606","reported_seq":19624,"reported_epoch":59,"state":"active+clean","last_fresh":"2026-02-24T07:49:40.731932+0000","last_change":"2026-02-24T07:48:57.905290+0000","last_active":"2026-02-24T07:49:40.731932+0000","last_peered":"2026-02-24T07:49:40.731932+0000","last_clean":"2026-02-24T07:49:40.731932+0000","last_became_active":"2026-02-24T07:48:57.904683+0000","last_became_peered":"2026-02-24T07:48:57.904683+0000","last_unstale":"2026-02-24T07:49:40.731932+0000","last_undegraded":"2026-02-24T07:49:40.731932+0000","last_fullsized":"2026-02-24T07:49:40.731932+0000","mapping_epoch":55,"log_start":"59'19603","ondisk_log_start":"59'19603","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:48:56.890768+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:50:16.643522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1284898816,"num_objects":19606,"num_object_clones":0,"num_object_copies":58818,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19606,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19606,"num_write_kb":1254784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.b","version":"59'19631","reported_seq":19649,"reported_epoch":59,"state":"active+clean","last_fresh":"2026-02-24T07:49:40.737697+0000","last_change":"2026-02-24T07:48:57.902025+0000","last_active":"2026-02-24T07:49:40.737697+0000","last_peered":"2026-02-24T07:49:40.737697+0000","last_clean":"2026-02-24T07:49:40.737697+0000","last_became_active":"2026-02-24T07:48:57.901700+0000","last_became_peered":"2026-02-24T07:48:57.901700+0000","last_unstale":"2026-02-24T07:49:40.737697+0000","last_undegraded":"2026-02-24T07:49:40.737697+0000","last_fullsized":"2026-02-24T07:49:40.737697+0000","mapping_epoch":55,"log_start":"59'19628","ondisk_log_start":"59'19628","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:48:56.890768+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:50:23.370319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1286537216,"num_objects":19631,"num_object_clones":0,"num_object_copies":58893,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19631,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19631,"num_write_kb":1256384,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.a","version":"59'19925","reported_seq":19941,"reported_epoch":59,"state":"active+clean","last_fresh":"2026-02-24T07:49:40.716206+0000","last_change":"2026-02-24T07:48:57.903698+0000","last_active":"2026-02-24T07:49:40.716206+0000","last_peered":"2026-02-24T07:49:40.716206+0000","last_clean":"2026-02-24T07:49:40.716206+0000","last_became_active":"2026-02-24T07:48:57.903228+0000","last_became_peered":"2026-02-24T07:48:57.903228+0000","last_unstale":"2026-02-24T07:49:40.716206+0000","last_undegraded":"2026-02-24T07:49:40.716206+0000","last_fullsized":"2026-02-24T07:49:40.716206+0000","mapping_epoch":55,"log_start":"59'19921","ondisk_log_start":"59'19921","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:48:56.890768+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":3000,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:50:26.780637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1305804800,"num_objects":19925,"num_object_clones":0,"num_object_copies":59775,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19925,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19925,"num_write_kb":1275200,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"59'19790","reported_seq":19808,"reported_epoch":59,"state":"active+clean","last_fresh":"2026-02-24T07:49:40.713440+0000","last_change":"2026-02-24T07:48:57.902011+0000","last_active":"2026-02-24T07:49:40.713440+0000","last_peered":"2026-02-24T07:49:40.713440+0000","last_clean":"2026-02-24T07:49:40.713440+0000","last_became_active":"2026-02-24T07:48:57.901730+0000","last_became_peered":"2026-02-24T07:48:57.901730+0000","last_unstale":"2026-02-24T07:49:40.713440+0000","last_undegraded":"2026-02-24T07:49:40.713440+0000","last_fullsized":"2026-02-24T07:49:40.713440+0000","mapping_epoch":55,"log_start":"59'19787","ondisk_log_start":"59'19787","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:48:56.890768+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:50:05.273318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1296957440,"num_objects":19790,"num_object_clones":0,"num_object_copies":59370,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19790,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19790,"num_write_kb":1266560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"59'19745","reported_seq":19763,"reported_epoch":59,"state":"active+clean","last_fresh":"2026-02-24T07:49:40.731372+0000","last_change":"2026-02-24T07:48:57.904924+0000","last_active":"2026-02-24T07:49:40.731372+0000","last_peered":"2026-02-24T07:49:40.731372+0000","last_clean":"2026-02-24T07:49:40.731372+0000","last_became_active":"2026-02-24T07:48:57.904376+0000","last_became_peered":"2026-02-24T07:48:57.904376+0000","last_unstale":"2026-02-24T07:49:40.731372+0000","last_undegraded":"2026-02-24T07:49:40.731372+0000","last_fullsized":"2026-02-24T07:49:40.731372+0000","mapping_epoch":55,"log_start":"59'19742","ondisk_log_start":"59'19742","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:48:56.890768+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:49:57.806612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1294008320,"num_objects":19745,"num_object_clones":0,"num_object_copies":59235,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19745,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19745,"num_write_kb":1263680,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.7","version":"59'19840","reported_seq":19858,"reported_epoch":59,"state":"active+clean","last_fresh":"2026-02-24T07:49:40.801382+0000","last_change":"2026-02-24T07:48:57.902698+0000","last_active":"2026-02-24T07:49:40.801382+0000","last_peered":"2026-02-24T07:49:40.801382+0000","last_clean":"2026-02-24T07:49:40.801382+0000","last_became_active":"2026-02-24T07:48:57.902484+0000","last_became_peered":"2026-02-24T07:48:57.902484+0000","last_unstale":"2026-02-24T07:49:40.801382+0000","last_undegraded":"2026-02-24T07:49:40.801382+0000","last_fullsized":"2026-02-24T07:49:40.801382+0000","mapping_epoch":55,"log_start":"59'19837","ondisk_log_start":"59'19837","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:48:56.890768+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:50:03.263698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1300234240,"num_objects":19840,"num_object_clones":0,"num_object_copies":59520,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19840,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19840,"num_write_kb":1269760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.6","version":"59'19481","reported_seq":19498,"reported_epoch":59,"state":"active+clean","last_fresh":"2026-02-24T07:49:40.731562+0000","last_change":"2026-02-24T07:48:57.901839+0000","last_active":"2026-02-24T07:49:40.731562+0000","last_peered":"2026-02-24T07:49:40.731562+0000","last_clean":"2026-02-24T07:49:40.731562+0000","last_became_active":"2026-02-24T07:48:57.901565+0000","last_became_peered":"2026-02-24T07:48:57.901565+0000","last_unstale":"2026-02-24T07:49:40.731562+0000","last_undegraded":"2026-02-24T07:49:40.731562+0000","last_fullsized":"2026-02-24T07:49:40.731562+0000","mapping_epoch":55,"log_start":"59'19478","ondisk_log_start":"59'19478","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:48:56.890768+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:50:09.631523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1276706816,"num_objects":19481,"num_object_clones":0,"num_object_copies":58443,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19481,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19481,"num_write_kb":1246784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"59'19941","reported_seq":19959,"reported_epoch":59,"state":"active+clean","last_fresh":"2026-02-24T07:49:40.807618+0000","last_change":"2026-02-24T07:48:57.902165+0000","last_active":"2026-02-24T07:49:40.807618+0000","last_peered":"2026-02-24T07:49:40.807618+0000","last_clean":"2026-02-24T07:49:40.807618+0000","last_became_active":"2026-02-24T07:48:57.901824+0000","last_became_peered":"2026-02-24T07:48:57.901824+0000","last_unstale":"2026-02-24T07:49:40.807618+0000","last_undegraded":"2026-02-24T07:49:40.807618+0000","last_fullsized":"2026-02-24T07:49:40.807618+0000","mapping_epoch":55,"log_start":"59'19938","ondisk_log_start":"59'19938","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:48:56.890768+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:50:12.091202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1306853376,"num_objects":19941,"num_object_clones":0,"num_object_copies":59823,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19941,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19941,"num_write_kb":1276224,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[1,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"59'19720","reported_seq":19738,"reported_epoch":59,"state":"active+clean","last_fresh":"2026-02-24T07:49:40.735441+0000","last_change":"2026-02-24T07:48:57.902537+0000","last_active":"2026-02-24T07:49:40.735441+0000","last_peered":"2026-02-24T07:49:40.735441+0000","last_clean":"2026-02-24T07:49:40.735441+0000","last_became_active":"2026-02-24T07:48:57.902224+0000","last_became_peered":"2026-02-24T07:48:57.902224+0000","last_unstale":"2026-02-24T07:49:40.735441+0000","last_undegraded":"2026-02-24T07:49:40.735441+0000","last_fullsized":"2026-02-24T07:49:40.735441+0000","mapping_epoch":55,"log_start":"59'19717","ondisk_log_start":"59'19717","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:48:56.890768+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:50:06.186992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1292369920,"num_objects":19720,"num_object_clones":0,"num_object_copies":59160,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19720,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19720,"num_write_kb":1262080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"59'19785","reported_seq":19803,"reported_epoch":59,"state":"active+clean","last_fresh":"2026-02-24T07:49:40.808510+0000","last_change":"2026-02-24T07:48:57.902384+0000","last_active":"2026-02-24T07:49:40.808510+0000","last_peered":"2026-02-24T07:49:40.808510+0000","last_clean":"2026-02-24T07:49:40.808510+0000","last_became_active":"2026-02-24T07:48:57.902125+0000","last_became_peered":"2026-02-24T07:48:57.902125+0000","last_unstale":"2026-02-24T07:49:40.808510+0000","last_undegraded":"2026-02-24T07:49:40.808510+0000","last_fullsized":"2026-02-24T07:49:40.808510+0000","mapping_epoch":55,"log_start":"59'19782","ondisk_log_start":"59'19782","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:48:56.890768+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:49:57.995484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1296629760,"num_objects":19785,"num_object_clones":0,"num_object_copies":59355,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19785,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19785,"num_write_kb":1266240,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1","version":"59'19802","reported_seq":19820,"reported_epoch":59,"state":"active+clean","last_fresh":"2026-02-24T07:49:40.709453+0000","last_change":"2026-02-24T07:48:57.903745+0000","last_active":"2026-02-24T07:49:40.709453+0000","last_peered":"2026-02-24T07:49:40.709453+0000","last_clean":"2026-02-24T07:49:40.709453+0000","last_became_active":"2026-02-24T07:48:57.903439+0000","last_became_peered":"2026-02-24T07:48:57.903439+0000","last_unstale":"2026-02-24T07:49:40.709453+0000","last_undegraded":"2026-02-24T07:49:40.709453+0000","last_fullsized":"2026-02-24T07:49:40.709453+0000","mapping_epoch":55,"log_start":"59'19799","ondisk_log_start":"59'19799","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:48:56.890768+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:50:24.058110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1297743872,"num_objects":19802,"num_object_clones":0,"num_object_copies":59406,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19802,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19802,"num_write_kb":1267328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.0","version":"59'19752","reported_seq":19768,"reported_epoch":59,"state":"active+clean","last_fresh":"2026-02-24T07:49:40.736692+0000","last_change":"2026-02-24T07:48:57.902838+0000","last_active":"2026-02-24T07:49:40.736692+0000","last_peered":"2026-02-24T07:49:40.736692+0000","last_clean":"2026-02-24T07:49:40.736692+0000","last_became_active":"2026-02-24T07:48:57.902565+0000","last_became_peered":"2026-02-24T07:48:57.902565+0000","last_unstale":"2026-02-24T07:49:40.736692+0000","last_undegraded":"2026-02-24T07:49:40.736692+0000","last_fullsized":"2026-02-24T07:49:40.736692+0000","mapping_epoch":55,"log_start":"59'19749","ondisk_log_start":"59'19749","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:48:56.890768+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:50:25.079534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1294467072,"num_objects":19752,"num_object_clones":0,"num_object_copies":59256,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19752,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19752,"num_write_kb":1264128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,3],"acting":[0,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.5","version":"59'19753","reported_seq":19771,"reported_epoch":59,"state":"active+clean","last_fresh":"2026-02-24T07:49:40.720547+0000","last_change":"2026-02-24T07:48:57.905643+0000","last_active":"2026-02-24T07:49:40.720547+0000","last_peered":"2026-02-24T07:49:40.720547+0000","last_clean":"2026-02-24T07:49:40.720547+0000","last_became_active":"2026-02-24T07:48:57.905127+0000","last_became_peered":"2026-02-24T07:48:57.905127+0000","last_unstale":"2026-02-24T07:49:40.720547+0000","last_undegraded":"2026-02-24T07:49:40.720547+0000","last_fullsized":"2026-02-24T07:49:40.720547+0000","mapping_epoch":55,"log_start":"59'19750","ondisk_log_start":"59'19750","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:48:56.890768+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:49:59.684568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1294532608,"num_objects":19753,"num_object_clones":0,"num_object_copies":59259,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19753,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19753,"num_write_kb":1264192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"29'99","reported_seq":281,"reported_epoch":59,"state":"active+clean","last_fresh":"2026-02-24T07:49:39.884150+0000","last_change":"2026-02-24T07:49:06.478572+0000","last_active":"2026-02-24T07:49:39.884150+0000","last_peered":"2026-02-24T07:49:39.884150+0000","last_clean":"2026-02-24T07:49:39.884150+0000","last_became_active":"2026-02-24T07:42:54.186719+0000","last_became_peered":"2026-02-24T07:42:54.186719+0000","last_unstale":"2026-02-24T07:49:39.884150+0000","last_undegraded":"2026-02-24T07:49:39.884150+0000","last_fullsized":"2026-02-24T07:49:39.884150+0000","mapping_epoch":27,"log_start":"29'96","ondisk_log_start":"29'96","created":20,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'99","last_scrub_stamp":"2026-02-24T07:49:06.478534+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:42:43.640776+0000","last_clean_scrub_stamp":"2026-02-24T07:49:06.478534+0000","objects_scrubbed":2,"log_size":3,"log_dups_size":96,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:50:26.893586+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":240,"num_write":135,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":16,"stat_sum":{"num_bytes":20714881024,"num_objects":316084,"num_object_clones":0,"num_object_copies":948252,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":316084,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":316084,"num_write_kb":20229376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":62140121088,"data_stored":62140121088,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":49,"ondisk_log_size":49,"up":48,"acting":48,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":240,"num_write":135,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":3,"up_from":25,"seq":107374182484,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":16191792,"kb_used_data":15059228,"kb_used_omap":10,"kb_used_meta":1132533,"kb_avail":717811408,"statfs":{"total":751619276800,"available":735038881792,"internally_reserved":0,"allocated":15420649472,"data_stored":15420251746,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":10649,"internal_metadata":1159714407},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[7,2],"upper_bound":4},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":19,"seq":81604378710,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":19052080,"kb_used_data":17601692,"kb_used_omap":10,"kb_used_meta":1450357,"kb_avail":714951120,"statfs":{"total":751619276800,"available":732109946880,"internally_reserved":0,"allocated":18024132608,"data_stored":18023734882,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":10700,"internal_metadata":1485166132},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542231,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":14197356,"kb_used_data":12636760,"kb_used_omap":11,"kb_used_meta":1560564,"kb_avail":719805844,"statfs":{"total":751619276800,"available":737081184256,"internally_reserved":0,"allocated":12940042240,"data_stored":12939648066,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12199,"internal_metadata":1598017625},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705754,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":16182064,"kb_used_data":15061084,"kb_used_omap":9,"kb_used_meta":1120950,"kb_avail":717821136,"statfs":{"total":751619276800,"available":735048843264,"internally_reserved":0,"allocated":15422550016,"data_stored":15422152290,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9230,"internal_metadata":1147853810},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":15516434432,"data_stored":15516434432,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12976324608,"data_stored":12976324608,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":18099798016,"data_stored":18099798016,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":15547564032,"data_stored":15547564032,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-24T07:49:42.896 INFO:tasks.cephadm.ceph_manager.ceph:recovered! 2026-02-24T07:49:42.896 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2026-02-24T07:49:42.896 INFO:tasks.thrashosds.thrasher:fixing pg num pool unique_pool_2 2026-02-24T07:49:42.896 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph pg dump --format=json 2026-02-24T07:49:42.954 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:49:42 trial035 ceph-mon[39284]: pgmap v235: 17 pgs: 17 active+clean; 19 GiB data, 63 GiB used, 2.7 TiB / 2.7 TiB avail; 780 MiB/s wr, 12.48k op/s 2026-02-24T07:49:42.954 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:49:42 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/2878273769' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 3, "weight": 1.0}]': finished 2026-02-24T07:49:42.954 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:49:42 trial035 ceph-mon[39284]: osdmap e61: 4 total, 4 up, 4 in 2026-02-24T07:49:42.954 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:49:42 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/2766267834' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2026-02-24T07:49:42.996 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 44 16 333591 333575 473.792 489.75 0.00130059 0.00210941 2026-02-24T07:49:43.037 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:49:43.161 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:49:43.159+0000 7f2bfc86e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:49:43.162 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:49:43.159+0000 7f2bfc86e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:49:43.301 INFO:teuthology.orchestra.run.trial035.stdout: 2026-02-24T07:49:43.302 INFO:teuthology.orchestra.run.trial035.stderr:dumped all 2026-02-24T07:49:43.349 INFO:teuthology.orchestra.run.trial035.stdout:{"pg_ready":true,"pg_map":{"version":236,"stamp":"2026-02-24T07:49:41.888072+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":20715471392,"num_objects":316086,"num_object_clones":0,"num_object_copies":948258,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":316086,"num_whiteouts":0,"num_read":136,"num_read_kb":240,"num_write":316219,"num_write_kb":20231526,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":52,"ondisk_log_size":52,"up":51,"acting":51,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":51,"num_osds":4,"num_per_pool_osds":4,"num_per_pool_omap_osds":4,"kb":2936012800,"kb_used":65623292,"kb_used_data":60358764,"kb_used_omap":41,"kb_used_meta":5264406,"kb_avail":2870389508,"statfs":{"total":3006477107200,"available":2939278856192,"internally_reserved":0,"allocated":61807374336,"data_stored":61805786984,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":42778,"internal_metadata":5390751974},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[7,2],"upper_bound":4},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2322923520,"num_objects":35445,"num_object_clones":0,"num_object_copies":106335,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35445,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":35445,"num_write_kb":2268480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.176978"},"pg_stats":[{"pgid":"4.f","version":"59'19867","reported_seq":19883,"reported_epoch":59,"state":"active+clean","last_fresh":"2026-02-24T07:49:40.810230+0000","last_change":"2026-02-24T07:48:57.902569+0000","last_active":"2026-02-24T07:49:40.810230+0000","last_peered":"2026-02-24T07:49:40.810230+0000","last_clean":"2026-02-24T07:49:40.810230+0000","last_became_active":"2026-02-24T07:48:57.902313+0000","last_became_peered":"2026-02-24T07:48:57.902313+0000","last_unstale":"2026-02-24T07:49:40.810230+0000","last_undegraded":"2026-02-24T07:49:40.810230+0000","last_fullsized":"2026-02-24T07:49:40.810230+0000","mapping_epoch":55,"log_start":"59'19864","ondisk_log_start":"59'19864","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:48:56.890768+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:49:59.578048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1302003712,"num_objects":19867,"num_object_clones":0,"num_object_copies":59601,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19867,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19867,"num_write_kb":1271488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"59'19854","reported_seq":19872,"reported_epoch":59,"state":"active+clean","last_fresh":"2026-02-24T07:49:40.706494+0000","last_change":"2026-02-24T07:48:57.903920+0000","last_active":"2026-02-24T07:49:40.706494+0000","last_peered":"2026-02-24T07:49:40.706494+0000","last_clean":"2026-02-24T07:49:40.706494+0000","last_became_active":"2026-02-24T07:48:57.903584+0000","last_became_peered":"2026-02-24T07:48:57.903584+0000","last_unstale":"2026-02-24T07:49:40.706494+0000","last_undegraded":"2026-02-24T07:49:40.706494+0000","last_fullsized":"2026-02-24T07:49:40.706494+0000","mapping_epoch":55,"log_start":"59'19851","ondisk_log_start":"59'19851","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:48:56.890768+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:50:17.159571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1301151744,"num_objects":19854,"num_object_clones":0,"num_object_copies":59562,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19854,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19854,"num_write_kb":1270656,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.d","version":"59'19592","reported_seq":19610,"reported_epoch":59,"state":"active+clean","last_fresh":"2026-02-24T07:49:40.729965+0000","last_change":"2026-02-24T07:48:57.905520+0000","last_active":"2026-02-24T07:49:40.729965+0000","last_peered":"2026-02-24T07:49:40.729965+0000","last_clean":"2026-02-24T07:49:40.729965+0000","last_became_active":"2026-02-24T07:48:57.905031+0000","last_became_peered":"2026-02-24T07:48:57.905031+0000","last_unstale":"2026-02-24T07:49:40.729965+0000","last_undegraded":"2026-02-24T07:49:40.729965+0000","last_fullsized":"2026-02-24T07:49:40.729965+0000","mapping_epoch":55,"log_start":"59'19589","ondisk_log_start":"59'19589","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:48:56.890768+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:50:02.301170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1283981312,"num_objects":19592,"num_object_clones":0,"num_object_copies":58776,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19592,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19592,"num_write_kb":1253888,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"59'19606","reported_seq":19624,"reported_epoch":59,"state":"active+clean","last_fresh":"2026-02-24T07:49:40.731932+0000","last_change":"2026-02-24T07:48:57.905290+0000","last_active":"2026-02-24T07:49:40.731932+0000","last_peered":"2026-02-24T07:49:40.731932+0000","last_clean":"2026-02-24T07:49:40.731932+0000","last_became_active":"2026-02-24T07:48:57.904683+0000","last_became_peered":"2026-02-24T07:48:57.904683+0000","last_unstale":"2026-02-24T07:49:40.731932+0000","last_undegraded":"2026-02-24T07:49:40.731932+0000","last_fullsized":"2026-02-24T07:49:40.731932+0000","mapping_epoch":55,"log_start":"59'19603","ondisk_log_start":"59'19603","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:48:56.890768+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:50:16.643522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1284898816,"num_objects":19606,"num_object_clones":0,"num_object_copies":58818,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19606,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19606,"num_write_kb":1254784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.b","version":"59'19631","reported_seq":19649,"reported_epoch":59,"state":"active+clean","last_fresh":"2026-02-24T07:49:40.737697+0000","last_change":"2026-02-24T07:48:57.902025+0000","last_active":"2026-02-24T07:49:40.737697+0000","last_peered":"2026-02-24T07:49:40.737697+0000","last_clean":"2026-02-24T07:49:40.737697+0000","last_became_active":"2026-02-24T07:48:57.901700+0000","last_became_peered":"2026-02-24T07:48:57.901700+0000","last_unstale":"2026-02-24T07:49:40.737697+0000","last_undegraded":"2026-02-24T07:49:40.737697+0000","last_fullsized":"2026-02-24T07:49:40.737697+0000","mapping_epoch":55,"log_start":"59'19628","ondisk_log_start":"59'19628","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:48:56.890768+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:50:23.370319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1286537216,"num_objects":19631,"num_object_clones":0,"num_object_copies":58893,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19631,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19631,"num_write_kb":1256384,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.a","version":"59'19925","reported_seq":19941,"reported_epoch":59,"state":"active+clean","last_fresh":"2026-02-24T07:49:40.716206+0000","last_change":"2026-02-24T07:48:57.903698+0000","last_active":"2026-02-24T07:49:40.716206+0000","last_peered":"2026-02-24T07:49:40.716206+0000","last_clean":"2026-02-24T07:49:40.716206+0000","last_became_active":"2026-02-24T07:48:57.903228+0000","last_became_peered":"2026-02-24T07:48:57.903228+0000","last_unstale":"2026-02-24T07:49:40.716206+0000","last_undegraded":"2026-02-24T07:49:40.716206+0000","last_fullsized":"2026-02-24T07:49:40.716206+0000","mapping_epoch":55,"log_start":"59'19921","ondisk_log_start":"59'19921","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:48:56.890768+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":3000,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:50:26.780637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1305804800,"num_objects":19925,"num_object_clones":0,"num_object_copies":59775,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19925,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19925,"num_write_kb":1275200,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"59'19790","reported_seq":19808,"reported_epoch":59,"state":"active+clean","last_fresh":"2026-02-24T07:49:40.713440+0000","last_change":"2026-02-24T07:48:57.902011+0000","last_active":"2026-02-24T07:49:40.713440+0000","last_peered":"2026-02-24T07:49:40.713440+0000","last_clean":"2026-02-24T07:49:40.713440+0000","last_became_active":"2026-02-24T07:48:57.901730+0000","last_became_peered":"2026-02-24T07:48:57.901730+0000","last_unstale":"2026-02-24T07:49:40.713440+0000","last_undegraded":"2026-02-24T07:49:40.713440+0000","last_fullsized":"2026-02-24T07:49:40.713440+0000","mapping_epoch":55,"log_start":"59'19787","ondisk_log_start":"59'19787","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:48:56.890768+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:50:05.273318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1296957440,"num_objects":19790,"num_object_clones":0,"num_object_copies":59370,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19790,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19790,"num_write_kb":1266560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"59'19745","reported_seq":19763,"reported_epoch":59,"state":"active+clean","last_fresh":"2026-02-24T07:49:40.731372+0000","last_change":"2026-02-24T07:48:57.904924+0000","last_active":"2026-02-24T07:49:40.731372+0000","last_peered":"2026-02-24T07:49:40.731372+0000","last_clean":"2026-02-24T07:49:40.731372+0000","last_became_active":"2026-02-24T07:48:57.904376+0000","last_became_peered":"2026-02-24T07:48:57.904376+0000","last_unstale":"2026-02-24T07:49:40.731372+0000","last_undegraded":"2026-02-24T07:49:40.731372+0000","last_fullsized":"2026-02-24T07:49:40.731372+0000","mapping_epoch":55,"log_start":"59'19742","ondisk_log_start":"59'19742","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:48:56.890768+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:49:57.806612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1294008320,"num_objects":19745,"num_object_clones":0,"num_object_copies":59235,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19745,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19745,"num_write_kb":1263680,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.7","version":"59'19840","reported_seq":19858,"reported_epoch":59,"state":"active+clean","last_fresh":"2026-02-24T07:49:40.801382+0000","last_change":"2026-02-24T07:48:57.902698+0000","last_active":"2026-02-24T07:49:40.801382+0000","last_peered":"2026-02-24T07:49:40.801382+0000","last_clean":"2026-02-24T07:49:40.801382+0000","last_became_active":"2026-02-24T07:48:57.902484+0000","last_became_peered":"2026-02-24T07:48:57.902484+0000","last_unstale":"2026-02-24T07:49:40.801382+0000","last_undegraded":"2026-02-24T07:49:40.801382+0000","last_fullsized":"2026-02-24T07:49:40.801382+0000","mapping_epoch":55,"log_start":"59'19837","ondisk_log_start":"59'19837","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:48:56.890768+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:50:03.263698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1300234240,"num_objects":19840,"num_object_clones":0,"num_object_copies":59520,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19840,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19840,"num_write_kb":1269760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.6","version":"59'19481","reported_seq":19498,"reported_epoch":59,"state":"active+clean","last_fresh":"2026-02-24T07:49:40.731562+0000","last_change":"2026-02-24T07:48:57.901839+0000","last_active":"2026-02-24T07:49:40.731562+0000","last_peered":"2026-02-24T07:49:40.731562+0000","last_clean":"2026-02-24T07:49:40.731562+0000","last_became_active":"2026-02-24T07:48:57.901565+0000","last_became_peered":"2026-02-24T07:48:57.901565+0000","last_unstale":"2026-02-24T07:49:40.731562+0000","last_undegraded":"2026-02-24T07:49:40.731562+0000","last_fullsized":"2026-02-24T07:49:40.731562+0000","mapping_epoch":55,"log_start":"59'19478","ondisk_log_start":"59'19478","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:48:56.890768+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:50:09.631523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1276706816,"num_objects":19481,"num_object_clones":0,"num_object_copies":58443,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19481,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19481,"num_write_kb":1246784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"59'19941","reported_seq":19959,"reported_epoch":59,"state":"active+clean","last_fresh":"2026-02-24T07:49:40.807618+0000","last_change":"2026-02-24T07:48:57.902165+0000","last_active":"2026-02-24T07:49:40.807618+0000","last_peered":"2026-02-24T07:49:40.807618+0000","last_clean":"2026-02-24T07:49:40.807618+0000","last_became_active":"2026-02-24T07:48:57.901824+0000","last_became_peered":"2026-02-24T07:48:57.901824+0000","last_unstale":"2026-02-24T07:49:40.807618+0000","last_undegraded":"2026-02-24T07:49:40.807618+0000","last_fullsized":"2026-02-24T07:49:40.807618+0000","mapping_epoch":55,"log_start":"59'19938","ondisk_log_start":"59'19938","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:48:56.890768+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:50:12.091202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1306853376,"num_objects":19941,"num_object_clones":0,"num_object_copies":59823,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19941,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19941,"num_write_kb":1276224,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[1,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"59'19720","reported_seq":19738,"reported_epoch":59,"state":"active+clean","last_fresh":"2026-02-24T07:49:40.735441+0000","last_change":"2026-02-24T07:48:57.902537+0000","last_active":"2026-02-24T07:49:40.735441+0000","last_peered":"2026-02-24T07:49:40.735441+0000","last_clean":"2026-02-24T07:49:40.735441+0000","last_became_active":"2026-02-24T07:48:57.902224+0000","last_became_peered":"2026-02-24T07:48:57.902224+0000","last_unstale":"2026-02-24T07:49:40.735441+0000","last_undegraded":"2026-02-24T07:49:40.735441+0000","last_fullsized":"2026-02-24T07:49:40.735441+0000","mapping_epoch":55,"log_start":"59'19717","ondisk_log_start":"59'19717","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:48:56.890768+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:50:06.186992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1292369920,"num_objects":19720,"num_object_clones":0,"num_object_copies":59160,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19720,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19720,"num_write_kb":1262080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"59'19785","reported_seq":19803,"reported_epoch":59,"state":"active+clean","last_fresh":"2026-02-24T07:49:40.808510+0000","last_change":"2026-02-24T07:48:57.902384+0000","last_active":"2026-02-24T07:49:40.808510+0000","last_peered":"2026-02-24T07:49:40.808510+0000","last_clean":"2026-02-24T07:49:40.808510+0000","last_became_active":"2026-02-24T07:48:57.902125+0000","last_became_peered":"2026-02-24T07:48:57.902125+0000","last_unstale":"2026-02-24T07:49:40.808510+0000","last_undegraded":"2026-02-24T07:49:40.808510+0000","last_fullsized":"2026-02-24T07:49:40.808510+0000","mapping_epoch":55,"log_start":"59'19782","ondisk_log_start":"59'19782","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:48:56.890768+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:49:57.995484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1296629760,"num_objects":19785,"num_object_clones":0,"num_object_copies":59355,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19785,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19785,"num_write_kb":1266240,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1","version":"59'19802","reported_seq":19820,"reported_epoch":59,"state":"active+clean","last_fresh":"2026-02-24T07:49:40.709453+0000","last_change":"2026-02-24T07:48:57.903745+0000","last_active":"2026-02-24T07:49:40.709453+0000","last_peered":"2026-02-24T07:49:40.709453+0000","last_clean":"2026-02-24T07:49:40.709453+0000","last_became_active":"2026-02-24T07:48:57.903439+0000","last_became_peered":"2026-02-24T07:48:57.903439+0000","last_unstale":"2026-02-24T07:49:40.709453+0000","last_undegraded":"2026-02-24T07:49:40.709453+0000","last_fullsized":"2026-02-24T07:49:40.709453+0000","mapping_epoch":55,"log_start":"59'19799","ondisk_log_start":"59'19799","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:48:56.890768+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:50:24.058110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1297743872,"num_objects":19802,"num_object_clones":0,"num_object_copies":59406,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19802,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19802,"num_write_kb":1267328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.0","version":"59'19752","reported_seq":19768,"reported_epoch":59,"state":"active+clean","last_fresh":"2026-02-24T07:49:40.736692+0000","last_change":"2026-02-24T07:48:57.902838+0000","last_active":"2026-02-24T07:49:40.736692+0000","last_peered":"2026-02-24T07:49:40.736692+0000","last_clean":"2026-02-24T07:49:40.736692+0000","last_became_active":"2026-02-24T07:48:57.902565+0000","last_became_peered":"2026-02-24T07:48:57.902565+0000","last_unstale":"2026-02-24T07:49:40.736692+0000","last_undegraded":"2026-02-24T07:49:40.736692+0000","last_fullsized":"2026-02-24T07:49:40.736692+0000","mapping_epoch":55,"log_start":"59'19749","ondisk_log_start":"59'19749","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:48:56.890768+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:50:25.079534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1294467072,"num_objects":19752,"num_object_clones":0,"num_object_copies":59256,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19752,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19752,"num_write_kb":1264128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,3],"acting":[0,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.5","version":"59'19753","reported_seq":19771,"reported_epoch":59,"state":"active+clean","last_fresh":"2026-02-24T07:49:40.720547+0000","last_change":"2026-02-24T07:48:57.905643+0000","last_active":"2026-02-24T07:49:40.720547+0000","last_peered":"2026-02-24T07:49:40.720547+0000","last_clean":"2026-02-24T07:49:40.720547+0000","last_became_active":"2026-02-24T07:48:57.905127+0000","last_became_peered":"2026-02-24T07:48:57.905127+0000","last_unstale":"2026-02-24T07:49:40.720547+0000","last_undegraded":"2026-02-24T07:49:40.720547+0000","last_fullsized":"2026-02-24T07:49:40.720547+0000","mapping_epoch":55,"log_start":"59'19750","ondisk_log_start":"59'19750","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:48:56.890768+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:49:59.684568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1294532608,"num_objects":19753,"num_object_clones":0,"num_object_copies":59259,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19753,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19753,"num_write_kb":1264192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"29'99","reported_seq":281,"reported_epoch":59,"state":"active+clean","last_fresh":"2026-02-24T07:49:39.884150+0000","last_change":"2026-02-24T07:49:06.478572+0000","last_active":"2026-02-24T07:49:39.884150+0000","last_peered":"2026-02-24T07:49:39.884150+0000","last_clean":"2026-02-24T07:49:39.884150+0000","last_became_active":"2026-02-24T07:42:54.186719+0000","last_became_peered":"2026-02-24T07:42:54.186719+0000","last_unstale":"2026-02-24T07:49:39.884150+0000","last_undegraded":"2026-02-24T07:49:39.884150+0000","last_fullsized":"2026-02-24T07:49:39.884150+0000","mapping_epoch":27,"log_start":"29'96","ondisk_log_start":"29'96","created":20,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'99","last_scrub_stamp":"2026-02-24T07:49:06.478534+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:42:43.640776+0000","last_clean_scrub_stamp":"2026-02-24T07:49:06.478534+0000","objects_scrubbed":2,"log_size":3,"log_dups_size":96,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:50:26.893586+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":240,"num_write":135,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":16,"stat_sum":{"num_bytes":20714881024,"num_objects":316084,"num_object_clones":0,"num_object_copies":948252,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":316084,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":316084,"num_write_kb":20229376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":62140121088,"data_stored":62140121088,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":49,"ondisk_log_size":49,"up":48,"acting":48,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":240,"num_write":135,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":3,"up_from":25,"seq":107374182484,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":16191792,"kb_used_data":15059228,"kb_used_omap":10,"kb_used_meta":1132533,"kb_avail":717811408,"statfs":{"total":751619276800,"available":735038881792,"internally_reserved":0,"allocated":15420649472,"data_stored":15420251746,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":10649,"internal_metadata":1159714407},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[7,2],"upper_bound":4},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":19,"seq":81604378710,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":19052080,"kb_used_data":17601692,"kb_used_omap":10,"kb_used_meta":1450357,"kb_avail":714951120,"statfs":{"total":751619276800,"available":732109946880,"internally_reserved":0,"allocated":18024132608,"data_stored":18023734882,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":10700,"internal_metadata":1485166132},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542231,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":14197356,"kb_used_data":12636760,"kb_used_omap":11,"kb_used_meta":1560564,"kb_avail":719805844,"statfs":{"total":751619276800,"available":737081184256,"internally_reserved":0,"allocated":12940042240,"data_stored":12939648066,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12199,"internal_metadata":1598017625},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705754,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":16182064,"kb_used_data":15061084,"kb_used_omap":9,"kb_used_meta":1120950,"kb_avail":717821136,"statfs":{"total":751619276800,"available":735048843264,"internally_reserved":0,"allocated":15422550016,"data_stored":15422152290,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9230,"internal_metadata":1147853810},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":15516434432,"data_stored":15516434432,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12976324608,"data_stored":12976324608,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":18099798016,"data_stored":18099798016,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":15547564032,"data_stored":15547564032,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-24T07:49:43.349 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph osd pool set unique_pool_2 pgp_num 26 2026-02-24T07:49:43.492 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:49:43.617 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:49:43.615+0000 7f7f8dc97640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:49:43.617 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:49:43.615+0000 7f7f8dc97640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:49:43.761 INFO:teuthology.orchestra.run.trial035.stderr:Error EPERM: crimson_allow_pg_split is false; pgp_num increase denied 2026-02-24T07:49:43.799 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-24T07:49:43.997 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 45 16 341812 341796 474.681 513.812 0.00106697 0.00210549 2026-02-24T07:49:44.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:49:43 trial035 ceph-mon[39284]: from='client.14931 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2026-02-24T07:49:44.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:49:43 trial035 ceph-mon[39284]: from='client.14935 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2026-02-24T07:49:44.205 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:49:43 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/3320383007' entity='client.admin' cmd={"prefix": "osd pool set", "pool": "unique_pool_2", "var": "pgp_num", "val": "26"} : dispatch 2026-02-24T07:49:44.469 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2026-02-24T07:49:44.623 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:49:44.751 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:49:44.749+0000 7fa70bfac640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:49:44.752 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:49:44.749+0000 7fa70bfac640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:49:44.767 INFO:teuthology.orchestra.run.trial035.stdout:osd.0: {} 2026-02-24T07:49:44.768 INFO:teuthology.orchestra.run.trial035.stderr:osd.0: osd_enable_op_tracker = '' 2026-02-24T07:49:44.781 INFO:teuthology.orchestra.run.trial035.stdout:osd.1: {} 2026-02-24T07:49:44.781 INFO:teuthology.orchestra.run.trial035.stderr:osd.1: osd_enable_op_tracker = '' 2026-02-24T07:49:44.795 INFO:teuthology.orchestra.run.trial035.stdout:osd.2: {} 2026-02-24T07:49:44.795 INFO:teuthology.orchestra.run.trial035.stderr:osd.2: osd_enable_op_tracker = '' 2026-02-24T07:49:44.810 INFO:teuthology.orchestra.run.trial035.stdout:osd.3: {} 2026-02-24T07:49:44.811 INFO:teuthology.orchestra.run.trial035.stderr:osd.3: osd_enable_op_tracker = '' 2026-02-24T07:49:44.998 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 46 16 349962 349946 475.429 509.375 0.00150025 0.00210217 2026-02-24T07:49:45.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:49:44 trial035 ceph-mon[39284]: from='client.14937 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2026-02-24T07:49:45.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:49:44 trial035 ceph-mon[39284]: pgmap v237: 17 pgs: 17 active+clean; 19 GiB data, 63 GiB used, 2.7 TiB / 2.7 TiB avail; 380 MiB/s wr, 6.08k op/s 2026-02-24T07:49:45.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:49:44 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/318794569' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2026-02-24T07:49:45.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:49:44 trial035 ceph-mon[39284]: from='mgr.14284 10.20.193.35:0/3681373222' entity='mgr.x' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2026-02-24T07:49:45.998 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 47 16 357955 357939 475.942 499.562 0.0018028 0.00209991 2026-02-24T07:49:46.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:49:45 trial035 ceph-mon[39284]: pgmap v238: 17 pgs: 17 active+clean; 19 GiB data, 63 GiB used, 2.7 TiB / 2.7 TiB avail; 369 MiB/s wr, 5.91k op/s 2026-02-24T07:49:46.855 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2026-02-24T07:49:46.998 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 48 15 366136 366121 476.68 511.375 0.00253007 0.00209665 2026-02-24T07:49:47.001 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:49:47.204 INFO:journalctl@ceph.mgr.x.trial035.stdout:Feb 24 07:49:46 trial035 ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-mgr-x[39621]: ::ffff:10.20.193.91 - - [24/Feb/2026:07:49:46] "GET /metrics HTTP/1.1" 200 4210 "" "Prometheus/3.6.0" 2026-02-24T07:49:47.264 INFO:journalctl@ceph.prometheus.a.trial091.stdout:Feb 24 07:49:46 trial091 ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-prometheus-a[66007]: time=2026-02-24T07:49:46.973Z level=ERROR source=scrape.go:1630 msg="Failed to determine correct type of scrape target." component="scrape manager" scrape_pool=ceph target=http://trial035:9283/metrics content_type="text/html;charset=utf-8" fallback_media_type="" err="received unsupported Content-Type \"text/html;charset=utf-8\" and no fallback_scrape_protocol specified for target" 2026-02-24T07:49:47.474 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:49:47.454+0000 7feb84f47640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:49:47.474 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:49:47.454+0000 7feb84f47640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:49:47.487 INFO:teuthology.orchestra.run.trial035.stdout:osd.0: {} 2026-02-24T07:49:47.488 INFO:teuthology.orchestra.run.trial035.stderr:osd.0: osd_enable_op_tracker = '' 2026-02-24T07:49:47.499 INFO:teuthology.orchestra.run.trial035.stdout:osd.1: {} 2026-02-24T07:49:47.500 INFO:teuthology.orchestra.run.trial035.stderr:osd.1: osd_enable_op_tracker = '' 2026-02-24T07:49:47.516 INFO:teuthology.orchestra.run.trial035.stdout:osd.2: {} 2026-02-24T07:49:47.517 INFO:teuthology.orchestra.run.trial035.stderr:osd.2: osd_enable_op_tracker = '' 2026-02-24T07:49:47.530 INFO:teuthology.orchestra.run.trial035.stdout:osd.3: {} 2026-02-24T07:49:47.530 INFO:teuthology.orchestra.run.trial035.stderr:osd.3: osd_enable_op_tracker = '' 2026-02-24T07:49:47.531 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:49:47 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/990145609' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2026-02-24T07:49:47.998 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 49 15 371785 371770 474.156 353.062 0.00268859 0.00210781 2026-02-24T07:49:48.800 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2026-02-24T07:49:48.801 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2026-02-24T07:49:48.801 INFO:tasks.thrashosds.thrasher:trim_stale_osdmaps 2026-02-24T07:49:48.801 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph tell osd.0 'trim stale osdmaps' 2026-02-24T07:49:48.825 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:49:48 trial035 ceph-mon[39284]: pgmap v239: 17 pgs: 17 active+clean; 22 GiB data, 70 GiB used, 2.7 TiB / 2.7 TiB avail; 601 MiB/s wr, 9.62k op/s 2026-02-24T07:49:48.940 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:49:48.998 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 50 16 379047 379031 473.749 453.812 0.00211987 0.00210964 2026-02-24T07:49:49.074 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:49:49.072+0000 7faac49c8640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:49:49.074 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:49:49.072+0000 7faac49c8640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:49:49.096 INFO:teuthology.orchestra.run.trial035.stderr:no valid command found; 10 closest matches: 2026-02-24T07:49:49.096 INFO:teuthology.orchestra.run.trial035.stderr:0 2026-02-24T07:49:49.096 INFO:teuthology.orchestra.run.trial035.stderr:1 2026-02-24T07:49:49.096 INFO:teuthology.orchestra.run.trial035.stderr:2 2026-02-24T07:49:49.096 INFO:teuthology.orchestra.run.trial035.stderr:abort 2026-02-24T07:49:49.097 INFO:teuthology.orchestra.run.trial035.stderr:assert 2026-02-24T07:49:49.097 INFO:teuthology.orchestra.run.trial035.stderr:bench [] [] [] [] 2026-02-24T07:49:49.097 INFO:teuthology.orchestra.run.trial035.stderr:bluefs debug_inject_read_zeros 2026-02-24T07:49:49.097 INFO:teuthology.orchestra.run.trial035.stderr:bluefs files list 2026-02-24T07:49:49.097 INFO:teuthology.orchestra.run.trial035.stderr:bluefs stats 2026-02-24T07:49:49.097 INFO:teuthology.orchestra.run.trial035.stderr:bluestore allocator dump block 2026-02-24T07:49:49.097 INFO:teuthology.orchestra.run.trial035.stderr:Error EINVAL: invalid command 2026-02-24T07:49:49.133 DEBUG:teuthology.orchestra.run:got remote process result: 22 2026-02-24T07:49:49.134 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2026-02-24T07:49:49.134 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph tell osd.1 'trim stale osdmaps' 2026-02-24T07:49:49.310 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:49:49.434 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:49:49.432+0000 7f3362dd0640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:49:49.435 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:49:49.432+0000 7f3362dd0640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:49:49.455 INFO:teuthology.orchestra.run.trial035.stderr:no valid command found; 10 closest matches: 2026-02-24T07:49:49.456 INFO:teuthology.orchestra.run.trial035.stderr:0 2026-02-24T07:49:49.456 INFO:teuthology.orchestra.run.trial035.stderr:1 2026-02-24T07:49:49.456 INFO:teuthology.orchestra.run.trial035.stderr:2 2026-02-24T07:49:49.456 INFO:teuthology.orchestra.run.trial035.stderr:abort 2026-02-24T07:49:49.456 INFO:teuthology.orchestra.run.trial035.stderr:assert 2026-02-24T07:49:49.456 INFO:teuthology.orchestra.run.trial035.stderr:bench [] [] [] [] 2026-02-24T07:49:49.456 INFO:teuthology.orchestra.run.trial035.stderr:bluefs debug_inject_read_zeros 2026-02-24T07:49:49.456 INFO:teuthology.orchestra.run.trial035.stderr:bluefs files list 2026-02-24T07:49:49.456 INFO:teuthology.orchestra.run.trial035.stderr:bluefs stats 2026-02-24T07:49:49.456 INFO:teuthology.orchestra.run.trial035.stderr:bluestore allocator dump block 2026-02-24T07:49:49.457 INFO:teuthology.orchestra.run.trial035.stderr:Error EINVAL: invalid command 2026-02-24T07:49:49.493 DEBUG:teuthology.orchestra.run:got remote process result: 22 2026-02-24T07:49:49.494 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2026-02-24T07:49:49.495 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph tell osd.2 'trim stale osdmaps' 2026-02-24T07:49:49.642 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:49:49.767 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:49:49.764+0000 7fd57e3a6640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:49:49.767 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:49:49.765+0000 7fd57e3a6640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:49:49.792 INFO:teuthology.orchestra.run.trial035.stderr:no valid command found; 10 closest matches: 2026-02-24T07:49:49.792 INFO:teuthology.orchestra.run.trial035.stderr:0 2026-02-24T07:49:49.792 INFO:teuthology.orchestra.run.trial035.stderr:1 2026-02-24T07:49:49.792 INFO:teuthology.orchestra.run.trial035.stderr:2 2026-02-24T07:49:49.792 INFO:teuthology.orchestra.run.trial035.stderr:abort 2026-02-24T07:49:49.792 INFO:teuthology.orchestra.run.trial035.stderr:assert 2026-02-24T07:49:49.793 INFO:teuthology.orchestra.run.trial035.stderr:bench [] [] [] [] 2026-02-24T07:49:49.793 INFO:teuthology.orchestra.run.trial035.stderr:bluefs debug_inject_read_zeros 2026-02-24T07:49:49.793 INFO:teuthology.orchestra.run.trial035.stderr:bluefs files list 2026-02-24T07:49:49.793 INFO:teuthology.orchestra.run.trial035.stderr:bluefs stats 2026-02-24T07:49:49.793 INFO:teuthology.orchestra.run.trial035.stderr:bluestore allocator dump block 2026-02-24T07:49:49.793 INFO:teuthology.orchestra.run.trial035.stderr:Error EINVAL: invalid command 2026-02-24T07:49:49.826 DEBUG:teuthology.orchestra.run:got remote process result: 22 2026-02-24T07:49:49.827 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2026-02-24T07:49:49.827 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph tell osd.3 'trim stale osdmaps' 2026-02-24T07:49:49.969 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:49:49.998 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 51 16 387114 387098 474.345 504.188 0.00163171 0.00210697 2026-02-24T07:49:50.094 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:49:50.092+0000 7fb534a6c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:49:50.095 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:49:50.092+0000 7fb534a6c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:49:50.116 INFO:teuthology.orchestra.run.trial035.stderr:no valid command found; 10 closest matches: 2026-02-24T07:49:50.116 INFO:teuthology.orchestra.run.trial035.stderr:0 2026-02-24T07:49:50.116 INFO:teuthology.orchestra.run.trial035.stderr:1 2026-02-24T07:49:50.117 INFO:teuthology.orchestra.run.trial035.stderr:2 2026-02-24T07:49:50.117 INFO:teuthology.orchestra.run.trial035.stderr:abort 2026-02-24T07:49:50.117 INFO:teuthology.orchestra.run.trial035.stderr:assert 2026-02-24T07:49:50.117 INFO:teuthology.orchestra.run.trial035.stderr:bench [] [] [] [] 2026-02-24T07:49:50.117 INFO:teuthology.orchestra.run.trial035.stderr:bluefs debug_inject_read_zeros 2026-02-24T07:49:50.117 INFO:teuthology.orchestra.run.trial035.stderr:bluefs files list 2026-02-24T07:49:50.117 INFO:teuthology.orchestra.run.trial035.stderr:bluefs stats 2026-02-24T07:49:50.117 INFO:teuthology.orchestra.run.trial035.stderr:bluestore allocator dump block 2026-02-24T07:49:50.117 INFO:teuthology.orchestra.run.trial035.stderr:Error EINVAL: invalid command 2026-02-24T07:49:50.152 DEBUG:teuthology.orchestra.run:got remote process result: 22 2026-02-24T07:49:50.153 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2026-02-24T07:49:50.212 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2026-02-24T07:49:50.349 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:49:50.463 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:49:50.461+0000 7fadf2017640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:49:50.463 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:49:50.461+0000 7fadf2017640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:49:50.480 INFO:teuthology.orchestra.run.trial035.stdout:osd.0: {} 2026-02-24T07:49:50.480 INFO:teuthology.orchestra.run.trial035.stderr:osd.0: osd_enable_op_tracker = '' 2026-02-24T07:49:50.493 INFO:teuthology.orchestra.run.trial035.stdout:osd.1: {} 2026-02-24T07:49:50.493 INFO:teuthology.orchestra.run.trial035.stderr:osd.1: osd_enable_op_tracker = '' 2026-02-24T07:49:50.507 INFO:teuthology.orchestra.run.trial035.stdout:osd.2: {} 2026-02-24T07:49:50.507 INFO:teuthology.orchestra.run.trial035.stderr:osd.2: osd_enable_op_tracker = '' 2026-02-24T07:49:50.520 INFO:teuthology.orchestra.run.trial035.stdout:osd.3: {} 2026-02-24T07:49:50.520 INFO:teuthology.orchestra.run.trial035.stderr:osd.3: osd_enable_op_tracker = '' 2026-02-24T07:49:50.998 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 52 16 395032 395016 474.739 494.875 0.00211779 0.00210523 2026-02-24T07:49:51.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:49:50 trial035 ceph-mon[39284]: pgmap v240: 17 pgs: 17 active+clean; 22 GiB data, 70 GiB used, 2.7 TiB / 2.7 TiB avail; 282 MiB/s wr, 4.51k op/s 2026-02-24T07:49:51.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:49:50 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/3147310111' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2026-02-24T07:49:51.998 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 53 16 402744 402728 474.875 482 0.0015749 0.00210463 2026-02-24T07:49:52.565 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2026-02-24T07:49:52.717 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:49:52.848 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:49:52 trial035 ceph-mon[39284]: pgmap v241: 17 pgs: 17 active+clean; 24 GiB data, 77 GiB used, 2.7 TiB / 2.7 TiB avail; 481 MiB/s wr, 7.70k op/s 2026-02-24T07:49:52.849 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:49:52.846+0000 7fbbd9291640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:49:52.849 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:49:52.846+0000 7fbbd9291640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:49:52.864 INFO:teuthology.orchestra.run.trial035.stdout:osd.0: {} 2026-02-24T07:49:52.865 INFO:teuthology.orchestra.run.trial035.stderr:osd.0: osd_enable_op_tracker = '' 2026-02-24T07:49:52.879 INFO:teuthology.orchestra.run.trial035.stdout:osd.1: {} 2026-02-24T07:49:52.879 INFO:teuthology.orchestra.run.trial035.stderr:osd.1: osd_enable_op_tracker = '' 2026-02-24T07:49:52.890 INFO:teuthology.orchestra.run.trial035.stdout:osd.2: {} 2026-02-24T07:49:52.890 INFO:teuthology.orchestra.run.trial035.stderr:osd.2: osd_enable_op_tracker = '' 2026-02-24T07:49:52.902 INFO:teuthology.orchestra.run.trial035.stdout:osd.3: {} 2026-02-24T07:49:52.902 INFO:teuthology.orchestra.run.trial035.stderr:osd.3: osd_enable_op_tracker = '' 2026-02-24T07:49:52.998 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 54 16 410746 410730 475.342 500.125 0.00196192 0.00210256 2026-02-24T07:49:53.998 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 55 15 419168 419153 476.27 526.438 0.00200062 0.00209845 2026-02-24T07:49:54.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:49:53 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/940349849' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2026-02-24T07:49:54.945 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2026-02-24T07:49:54.998 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 56 15 427482 427467 477.044 519.625 0.00161013 0.00209505 2026-02-24T07:49:55.087 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:49:55.112 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:49:54 trial035 ceph-mon[39284]: pgmap v242: 17 pgs: 17 active+clean; 24 GiB data, 77 GiB used, 2.7 TiB / 2.7 TiB avail; 407 MiB/s wr, 6.51k op/s 2026-02-24T07:49:55.153 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2026-02-24T07:49:55.153 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2026-02-24T07:49:55.154 INFO:tasks.thrashosds.thrasher:fixing pg num pool .mgr 2026-02-24T07:49:55.154 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph pg dump --format=json 2026-02-24T07:49:55.210 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:49:55.208+0000 7f6791abb640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:49:55.210 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:49:55.208+0000 7f6791abb640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:49:55.225 INFO:teuthology.orchestra.run.trial035.stdout:osd.0: {} 2026-02-24T07:49:55.225 INFO:teuthology.orchestra.run.trial035.stderr:osd.0: osd_enable_op_tracker = '' 2026-02-24T07:49:55.238 INFO:teuthology.orchestra.run.trial035.stdout:osd.1: {} 2026-02-24T07:49:55.238 INFO:teuthology.orchestra.run.trial035.stderr:osd.1: osd_enable_op_tracker = '' 2026-02-24T07:49:55.251 INFO:teuthology.orchestra.run.trial035.stdout:osd.2: {} 2026-02-24T07:49:55.252 INFO:teuthology.orchestra.run.trial035.stderr:osd.2: osd_enable_op_tracker = '' 2026-02-24T07:49:55.265 INFO:teuthology.orchestra.run.trial035.stdout:osd.3: {} 2026-02-24T07:49:55.265 INFO:teuthology.orchestra.run.trial035.stderr:osd.3: osd_enable_op_tracker = '' 2026-02-24T07:49:55.305 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:49:55.437 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:49:55.435+0000 7f1da20bf640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:49:55.438 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:49:55.435+0000 7f1da20bf640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:49:55.584 INFO:teuthology.orchestra.run.trial035.stdout: 2026-02-24T07:49:55.585 INFO:teuthology.orchestra.run.trial035.stderr:dumped all 2026-02-24T07:49:55.629 INFO:teuthology.orchestra.run.trial035.stdout:{"pg_ready":true,"pg_map":{"version":242,"stamp":"2026-02-24T07:49:53.713215+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":25759384096,"num_objects":393050,"num_object_clones":0,"num_object_copies":1179150,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":393050,"num_whiteouts":0,"num_read":136,"num_read_kb":240,"num_write":393183,"num_write_kb":25157222,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":51,"ondisk_log_size":51,"up":51,"acting":51,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":51,"num_osds":4,"num_per_pool_osds":4,"num_per_pool_omap_osds":4,"kb":2936012800,"kb_used":81161692,"kb_used_data":75048844,"kb_used_omap":47,"kb_used_meta":6112656,"kb_avail":2854851108,"statfs":{"total":3006477107200,"available":2923367534592,"internally_reserved":0,"allocated":76850016256,"data_stored":76848365872,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":48646,"internal_metadata":6259360250},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[7,1,2],"upper_bound":8},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":5043912704,"num_objects":76964,"num_object_clones":0,"num_object_copies":230892,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":76964,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":76964,"num_write_kb":4925696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"11.825143"},"pg_stats":[{"pgid":"4.f","version":"61'24776","reported_seq":24798,"reported_epoch":61,"state":"active+clean","last_fresh":"2026-02-24T07:49:50.807068+0000","last_change":"2026-02-24T07:48:57.902569+0000","last_active":"2026-02-24T07:49:50.807068+0000","last_peered":"2026-02-24T07:49:50.807068+0000","last_clean":"2026-02-24T07:49:50.807068+0000","last_became_active":"2026-02-24T07:48:57.902313+0000","last_became_peered":"2026-02-24T07:48:57.902313+0000","last_unstale":"2026-02-24T07:49:50.807068+0000","last_undegraded":"2026-02-24T07:49:50.807068+0000","last_fullsized":"2026-02-24T07:49:50.807068+0000","mapping_epoch":55,"log_start":"61'24773","ondisk_log_start":"61'24773","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:48:56.890768+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:49:59.578048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1623719936,"num_objects":24776,"num_object_clones":0,"num_object_copies":74328,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24776,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24776,"num_write_kb":1585664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"61'24689","reported_seq":24710,"reported_epoch":61,"state":"active+clean","last_fresh":"2026-02-24T07:49:50.716531+0000","last_change":"2026-02-24T07:48:57.903920+0000","last_active":"2026-02-24T07:49:50.716531+0000","last_peered":"2026-02-24T07:49:50.716531+0000","last_clean":"2026-02-24T07:49:50.716531+0000","last_became_active":"2026-02-24T07:48:57.903584+0000","last_became_peered":"2026-02-24T07:48:57.903584+0000","last_unstale":"2026-02-24T07:49:50.716531+0000","last_undegraded":"2026-02-24T07:49:50.716531+0000","last_fullsized":"2026-02-24T07:49:50.716531+0000","mapping_epoch":55,"log_start":"61'24686","ondisk_log_start":"61'24686","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:48:56.890768+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:50:17.159571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1618018304,"num_objects":24689,"num_object_clones":0,"num_object_copies":74067,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24689,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24689,"num_write_kb":1580096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.d","version":"61'24490","reported_seq":24512,"reported_epoch":61,"state":"active+clean","last_fresh":"2026-02-24T07:49:50.731044+0000","last_change":"2026-02-24T07:48:57.905520+0000","last_active":"2026-02-24T07:49:50.731044+0000","last_peered":"2026-02-24T07:49:50.731044+0000","last_clean":"2026-02-24T07:49:50.731044+0000","last_became_active":"2026-02-24T07:48:57.905031+0000","last_became_peered":"2026-02-24T07:48:57.905031+0000","last_unstale":"2026-02-24T07:49:50.731044+0000","last_undegraded":"2026-02-24T07:49:50.731044+0000","last_fullsized":"2026-02-24T07:49:50.731044+0000","mapping_epoch":55,"log_start":"61'24487","ondisk_log_start":"61'24487","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:48:56.890768+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:50:02.301170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1604976640,"num_objects":24490,"num_object_clones":0,"num_object_copies":73470,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24490,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24490,"num_write_kb":1567360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"61'24332","reported_seq":24354,"reported_epoch":61,"state":"active+clean","last_fresh":"2026-02-24T07:49:50.729797+0000","last_change":"2026-02-24T07:48:57.905290+0000","last_active":"2026-02-24T07:49:50.729797+0000","last_peered":"2026-02-24T07:49:50.729797+0000","last_clean":"2026-02-24T07:49:50.729797+0000","last_became_active":"2026-02-24T07:48:57.904683+0000","last_became_peered":"2026-02-24T07:48:57.904683+0000","last_unstale":"2026-02-24T07:49:50.729797+0000","last_undegraded":"2026-02-24T07:49:50.729797+0000","last_fullsized":"2026-02-24T07:49:50.729797+0000","mapping_epoch":55,"log_start":"61'24329","ondisk_log_start":"61'24329","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:48:56.890768+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:50:16.643522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1594621952,"num_objects":24332,"num_object_clones":0,"num_object_copies":72996,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24332,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24332,"num_write_kb":1557248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.b","version":"61'24318","reported_seq":24340,"reported_epoch":61,"state":"active+clean","last_fresh":"2026-02-24T07:49:50.736002+0000","last_change":"2026-02-24T07:48:57.902025+0000","last_active":"2026-02-24T07:49:50.736002+0000","last_peered":"2026-02-24T07:49:50.736002+0000","last_clean":"2026-02-24T07:49:50.736002+0000","last_became_active":"2026-02-24T07:48:57.901700+0000","last_became_peered":"2026-02-24T07:48:57.901700+0000","last_unstale":"2026-02-24T07:49:50.736002+0000","last_undegraded":"2026-02-24T07:49:50.736002+0000","last_fullsized":"2026-02-24T07:49:50.736002+0000","mapping_epoch":55,"log_start":"61'24315","ondisk_log_start":"61'24315","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:48:56.890768+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:50:23.370319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1593704448,"num_objects":24318,"num_object_clones":0,"num_object_copies":72954,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24318,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24318,"num_write_kb":1556352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.a","version":"61'24764","reported_seq":24786,"reported_epoch":61,"state":"active+clean","last_fresh":"2026-02-24T07:49:50.716606+0000","last_change":"2026-02-24T07:48:57.903698+0000","last_active":"2026-02-24T07:49:50.716606+0000","last_peered":"2026-02-24T07:49:50.716606+0000","last_clean":"2026-02-24T07:49:50.716606+0000","last_became_active":"2026-02-24T07:48:57.903228+0000","last_became_peered":"2026-02-24T07:48:57.903228+0000","last_unstale":"2026-02-24T07:49:50.716606+0000","last_undegraded":"2026-02-24T07:49:50.716606+0000","last_fullsized":"2026-02-24T07:49:50.716606+0000","mapping_epoch":55,"log_start":"61'24761","ondisk_log_start":"61'24761","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:48:56.890768+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:50:26.780637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1622933504,"num_objects":24764,"num_object_clones":0,"num_object_copies":74292,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24764,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24764,"num_write_kb":1584896,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"61'24685","reported_seq":24707,"reported_epoch":61,"state":"active+clean","last_fresh":"2026-02-24T07:49:50.717242+0000","last_change":"2026-02-24T07:48:57.902011+0000","last_active":"2026-02-24T07:49:50.717242+0000","last_peered":"2026-02-24T07:49:50.717242+0000","last_clean":"2026-02-24T07:49:50.717242+0000","last_became_active":"2026-02-24T07:48:57.901730+0000","last_became_peered":"2026-02-24T07:48:57.901730+0000","last_unstale":"2026-02-24T07:49:50.717242+0000","last_undegraded":"2026-02-24T07:49:50.717242+0000","last_fullsized":"2026-02-24T07:49:50.717242+0000","mapping_epoch":55,"log_start":"61'24682","ondisk_log_start":"61'24682","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:48:56.890768+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:50:05.273318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1617756160,"num_objects":24685,"num_object_clones":0,"num_object_copies":74055,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24685,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24685,"num_write_kb":1579840,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"61'24510","reported_seq":24532,"reported_epoch":61,"state":"active+clean","last_fresh":"2026-02-24T07:49:50.732107+0000","last_change":"2026-02-24T07:48:57.904924+0000","last_active":"2026-02-24T07:49:50.732107+0000","last_peered":"2026-02-24T07:49:50.732107+0000","last_clean":"2026-02-24T07:49:50.732107+0000","last_became_active":"2026-02-24T07:48:57.904376+0000","last_became_peered":"2026-02-24T07:48:57.904376+0000","last_unstale":"2026-02-24T07:49:50.732107+0000","last_undegraded":"2026-02-24T07:49:50.732107+0000","last_fullsized":"2026-02-24T07:49:50.732107+0000","mapping_epoch":55,"log_start":"61'24507","ondisk_log_start":"61'24507","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:48:56.890768+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:49:57.806612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1606287360,"num_objects":24510,"num_object_clones":0,"num_object_copies":73530,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24510,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24510,"num_write_kb":1568640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.7","version":"61'24650","reported_seq":24672,"reported_epoch":61,"state":"active+clean","last_fresh":"2026-02-24T07:49:50.810087+0000","last_change":"2026-02-24T07:48:57.902698+0000","last_active":"2026-02-24T07:49:50.810087+0000","last_peered":"2026-02-24T07:49:50.810087+0000","last_clean":"2026-02-24T07:49:50.810087+0000","last_became_active":"2026-02-24T07:48:57.902484+0000","last_became_peered":"2026-02-24T07:48:57.902484+0000","last_unstale":"2026-02-24T07:49:50.810087+0000","last_undegraded":"2026-02-24T07:49:50.810087+0000","last_fullsized":"2026-02-24T07:49:50.810087+0000","mapping_epoch":55,"log_start":"61'24647","ondisk_log_start":"61'24647","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:48:56.890768+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:50:03.263698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1615462400,"num_objects":24650,"num_object_clones":0,"num_object_copies":73950,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24650,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24650,"num_write_kb":1577600,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.6","version":"61'24386","reported_seq":24408,"reported_epoch":61,"state":"active+clean","last_fresh":"2026-02-24T07:49:50.733497+0000","last_change":"2026-02-24T07:48:57.901839+0000","last_active":"2026-02-24T07:49:50.733497+0000","last_peered":"2026-02-24T07:49:50.733497+0000","last_clean":"2026-02-24T07:49:50.733497+0000","last_became_active":"2026-02-24T07:48:57.901565+0000","last_became_peered":"2026-02-24T07:48:57.901565+0000","last_unstale":"2026-02-24T07:49:50.733497+0000","last_undegraded":"2026-02-24T07:49:50.733497+0000","last_fullsized":"2026-02-24T07:49:50.733497+0000","mapping_epoch":55,"log_start":"61'24383","ondisk_log_start":"61'24383","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:48:56.890768+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:50:09.631523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1598160896,"num_objects":24386,"num_object_clones":0,"num_object_copies":73158,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24386,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24386,"num_write_kb":1560704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"61'24804","reported_seq":24826,"reported_epoch":61,"state":"active+clean","last_fresh":"2026-02-24T07:49:50.809180+0000","last_change":"2026-02-24T07:48:57.902165+0000","last_active":"2026-02-24T07:49:50.809180+0000","last_peered":"2026-02-24T07:49:50.809180+0000","last_clean":"2026-02-24T07:49:50.809180+0000","last_became_active":"2026-02-24T07:48:57.901824+0000","last_became_peered":"2026-02-24T07:48:57.901824+0000","last_unstale":"2026-02-24T07:49:50.809180+0000","last_undegraded":"2026-02-24T07:49:50.809180+0000","last_fullsized":"2026-02-24T07:49:50.809180+0000","mapping_epoch":55,"log_start":"61'24801","ondisk_log_start":"61'24801","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:48:56.890768+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:50:12.091202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1625554944,"num_objects":24804,"num_object_clones":0,"num_object_copies":74412,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24804,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24804,"num_write_kb":1587456,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[1,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"61'24449","reported_seq":24471,"reported_epoch":61,"state":"active+clean","last_fresh":"2026-02-24T07:49:50.732133+0000","last_change":"2026-02-24T07:48:57.902537+0000","last_active":"2026-02-24T07:49:50.732133+0000","last_peered":"2026-02-24T07:49:50.732133+0000","last_clean":"2026-02-24T07:49:50.732133+0000","last_became_active":"2026-02-24T07:48:57.902224+0000","last_became_peered":"2026-02-24T07:48:57.902224+0000","last_unstale":"2026-02-24T07:49:50.732133+0000","last_undegraded":"2026-02-24T07:49:50.732133+0000","last_fullsized":"2026-02-24T07:49:50.732133+0000","mapping_epoch":55,"log_start":"61'24446","ondisk_log_start":"61'24446","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:48:56.890768+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:50:06.186992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1602289664,"num_objects":24449,"num_object_clones":0,"num_object_copies":73347,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24449,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24449,"num_write_kb":1564736,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"61'24565","reported_seq":24587,"reported_epoch":61,"state":"active+clean","last_fresh":"2026-02-24T07:49:50.808009+0000","last_change":"2026-02-24T07:48:57.902384+0000","last_active":"2026-02-24T07:49:50.808009+0000","last_peered":"2026-02-24T07:49:50.808009+0000","last_clean":"2026-02-24T07:49:50.808009+0000","last_became_active":"2026-02-24T07:48:57.902125+0000","last_became_peered":"2026-02-24T07:48:57.902125+0000","last_unstale":"2026-02-24T07:49:50.808009+0000","last_undegraded":"2026-02-24T07:49:50.808009+0000","last_fullsized":"2026-02-24T07:49:50.808009+0000","mapping_epoch":55,"log_start":"61'24562","ondisk_log_start":"61'24562","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:48:56.890768+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:49:57.995484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1609891840,"num_objects":24565,"num_object_clones":0,"num_object_copies":73695,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24565,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24565,"num_write_kb":1572160,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1","version":"61'24623","reported_seq":24645,"reported_epoch":61,"state":"active+clean","last_fresh":"2026-02-24T07:49:50.714479+0000","last_change":"2026-02-24T07:48:57.903745+0000","last_active":"2026-02-24T07:49:50.714479+0000","last_peered":"2026-02-24T07:49:50.714479+0000","last_clean":"2026-02-24T07:49:50.714479+0000","last_became_active":"2026-02-24T07:48:57.903439+0000","last_became_peered":"2026-02-24T07:48:57.903439+0000","last_unstale":"2026-02-24T07:49:50.714479+0000","last_undegraded":"2026-02-24T07:49:50.714479+0000","last_fullsized":"2026-02-24T07:49:50.714479+0000","mapping_epoch":55,"log_start":"61'24620","ondisk_log_start":"61'24620","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:48:56.890768+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:50:24.058110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1613692928,"num_objects":24623,"num_object_clones":0,"num_object_copies":73869,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24623,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24623,"num_write_kb":1575872,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.0","version":"61'24501","reported_seq":24523,"reported_epoch":61,"state":"active+clean","last_fresh":"2026-02-24T07:49:50.735320+0000","last_change":"2026-02-24T07:48:57.902838+0000","last_active":"2026-02-24T07:49:50.735320+0000","last_peered":"2026-02-24T07:49:50.735320+0000","last_clean":"2026-02-24T07:49:50.735320+0000","last_became_active":"2026-02-24T07:48:57.902565+0000","last_became_peered":"2026-02-24T07:48:57.902565+0000","last_unstale":"2026-02-24T07:49:50.735320+0000","last_undegraded":"2026-02-24T07:49:50.735320+0000","last_fullsized":"2026-02-24T07:49:50.735320+0000","mapping_epoch":55,"log_start":"61'24498","ondisk_log_start":"61'24498","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:48:56.890768+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:50:25.079534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1605697536,"num_objects":24501,"num_object_clones":0,"num_object_copies":73503,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24501,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24501,"num_write_kb":1568064,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,3],"acting":[0,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.5","version":"61'24506","reported_seq":24528,"reported_epoch":61,"state":"active+clean","last_fresh":"2026-02-24T07:49:50.731811+0000","last_change":"2026-02-24T07:48:57.905643+0000","last_active":"2026-02-24T07:49:50.731811+0000","last_peered":"2026-02-24T07:49:50.731811+0000","last_clean":"2026-02-24T07:49:50.731811+0000","last_became_active":"2026-02-24T07:48:57.905127+0000","last_became_peered":"2026-02-24T07:48:57.905127+0000","last_unstale":"2026-02-24T07:49:50.731811+0000","last_undegraded":"2026-02-24T07:49:50.731811+0000","last_fullsized":"2026-02-24T07:49:50.731811+0000","mapping_epoch":55,"log_start":"61'24503","ondisk_log_start":"61'24503","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:48:56.890768+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:49:59.684568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1606025216,"num_objects":24506,"num_object_clones":0,"num_object_copies":73518,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24506,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24506,"num_write_kb":1568384,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"29'99","reported_seq":285,"reported_epoch":61,"state":"active+clean","last_fresh":"2026-02-24T07:49:41.888452+0000","last_change":"2026-02-24T07:49:06.478572+0000","last_active":"2026-02-24T07:49:41.888452+0000","last_peered":"2026-02-24T07:49:41.888452+0000","last_clean":"2026-02-24T07:49:41.888452+0000","last_became_active":"2026-02-24T07:42:54.186719+0000","last_became_peered":"2026-02-24T07:42:54.186719+0000","last_unstale":"2026-02-24T07:49:41.888452+0000","last_undegraded":"2026-02-24T07:49:41.888452+0000","last_fullsized":"2026-02-24T07:49:41.888452+0000","mapping_epoch":27,"log_start":"29'96","ondisk_log_start":"29'96","created":20,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'99","last_scrub_stamp":"2026-02-24T07:49:06.478534+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:42:43.640776+0000","last_clean_scrub_stamp":"2026-02-24T07:49:06.478534+0000","objects_scrubbed":2,"log_size":3,"log_dups_size":96,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:50:26.893586+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":240,"num_write":135,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":16,"stat_sum":{"num_bytes":25758793728,"num_objects":393048,"num_object_clones":0,"num_object_copies":1179144,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":393048,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":393048,"num_write_kb":25155072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":77276512256,"data_stored":77276512256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":48,"ondisk_log_size":48,"up":48,"acting":48,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":240,"num_write":135,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":3,"up_from":25,"seq":107374182486,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":20078536,"kb_used_data":18729396,"kb_used_omap":11,"kb_used_meta":1349108,"kb_avail":713924664,"statfs":{"total":751619276800,"available":731058855936,"internally_reserved":0,"allocated":19178901504,"data_stored":19178488020,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":11953,"internal_metadata":1381486927},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[4,0,2],"upper_bound":8},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":19,"seq":81604378712,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":23613576,"kb_used_data":21910516,"kb_used_omap":11,"kb_used_meta":1703028,"kb_avail":710389624,"statfs":{"total":751619276800,"available":727438974976,"internally_reserved":0,"allocated":22436368384,"data_stored":22435954900,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12004,"internal_metadata":1743900956},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542233,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":17433092,"kb_used_data":15673904,"kb_used_omap":13,"kb_used_meta":1759090,"kb_avail":716570108,"statfs":{"total":751619276800,"available":733767790592,"internally_reserved":0,"allocated":16050077696,"data_stored":16049667764,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13503,"internal_metadata":1801308993},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705756,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":20036488,"kb_used_data":18735028,"kb_used_omap":10,"kb_used_meta":1301429,"kb_avail":713966712,"statfs":{"total":751619276800,"available":731101913088,"internally_reserved":0,"allocated":19184668672,"data_stored":19184255188,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":11186,"internal_metadata":1332663374},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,1],"upper_bound":4},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":19295961088,"data_stored":19295961088,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16137912320,"data_stored":16137912320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":22513451008,"data_stored":22513451008,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":19329187840,"data_stored":19329187840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-24T07:49:55.630 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph osd pool set .mgr pgp_num 231 2026-02-24T07:49:55.775 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:49:55.895 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:49:55 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/3373496804' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2026-02-24T07:49:55.913 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:49:55.911+0000 7efe6bfeb640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:49:55.914 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:49:55.911+0000 7efe6bfeb640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:49:55.998 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 57 15 435446 435431 477.406 497.75 0.00150504 0.00209346 2026-02-24T07:49:56.055 INFO:teuthology.orchestra.run.trial035.stderr:Error EPERM: crimson_allow_pg_split is false; pgp_num increase denied 2026-02-24T07:49:56.095 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-24T07:49:56.998 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 58 16 443504 443488 477.857 503.562 0.0015278 0.0020915 2026-02-24T07:49:57.106 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:49:56 trial035 ceph-mon[39284]: from='client.14959 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2026-02-24T07:49:57.106 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:49:56 trial035 ceph-mon[39284]: pgmap v243: 17 pgs: 17 active+clean; 24 GiB data, 77 GiB used, 2.7 TiB / 2.7 TiB avail; 401 MiB/s wr, 6.41k op/s 2026-02-24T07:49:57.106 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:49:56 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/3735442556' entity='client.admin' cmd={"prefix": "osd pool set", "pool": ".mgr", "var": "pgp_num", "val": "231"} : dispatch 2026-02-24T07:49:57.106 INFO:journalctl@ceph.mgr.x.trial035.stdout:Feb 24 07:49:56 trial035 ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-mgr-x[39621]: ::ffff:10.20.193.91 - - [24/Feb/2026:07:49:56] "GET /metrics HTTP/1.1" 200 4210 "" "Prometheus/3.6.0" 2026-02-24T07:49:57.311 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2026-02-24T07:49:57.356 INFO:journalctl@ceph.prometheus.a.trial091.stdout:Feb 24 07:49:56 trial091 ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-prometheus-a[66007]: time=2026-02-24T07:49:56.973Z level=ERROR source=scrape.go:1630 msg="Failed to determine correct type of scrape target." component="scrape manager" scrape_pool=ceph target=http://trial035:9283/metrics content_type="text/html;charset=utf-8" fallback_media_type="" err="received unsupported Content-Type \"text/html;charset=utf-8\" and no fallback_scrape_protocol specified for target" 2026-02-24T07:49:57.456 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:49:57.578 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:49:57.575+0000 7f10fa57a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:49:57.578 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:49:57.575+0000 7f10fa57a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:49:57.594 INFO:teuthology.orchestra.run.trial035.stdout:osd.0: {} 2026-02-24T07:49:57.595 INFO:teuthology.orchestra.run.trial035.stderr:osd.0: osd_enable_op_tracker = '' 2026-02-24T07:49:57.609 INFO:teuthology.orchestra.run.trial035.stdout:osd.1: {} 2026-02-24T07:49:57.609 INFO:teuthology.orchestra.run.trial035.stderr:osd.1: osd_enable_op_tracker = '' 2026-02-24T07:49:57.624 INFO:teuthology.orchestra.run.trial035.stdout:osd.2: {} 2026-02-24T07:49:57.624 INFO:teuthology.orchestra.run.trial035.stderr:osd.2: osd_enable_op_tracker = '' 2026-02-24T07:49:57.637 INFO:teuthology.orchestra.run.trial035.stdout:osd.3: {} 2026-02-24T07:49:57.637 INFO:teuthology.orchestra.run.trial035.stderr:osd.3: osd_enable_op_tracker = '' 2026-02-24T07:49:57.998 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 59 15 451529 451514 478.259 501.625 0.00162659 0.00208973 2026-02-24T07:49:58.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:49:57 trial035 ceph-mon[39284]: purged_snaps scrub starts 2026-02-24T07:49:58.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:49:57 trial035 ceph-mon[39284]: purged_snaps scrub ok 2026-02-24T07:49:58.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:49:57 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/1266127127' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2026-02-24T07:49:58.998 INFO:tasks.radosbench.radosbench.0.trial035.stdout:2026-02-24T07:49:58.998314+0000 min lat: 0.000682058 max lat: 0.0426186 avg lat: 0.00208864 2026-02-24T07:49:58.998 INFO:tasks.radosbench.radosbench.0.trial035.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2026-02-24T07:49:58.998 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 60 15 459421 459406 478.508 493.25 0.00185637 0.00208864 2026-02-24T07:49:59.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:49:58 trial035 ceph-mon[39284]: pgmap v244: 17 pgs: 17 active+clean; 26 GiB data, 85 GiB used, 2.7 TiB / 2.7 TiB avail; 611 MiB/s wr, 9.78k op/s 2026-02-24T07:49:59.679 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2026-02-24T07:49:59.818 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:49:59.948 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:49:59.945+0000 7f11b5493640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:49:59.948 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:49:59.945+0000 7f11b5493640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:49:59.962 INFO:teuthology.orchestra.run.trial035.stdout:osd.0: {} 2026-02-24T07:49:59.963 INFO:teuthology.orchestra.run.trial035.stderr:osd.0: osd_enable_op_tracker = '' 2026-02-24T07:49:59.963 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:49:59 trial035 ceph-mon[39284]: 4.2 scrub starts 2026-02-24T07:49:59.978 INFO:teuthology.orchestra.run.trial035.stdout:osd.1: {} 2026-02-24T07:49:59.978 INFO:teuthology.orchestra.run.trial035.stderr:osd.1: osd_enable_op_tracker = '' 2026-02-24T07:49:59.989 INFO:teuthology.orchestra.run.trial035.stdout:osd.2: {} 2026-02-24T07:49:59.990 INFO:teuthology.orchestra.run.trial035.stderr:osd.2: osd_enable_op_tracker = '' 2026-02-24T07:49:59.998 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 61 16 466741 466725 478.162 457.438 0.00152443 0.00209013 2026-02-24T07:50:00.006 INFO:teuthology.orchestra.run.trial035.stdout:osd.3: {} 2026-02-24T07:50:00.006 INFO:teuthology.orchestra.run.trial035.stderr:osd.3: osd_enable_op_tracker = '' 2026-02-24T07:50:00.999 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 62 16 471913 471897 475.663 323.25 0.00144058 0.00210113 2026-02-24T07:50:01.096 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2026-02-24T07:50:01.097 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2026-02-24T07:50:01.097 INFO:tasks.thrashosds.thrasher:fixing pg num pool unique_pool_2 2026-02-24T07:50:01.097 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph pg dump --format=json 2026-02-24T07:50:01.123 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:50:00 trial035 ceph-mon[39284]: pgmap v245: 17 pgs: 17 active+clean; 26 GiB data, 85 GiB used, 2.7 TiB / 2.7 TiB avail; 404 MiB/s wr, 6.46k op/s 2026-02-24T07:50:01.124 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:50:00 trial035 ceph-mon[39284]: from='mgr.14284 10.20.193.35:0/3681373222' entity='mgr.x' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2026-02-24T07:50:01.124 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:50:00 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/3638943767' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2026-02-24T07:50:01.124 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:50:00 trial035 ceph-mon[39284]: 4.8 scrub starts 2026-02-24T07:50:01.237 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:50:01.522 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:50:01.519+0000 7fe51978c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:50:01.522 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:50:01.519+0000 7fe51978c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:50:01.664 INFO:teuthology.orchestra.run.trial035.stdout: 2026-02-24T07:50:01.665 INFO:teuthology.orchestra.run.trial035.stderr:dumped all 2026-02-24T07:50:01.711 INFO:teuthology.orchestra.run.trial035.stdout:{"pg_ready":true,"pg_map":{"version":245,"stamp":"2026-02-24T07:49:59.714016+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":28410577440,"num_objects":433504,"num_object_clones":0,"num_object_copies":1300512,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":433504,"num_whiteouts":0,"num_read":136,"num_read_kb":240,"num_write":433637,"num_write_kb":27746278,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":51,"ondisk_log_size":51,"up":51,"acting":51,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":51,"num_osds":4,"num_per_pool_osds":4,"num_per_pool_omap_osds":4,"kb":2936012800,"kb_used":89174364,"kb_used_data":82836556,"kb_used_omap":47,"kb_used_meta":6337680,"kb_avail":2846838436,"statfs":{"total":3006477107200,"available":2915162558464,"internally_reserved":0,"allocated":84824633344,"data_stored":84822982960,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":48646,"internal_metadata":6489784826},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[8,2,2],"upper_bound":8},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":5084348416,"num_objects":77581,"num_object_clones":0,"num_object_copies":232743,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":77581,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":77581,"num_write_kb":4965184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001615"},"pg_stats":[{"pgid":"4.f","version":"61'27321","reported_seq":27343,"reported_epoch":61,"state":"active+clean","last_fresh":"2026-02-24T07:49:55.810526+0000","last_change":"2026-02-24T07:48:57.902569+0000","last_active":"2026-02-24T07:49:55.810526+0000","last_peered":"2026-02-24T07:49:55.810526+0000","last_clean":"2026-02-24T07:49:55.810526+0000","last_became_active":"2026-02-24T07:48:57.902313+0000","last_became_peered":"2026-02-24T07:48:57.902313+0000","last_unstale":"2026-02-24T07:49:55.810526+0000","last_undegraded":"2026-02-24T07:49:55.810526+0000","last_fullsized":"2026-02-24T07:49:55.810526+0000","mapping_epoch":55,"log_start":"61'27318","ondisk_log_start":"61'27318","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:48:56.890768+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:49:59.578048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1790509056,"num_objects":27321,"num_object_clones":0,"num_object_copies":81963,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27321,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":27321,"num_write_kb":1748544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"61'27243","reported_seq":27265,"reported_epoch":61,"state":"active+clean","last_fresh":"2026-02-24T07:49:55.715112+0000","last_change":"2026-02-24T07:48:57.903920+0000","last_active":"2026-02-24T07:49:55.715112+0000","last_peered":"2026-02-24T07:49:55.715112+0000","last_clean":"2026-02-24T07:49:55.715112+0000","last_became_active":"2026-02-24T07:48:57.903584+0000","last_became_peered":"2026-02-24T07:48:57.903584+0000","last_unstale":"2026-02-24T07:49:55.715112+0000","last_undegraded":"2026-02-24T07:49:55.715112+0000","last_fullsized":"2026-02-24T07:49:55.715112+0000","mapping_epoch":55,"log_start":"61'27240","ondisk_log_start":"61'27240","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:48:56.890768+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:50:17.159571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1785397248,"num_objects":27243,"num_object_clones":0,"num_object_copies":81729,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27243,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":27243,"num_write_kb":1743552,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.d","version":"61'27047","reported_seq":27067,"reported_epoch":61,"state":"active+clean","last_fresh":"2026-02-24T07:49:55.733409+0000","last_change":"2026-02-24T07:48:57.905520+0000","last_active":"2026-02-24T07:49:55.733409+0000","last_peered":"2026-02-24T07:49:55.733409+0000","last_clean":"2026-02-24T07:49:55.733409+0000","last_became_active":"2026-02-24T07:48:57.905031+0000","last_became_peered":"2026-02-24T07:48:57.905031+0000","last_unstale":"2026-02-24T07:49:55.733409+0000","last_undegraded":"2026-02-24T07:49:55.733409+0000","last_fullsized":"2026-02-24T07:49:55.733409+0000","mapping_epoch":55,"log_start":"61'27044","ondisk_log_start":"61'27044","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:48:56.890768+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:50:02.301170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1772552192,"num_objects":27047,"num_object_clones":0,"num_object_copies":81141,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27047,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":27047,"num_write_kb":1731008,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"61'26843","reported_seq":26865,"reported_epoch":61,"state":"active+clean","last_fresh":"2026-02-24T07:49:55.727661+0000","last_change":"2026-02-24T07:48:57.905290+0000","last_active":"2026-02-24T07:49:55.727661+0000","last_peered":"2026-02-24T07:49:55.727661+0000","last_clean":"2026-02-24T07:49:55.727661+0000","last_became_active":"2026-02-24T07:48:57.904683+0000","last_became_peered":"2026-02-24T07:48:57.904683+0000","last_unstale":"2026-02-24T07:49:55.727661+0000","last_undegraded":"2026-02-24T07:49:55.727661+0000","last_fullsized":"2026-02-24T07:49:55.727661+0000","mapping_epoch":55,"log_start":"61'26840","ondisk_log_start":"61'26840","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:48:56.890768+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:50:16.643522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1759182848,"num_objects":26843,"num_object_clones":0,"num_object_copies":80529,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26843,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":26843,"num_write_kb":1717952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.b","version":"61'26903","reported_seq":26924,"reported_epoch":61,"state":"active+clean","last_fresh":"2026-02-24T07:49:55.735050+0000","last_change":"2026-02-24T07:48:57.902025+0000","last_active":"2026-02-24T07:49:55.735050+0000","last_peered":"2026-02-24T07:49:55.735050+0000","last_clean":"2026-02-24T07:49:55.735050+0000","last_became_active":"2026-02-24T07:48:57.901700+0000","last_became_peered":"2026-02-24T07:48:57.901700+0000","last_unstale":"2026-02-24T07:49:55.735050+0000","last_undegraded":"2026-02-24T07:49:55.735050+0000","last_fullsized":"2026-02-24T07:49:55.735050+0000","mapping_epoch":55,"log_start":"61'26900","ondisk_log_start":"61'26900","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:48:56.890768+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:50:23.370319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1763115008,"num_objects":26903,"num_object_clones":0,"num_object_copies":80709,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26903,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":26903,"num_write_kb":1721792,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.a","version":"61'27300","reported_seq":27322,"reported_epoch":61,"state":"active+clean","last_fresh":"2026-02-24T07:49:55.717374+0000","last_change":"2026-02-24T07:48:57.903698+0000","last_active":"2026-02-24T07:49:55.717374+0000","last_peered":"2026-02-24T07:49:55.717374+0000","last_clean":"2026-02-24T07:49:55.717374+0000","last_became_active":"2026-02-24T07:48:57.903228+0000","last_became_peered":"2026-02-24T07:48:57.903228+0000","last_unstale":"2026-02-24T07:49:55.717374+0000","last_undegraded":"2026-02-24T07:49:55.717374+0000","last_fullsized":"2026-02-24T07:49:55.717374+0000","mapping_epoch":55,"log_start":"61'27297","ondisk_log_start":"61'27297","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:48:56.890768+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:50:26.780637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1789132800,"num_objects":27300,"num_object_clones":0,"num_object_copies":81900,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27300,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":27300,"num_write_kb":1747200,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"61'27148","reported_seq":27170,"reported_epoch":61,"state":"active+clean","last_fresh":"2026-02-24T07:49:55.716942+0000","last_change":"2026-02-24T07:48:57.902011+0000","last_active":"2026-02-24T07:49:55.716942+0000","last_peered":"2026-02-24T07:49:55.716942+0000","last_clean":"2026-02-24T07:49:55.716942+0000","last_became_active":"2026-02-24T07:48:57.901730+0000","last_became_peered":"2026-02-24T07:48:57.901730+0000","last_unstale":"2026-02-24T07:49:55.716942+0000","last_undegraded":"2026-02-24T07:49:55.716942+0000","last_fullsized":"2026-02-24T07:49:55.716942+0000","mapping_epoch":55,"log_start":"61'27145","ondisk_log_start":"61'27145","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:48:56.890768+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:50:05.273318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1779171328,"num_objects":27148,"num_object_clones":0,"num_object_copies":81444,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27148,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":27148,"num_write_kb":1737472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"61'27012","reported_seq":27033,"reported_epoch":61,"state":"active+clean","last_fresh":"2026-02-24T07:49:55.733794+0000","last_change":"2026-02-24T07:48:57.904924+0000","last_active":"2026-02-24T07:49:55.733794+0000","last_peered":"2026-02-24T07:49:55.733794+0000","last_clean":"2026-02-24T07:49:55.733794+0000","last_became_active":"2026-02-24T07:48:57.904376+0000","last_became_peered":"2026-02-24T07:48:57.904376+0000","last_unstale":"2026-02-24T07:49:55.733794+0000","last_undegraded":"2026-02-24T07:49:55.733794+0000","last_fullsized":"2026-02-24T07:49:55.733794+0000","mapping_epoch":55,"log_start":"61'27009","ondisk_log_start":"61'27009","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:48:56.890768+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:49:57.806612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1770258432,"num_objects":27012,"num_object_clones":0,"num_object_copies":81036,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27012,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":27012,"num_write_kb":1728768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.7","version":"61'27122","reported_seq":27144,"reported_epoch":61,"state":"active+clean","last_fresh":"2026-02-24T07:49:55.810693+0000","last_change":"2026-02-24T07:48:57.902698+0000","last_active":"2026-02-24T07:49:55.810693+0000","last_peered":"2026-02-24T07:49:55.810693+0000","last_clean":"2026-02-24T07:49:55.810693+0000","last_became_active":"2026-02-24T07:48:57.902484+0000","last_became_peered":"2026-02-24T07:48:57.902484+0000","last_unstale":"2026-02-24T07:49:55.810693+0000","last_undegraded":"2026-02-24T07:49:55.810693+0000","last_fullsized":"2026-02-24T07:49:55.810693+0000","mapping_epoch":55,"log_start":"61'27119","ondisk_log_start":"61'27119","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:48:56.890768+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:50:03.263698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1777467392,"num_objects":27122,"num_object_clones":0,"num_object_copies":81366,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27122,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":27122,"num_write_kb":1735808,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.6","version":"61'26898","reported_seq":26920,"reported_epoch":61,"state":"active+clean","last_fresh":"2026-02-24T07:49:55.739205+0000","last_change":"2026-02-24T07:48:57.901839+0000","last_active":"2026-02-24T07:49:55.739205+0000","last_peered":"2026-02-24T07:49:55.739205+0000","last_clean":"2026-02-24T07:49:55.739205+0000","last_became_active":"2026-02-24T07:48:57.901565+0000","last_became_peered":"2026-02-24T07:48:57.901565+0000","last_unstale":"2026-02-24T07:49:55.739205+0000","last_undegraded":"2026-02-24T07:49:55.739205+0000","last_fullsized":"2026-02-24T07:49:55.739205+0000","mapping_epoch":55,"log_start":"61'26895","ondisk_log_start":"61'26895","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:48:56.890768+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:50:09.631523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1762787328,"num_objects":26898,"num_object_clones":0,"num_object_copies":80694,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26898,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":26898,"num_write_kb":1721472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"61'27279","reported_seq":27301,"reported_epoch":61,"state":"active+clean","last_fresh":"2026-02-24T07:49:55.807572+0000","last_change":"2026-02-24T07:48:57.902165+0000","last_active":"2026-02-24T07:49:55.807572+0000","last_peered":"2026-02-24T07:49:55.807572+0000","last_clean":"2026-02-24T07:49:55.807572+0000","last_became_active":"2026-02-24T07:48:57.901824+0000","last_became_peered":"2026-02-24T07:48:57.901824+0000","last_unstale":"2026-02-24T07:49:55.807572+0000","last_undegraded":"2026-02-24T07:49:55.807572+0000","last_fullsized":"2026-02-24T07:49:55.807572+0000","mapping_epoch":55,"log_start":"61'27276","ondisk_log_start":"61'27276","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:48:56.890768+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:50:12.091202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1787756544,"num_objects":27279,"num_object_clones":0,"num_object_copies":81837,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27279,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":27279,"num_write_kb":1745856,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[1,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"61'26962","reported_seq":26984,"reported_epoch":61,"state":"active+clean","last_fresh":"2026-02-24T07:49:55.735346+0000","last_change":"2026-02-24T07:48:57.902537+0000","last_active":"2026-02-24T07:49:55.735346+0000","last_peered":"2026-02-24T07:49:55.735346+0000","last_clean":"2026-02-24T07:49:55.735346+0000","last_became_active":"2026-02-24T07:48:57.902224+0000","last_became_peered":"2026-02-24T07:48:57.902224+0000","last_unstale":"2026-02-24T07:49:55.735346+0000","last_undegraded":"2026-02-24T07:49:55.735346+0000","last_fullsized":"2026-02-24T07:49:55.735346+0000","mapping_epoch":55,"log_start":"61'26959","ondisk_log_start":"61'26959","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:48:56.890768+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:50:06.186992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1766981632,"num_objects":26962,"num_object_clones":0,"num_object_copies":80886,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26962,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":26962,"num_write_kb":1725568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"61'27155","reported_seq":27176,"reported_epoch":61,"state":"active+clean","last_fresh":"2026-02-24T07:49:55.812365+0000","last_change":"2026-02-24T07:48:57.902384+0000","last_active":"2026-02-24T07:49:55.812365+0000","last_peered":"2026-02-24T07:49:55.812365+0000","last_clean":"2026-02-24T07:49:55.812365+0000","last_became_active":"2026-02-24T07:48:57.902125+0000","last_became_peered":"2026-02-24T07:48:57.902125+0000","last_unstale":"2026-02-24T07:49:55.812365+0000","last_undegraded":"2026-02-24T07:49:55.812365+0000","last_fullsized":"2026-02-24T07:49:55.812365+0000","mapping_epoch":55,"log_start":"61'27152","ondisk_log_start":"61'27152","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:48:56.890768+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:49:57.995484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1779630080,"num_objects":27155,"num_object_clones":0,"num_object_copies":81465,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27155,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":27155,"num_write_kb":1737920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1","version":"61'27255","reported_seq":27277,"reported_epoch":61,"state":"active+clean","last_fresh":"2026-02-24T07:49:55.715171+0000","last_change":"2026-02-24T07:48:57.903745+0000","last_active":"2026-02-24T07:49:55.715171+0000","last_peered":"2026-02-24T07:49:55.715171+0000","last_clean":"2026-02-24T07:49:55.715171+0000","last_became_active":"2026-02-24T07:48:57.903439+0000","last_became_peered":"2026-02-24T07:48:57.903439+0000","last_unstale":"2026-02-24T07:49:55.715171+0000","last_undegraded":"2026-02-24T07:49:55.715171+0000","last_fullsized":"2026-02-24T07:49:55.715171+0000","mapping_epoch":55,"log_start":"61'27252","ondisk_log_start":"61'27252","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:48:56.890768+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:50:24.058110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1786183680,"num_objects":27255,"num_object_clones":0,"num_object_copies":81765,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27255,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":27255,"num_write_kb":1744320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.0","version":"61'27024","reported_seq":27046,"reported_epoch":61,"state":"active+clean","last_fresh":"2026-02-24T07:49:55.735763+0000","last_change":"2026-02-24T07:48:57.902838+0000","last_active":"2026-02-24T07:49:55.735763+0000","last_peered":"2026-02-24T07:49:55.735763+0000","last_clean":"2026-02-24T07:49:55.735763+0000","last_became_active":"2026-02-24T07:48:57.902565+0000","last_became_peered":"2026-02-24T07:48:57.902565+0000","last_unstale":"2026-02-24T07:49:55.735763+0000","last_undegraded":"2026-02-24T07:49:55.735763+0000","last_fullsized":"2026-02-24T07:49:55.735763+0000","mapping_epoch":55,"log_start":"61'27021","ondisk_log_start":"61'27021","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:48:56.890768+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:50:25.079534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1771044864,"num_objects":27024,"num_object_clones":0,"num_object_copies":81072,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27024,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":27024,"num_write_kb":1729536,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,3],"acting":[0,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.5","version":"61'26990","reported_seq":27012,"reported_epoch":61,"state":"active+clean","last_fresh":"2026-02-24T07:49:55.731240+0000","last_change":"2026-02-24T07:48:57.905643+0000","last_active":"2026-02-24T07:49:55.731240+0000","last_peered":"2026-02-24T07:49:55.731240+0000","last_clean":"2026-02-24T07:49:55.731240+0000","last_became_active":"2026-02-24T07:48:57.905127+0000","last_became_peered":"2026-02-24T07:48:57.905127+0000","last_unstale":"2026-02-24T07:49:55.731240+0000","last_undegraded":"2026-02-24T07:49:55.731240+0000","last_fullsized":"2026-02-24T07:49:55.731240+0000","mapping_epoch":55,"log_start":"61'26987","ondisk_log_start":"61'26987","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:48:56.890768+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:49:59.684568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1768816640,"num_objects":26990,"num_object_clones":0,"num_object_copies":80970,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26990,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":26990,"num_write_kb":1727360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"29'99","reported_seq":285,"reported_epoch":61,"state":"active+clean","last_fresh":"2026-02-24T07:49:41.888452+0000","last_change":"2026-02-24T07:49:06.478572+0000","last_active":"2026-02-24T07:49:41.888452+0000","last_peered":"2026-02-24T07:49:41.888452+0000","last_clean":"2026-02-24T07:49:41.888452+0000","last_became_active":"2026-02-24T07:42:54.186719+0000","last_became_peered":"2026-02-24T07:42:54.186719+0000","last_unstale":"2026-02-24T07:49:41.888452+0000","last_undegraded":"2026-02-24T07:49:41.888452+0000","last_fullsized":"2026-02-24T07:49:41.888452+0000","mapping_epoch":27,"log_start":"29'96","ondisk_log_start":"29'96","created":20,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'99","last_scrub_stamp":"2026-02-24T07:49:06.478534+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:42:43.640776+0000","last_clean_scrub_stamp":"2026-02-24T07:49:06.478534+0000","objects_scrubbed":2,"log_size":3,"log_dups_size":96,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:50:26.893586+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":240,"num_write":135,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":16,"stat_sum":{"num_bytes":28409987072,"num_objects":433502,"num_object_clones":0,"num_object_copies":1300506,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":433502,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":433502,"num_write_kb":27744128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":85221441536,"data_stored":85221441536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":48,"ondisk_log_size":48,"up":48,"acting":48,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":240,"num_write":135,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":3,"up_from":25,"seq":107374182487,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":22136904,"kb_used_data":20679476,"kb_used_omap":11,"kb_used_meta":1457396,"kb_avail":711866296,"statfs":{"total":751619276800,"available":728951087104,"internally_reserved":0,"allocated":21175783424,"data_stored":21175369940,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":11953,"internal_metadata":1492373839},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,2,1],"upper_bound":8},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":19,"seq":81604378713,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":26027400,"kb_used_data":24180404,"kb_used_omap":11,"kb_used_meta":1846964,"kb_avail":707975800,"statfs":{"total":751619276800,"available":724967219200,"internally_reserved":0,"allocated":24760733696,"data_stored":24760320212,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12004,"internal_metadata":1891291420},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542234,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":18863812,"kb_used_data":17276272,"kb_used_omap":13,"kb_used_meta":1587506,"kb_avail":715139388,"statfs":{"total":751619276800,"available":732302733312,"internally_reserved":0,"allocated":17690902528,"data_stored":17690492596,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13503,"internal_metadata":1625606977},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705757,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":22146248,"kb_used_data":20700404,"kb_used_omap":10,"kb_used_meta":1445813,"kb_avail":711856952,"statfs":{"total":751619276800,"available":728941518848,"internally_reserved":0,"allocated":21197213696,"data_stored":21196800212,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":11186,"internal_metadata":1480512590},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[6,0,1],"upper_bound":8},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":21280063488,"data_stored":21280063488,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":17789222912,"data_stored":17789222912,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":24831655936,"data_stored":24831655936,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":21320499200,"data_stored":21320499200,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-24T07:50:01.712 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph osd pool set unique_pool_2 pgp_num 26 2026-02-24T07:50:01.890 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:50:01.915 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:50:01 trial035 ceph-mon[39284]: 4.f scrub starts 2026-02-24T07:50:01.916 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:50:01 trial035 ceph-mon[39284]: 4.5 scrub starts 2026-02-24T07:50:01.999 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 63 16 477104 477088 473.262 324.438 0.00139444 0.00211177 2026-02-24T07:50:02.017 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:50:02.015+0000 7f0c84017640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:50:02.017 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:50:02.015+0000 7f0c84017640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:50:02.157 INFO:teuthology.orchestra.run.trial035.stderr:Error EPERM: crimson_allow_pg_split is false; pgp_num increase denied 2026-02-24T07:50:02.200 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-24T07:50:02.591 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2026-02-24T07:50:02.735 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:50:02.853 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:50:02.851+0000 7ff0f8221640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:50:02.853 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:50:02.852+0000 7ff0f8221640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:50:02.869 INFO:teuthology.orchestra.run.trial035.stdout:osd.0: {} 2026-02-24T07:50:02.869 INFO:teuthology.orchestra.run.trial035.stderr:osd.0: osd_enable_op_tracker = '' 2026-02-24T07:50:02.883 INFO:teuthology.orchestra.run.trial035.stdout:osd.1: {} 2026-02-24T07:50:02.883 INFO:teuthology.orchestra.run.trial035.stderr:osd.1: osd_enable_op_tracker = '' 2026-02-24T07:50:02.895 INFO:teuthology.orchestra.run.trial035.stdout:osd.2: {} 2026-02-24T07:50:02.896 INFO:teuthology.orchestra.run.trial035.stderr:osd.2: osd_enable_op_tracker = '' 2026-02-24T07:50:02.909 INFO:teuthology.orchestra.run.trial035.stdout:osd.3: {} 2026-02-24T07:50:02.909 INFO:teuthology.orchestra.run.trial035.stderr:osd.3: osd_enable_op_tracker = '' 2026-02-24T07:50:02.999 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 64 16 482716 482700 471.348 350.75 0.00137501 0.00212036 2026-02-24T07:50:03.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:50:02 trial035 ceph-mon[39284]: 4.2 scrub ok 2026-02-24T07:50:03.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:50:02 trial035 ceph-mon[39284]: from='client.14967 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2026-02-24T07:50:03.205 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:50:02 trial035 ceph-mon[39284]: pgmap v246: 17 pgs: 3 active+clean+scrubbing, 14 active+clean; 29 GiB data, 92 GiB used, 2.6 TiB / 2.7 TiB avail; 597 MiB/s wr, 9.54k op/s 2026-02-24T07:50:03.205 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:50:02 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/2169611563' entity='client.admin' cmd={"prefix": "osd pool set", "pool": "unique_pool_2", "var": "pgp_num", "val": "26"} : dispatch 2026-02-24T07:50:03.205 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:50:02 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/1491443868' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2026-02-24T07:50:03.999 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 65 16 488860 488844 470.003 384 0.00133113 0.00212637 2026-02-24T07:50:04.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:50:03 trial035 ceph-mon[39284]: 4.8 scrub ok 2026-02-24T07:50:04.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:50:03 trial035 ceph-mon[39284]: 4.d scrub starts 2026-02-24T07:50:04.954 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2026-02-24T07:50:04.999 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 66 16 495361 495345 469.038 406.312 0.00100696 0.0021308 2026-02-24T07:50:05.098 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:50:05.156 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:50:04 trial035 ceph-mon[39284]: 4.f scrub ok 2026-02-24T07:50:05.156 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:50:04 trial035 ceph-mon[39284]: pgmap v247: 17 pgs: 3 active+clean+scrubbing, 14 active+clean; 29 GiB data, 92 GiB used, 2.6 TiB / 2.7 TiB avail; 403 MiB/s wr, 6.45k op/s 2026-02-24T07:50:05.223 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:50:05.222+0000 7fbcb0d5b640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:50:05.224 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:50:05.222+0000 7fbcb0d5b640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:50:05.239 INFO:teuthology.orchestra.run.trial035.stdout:osd.0: {} 2026-02-24T07:50:05.240 INFO:teuthology.orchestra.run.trial035.stderr:osd.0: osd_enable_op_tracker = '' 2026-02-24T07:50:05.255 INFO:teuthology.orchestra.run.trial035.stdout:osd.1: {} 2026-02-24T07:50:05.255 INFO:teuthology.orchestra.run.trial035.stderr:osd.1: osd_enable_op_tracker = '' 2026-02-24T07:50:05.267 INFO:teuthology.orchestra.run.trial035.stdout:osd.2: {} 2026-02-24T07:50:05.267 INFO:teuthology.orchestra.run.trial035.stderr:osd.2: osd_enable_op_tracker = '' 2026-02-24T07:50:05.279 INFO:teuthology.orchestra.run.trial035.stdout:osd.3: {} 2026-02-24T07:50:05.279 INFO:teuthology.orchestra.run.trial035.stderr:osd.3: osd_enable_op_tracker = '' 2026-02-24T07:50:05.999 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 67 16 502559 502543 468.751 449.875 0.00143815 0.00213213 2026-02-24T07:50:06.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:50:05 trial035 ceph-mon[39284]: 4.5 scrub ok 2026-02-24T07:50:06.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:50:05 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/453765665' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2026-02-24T07:50:06.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:50:05 trial035 ceph-mon[39284]: purged_snaps scrub starts 2026-02-24T07:50:06.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:50:05 trial035 ceph-mon[39284]: purged_snaps scrub ok 2026-02-24T07:50:07.000 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 68 16 509823 509807 468.534 454 0.00170257 0.00213311 2026-02-24T07:50:07.201 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2026-02-24T07:50:07.201 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2026-02-24T07:50:07.202 INFO:tasks.thrashosds.thrasher:Growing pool unique_pool_2 2026-02-24T07:50:07.202 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph pg dump --format=json 2026-02-24T07:50:07.205 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:50:06 trial035 ceph-mon[39284]: pgmap v248: 17 pgs: 3 active+clean+scrubbing, 14 active+clean; 29 GiB data, 92 GiB used, 2.6 TiB / 2.7 TiB avail; 403 MiB/s wr, 6.45k op/s 2026-02-24T07:50:07.205 INFO:journalctl@ceph.mgr.x.trial035.stdout:Feb 24 07:50:06 trial035 ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-mgr-x[39621]: ::ffff:10.20.193.91 - - [24/Feb/2026:07:50:06] "GET /metrics HTTP/1.1" 200 4213 "" "Prometheus/3.6.0" 2026-02-24T07:50:07.326 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2026-02-24T07:50:07.340 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:50:07.356 INFO:journalctl@ceph.prometheus.a.trial091.stdout:Feb 24 07:50:06 trial091 ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-prometheus-a[66007]: time=2026-02-24T07:50:06.973Z level=ERROR source=scrape.go:1630 msg="Failed to determine correct type of scrape target." component="scrape manager" scrape_pool=ceph target=http://trial035:9283/metrics content_type="text/html;charset=utf-8" fallback_media_type="" err="received unsupported Content-Type \"text/html;charset=utf-8\" and no fallback_scrape_protocol specified for target" 2026-02-24T07:50:07.584 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:50:07.843 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:50:07.782+0000 7f0a894c5640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:50:07.843 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:50:07.782+0000 7f0a894c5640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:50:07.925 INFO:teuthology.orchestra.run.trial035.stdout: 2026-02-24T07:50:07.926 INFO:teuthology.orchestra.run.trial035.stderr:dumped all 2026-02-24T07:50:08.000 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 69 16 515296 515280 466.698 342.062 0.00129432 0.0021411 2026-02-24T07:50:08.062 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:50:07 trial035 ceph-mon[39284]: 4.3 scrub starts 2026-02-24T07:50:08.063 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:50:07 trial035 ceph-mon[39284]: 4.d scrub ok 2026-02-24T07:50:08.063 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:50:07 trial035 ceph-mon[39284]: from='mgr.14284 10.20.193.35:0/3681373222' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2026-02-24T07:50:08.063 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:50:07 trial035 ceph-mon[39284]: from='mgr.14284 10.20.193.35:0/3681373222' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-24T07:50:08.063 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:50:07 trial035 ceph-mon[39284]: from='mgr.14284 10.20.193.35:0/3681373222' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2026-02-24T07:50:08.063 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:50:07 trial035 ceph-mon[39284]: from='mgr.14284 10.20.193.35:0/3681373222' entity='mgr.x' 2026-02-24T07:50:08.063 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:50:07 trial035 ceph-mon[39284]: from='mgr.14284 10.20.193.35:0/3681373222' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2026-02-24T07:50:08.827 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:50:08.774+0000 7f21b661a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:50:08.828 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:50:08.774+0000 7f21b661a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:50:08.828 INFO:teuthology.orchestra.run.trial035.stderr:osd.0: osd_enable_op_tracker = '' 2026-02-24T07:50:08.828 INFO:teuthology.orchestra.run.trial035.stderr:osd.1: osd_enable_op_tracker = '' 2026-02-24T07:50:08.828 INFO:teuthology.orchestra.run.trial035.stderr:osd.2: osd_enable_op_tracker = '' 2026-02-24T07:50:08.828 INFO:teuthology.orchestra.run.trial035.stdout:osd.0: {} 2026-02-24T07:50:08.828 INFO:teuthology.orchestra.run.trial035.stdout:osd.1: {} 2026-02-24T07:50:08.828 INFO:teuthology.orchestra.run.trial035.stdout:osd.2: {} 2026-02-24T07:50:08.839 INFO:teuthology.orchestra.run.trial035.stdout:osd.3: {} 2026-02-24T07:50:08.839 INFO:teuthology.orchestra.run.trial035.stderr:osd.3: osd_enable_op_tracker = '' 2026-02-24T07:50:09.000 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 70 16 519690 519674 463.953 274.625 0.00162082 0.00215374 2026-02-24T07:50:09.167 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:50:08 trial035 ceph-mon[39284]: 4.7 scrub starts 2026-02-24T07:50:09.168 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:50:08 trial035 ceph-mon[39284]: pgmap v249: 17 pgs: 1 active+clean+scrubbing, 16 active+clean; 31 GiB data, 98 GiB used, 2.6 TiB / 2.7 TiB avail; 561 MiB/s wr, 8.98k op/s 2026-02-24T07:50:09.168 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:50:08 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/3301583812' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2026-02-24T07:50:09.745 INFO:teuthology.orchestra.run.trial035.stdout:{"pg_ready":true,"pg_map":{"version":249,"stamp":"2026-02-24T07:50:07.715091+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":32823837216,"num_objects":500845,"num_object_clones":0,"num_object_copies":1502535,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":500845,"num_whiteouts":0,"num_read":136,"num_read_kb":240,"num_write":500978,"num_write_kb":32056102,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":51,"ondisk_log_size":51,"up":51,"acting":51,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":51,"num_osds":4,"num_per_pool_osds":4,"num_per_pool_omap_osds":4,"kb":2936012800,"kb_used":102503068,"kb_used_data":95695884,"kb_used_omap":49,"kb_used_meta":6806990,"kb_avail":2833509732,"statfs":{"total":3006477107200,"available":2901513965568,"internally_reserved":0,"allocated":97992585216,"data_stored":97990934832,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":50934,"internal_metadata":6970358026},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[5,1],"upper_bound":4},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":7064453120,"num_objects":107795,"num_object_clones":0,"num_object_copies":323385,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":107795,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":107795,"num_write_kb":6898880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001598"},"pg_stats":[{"pgid":"4.f","version":"61'31575","reported_seq":31908,"reported_epoch":61,"state":"active+clean","last_fresh":"2026-02-24T07:50:05.822237+0000","last_change":"2026-02-24T07:50:03.071459+0000","last_active":"2026-02-24T07:50:05.822237+0000","last_peered":"2026-02-24T07:50:05.822237+0000","last_clean":"2026-02-24T07:50:05.822237+0000","last_became_active":"2026-02-24T07:48:57.902313+0000","last_became_peered":"2026-02-24T07:48:57.902313+0000","last_unstale":"2026-02-24T07:50:05.822237+0000","last_undegraded":"2026-02-24T07:50:05.822237+0000","last_fullsized":"2026-02-24T07:50:05.822237+0000","mapping_epoch":55,"log_start":"61'31572","ondisk_log_start":"61'31572","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"61'30419","last_scrub_stamp":"2026-02-24T07:50:03.071380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:50:03.071380+0000","objects_scrubbed":29975,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:51:16.123453+0000","scrub_duration":2568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2069299200,"num_objects":31575,"num_object_clones":0,"num_object_copies":94725,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31575,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":31575,"num_write_kb":2020800,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"61'31450","reported_seq":31472,"reported_epoch":61,"state":"active+clean","last_fresh":"2026-02-24T07:50:05.715102+0000","last_change":"2026-02-24T07:48:57.903920+0000","last_active":"2026-02-24T07:50:05.715102+0000","last_peered":"2026-02-24T07:50:05.715102+0000","last_clean":"2026-02-24T07:50:05.715102+0000","last_became_active":"2026-02-24T07:48:57.903584+0000","last_became_peered":"2026-02-24T07:48:57.903584+0000","last_unstale":"2026-02-24T07:50:05.715102+0000","last_undegraded":"2026-02-24T07:50:05.715102+0000","last_fullsized":"2026-02-24T07:50:05.715102+0000","mapping_epoch":55,"log_start":"61'31447","ondisk_log_start":"61'31447","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:48:56.890768+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:50:17.159571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2061107200,"num_objects":31450,"num_object_clones":0,"num_object_copies":94350,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31450,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":31450,"num_write_kb":2012800,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.d","version":"61'31229","reported_seq":31462,"reported_epoch":61,"state":"active+clean+scrubbing","last_fresh":"2026-02-24T07:50:05.744231+0000","last_change":"2026-02-24T07:50:03.205888+0000","last_active":"2026-02-24T07:50:05.744231+0000","last_peered":"2026-02-24T07:50:05.744231+0000","last_clean":"2026-02-24T07:50:05.744231+0000","last_became_active":"2026-02-24T07:48:57.905031+0000","last_became_peered":"2026-02-24T07:48:57.905031+0000","last_unstale":"2026-02-24T07:50:05.744231+0000","last_undegraded":"2026-02-24T07:50:05.744231+0000","last_fullsized":"2026-02-24T07:50:05.744231+0000","mapping_epoch":55,"log_start":"61'31226","ondisk_log_start":"61'31226","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:48:56.890768+0000","objects_scrubbed":20599,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"scrubbing for 3s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2046623744,"num_objects":31229,"num_object_clones":0,"num_object_copies":93687,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31229,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":31229,"num_write_kb":1998656,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"61'31222","reported_seq":31244,"reported_epoch":61,"state":"active+clean","last_fresh":"2026-02-24T07:50:05.739137+0000","last_change":"2026-02-24T07:48:57.905290+0000","last_active":"2026-02-24T07:50:05.739137+0000","last_peered":"2026-02-24T07:50:05.739137+0000","last_clean":"2026-02-24T07:50:05.739137+0000","last_became_active":"2026-02-24T07:48:57.904683+0000","last_became_peered":"2026-02-24T07:48:57.904683+0000","last_unstale":"2026-02-24T07:50:05.739137+0000","last_undegraded":"2026-02-24T07:50:05.739137+0000","last_fullsized":"2026-02-24T07:50:05.739137+0000","mapping_epoch":55,"log_start":"61'31219","ondisk_log_start":"61'31219","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:48:56.890768+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:50:16.643522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2046164992,"num_objects":31222,"num_object_clones":0,"num_object_copies":93666,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31222,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":31222,"num_write_kb":1998208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.b","version":"61'31115","reported_seq":31137,"reported_epoch":61,"state":"active+clean","last_fresh":"2026-02-24T07:50:05.736016+0000","last_change":"2026-02-24T07:48:57.902025+0000","last_active":"2026-02-24T07:50:05.736016+0000","last_peered":"2026-02-24T07:50:05.736016+0000","last_clean":"2026-02-24T07:50:05.736016+0000","last_became_active":"2026-02-24T07:48:57.901700+0000","last_became_peered":"2026-02-24T07:48:57.901700+0000","last_unstale":"2026-02-24T07:50:05.736016+0000","last_undegraded":"2026-02-24T07:50:05.736016+0000","last_fullsized":"2026-02-24T07:50:05.736016+0000","mapping_epoch":55,"log_start":"61'31112","ondisk_log_start":"61'31112","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:48:56.890768+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:50:23.370319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2039152640,"num_objects":31115,"num_object_clones":0,"num_object_copies":93345,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31115,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":31115,"num_write_kb":1991360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.a","version":"61'31423","reported_seq":31445,"reported_epoch":61,"state":"active+clean","last_fresh":"2026-02-24T07:50:05.711025+0000","last_change":"2026-02-24T07:48:57.903698+0000","last_active":"2026-02-24T07:50:05.711025+0000","last_peered":"2026-02-24T07:50:05.711025+0000","last_clean":"2026-02-24T07:50:05.711025+0000","last_became_active":"2026-02-24T07:48:57.903228+0000","last_became_peered":"2026-02-24T07:48:57.903228+0000","last_unstale":"2026-02-24T07:50:05.711025+0000","last_undegraded":"2026-02-24T07:50:05.711025+0000","last_fullsized":"2026-02-24T07:50:05.711025+0000","mapping_epoch":55,"log_start":"61'31420","ondisk_log_start":"61'31420","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:48:56.890768+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:50:26.780637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2059337728,"num_objects":31423,"num_object_clones":0,"num_object_copies":94269,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31423,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":31423,"num_write_kb":2011072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"61'31315","reported_seq":31337,"reported_epoch":61,"state":"active+clean","last_fresh":"2026-02-24T07:50:05.715321+0000","last_change":"2026-02-24T07:48:57.902011+0000","last_active":"2026-02-24T07:50:05.715321+0000","last_peered":"2026-02-24T07:50:05.715321+0000","last_clean":"2026-02-24T07:50:05.715321+0000","last_became_active":"2026-02-24T07:48:57.901730+0000","last_became_peered":"2026-02-24T07:48:57.901730+0000","last_unstale":"2026-02-24T07:50:05.715321+0000","last_undegraded":"2026-02-24T07:50:05.715321+0000","last_fullsized":"2026-02-24T07:50:05.715321+0000","mapping_epoch":55,"log_start":"61'31312","ondisk_log_start":"61'31312","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:48:56.890768+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2052259840,"num_objects":31315,"num_object_clones":0,"num_object_copies":93945,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31315,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":31315,"num_write_kb":2004160,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"61'31182","reported_seq":31511,"reported_epoch":61,"state":"active+clean","last_fresh":"2026-02-24T07:50:05.740509+0000","last_change":"2026-02-24T07:50:03.153499+0000","last_active":"2026-02-24T07:50:05.740509+0000","last_peered":"2026-02-24T07:50:05.740509+0000","last_clean":"2026-02-24T07:50:05.740509+0000","last_became_active":"2026-02-24T07:48:57.904376+0000","last_became_peered":"2026-02-24T07:48:57.904376+0000","last_unstale":"2026-02-24T07:50:05.740509+0000","last_undegraded":"2026-02-24T07:50:05.740509+0000","last_fullsized":"2026-02-24T07:50:05.740509+0000","mapping_epoch":55,"log_start":"61'31179","ondisk_log_start":"61'31179","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"61'30163","last_scrub_stamp":"2026-02-24T07:50:03.153454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:50:03.153454+0000","objects_scrubbed":29681,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:51:10.266136+0000","scrub_duration":2916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2043543552,"num_objects":31182,"num_object_clones":0,"num_object_copies":93546,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31182,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":31182,"num_write_kb":1995648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.7","version":"61'31176","reported_seq":31198,"reported_epoch":61,"state":"active+clean","last_fresh":"2026-02-24T07:50:05.823200+0000","last_change":"2026-02-24T07:48:57.902698+0000","last_active":"2026-02-24T07:50:05.823200+0000","last_peered":"2026-02-24T07:50:05.823200+0000","last_clean":"2026-02-24T07:50:05.823200+0000","last_became_active":"2026-02-24T07:48:57.902484+0000","last_became_peered":"2026-02-24T07:48:57.902484+0000","last_unstale":"2026-02-24T07:50:05.823200+0000","last_undegraded":"2026-02-24T07:50:05.823200+0000","last_fullsized":"2026-02-24T07:50:05.823200+0000","mapping_epoch":55,"log_start":"61'31173","ondisk_log_start":"61'31173","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:48:56.890768+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2043150336,"num_objects":31176,"num_object_clones":0,"num_object_copies":93528,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31176,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":31176,"num_write_kb":1995264,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.6","version":"61'31077","reported_seq":31099,"reported_epoch":61,"state":"active+clean","last_fresh":"2026-02-24T07:50:05.729744+0000","last_change":"2026-02-24T07:48:57.901839+0000","last_active":"2026-02-24T07:50:05.729744+0000","last_peered":"2026-02-24T07:50:05.729744+0000","last_clean":"2026-02-24T07:50:05.729744+0000","last_became_active":"2026-02-24T07:48:57.901565+0000","last_became_peered":"2026-02-24T07:48:57.901565+0000","last_unstale":"2026-02-24T07:50:05.729744+0000","last_undegraded":"2026-02-24T07:50:05.729744+0000","last_fullsized":"2026-02-24T07:50:05.729744+0000","mapping_epoch":55,"log_start":"61'31074","ondisk_log_start":"61'31074","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:48:56.890768+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:50:09.631523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2036662272,"num_objects":31077,"num_object_clones":0,"num_object_copies":93231,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31077,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":31077,"num_write_kb":1988928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"61'31563","reported_seq":31585,"reported_epoch":61,"state":"active+clean","last_fresh":"2026-02-24T07:50:05.818442+0000","last_change":"2026-02-24T07:48:57.902165+0000","last_active":"2026-02-24T07:50:05.818442+0000","last_peered":"2026-02-24T07:50:05.818442+0000","last_clean":"2026-02-24T07:50:05.818442+0000","last_became_active":"2026-02-24T07:48:57.901824+0000","last_became_peered":"2026-02-24T07:48:57.901824+0000","last_unstale":"2026-02-24T07:50:05.818442+0000","last_undegraded":"2026-02-24T07:50:05.818442+0000","last_fullsized":"2026-02-24T07:50:05.818442+0000","mapping_epoch":55,"log_start":"61'31560","ondisk_log_start":"61'31560","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:48:56.890768+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:50:12.091202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2068512768,"num_objects":31563,"num_object_clones":0,"num_object_copies":94689,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31563,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":31563,"num_write_kb":2020032,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[1,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"61'31064","reported_seq":31086,"reported_epoch":61,"state":"active+clean","last_fresh":"2026-02-24T07:50:05.724792+0000","last_change":"2026-02-24T07:48:57.902537+0000","last_active":"2026-02-24T07:50:05.724792+0000","last_peered":"2026-02-24T07:50:05.724792+0000","last_clean":"2026-02-24T07:50:05.724792+0000","last_became_active":"2026-02-24T07:48:57.902224+0000","last_became_peered":"2026-02-24T07:48:57.902224+0000","last_unstale":"2026-02-24T07:50:05.724792+0000","last_undegraded":"2026-02-24T07:50:05.724792+0000","last_fullsized":"2026-02-24T07:50:05.724792+0000","mapping_epoch":55,"log_start":"61'31061","ondisk_log_start":"61'31061","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:48:56.890768+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:50:06.186992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2035810304,"num_objects":31064,"num_object_clones":0,"num_object_copies":93192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31064,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":31064,"num_write_kb":1988096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"61'31427","reported_seq":31752,"reported_epoch":61,"state":"active+clean","last_fresh":"2026-02-24T07:50:05.818913+0000","last_change":"2026-02-24T07:50:01.398116+0000","last_active":"2026-02-24T07:50:05.818913+0000","last_peered":"2026-02-24T07:50:05.818913+0000","last_clean":"2026-02-24T07:50:05.818913+0000","last_became_active":"2026-02-24T07:48:57.902125+0000","last_became_peered":"2026-02-24T07:48:57.902125+0000","last_unstale":"2026-02-24T07:50:05.818913+0000","last_undegraded":"2026-02-24T07:50:05.818913+0000","last_fullsized":"2026-02-24T07:50:05.818913+0000","mapping_epoch":55,"log_start":"61'31424","ondisk_log_start":"61'31424","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"61'29707","last_scrub_stamp":"2026-02-24T07:50:01.398090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:50:01.398090+0000","objects_scrubbed":29171,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:51:10.566529+0000","scrub_duration":2880,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2059599872,"num_objects":31427,"num_object_clones":0,"num_object_copies":94281,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31427,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":31427,"num_write_kb":2011328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1","version":"61'31476","reported_seq":31498,"reported_epoch":61,"state":"active+clean","last_fresh":"2026-02-24T07:50:05.719982+0000","last_change":"2026-02-24T07:48:57.903745+0000","last_active":"2026-02-24T07:50:05.719982+0000","last_peered":"2026-02-24T07:50:05.719982+0000","last_clean":"2026-02-24T07:50:05.719982+0000","last_became_active":"2026-02-24T07:48:57.903439+0000","last_became_peered":"2026-02-24T07:48:57.903439+0000","last_unstale":"2026-02-24T07:50:05.719982+0000","last_undegraded":"2026-02-24T07:50:05.719982+0000","last_fullsized":"2026-02-24T07:50:05.719982+0000","mapping_epoch":55,"log_start":"61'31473","ondisk_log_start":"61'31473","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:48:56.890768+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:50:24.058110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2062811136,"num_objects":31476,"num_object_clones":0,"num_object_copies":94428,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31476,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":31476,"num_write_kb":2014464,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.0","version":"61'31318","reported_seq":31340,"reported_epoch":61,"state":"active+clean","last_fresh":"2026-02-24T07:50:05.737928+0000","last_change":"2026-02-24T07:48:57.902838+0000","last_active":"2026-02-24T07:50:05.737928+0000","last_peered":"2026-02-24T07:50:05.737928+0000","last_clean":"2026-02-24T07:50:05.737928+0000","last_became_active":"2026-02-24T07:48:57.902565+0000","last_became_peered":"2026-02-24T07:48:57.902565+0000","last_unstale":"2026-02-24T07:50:05.737928+0000","last_undegraded":"2026-02-24T07:50:05.737928+0000","last_fullsized":"2026-02-24T07:50:05.737928+0000","mapping_epoch":55,"log_start":"61'31315","ondisk_log_start":"61'31315","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:48:56.890768+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:50:25.079534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2052456448,"num_objects":31318,"num_object_clones":0,"num_object_copies":93954,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31318,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":31318,"num_write_kb":2004352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,3],"acting":[0,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.5","version":"61'31231","reported_seq":31566,"reported_epoch":61,"state":"active+clean","last_fresh":"2026-02-24T07:50:05.744251+0000","last_change":"2026-02-24T07:50:04.821914+0000","last_active":"2026-02-24T07:50:05.744251+0000","last_peered":"2026-02-24T07:50:05.744251+0000","last_clean":"2026-02-24T07:50:05.744251+0000","last_became_active":"2026-02-24T07:48:57.905127+0000","last_became_peered":"2026-02-24T07:48:57.905127+0000","last_unstale":"2026-02-24T07:50:05.744251+0000","last_undegraded":"2026-02-24T07:50:05.744251+0000","last_fullsized":"2026-02-24T07:50:05.744251+0000","mapping_epoch":55,"log_start":"61'31228","ondisk_log_start":"61'31228","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"61'30833","last_scrub_stamp":"2026-02-24T07:50:04.821861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:50:04.821861+0000","objects_scrubbed":30132,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:51:15.928207+0000","scrub_duration":3578,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2046754816,"num_objects":31231,"num_object_clones":0,"num_object_copies":93693,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31231,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":31231,"num_write_kb":1998784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"29'99","reported_seq":285,"reported_epoch":61,"state":"active+clean","last_fresh":"2026-02-24T07:49:41.888452+0000","last_change":"2026-02-24T07:49:06.478572+0000","last_active":"2026-02-24T07:49:41.888452+0000","last_peered":"2026-02-24T07:49:41.888452+0000","last_clean":"2026-02-24T07:49:41.888452+0000","last_became_active":"2026-02-24T07:42:54.186719+0000","last_became_peered":"2026-02-24T07:42:54.186719+0000","last_unstale":"2026-02-24T07:49:41.888452+0000","last_undegraded":"2026-02-24T07:49:41.888452+0000","last_fullsized":"2026-02-24T07:49:41.888452+0000","mapping_epoch":27,"log_start":"29'96","ondisk_log_start":"29'96","created":20,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'99","last_scrub_stamp":"2026-02-24T07:49:06.478534+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:42:43.640776+0000","last_clean_scrub_stamp":"2026-02-24T07:49:06.478534+0000","objects_scrubbed":2,"log_size":3,"log_dups_size":96,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:50:26.893586+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":240,"num_write":135,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":16,"stat_sum":{"num_bytes":32823246848,"num_objects":500843,"num_object_clones":0,"num_object_copies":1502529,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":500843,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":500843,"num_write_kb":32053952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":98464497664,"data_stored":98464497664,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":48,"ondisk_log_size":48,"up":48,"acting":48,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":240,"num_write":135,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":3,"up_from":25,"seq":107374182489,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":25540040,"kb_used_data":23866036,"kb_used_omap":12,"kb_used_meta":1673971,"kb_avail":708463160,"statfs":{"total":751619276800,"available":725466275840,"internally_reserved":0,"allocated":24438820864,"data_stored":24438407380,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12605,"internal_metadata":1714147011},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":19,"seq":81604378715,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":29676040,"kb_used_data":27901108,"kb_used_omap":13,"kb_used_meta":1774834,"kb_avail":704327160,"statfs":{"total":751619276800,"available":721231011840,"internally_reserved":0,"allocated":28570734592,"data_stored":28570321108,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13640,"internal_metadata":1817430712},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542236,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":21722372,"kb_used_data":19990448,"kb_used_omap":13,"kb_used_meta":1731890,"kb_avail":712280828,"statfs":{"total":751619276800,"available":729375567872,"internally_reserved":0,"allocated":20470218752,"data_stored":20469808820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13503,"internal_metadata":1773456193},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[5,1],"upper_bound":4},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705759,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":25564616,"kb_used_data":23938292,"kb_used_omap":10,"kb_used_meta":1626293,"kb_avail":708438584,"statfs":{"total":751619276800,"available":725441110016,"internally_reserved":0,"allocated":24512811008,"data_stored":24512397524,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":11186,"internal_metadata":1665324110},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":24581111808,"data_stored":24581111808,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":20552286208,"data_stored":20552286208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":28683927552,"data_stored":28683927552,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":24647172096,"data_stored":24647172096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-24T07:50:09.747 INFO:tasks.cephadm.ceph_manager.ceph:increase pool size by 10 2026-02-24T07:50:09.748 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph osd pool set unique_pool_2 pg_num 36 2026-02-24T07:50:09.902 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:50:10.000 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 71 16 524378 524362 461.545 293 0.00248098 0.00216526 2026-02-24T07:50:10.182 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:50:09 trial035 ceph-mon[39284]: from='client.14975 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2026-02-24T07:50:10.182 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:50:09 trial035 ceph-mon[39284]: 4.9 scrub starts 2026-02-24T07:50:10.255 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:50:10.254+0000 7fbaab930640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:50:10.255 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:50:10.254+0000 7fbaab930640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:50:10.395 INFO:teuthology.orchestra.run.trial035.stderr:Error EPERM: crimson_allow_pg_split is false; pg_num increase denied for crimson pool 2026-02-24T07:50:10.435 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-24T07:50:11.000 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 72 16 530113 530097 460.113 358.438 0.00626365 0.00217216 2026-02-24T07:50:11.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:50:10 trial035 ceph-mon[39284]: pgmap v250: 17 pgs: 1 active+clean+scrubbing, 16 active+clean; 31 GiB data, 98 GiB used, 2.6 TiB / 2.7 TiB avail; 351 MiB/s wr, 5.61k op/s 2026-02-24T07:50:11.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:50:10 trial035 ceph-mon[39284]: 4.3 scrub ok 2026-02-24T07:50:11.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:50:10 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/3713756270' entity='client.admin' cmd={"prefix": "osd pool set", "pool": "unique_pool_2", "var": "pg_num", "val": "36"} : dispatch 2026-02-24T07:50:11.646 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2026-02-24T07:50:11.791 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:50:11.943 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:50:11 trial035 ceph-mon[39284]: 4.6 scrub starts 2026-02-24T07:50:12.000 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 73 16 536189 536173 459.012 379.75 0.00563235 0.00217703 2026-02-24T07:50:12.241 INFO:teuthology.orchestra.run.trial035.stdout:osd.0: {} 2026-02-24T07:50:12.241 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:50:12.195+0000 7fe407452640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:50:12.242 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:50:12.196+0000 7fe407452640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:50:12.242 INFO:teuthology.orchestra.run.trial035.stderr:osd.0: osd_enable_op_tracker = '' 2026-02-24T07:50:12.242 INFO:teuthology.orchestra.run.trial035.stdout:osd.1: {} 2026-02-24T07:50:12.242 INFO:teuthology.orchestra.run.trial035.stderr:osd.1: osd_enable_op_tracker = '' 2026-02-24T07:50:12.257 INFO:teuthology.orchestra.run.trial035.stdout:osd.2: {} 2026-02-24T07:50:12.257 INFO:teuthology.orchestra.run.trial035.stderr:osd.2: osd_enable_op_tracker = '' 2026-02-24T07:50:12.270 INFO:teuthology.orchestra.run.trial035.stdout:osd.3: {} 2026-02-24T07:50:12.270 INFO:teuthology.orchestra.run.trial035.stderr:osd.3: osd_enable_op_tracker = '' 2026-02-24T07:50:13.000 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 74 15 540754 540739 456.665 285.375 0.00117641 0.00218857 2026-02-24T07:50:13.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:50:12 trial035 ceph-mon[39284]: 4.7 scrub ok 2026-02-24T07:50:13.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:50:12 trial035 ceph-mon[39284]: pgmap v251: 17 pgs: 3 active+clean+scrubbing, 14 active+clean; 32 GiB data, 103 GiB used, 2.6 TiB / 2.7 TiB avail; 496 MiB/s wr, 7.93k op/s 2026-02-24T07:50:13.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:50:12 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/234641904' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2026-02-24T07:50:14.000 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 75 15 547731 547716 456.39 436.062 0.00153487 0.00218991 2026-02-24T07:50:14.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:50:13 trial035 ceph-mon[39284]: 4.9 scrub ok 2026-02-24T07:50:14.832 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2026-02-24T07:50:14.979 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:50:15.000 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 76 15 554601 554586 456.034 429.375 0.00159082 0.00219161 2026-02-24T07:50:15.115 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:50:15.113+0000 7f250ded5640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:50:15.115 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:50:15.113+0000 7f250ded5640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:50:15.116 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:50:14 trial035 ceph-mon[39284]: 4.4 scrub starts 2026-02-24T07:50:15.116 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:50:14 trial035 ceph-mon[39284]: pgmap v252: 17 pgs: 3 active+clean+scrubbing, 14 active+clean; 32 GiB data, 103 GiB used, 2.6 TiB / 2.7 TiB avail; 303 MiB/s wr, 4.85k op/s 2026-02-24T07:50:15.116 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:50:14 trial035 ceph-mon[39284]: purged_snaps scrub starts 2026-02-24T07:50:15.116 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:50:14 trial035 ceph-mon[39284]: purged_snaps scrub ok 2026-02-24T07:50:15.116 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:50:14 trial035 ceph-mon[39284]: from='mgr.14284 10.20.193.35:0/3681373222' entity='mgr.x' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2026-02-24T07:50:15.133 INFO:teuthology.orchestra.run.trial035.stdout:osd.0: {} 2026-02-24T07:50:15.134 INFO:teuthology.orchestra.run.trial035.stderr:osd.0: osd_enable_op_tracker = '' 2026-02-24T07:50:15.149 INFO:teuthology.orchestra.run.trial035.stdout:osd.1: {} 2026-02-24T07:50:15.149 INFO:teuthology.orchestra.run.trial035.stderr:osd.1: osd_enable_op_tracker = '' 2026-02-24T07:50:15.163 INFO:teuthology.orchestra.run.trial035.stdout:osd.2: {} 2026-02-24T07:50:15.163 INFO:teuthology.orchestra.run.trial035.stderr:osd.2: osd_enable_op_tracker = '' 2026-02-24T07:50:15.176 INFO:teuthology.orchestra.run.trial035.stdout:osd.3: {} 2026-02-24T07:50:15.176 INFO:teuthology.orchestra.run.trial035.stderr:osd.3: osd_enable_op_tracker = '' 2026-02-24T07:50:15.436 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2026-02-24T07:50:15.437 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2026-02-24T07:50:15.437 INFO:tasks.thrashosds.thrasher:Growing pool unique_pool_2 2026-02-24T07:50:15.437 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph pg dump --format=json 2026-02-24T07:50:15.580 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:50:15.711 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:50:15.708+0000 7f1883887640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:50:15.711 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:50:15.708+0000 7f1883887640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:50:15.846 INFO:teuthology.orchestra.run.trial035.stdout: 2026-02-24T07:50:15.847 INFO:teuthology.orchestra.run.trial035.stderr:dumped all 2026-02-24T07:50:15.892 INFO:teuthology.orchestra.run.trial035.stdout:{"pg_ready":true,"pg_map":{"version":253,"stamp":"2026-02-24T07:50:15.716182+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":34649211424,"num_objects":528698,"num_object_clones":0,"num_object_copies":1586094,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":528698,"num_whiteouts":0,"num_read":136,"num_read_kb":240,"num_write":528831,"num_write_kb":33838694,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":51,"ondisk_log_size":51,"up":51,"acting":51,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":51,"num_osds":4,"num_per_pool_osds":4,"num_per_pool_omap_osds":4,"kb":2936012800,"kb_used":108218588,"kb_used_data":101104652,"kb_used_omap":49,"kb_used_meta":7113806,"kb_avail":2827794212,"statfs":{"total":3006477107200,"available":2895661273088,"internally_reserved":0,"allocated":103531163648,"data_stored":103529513264,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":50934,"internal_metadata":7284537610},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[7,4,9,2,1],"upper_bound":32},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":3816095744,"num_objects":58229,"num_object_clones":0,"num_object_copies":174687,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58229,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":58229,"num_write_kb":3726656,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001594"},"pg_stats":[{"pgid":"4.f","version":"61'33289","reported_seq":33622,"reported_epoch":61,"state":"active+clean","last_fresh":"2026-02-24T07:50:10.826468+0000","last_change":"2026-02-24T07:50:03.071459+0000","last_active":"2026-02-24T07:50:10.826468+0000","last_peered":"2026-02-24T07:50:10.826468+0000","last_clean":"2026-02-24T07:50:10.826468+0000","last_became_active":"2026-02-24T07:48:57.902313+0000","last_became_peered":"2026-02-24T07:48:57.902313+0000","last_unstale":"2026-02-24T07:50:10.826468+0000","last_undegraded":"2026-02-24T07:50:10.826468+0000","last_fullsized":"2026-02-24T07:50:10.826468+0000","mapping_epoch":55,"log_start":"61'33286","ondisk_log_start":"61'33286","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"61'30419","last_scrub_stamp":"2026-02-24T07:50:03.071380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:50:03.071380+0000","objects_scrubbed":29975,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:51:16.123453+0000","scrub_duration":2568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2181627904,"num_objects":33289,"num_object_clones":0,"num_object_copies":99867,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33289,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":33289,"num_write_kb":2130496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"61'33131","reported_seq":33153,"reported_epoch":61,"state":"active+clean","last_fresh":"2026-02-24T07:50:10.719558+0000","last_change":"2026-02-24T07:48:57.903920+0000","last_active":"2026-02-24T07:50:10.719558+0000","last_peered":"2026-02-24T07:50:10.719558+0000","last_clean":"2026-02-24T07:50:10.719558+0000","last_became_active":"2026-02-24T07:48:57.903584+0000","last_became_peered":"2026-02-24T07:48:57.903584+0000","last_unstale":"2026-02-24T07:50:10.719558+0000","last_undegraded":"2026-02-24T07:50:10.719558+0000","last_fullsized":"2026-02-24T07:50:10.719558+0000","mapping_epoch":55,"log_start":"61'33128","ondisk_log_start":"61'33128","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:48:56.890768+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:50:17.159571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2171273216,"num_objects":33131,"num_object_clones":0,"num_object_copies":99393,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33131,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":33131,"num_write_kb":2120384,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.d","version":"61'33018","reported_seq":33360,"reported_epoch":61,"state":"active+clean","last_fresh":"2026-02-24T07:50:10.737044+0000","last_change":"2026-02-24T07:50:06.701973+0000","last_active":"2026-02-24T07:50:10.737044+0000","last_peered":"2026-02-24T07:50:10.737044+0000","last_clean":"2026-02-24T07:50:10.737044+0000","last_became_active":"2026-02-24T07:48:57.905031+0000","last_became_peered":"2026-02-24T07:48:57.905031+0000","last_unstale":"2026-02-24T07:50:10.737044+0000","last_undegraded":"2026-02-24T07:50:10.737044+0000","last_fullsized":"2026-02-24T07:50:10.737044+0000","mapping_epoch":55,"log_start":"61'33015","ondisk_log_start":"61'33015","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"61'31648","last_scrub_stamp":"2026-02-24T07:50:06.701883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:50:06.701883+0000","objects_scrubbed":30990,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:51:19.224966+0000","scrub_duration":3493,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2163867648,"num_objects":33018,"num_object_clones":0,"num_object_copies":99054,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33018,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":33018,"num_write_kb":2113152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"61'32974","reported_seq":32996,"reported_epoch":61,"state":"active+clean","last_fresh":"2026-02-24T07:50:10.745032+0000","last_change":"2026-02-24T07:48:57.905290+0000","last_active":"2026-02-24T07:50:10.745032+0000","last_peered":"2026-02-24T07:50:10.745032+0000","last_clean":"2026-02-24T07:50:10.745032+0000","last_became_active":"2026-02-24T07:48:57.904683+0000","last_became_peered":"2026-02-24T07:48:57.904683+0000","last_unstale":"2026-02-24T07:50:10.745032+0000","last_undegraded":"2026-02-24T07:50:10.745032+0000","last_fullsized":"2026-02-24T07:50:10.745032+0000","mapping_epoch":55,"log_start":"61'32971","ondisk_log_start":"61'32971","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:48:56.890768+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:50:16.643522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2160984064,"num_objects":32974,"num_object_clones":0,"num_object_copies":98922,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32974,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":32974,"num_write_kb":2110336,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.b","version":"61'32808","reported_seq":32830,"reported_epoch":61,"state":"active+clean","last_fresh":"2026-02-24T07:50:10.738584+0000","last_change":"2026-02-24T07:48:57.902025+0000","last_active":"2026-02-24T07:50:10.738584+0000","last_peered":"2026-02-24T07:50:10.738584+0000","last_clean":"2026-02-24T07:50:10.738584+0000","last_became_active":"2026-02-24T07:48:57.901700+0000","last_became_peered":"2026-02-24T07:48:57.901700+0000","last_unstale":"2026-02-24T07:50:10.738584+0000","last_undegraded":"2026-02-24T07:50:10.738584+0000","last_fullsized":"2026-02-24T07:50:10.738584+0000","mapping_epoch":55,"log_start":"61'32805","ondisk_log_start":"61'32805","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:48:56.890768+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:50:23.370319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2150105088,"num_objects":32808,"num_object_clones":0,"num_object_copies":98424,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32808,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":32808,"num_write_kb":2099712,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.a","version":"61'33170","reported_seq":33192,"reported_epoch":61,"state":"active+clean","last_fresh":"2026-02-24T07:50:10.711900+0000","last_change":"2026-02-24T07:48:57.903698+0000","last_active":"2026-02-24T07:50:10.711900+0000","last_peered":"2026-02-24T07:50:10.711900+0000","last_clean":"2026-02-24T07:50:10.711900+0000","last_became_active":"2026-02-24T07:48:57.903228+0000","last_became_peered":"2026-02-24T07:48:57.903228+0000","last_unstale":"2026-02-24T07:50:10.711900+0000","last_undegraded":"2026-02-24T07:50:10.711900+0000","last_fullsized":"2026-02-24T07:50:10.711900+0000","mapping_epoch":55,"log_start":"61'33167","ondisk_log_start":"61'33167","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:48:56.890768+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:50:26.780637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2173829120,"num_objects":33170,"num_object_clones":0,"num_object_copies":99510,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33170,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":33170,"num_write_kb":2122880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"61'33050","reported_seq":33198,"reported_epoch":61,"state":"active+clean+scrubbing","last_fresh":"2026-02-24T07:50:10.719970+0000","last_change":"2026-02-24T07:50:09.388814+0000","last_active":"2026-02-24T07:50:10.719970+0000","last_peered":"2026-02-24T07:50:10.719970+0000","last_clean":"2026-02-24T07:50:10.719970+0000","last_became_active":"2026-02-24T07:48:57.901730+0000","last_became_peered":"2026-02-24T07:48:57.901730+0000","last_unstale":"2026-02-24T07:50:10.719970+0000","last_undegraded":"2026-02-24T07:50:10.719970+0000","last_fullsized":"2026-02-24T07:50:10.719970+0000","mapping_epoch":55,"log_start":"61'33047","ondisk_log_start":"61'33047","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:48:56.890768+0000","objects_scrubbed":12299,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"scrubbing for 2s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2165964800,"num_objects":33050,"num_object_clones":0,"num_object_copies":99150,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33050,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":33050,"num_write_kb":2115200,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"61'32936","reported_seq":33265,"reported_epoch":61,"state":"active+clean","last_fresh":"2026-02-24T07:50:10.740219+0000","last_change":"2026-02-24T07:50:03.153499+0000","last_active":"2026-02-24T07:50:10.740219+0000","last_peered":"2026-02-24T07:50:10.740219+0000","last_clean":"2026-02-24T07:50:10.740219+0000","last_became_active":"2026-02-24T07:48:57.904376+0000","last_became_peered":"2026-02-24T07:48:57.904376+0000","last_unstale":"2026-02-24T07:50:10.740219+0000","last_undegraded":"2026-02-24T07:50:10.740219+0000","last_fullsized":"2026-02-24T07:50:10.740219+0000","mapping_epoch":55,"log_start":"61'32933","ondisk_log_start":"61'32933","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"61'30163","last_scrub_stamp":"2026-02-24T07:50:03.153454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:50:03.153454+0000","objects_scrubbed":29681,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:51:10.266136+0000","scrub_duration":2916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2158493696,"num_objects":32936,"num_object_clones":0,"num_object_copies":98808,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32936,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":32936,"num_write_kb":2107904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.7","version":"61'32851","reported_seq":33154,"reported_epoch":61,"state":"active+clean+scrubbing","last_fresh":"2026-02-24T07:50:10.826924+0000","last_change":"2026-02-24T07:50:07.527150+0000","last_active":"2026-02-24T07:50:10.826924+0000","last_peered":"2026-02-24T07:50:10.826924+0000","last_clean":"2026-02-24T07:50:10.826924+0000","last_became_active":"2026-02-24T07:48:57.902484+0000","last_became_peered":"2026-02-24T07:48:57.902484+0000","last_unstale":"2026-02-24T07:50:10.826924+0000","last_undegraded":"2026-02-24T07:50:10.826924+0000","last_fullsized":"2026-02-24T07:50:10.826924+0000","mapping_epoch":55,"log_start":"61'32848","ondisk_log_start":"61'32848","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:48:56.890768+0000","objects_scrubbed":27849,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"scrubbing for 4s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2152923136,"num_objects":32851,"num_object_clones":0,"num_object_copies":98553,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32851,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":32851,"num_write_kb":2102464,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.6","version":"61'32864","reported_seq":32904,"reported_epoch":61,"state":"active+clean+scrubbing","last_fresh":"2026-02-24T07:50:10.735642+0000","last_change":"2026-02-24T07:50:10.596389+0000","last_active":"2026-02-24T07:50:10.735642+0000","last_peered":"2026-02-24T07:50:10.735642+0000","last_clean":"2026-02-24T07:50:10.735642+0000","last_became_active":"2026-02-24T07:48:57.901565+0000","last_became_peered":"2026-02-24T07:48:57.901565+0000","last_unstale":"2026-02-24T07:50:10.735642+0000","last_undegraded":"2026-02-24T07:50:10.735642+0000","last_fullsized":"2026-02-24T07:50:10.735642+0000","mapping_epoch":55,"log_start":"61'32861","ondisk_log_start":"61'32861","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:48:56.890768+0000","objects_scrubbed":1499,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"scrubbing for 1s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2153775104,"num_objects":32864,"num_object_clones":0,"num_object_copies":98592,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32864,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":32864,"num_write_kb":2103296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"61'33290","reported_seq":33312,"reported_epoch":61,"state":"active+clean","last_fresh":"2026-02-24T07:50:10.829145+0000","last_change":"2026-02-24T07:48:57.902165+0000","last_active":"2026-02-24T07:50:10.829145+0000","last_peered":"2026-02-24T07:50:10.829145+0000","last_clean":"2026-02-24T07:50:10.829145+0000","last_became_active":"2026-02-24T07:48:57.901824+0000","last_became_peered":"2026-02-24T07:48:57.901824+0000","last_unstale":"2026-02-24T07:50:10.829145+0000","last_undegraded":"2026-02-24T07:50:10.829145+0000","last_fullsized":"2026-02-24T07:50:10.829145+0000","mapping_epoch":55,"log_start":"61'33287","ondisk_log_start":"61'33287","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:48:56.890768+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:50:12.091202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2181693440,"num_objects":33290,"num_object_clones":0,"num_object_copies":99870,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33290,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":33290,"num_write_kb":2130560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[1,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"61'32849","reported_seq":33202,"reported_epoch":61,"state":"active+clean","last_fresh":"2026-02-24T07:50:10.724879+0000","last_change":"2026-02-24T07:50:10.124042+0000","last_active":"2026-02-24T07:50:10.724879+0000","last_peered":"2026-02-24T07:50:10.724879+0000","last_clean":"2026-02-24T07:50:10.724879+0000","last_became_active":"2026-02-24T07:48:57.902224+0000","last_became_peered":"2026-02-24T07:48:57.902224+0000","last_unstale":"2026-02-24T07:50:10.724879+0000","last_undegraded":"2026-02-24T07:50:10.724879+0000","last_fullsized":"2026-02-24T07:50:10.724879+0000","mapping_epoch":55,"log_start":"61'32846","ondisk_log_start":"61'32846","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"61'32620","last_scrub_stamp":"2026-02-24T07:50:10.124005+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:50:10.124005+0000","objects_scrubbed":32078,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:51:12.877892+0000","scrub_duration":3559,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2152792064,"num_objects":32849,"num_object_clones":0,"num_object_copies":98547,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32849,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":32849,"num_write_kb":2102336,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"61'33193","reported_seq":33518,"reported_epoch":61,"state":"active+clean","last_fresh":"2026-02-24T07:50:10.829222+0000","last_change":"2026-02-24T07:50:01.398116+0000","last_active":"2026-02-24T07:50:10.829222+0000","last_peered":"2026-02-24T07:50:10.829222+0000","last_clean":"2026-02-24T07:50:10.829222+0000","last_became_active":"2026-02-24T07:48:57.902125+0000","last_became_peered":"2026-02-24T07:48:57.902125+0000","last_unstale":"2026-02-24T07:50:10.829222+0000","last_undegraded":"2026-02-24T07:50:10.829222+0000","last_fullsized":"2026-02-24T07:50:10.829222+0000","mapping_epoch":55,"log_start":"61'33190","ondisk_log_start":"61'33190","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"61'29707","last_scrub_stamp":"2026-02-24T07:50:01.398090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:50:01.398090+0000","objects_scrubbed":29171,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:51:10.566529+0000","scrub_duration":2880,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2175336448,"num_objects":33193,"num_object_clones":0,"num_object_copies":99579,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33193,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":33193,"num_write_kb":2124352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1","version":"61'33248","reported_seq":33270,"reported_epoch":61,"state":"active+clean","last_fresh":"2026-02-24T07:50:10.719985+0000","last_change":"2026-02-24T07:48:57.903745+0000","last_active":"2026-02-24T07:50:10.719985+0000","last_peered":"2026-02-24T07:50:10.719985+0000","last_clean":"2026-02-24T07:50:10.719985+0000","last_became_active":"2026-02-24T07:48:57.903439+0000","last_became_peered":"2026-02-24T07:48:57.903439+0000","last_unstale":"2026-02-24T07:50:10.719985+0000","last_undegraded":"2026-02-24T07:50:10.719985+0000","last_fullsized":"2026-02-24T07:50:10.719985+0000","mapping_epoch":55,"log_start":"61'33245","ondisk_log_start":"61'33245","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:48:56.890768+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:50:24.058110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2178940928,"num_objects":33248,"num_object_clones":0,"num_object_copies":99744,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33248,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":33248,"num_write_kb":2127872,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.0","version":"61'33060","reported_seq":33082,"reported_epoch":61,"state":"active+clean","last_fresh":"2026-02-24T07:50:10.741483+0000","last_change":"2026-02-24T07:48:57.902838+0000","last_active":"2026-02-24T07:50:10.741483+0000","last_peered":"2026-02-24T07:50:10.741483+0000","last_clean":"2026-02-24T07:50:10.741483+0000","last_became_active":"2026-02-24T07:48:57.902565+0000","last_became_peered":"2026-02-24T07:48:57.902565+0000","last_unstale":"2026-02-24T07:50:10.741483+0000","last_undegraded":"2026-02-24T07:50:10.741483+0000","last_fullsized":"2026-02-24T07:50:10.741483+0000","mapping_epoch":55,"log_start":"61'33057","ondisk_log_start":"61'33057","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:48:56.890768+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:50:25.079534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2166620160,"num_objects":33060,"num_object_clones":0,"num_object_copies":99180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33060,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":33060,"num_write_kb":2115840,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,3],"acting":[0,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.5","version":"61'32965","reported_seq":33300,"reported_epoch":61,"state":"active+clean","last_fresh":"2026-02-24T07:50:10.738872+0000","last_change":"2026-02-24T07:50:04.821914+0000","last_active":"2026-02-24T07:50:10.738872+0000","last_peered":"2026-02-24T07:50:10.738872+0000","last_clean":"2026-02-24T07:50:10.738872+0000","last_became_active":"2026-02-24T07:48:57.905127+0000","last_became_peered":"2026-02-24T07:48:57.905127+0000","last_unstale":"2026-02-24T07:50:10.738872+0000","last_undegraded":"2026-02-24T07:50:10.738872+0000","last_fullsized":"2026-02-24T07:50:10.738872+0000","mapping_epoch":55,"log_start":"61'32962","ondisk_log_start":"61'32962","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"61'30833","last_scrub_stamp":"2026-02-24T07:50:04.821861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:50:04.821861+0000","objects_scrubbed":30132,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:51:15.928207+0000","scrub_duration":3578,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2160394240,"num_objects":32965,"num_object_clones":0,"num_object_copies":98895,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32965,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":32965,"num_write_kb":2109760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"29'99","reported_seq":285,"reported_epoch":61,"state":"active+clean","last_fresh":"2026-02-24T07:49:41.888452+0000","last_change":"2026-02-24T07:49:06.478572+0000","last_active":"2026-02-24T07:49:41.888452+0000","last_peered":"2026-02-24T07:49:41.888452+0000","last_clean":"2026-02-24T07:49:41.888452+0000","last_became_active":"2026-02-24T07:42:54.186719+0000","last_became_peered":"2026-02-24T07:42:54.186719+0000","last_unstale":"2026-02-24T07:49:41.888452+0000","last_undegraded":"2026-02-24T07:49:41.888452+0000","last_fullsized":"2026-02-24T07:49:41.888452+0000","mapping_epoch":27,"log_start":"29'96","ondisk_log_start":"29'96","created":20,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'99","last_scrub_stamp":"2026-02-24T07:49:06.478534+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:42:43.640776+0000","last_clean_scrub_stamp":"2026-02-24T07:49:06.478534+0000","objects_scrubbed":2,"log_size":3,"log_dups_size":96,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:50:26.893586+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":240,"num_write":135,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":16,"stat_sum":{"num_bytes":34648621056,"num_objects":528696,"num_object_clones":0,"num_object_copies":1586088,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":528696,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":528696,"num_write_kb":33836544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":103941472256,"data_stored":103941472256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":48,"ondisk_log_size":48,"up":48,"acting":48,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":240,"num_write":135,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":3,"up_from":25,"seq":107374182490,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":26993480,"kb_used_data":25247284,"kb_used_omap":12,"kb_used_meta":1746163,"kb_avail":707009720,"statfs":{"total":751619276800,"available":723977953280,"internally_reserved":0,"allocated":25853218816,"data_stored":25852805332,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12605,"internal_metadata":1788071619},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,3,2],"upper_bound":8},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":19,"seq":81604378716,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":31333768,"kb_used_data":29468660,"kb_used_omap":13,"kb_used_meta":1865074,"kb_avail":702669432,"statfs":{"total":751619276800,"available":719533498368,"internally_reserved":0,"allocated":30175907840,"data_stored":30175494356,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13640,"internal_metadata":1909836472},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542237,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":22885636,"kb_used_data":21081520,"kb_used_omap":13,"kb_used_meta":1804082,"kb_avail":711117564,"statfs":{"total":751619276800,"available":728184385536,"internally_reserved":0,"allocated":21587476480,"data_stored":21587066548,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13503,"internal_metadata":1847380801},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1],"upper_bound":4},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705760,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":27005704,"kb_used_data":25307188,"kb_used_omap":10,"kb_used_meta":1698485,"kb_avail":706997496,"statfs":{"total":751619276800,"available":723965435904,"internally_reserved":0,"allocated":25914560512,"data_stored":25914147028,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":11186,"internal_metadata":1739248718},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,7,2,1],"upper_bound":32},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":25955205120,"data_stored":25955205120,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":21682257920,"data_stored":21682257920,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":30282547200,"data_stored":30282547200,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":26021462016,"data_stored":26021462016,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-24T07:50:15.894 INFO:tasks.cephadm.ceph_manager.ceph:increase pool size by 10 2026-02-24T07:50:15.894 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph osd pool set unique_pool_2 pg_num 46 2026-02-24T07:50:16.000 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 77 16 561836 561820 455.983 452.125 0.00267085 0.00219187 2026-02-24T07:50:16.043 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:50:16.102 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:50:15 trial035 ceph-mon[39284]: 4.6 scrub ok 2026-02-24T07:50:16.102 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:50:15 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/663563033' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2026-02-24T07:50:16.178 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:50:16.175+0000 7f4d548b2640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:50:16.178 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:50:16.176+0000 7f4d548b2640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:50:16.314 INFO:teuthology.orchestra.run.trial035.stderr:Error EPERM: crimson_allow_pg_split is false; pg_num increase denied for crimson pool 2026-02-24T07:50:16.353 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-24T07:50:17.000 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 78 16 569625 569609 456.377 486.812 0.00207958 0.00218997 2026-02-24T07:50:17.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:50:16 trial035 ceph-mon[39284]: pgmap v253: 17 pgs: 3 active+clean+scrubbing, 14 active+clean; 32 GiB data, 103 GiB used, 2.6 TiB / 2.7 TiB avail; 303 MiB/s wr, 4.85k op/s 2026-02-24T07:50:17.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:50:16 trial035 ceph-mon[39284]: from='client.14985 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2026-02-24T07:50:17.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:50:16 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/1045875616' entity='client.admin' cmd={"prefix": "osd pool set", "pool": "unique_pool_2", "var": "pg_num", "val": "46"} : dispatch 2026-02-24T07:50:17.204 INFO:journalctl@ceph.mgr.x.trial035.stdout:Feb 24 07:50:16 trial035 ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-mgr-x[39621]: ::ffff:10.20.193.91 - - [24/Feb/2026:07:50:16] "GET /metrics HTTP/1.1" 200 4214 "" "Prometheus/3.6.0" 2026-02-24T07:50:17.225 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2026-02-24T07:50:17.262 INFO:journalctl@ceph.prometheus.a.trial091.stdout:Feb 24 07:50:16 trial091 ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-prometheus-a[66007]: time=2026-02-24T07:50:16.973Z level=ERROR source=scrape.go:1630 msg="Failed to determine correct type of scrape target." component="scrape manager" scrape_pool=ceph target=http://trial035:9283/metrics content_type="text/html;charset=utf-8" fallback_media_type="" err="received unsupported Content-Type \"text/html;charset=utf-8\" and no fallback_scrape_protocol specified for target" 2026-02-24T07:50:17.369 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:50:17.490 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:50:17.488+0000 7f2ba3a8a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:50:17.490 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:50:17.488+0000 7f2ba3a8a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:50:17.506 INFO:teuthology.orchestra.run.trial035.stdout:osd.0: {} 2026-02-24T07:50:17.506 INFO:teuthology.orchestra.run.trial035.stderr:osd.0: osd_enable_op_tracker = '' 2026-02-24T07:50:17.521 INFO:teuthology.orchestra.run.trial035.stdout:osd.1: {} 2026-02-24T07:50:17.521 INFO:teuthology.orchestra.run.trial035.stderr:osd.1: osd_enable_op_tracker = '' 2026-02-24T07:50:17.535 INFO:teuthology.orchestra.run.trial035.stdout:osd.2: {} 2026-02-24T07:50:17.535 INFO:teuthology.orchestra.run.trial035.stderr:osd.2: osd_enable_op_tracker = '' 2026-02-24T07:50:17.550 INFO:teuthology.orchestra.run.trial035.stdout:osd.3: {} 2026-02-24T07:50:17.550 INFO:teuthology.orchestra.run.trial035.stderr:osd.3: osd_enable_op_tracker = '' 2026-02-24T07:50:18.000 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 79 16 576547 576531 456.076 432.625 0.00160175 0.0021914 2026-02-24T07:50:18.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:50:17 trial035 ceph-mon[39284]: 4.4 scrub ok 2026-02-24T07:50:18.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:50:17 trial035 ceph-mon[39284]: 4.e scrub starts 2026-02-24T07:50:18.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:50:17 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/2462899677' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2026-02-24T07:50:19.000 INFO:tasks.radosbench.radosbench.0.trial035.stdout:2026-02-24T07:50:19.000300+0000 min lat: 0.000682058 max lat: 0.0946675 avg lat: 0.00219232 2026-02-24T07:50:19.000 INFO:tasks.radosbench.radosbench.0.trial035.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2026-02-24T07:50:19.000 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 80 15 583608 583593 455.892 441.375 0.0018969 0.00219232 2026-02-24T07:50:19.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:50:18 trial035 ceph-mon[39284]: pgmap v254: 17 pgs: 1 active+clean+scrubbing, 16 active+clean; 34 GiB data, 109 GiB used, 2.6 TiB / 2.7 TiB avail; 467 MiB/s wr, 7.47k op/s 2026-02-24T07:50:19.591 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2026-02-24T07:50:19.730 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:50:20.001 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 81 16 590314 590298 455.437 419.062 0.00153807 0.00219447 2026-02-24T07:50:20.138 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:50:20.122+0000 7f18f7854640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:50:20.138 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:50:20.123+0000 7f18f7854640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:50:20.150 INFO:teuthology.orchestra.run.trial035.stdout:osd.0: {} 2026-02-24T07:50:20.150 INFO:teuthology.orchestra.run.trial035.stderr:osd.0: osd_enable_op_tracker = '' 2026-02-24T07:50:20.151 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:50:19 trial035 ceph-mon[39284]: pgmap v255: 17 pgs: 1 active+clean+scrubbing, 16 active+clean; 34 GiB data, 109 GiB used, 2.6 TiB / 2.7 TiB avail; 309 MiB/s wr, 4.94k op/s 2026-02-24T07:50:20.167 INFO:teuthology.orchestra.run.trial035.stdout:osd.1: {} 2026-02-24T07:50:20.167 INFO:teuthology.orchestra.run.trial035.stderr:osd.1: osd_enable_op_tracker = '' 2026-02-24T07:50:20.180 INFO:teuthology.orchestra.run.trial035.stdout:osd.2: {} 2026-02-24T07:50:20.181 INFO:teuthology.orchestra.run.trial035.stderr:osd.2: osd_enable_op_tracker = '' 2026-02-24T07:50:20.194 INFO:teuthology.orchestra.run.trial035.stdout:osd.3: {} 2026-02-24T07:50:20.195 INFO:teuthology.orchestra.run.trial035.stderr:osd.3: osd_enable_op_tracker = '' 2026-02-24T07:50:21.001 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 82 16 596730 596714 454.773 401 0.00189103 0.0021977 2026-02-24T07:50:21.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:50:20 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/1960251611' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2026-02-24T07:50:21.354 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2026-02-24T07:50:21.354 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2026-02-24T07:50:21.355 INFO:tasks.thrashosds.thrasher:Growing pool .mgr 2026-02-24T07:50:21.355 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph pg dump --format=json 2026-02-24T07:50:21.505 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:50:21.625 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:50:21.623+0000 7fae17640640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:50:21.626 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:50:21.623+0000 7fae17640640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:50:21.760 INFO:teuthology.orchestra.run.trial035.stdout: 2026-02-24T07:50:21.761 INFO:teuthology.orchestra.run.trial035.stderr:dumped all 2026-02-24T07:50:21.807 INFO:teuthology.orchestra.run.trial035.stdout:{"pg_ready":true,"pg_map":{"version":256,"stamp":"2026-02-24T07:50:21.717002+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":38995427872,"num_objects":595016,"num_object_clones":0,"num_object_copies":1785048,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":595016,"num_whiteouts":0,"num_read":136,"num_read_kb":240,"num_write":595149,"num_write_kb":38083046,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":52,"ondisk_log_size":52,"up":51,"acting":51,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":51,"num_osds":4,"num_per_pool_osds":4,"num_per_pool_omap_osds":4,"kb":2936012800,"kb_used":121619612,"kb_used_data":113801868,"kb_used_omap":51,"kb_used_meta":7817612,"kb_avail":2814393188,"statfs":{"total":3006477107200,"available":2881938624512,"internally_reserved":0,"allocated":116533112832,"data_stored":116531462448,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":52571,"internal_metadata":8005235365},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[6,3,1,1],"upper_bound":16},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":6171590656,"num_objects":94171,"num_object_clones":0,"num_object_copies":282513,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":94171,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":94171,"num_write_kb":6026944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001593"},"pg_stats":[{"pgid":"4.f","version":"61'37400","reported_seq":37733,"reported_epoch":61,"state":"active+clean","last_fresh":"2026-02-24T07:50:20.828225+0000","last_change":"2026-02-24T07:50:03.071459+0000","last_active":"2026-02-24T07:50:20.828225+0000","last_peered":"2026-02-24T07:50:20.828225+0000","last_clean":"2026-02-24T07:50:20.828225+0000","last_became_active":"2026-02-24T07:48:57.902313+0000","last_became_peered":"2026-02-24T07:48:57.902313+0000","last_unstale":"2026-02-24T07:50:20.828225+0000","last_undegraded":"2026-02-24T07:50:20.828225+0000","last_fullsized":"2026-02-24T07:50:20.828225+0000","mapping_epoch":55,"log_start":"61'37397","ondisk_log_start":"61'37397","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"61'30419","last_scrub_stamp":"2026-02-24T07:50:03.071380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:50:03.071380+0000","objects_scrubbed":29975,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:51:16.123453+0000","scrub_duration":2568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2451046400,"num_objects":37400,"num_object_clones":0,"num_object_copies":112200,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37400,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":37400,"num_write_kb":2393600,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"61'37327","reported_seq":37685,"reported_epoch":61,"state":"active+clean+scrubbing","last_fresh":"2026-02-24T07:50:20.727821+0000","last_change":"2026-02-24T07:50:17.422930+0000","last_active":"2026-02-24T07:50:20.727821+0000","last_peered":"2026-02-24T07:50:20.727821+0000","last_clean":"2026-02-24T07:50:20.727821+0000","last_became_active":"2026-02-24T07:48:57.903584+0000","last_became_peered":"2026-02-24T07:48:57.903584+0000","last_unstale":"2026-02-24T07:50:20.727821+0000","last_undegraded":"2026-02-24T07:50:20.727821+0000","last_fullsized":"2026-02-24T07:50:20.727821+0000","mapping_epoch":55,"log_start":"61'37324","ondisk_log_start":"61'37324","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:48:56.890768+0000","objects_scrubbed":33099,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"scrubbing for 4s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2446262272,"num_objects":37327,"num_object_clones":0,"num_object_copies":111981,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37327,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":37327,"num_write_kb":2388928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.d","version":"61'37130","reported_seq":37472,"reported_epoch":61,"state":"active+clean","last_fresh":"2026-02-24T07:50:20.742238+0000","last_change":"2026-02-24T07:50:06.701973+0000","last_active":"2026-02-24T07:50:20.742238+0000","last_peered":"2026-02-24T07:50:20.742238+0000","last_clean":"2026-02-24T07:50:20.742238+0000","last_became_active":"2026-02-24T07:48:57.905031+0000","last_became_peered":"2026-02-24T07:48:57.905031+0000","last_unstale":"2026-02-24T07:50:20.742238+0000","last_undegraded":"2026-02-24T07:50:20.742238+0000","last_fullsized":"2026-02-24T07:50:20.742238+0000","mapping_epoch":55,"log_start":"61'37127","ondisk_log_start":"61'37127","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"61'31648","last_scrub_stamp":"2026-02-24T07:50:06.701883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:50:06.701883+0000","objects_scrubbed":30990,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:51:19.224966+0000","scrub_duration":3493,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2433351680,"num_objects":37130,"num_object_clones":0,"num_object_copies":111390,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37130,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":37130,"num_write_kb":2376320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"61'36989","reported_seq":37011,"reported_epoch":61,"state":"active+clean","last_fresh":"2026-02-24T07:50:20.738423+0000","last_change":"2026-02-24T07:48:57.905290+0000","last_active":"2026-02-24T07:50:20.738423+0000","last_peered":"2026-02-24T07:50:20.738423+0000","last_clean":"2026-02-24T07:50:20.738423+0000","last_became_active":"2026-02-24T07:48:57.904683+0000","last_became_peered":"2026-02-24T07:48:57.904683+0000","last_unstale":"2026-02-24T07:50:20.738423+0000","last_undegraded":"2026-02-24T07:50:20.738423+0000","last_fullsized":"2026-02-24T07:50:20.738423+0000","mapping_epoch":55,"log_start":"61'36986","ondisk_log_start":"61'36986","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:48:56.890768+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2424111104,"num_objects":36989,"num_object_clones":0,"num_object_copies":110967,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36989,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":36989,"num_write_kb":2367296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.b","version":"61'36990","reported_seq":37012,"reported_epoch":61,"state":"active+clean","last_fresh":"2026-02-24T07:50:20.739530+0000","last_change":"2026-02-24T07:48:57.902025+0000","last_active":"2026-02-24T07:50:20.739530+0000","last_peered":"2026-02-24T07:50:20.739530+0000","last_clean":"2026-02-24T07:50:20.739530+0000","last_became_active":"2026-02-24T07:48:57.901700+0000","last_became_peered":"2026-02-24T07:48:57.901700+0000","last_unstale":"2026-02-24T07:50:20.739530+0000","last_undegraded":"2026-02-24T07:50:20.739530+0000","last_fullsized":"2026-02-24T07:50:20.739530+0000","mapping_epoch":55,"log_start":"61'36987","ondisk_log_start":"61'36987","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:48:56.890768+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:50:23.370319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2424176640,"num_objects":36990,"num_object_clones":0,"num_object_copies":110970,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36990,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":36990,"num_write_kb":2367360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.a","version":"61'37246","reported_seq":37268,"reported_epoch":61,"state":"active+clean","last_fresh":"2026-02-24T07:50:20.725357+0000","last_change":"2026-02-24T07:48:57.903698+0000","last_active":"2026-02-24T07:50:20.725357+0000","last_peered":"2026-02-24T07:50:20.725357+0000","last_clean":"2026-02-24T07:50:20.725357+0000","last_became_active":"2026-02-24T07:48:57.903228+0000","last_became_peered":"2026-02-24T07:48:57.903228+0000","last_unstale":"2026-02-24T07:50:20.725357+0000","last_undegraded":"2026-02-24T07:50:20.725357+0000","last_fullsized":"2026-02-24T07:50:20.725357+0000","mapping_epoch":55,"log_start":"61'37243","ondisk_log_start":"61'37243","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:48:56.890768+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:50:26.780637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2440953856,"num_objects":37246,"num_object_clones":0,"num_object_copies":111738,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37246,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":37246,"num_write_kb":2383744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"61'37179","reported_seq":37544,"reported_epoch":61,"state":"active+clean","last_fresh":"2026-02-24T07:50:20.724533+0000","last_change":"2026-02-24T07:50:13.165492+0000","last_active":"2026-02-24T07:50:20.724533+0000","last_peered":"2026-02-24T07:50:20.724533+0000","last_clean":"2026-02-24T07:50:20.724533+0000","last_became_active":"2026-02-24T07:48:57.901730+0000","last_became_peered":"2026-02-24T07:48:57.901730+0000","last_unstale":"2026-02-24T07:50:20.724533+0000","last_undegraded":"2026-02-24T07:50:20.724533+0000","last_fullsized":"2026-02-24T07:50:20.724533+0000","mapping_epoch":55,"log_start":"61'37176","ondisk_log_start":"61'37176","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"61'33943","last_scrub_stamp":"2026-02-24T07:50:13.165459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:50:13.165459+0000","objects_scrubbed":33253,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:51:33.820708+0000","scrub_duration":3776,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2436562944,"num_objects":37179,"num_object_clones":0,"num_object_copies":111537,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37179,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":37179,"num_write_kb":2379456,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"61'37208","reported_seq":37537,"reported_epoch":61,"state":"active+clean","last_fresh":"2026-02-24T07:50:20.743122+0000","last_change":"2026-02-24T07:50:03.153499+0000","last_active":"2026-02-24T07:50:20.743122+0000","last_peered":"2026-02-24T07:50:20.743122+0000","last_clean":"2026-02-24T07:50:20.743122+0000","last_became_active":"2026-02-24T07:48:57.904376+0000","last_became_peered":"2026-02-24T07:48:57.904376+0000","last_unstale":"2026-02-24T07:50:20.743122+0000","last_undegraded":"2026-02-24T07:50:20.743122+0000","last_fullsized":"2026-02-24T07:50:20.743122+0000","mapping_epoch":55,"log_start":"61'37205","ondisk_log_start":"61'37205","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"61'30163","last_scrub_stamp":"2026-02-24T07:50:03.153454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:50:03.153454+0000","objects_scrubbed":29681,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:51:10.266136+0000","scrub_duration":2916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2438463488,"num_objects":37208,"num_object_clones":0,"num_object_copies":111624,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37208,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":37208,"num_write_kb":2381312,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.7","version":"61'37004","reported_seq":37360,"reported_epoch":61,"state":"active+clean","last_fresh":"2026-02-24T07:50:20.832719+0000","last_change":"2026-02-24T07:50:11.227608+0000","last_active":"2026-02-24T07:50:20.832719+0000","last_peered":"2026-02-24T07:50:20.832719+0000","last_clean":"2026-02-24T07:50:20.832719+0000","last_became_active":"2026-02-24T07:48:57.902484+0000","last_became_peered":"2026-02-24T07:48:57.902484+0000","last_unstale":"2026-02-24T07:50:20.832719+0000","last_undegraded":"2026-02-24T07:50:20.832719+0000","last_fullsized":"2026-02-24T07:50:20.832719+0000","mapping_epoch":55,"log_start":"61'37001","ondisk_log_start":"61'37001","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"61'32990","last_scrub_stamp":"2026-02-24T07:50:11.227573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:50:11.227573+0000","objects_scrubbed":32460,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:51:29.781611+0000","scrub_duration":3701,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2425094144,"num_objects":37004,"num_object_clones":0,"num_object_copies":111012,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37004,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":37004,"num_write_kb":2368256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.6","version":"61'37008","reported_seq":37375,"reported_epoch":61,"state":"active+clean","last_fresh":"2026-02-24T07:50:20.739925+0000","last_change":"2026-02-24T07:50:14.386679+0000","last_active":"2026-02-24T07:50:20.739925+0000","last_peered":"2026-02-24T07:50:20.739925+0000","last_clean":"2026-02-24T07:50:20.739925+0000","last_became_active":"2026-02-24T07:48:57.901565+0000","last_became_peered":"2026-02-24T07:48:57.901565+0000","last_unstale":"2026-02-24T07:50:20.739925+0000","last_undegraded":"2026-02-24T07:50:20.739925+0000","last_fullsized":"2026-02-24T07:50:20.739925+0000","mapping_epoch":55,"log_start":"61'37005","ondisk_log_start":"61'37005","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"61'34244","last_scrub_stamp":"2026-02-24T07:50:14.386625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:50:14.386625+0000","objects_scrubbed":33505,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:51:16.739697+0000","scrub_duration":3790,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2425356288,"num_objects":37008,"num_object_clones":0,"num_object_copies":111024,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37008,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":37008,"num_write_kb":2368512,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"61'37410","reported_seq":37791,"reported_epoch":61,"state":"active+clean","last_fresh":"2026-02-24T07:50:20.831699+0000","last_change":"2026-02-24T07:50:16.399238+0000","last_active":"2026-02-24T07:50:20.831699+0000","last_peered":"2026-02-24T07:50:20.831699+0000","last_clean":"2026-02-24T07:50:20.831699+0000","last_became_active":"2026-02-24T07:48:57.901824+0000","last_became_peered":"2026-02-24T07:48:57.901824+0000","last_unstale":"2026-02-24T07:50:20.831699+0000","last_undegraded":"2026-02-24T07:50:20.831699+0000","last_fullsized":"2026-02-24T07:50:20.831699+0000","mapping_epoch":55,"log_start":"61'37407","ondisk_log_start":"61'37407","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"61'35510","last_scrub_stamp":"2026-02-24T07:50:16.399194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:50:16.399194+0000","objects_scrubbed":34844,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:51:44.770701+0000","scrub_duration":2971,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2451701760,"num_objects":37410,"num_object_clones":0,"num_object_copies":112230,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37410,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":37410,"num_write_kb":2394240,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[1,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"61'36971","reported_seq":37323,"reported_epoch":61,"state":"active+clean","last_fresh":"2026-02-24T07:50:20.742909+0000","last_change":"2026-02-24T07:50:10.124042+0000","last_active":"2026-02-24T07:50:20.742909+0000","last_peered":"2026-02-24T07:50:20.742909+0000","last_clean":"2026-02-24T07:50:20.742909+0000","last_became_active":"2026-02-24T07:48:57.902224+0000","last_became_peered":"2026-02-24T07:48:57.902224+0000","last_unstale":"2026-02-24T07:50:20.742909+0000","last_undegraded":"2026-02-24T07:50:20.742909+0000","last_fullsized":"2026-02-24T07:50:20.742909+0000","mapping_epoch":55,"log_start":"61'36967","ondisk_log_start":"61'36967","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"61'32620","last_scrub_stamp":"2026-02-24T07:50:10.124005+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:50:10.124005+0000","objects_scrubbed":32078,"log_size":4,"log_dups_size":3000,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:51:12.877892+0000","scrub_duration":3559,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2422931456,"num_objects":36971,"num_object_clones":0,"num_object_copies":110913,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36971,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":36971,"num_write_kb":2366144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"61'37386","reported_seq":37711,"reported_epoch":61,"state":"active+clean","last_fresh":"2026-02-24T07:50:20.826782+0000","last_change":"2026-02-24T07:50:01.398116+0000","last_active":"2026-02-24T07:50:20.826782+0000","last_peered":"2026-02-24T07:50:20.826782+0000","last_clean":"2026-02-24T07:50:20.826782+0000","last_became_active":"2026-02-24T07:48:57.902125+0000","last_became_peered":"2026-02-24T07:48:57.902125+0000","last_unstale":"2026-02-24T07:50:20.826782+0000","last_undegraded":"2026-02-24T07:50:20.826782+0000","last_fullsized":"2026-02-24T07:50:20.826782+0000","mapping_epoch":55,"log_start":"61'37383","ondisk_log_start":"61'37383","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"61'29707","last_scrub_stamp":"2026-02-24T07:50:01.398090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:50:01.398090+0000","objects_scrubbed":29171,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:51:10.566529+0000","scrub_duration":2880,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2450128896,"num_objects":37386,"num_object_clones":0,"num_object_copies":112158,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37386,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":37386,"num_write_kb":2392704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1","version":"61'37349","reported_seq":37371,"reported_epoch":61,"state":"active+clean","last_fresh":"2026-02-24T07:50:20.721551+0000","last_change":"2026-02-24T07:48:57.903745+0000","last_active":"2026-02-24T07:50:20.721551+0000","last_peered":"2026-02-24T07:50:20.721551+0000","last_clean":"2026-02-24T07:50:20.721551+0000","last_became_active":"2026-02-24T07:48:57.903439+0000","last_became_peered":"2026-02-24T07:48:57.903439+0000","last_unstale":"2026-02-24T07:50:20.721551+0000","last_undegraded":"2026-02-24T07:50:20.721551+0000","last_fullsized":"2026-02-24T07:50:20.721551+0000","mapping_epoch":55,"log_start":"61'37346","ondisk_log_start":"61'37346","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:48:56.890768+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:50:24.058110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2447704064,"num_objects":37349,"num_object_clones":0,"num_object_copies":112047,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37349,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":37349,"num_write_kb":2390336,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.0","version":"61'37167","reported_seq":37189,"reported_epoch":61,"state":"active+clean","last_fresh":"2026-02-24T07:50:20.732703+0000","last_change":"2026-02-24T07:48:57.902838+0000","last_active":"2026-02-24T07:50:20.732703+0000","last_peered":"2026-02-24T07:50:20.732703+0000","last_clean":"2026-02-24T07:50:20.732703+0000","last_became_active":"2026-02-24T07:48:57.902565+0000","last_became_peered":"2026-02-24T07:48:57.902565+0000","last_unstale":"2026-02-24T07:50:20.732703+0000","last_undegraded":"2026-02-24T07:50:20.732703+0000","last_fullsized":"2026-02-24T07:50:20.732703+0000","mapping_epoch":55,"log_start":"61'37164","ondisk_log_start":"61'37164","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:48:56.890768+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:50:25.079534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2435776512,"num_objects":37167,"num_object_clones":0,"num_object_copies":111501,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37167,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":37167,"num_write_kb":2378688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,3],"acting":[0,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.5","version":"61'37250","reported_seq":37585,"reported_epoch":61,"state":"active+clean","last_fresh":"2026-02-24T07:50:20.738810+0000","last_change":"2026-02-24T07:50:04.821914+0000","last_active":"2026-02-24T07:50:20.738810+0000","last_peered":"2026-02-24T07:50:20.738810+0000","last_clean":"2026-02-24T07:50:20.738810+0000","last_became_active":"2026-02-24T07:48:57.905127+0000","last_became_peered":"2026-02-24T07:48:57.905127+0000","last_unstale":"2026-02-24T07:50:20.738810+0000","last_undegraded":"2026-02-24T07:50:20.738810+0000","last_fullsized":"2026-02-24T07:50:20.738810+0000","mapping_epoch":55,"log_start":"61'37247","ondisk_log_start":"61'37247","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"61'30833","last_scrub_stamp":"2026-02-24T07:50:04.821861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:50:04.821861+0000","objects_scrubbed":30132,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:51:15.928207+0000","scrub_duration":3578,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2441216000,"num_objects":37250,"num_object_clones":0,"num_object_copies":111750,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37250,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":37250,"num_write_kb":2384000,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"29'99","reported_seq":285,"reported_epoch":61,"state":"active+clean","last_fresh":"2026-02-24T07:49:41.888452+0000","last_change":"2026-02-24T07:49:06.478572+0000","last_active":"2026-02-24T07:49:41.888452+0000","last_peered":"2026-02-24T07:49:41.888452+0000","last_clean":"2026-02-24T07:49:41.888452+0000","last_became_active":"2026-02-24T07:42:54.186719+0000","last_became_peered":"2026-02-24T07:42:54.186719+0000","last_unstale":"2026-02-24T07:49:41.888452+0000","last_undegraded":"2026-02-24T07:49:41.888452+0000","last_fullsized":"2026-02-24T07:49:41.888452+0000","mapping_epoch":27,"log_start":"29'96","ondisk_log_start":"29'96","created":20,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'99","last_scrub_stamp":"2026-02-24T07:49:06.478534+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:42:43.640776+0000","last_clean_scrub_stamp":"2026-02-24T07:49:06.478534+0000","objects_scrubbed":2,"log_size":3,"log_dups_size":96,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:50:26.893586+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":240,"num_write":135,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":16,"stat_sum":{"num_bytes":38994837504,"num_objects":595014,"num_object_clones":0,"num_object_copies":1785042,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":595014,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":595014,"num_write_kb":38080896,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":116982284288,"data_stored":116982284288,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":49,"ondisk_log_size":49,"up":48,"acting":48,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":240,"num_write":135,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":3,"up_from":25,"seq":107374182492,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":30348616,"kb_used_data":28421940,"kb_used_omap":12,"kb_used_meta":1926643,"kb_avail":703654584,"statfs":{"total":751619276800,"available":720542294016,"internally_reserved":0,"allocated":29104066560,"data_stored":29103653076,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12605,"internal_metadata":1972883139},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":19,"seq":81604378718,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":35289224,"kb_used_data":33171444,"kb_used_omap":13,"kb_used_meta":2117746,"kb_avail":698713976,"statfs":{"total":751619276800,"available":715483111424,"internally_reserved":0,"allocated":33967558656,"data_stored":33967145172,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13640,"internal_metadata":2168572600},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542239,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":25562052,"kb_used_data":23721776,"kb_used_omap":14,"kb_used_meta":1840241,"kb_avail":708441148,"statfs":{"total":751619276800,"available":725443735552,"internally_reserved":0,"allocated":24291098624,"data_stored":24290688692,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":15140,"internal_metadata":1884407004},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,1],"upper_bound":4},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705762,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":30419720,"kb_used_data":28486708,"kb_used_omap":10,"kb_used_meta":1932981,"kb_avail":703583480,"statfs":{"total":751619276800,"available":720469483520,"internally_reserved":0,"allocated":29170388992,"data_stored":29169975508,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":11186,"internal_metadata":1979372622},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,2,1,1],"upper_bound":16},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":29225320448,"data_stored":29225320448,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":24394661888,"data_stored":24394661888,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":34083569664,"data_stored":34083569664,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":29278732288,"data_stored":29278732288,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-24T07:50:21.809 INFO:tasks.cephadm.ceph_manager.ceph:increase pool size by 10 2026-02-24T07:50:21.809 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph osd pool set .mgr pg_num 241 2026-02-24T07:50:21.948 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:50:22.001 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 83 16 604031 604015 454.791 456.312 0.00130326 0.0021976 2026-02-24T07:50:22.010 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:50:21 trial035 ceph-mon[39284]: 4.e scrub ok 2026-02-24T07:50:22.010 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:50:21 trial035 ceph-mon[39284]: 4.c scrub starts 2026-02-24T07:50:22.010 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:50:21 trial035 ceph-mon[39284]: pgmap v256: 17 pgs: 1 active+clean+scrubbing, 16 active+clean; 36 GiB data, 116 GiB used, 2.6 TiB / 2.7 TiB avail; 490 MiB/s wr, 7.85k op/s 2026-02-24T07:50:22.011 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:50:21 trial035 ceph-mon[39284]: from='client.14993 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2026-02-24T07:50:22.084 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:50:22.082+0000 7f23f3fff640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:50:22.085 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:50:22.082+0000 7f23f3fff640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:50:22.232 INFO:teuthology.orchestra.run.trial035.stderr:Error EPERM: crimson_allow_pg_split is false; pg_num increase denied for crimson pool 2026-02-24T07:50:22.270 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-24T07:50:22.440 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2026-02-24T07:50:22.586 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:50:22.713 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:50:22.711+0000 7f1589f14640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:50:22.713 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:50:22.711+0000 7f1589f14640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:50:22.728 INFO:teuthology.orchestra.run.trial035.stdout:osd.0: {} 2026-02-24T07:50:22.728 INFO:teuthology.orchestra.run.trial035.stderr:osd.0: osd_enable_op_tracker = '' 2026-02-24T07:50:22.740 INFO:teuthology.orchestra.run.trial035.stdout:osd.1: {} 2026-02-24T07:50:22.741 INFO:teuthology.orchestra.run.trial035.stderr:osd.1: osd_enable_op_tracker = '' 2026-02-24T07:50:22.753 INFO:teuthology.orchestra.run.trial035.stdout:osd.2: {} 2026-02-24T07:50:22.754 INFO:teuthology.orchestra.run.trial035.stderr:osd.2: osd_enable_op_tracker = '' 2026-02-24T07:50:22.767 INFO:teuthology.orchestra.run.trial035.stdout:osd.3: {} 2026-02-24T07:50:22.767 INFO:teuthology.orchestra.run.trial035.stderr:osd.3: osd_enable_op_tracker = '' 2026-02-24T07:50:23.001 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 84 15 611143 611128 454.669 444.562 0.00289959 0.0021982 2026-02-24T07:50:23.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:50:22 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/3875619972' entity='client.admin' cmd={"prefix": "osd pool set", "pool": ".mgr", "var": "pg_num", "val": "241"} : dispatch 2026-02-24T07:50:23.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:50:22 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/973070523' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2026-02-24T07:50:24.001 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 85 16 617727 617711 454.158 411.438 0.00192379 0.00220067 2026-02-24T07:50:24.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:50:23 trial035 ceph-mon[39284]: pgmap v257: 17 pgs: 1 active+clean+scrubbing, 16 active+clean; 36 GiB data, 116 GiB used, 2.6 TiB / 2.7 TiB avail; 345 MiB/s wr, 5.53k op/s 2026-02-24T07:50:24.814 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2026-02-24T07:50:24.963 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:50:25.001 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 86 16 624269 624253 453.631 408.875 0.00199538 0.00220318 2026-02-24T07:50:25.092 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:50:25.090+0000 7f6120488640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:50:25.092 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:50:24 trial035 ceph-mon[39284]: 4.b scrub starts 2026-02-24T07:50:25.092 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:50:25.090+0000 7f6120488640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:50:25.109 INFO:teuthology.orchestra.run.trial035.stdout:osd.0: {} 2026-02-24T07:50:25.109 INFO:teuthology.orchestra.run.trial035.stderr:osd.0: osd_enable_op_tracker = '' 2026-02-24T07:50:25.123 INFO:teuthology.orchestra.run.trial035.stdout:osd.1: {} 2026-02-24T07:50:25.123 INFO:teuthology.orchestra.run.trial035.stderr:osd.1: osd_enable_op_tracker = '' 2026-02-24T07:50:25.138 INFO:teuthology.orchestra.run.trial035.stdout:osd.2: {} 2026-02-24T07:50:25.138 INFO:teuthology.orchestra.run.trial035.stderr:osd.2: osd_enable_op_tracker = '' 2026-02-24T07:50:25.151 INFO:teuthology.orchestra.run.trial035.stdout:osd.3: {} 2026-02-24T07:50:25.152 INFO:teuthology.orchestra.run.trial035.stderr:osd.3: osd_enable_op_tracker = '' 2026-02-24T07:50:26.001 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 87 15 629136 629121 451.913 304.25 0.00294626 0.0022116 2026-02-24T07:50:26.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:50:25 trial035 ceph-mon[39284]: 4.c scrub ok 2026-02-24T07:50:26.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:50:25 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/1685399284' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2026-02-24T07:50:26.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:50:25 trial035 ceph-mon[39284]: 4.1 scrub starts 2026-02-24T07:50:26.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:50:25 trial035 ceph-mon[39284]: pgmap v258: 17 pgs: 1 active+clean+scrubbing, 16 active+clean; 36 GiB data, 116 GiB used, 2.6 TiB / 2.7 TiB avail; 345 MiB/s wr, 5.53k op/s 2026-02-24T07:50:27.002 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 88 16 635057 635041 450.982 370 0.00152745 0.00221616 2026-02-24T07:50:27.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:50:26 trial035 ceph-mon[39284]: 4.0 scrub starts 2026-02-24T07:50:27.204 INFO:journalctl@ceph.mgr.x.trial035.stdout:Feb 24 07:50:26 trial035 ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-mgr-x[39621]: ::ffff:10.20.193.91 - - [24/Feb/2026:07:50:26] "GET /metrics HTTP/1.1" 200 4214 "" "Prometheus/3.6.0" 2026-02-24T07:50:27.271 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2026-02-24T07:50:27.271 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2026-02-24T07:50:27.271 INFO:tasks.thrashosds.thrasher:Growing pool unique_pool_2 2026-02-24T07:50:27.271 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph pg dump --format=json 2026-02-24T07:50:27.356 INFO:journalctl@ceph.prometheus.a.trial091.stdout:Feb 24 07:50:26 trial091 ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-prometheus-a[66007]: time=2026-02-24T07:50:26.973Z level=ERROR source=scrape.go:1630 msg="Failed to determine correct type of scrape target." component="scrape manager" scrape_pool=ceph target=http://trial035:9283/metrics content_type="text/html;charset=utf-8" fallback_media_type="" err="received unsupported Content-Type \"text/html;charset=utf-8\" and no fallback_scrape_protocol specified for target" 2026-02-24T07:50:27.405 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:50:27.520 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:50:27.518+0000 7fae78da9640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:50:27.520 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:50:27.518+0000 7fae78da9640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:50:27.679 INFO:teuthology.orchestra.run.trial035.stdout: 2026-02-24T07:50:27.680 INFO:teuthology.orchestra.run.trial035.stderr:dumped all 2026-02-24T07:50:27.723 INFO:teuthology.orchestra.run.trial035.stdout:{"pg_ready":true,"pg_map":{"version":258,"stamp":"2026-02-24T07:50:25.717600+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":38995427872,"num_objects":595016,"num_object_clones":0,"num_object_copies":1785048,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":595016,"num_whiteouts":0,"num_read":136,"num_read_kb":240,"num_write":595149,"num_write_kb":38083046,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":52,"ondisk_log_size":52,"up":51,"acting":51,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":51,"num_osds":4,"num_per_pool_osds":4,"num_per_pool_omap_osds":4,"kb":2936012800,"kb_used":121619612,"kb_used_data":113801868,"kb_used_omap":51,"kb_used_meta":7817612,"kb_avail":2814393188,"statfs":{"total":3006477107200,"available":2881938624512,"internally_reserved":0,"allocated":116533112832,"data_stored":116531462448,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":52571,"internal_metadata":8005235365},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[6,3,1,1],"upper_bound":16},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":4346216448,"num_objects":66318,"num_object_clones":0,"num_object_copies":198954,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66318,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":66318,"num_write_kb":4244352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001670"},"pg_stats":[{"pgid":"4.f","version":"61'37400","reported_seq":37733,"reported_epoch":61,"state":"active+clean","last_fresh":"2026-02-24T07:50:20.828225+0000","last_change":"2026-02-24T07:50:03.071459+0000","last_active":"2026-02-24T07:50:20.828225+0000","last_peered":"2026-02-24T07:50:20.828225+0000","last_clean":"2026-02-24T07:50:20.828225+0000","last_became_active":"2026-02-24T07:48:57.902313+0000","last_became_peered":"2026-02-24T07:48:57.902313+0000","last_unstale":"2026-02-24T07:50:20.828225+0000","last_undegraded":"2026-02-24T07:50:20.828225+0000","last_fullsized":"2026-02-24T07:50:20.828225+0000","mapping_epoch":55,"log_start":"61'37397","ondisk_log_start":"61'37397","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"61'30419","last_scrub_stamp":"2026-02-24T07:50:03.071380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:50:03.071380+0000","objects_scrubbed":29975,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:51:16.123453+0000","scrub_duration":2568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2451046400,"num_objects":37400,"num_object_clones":0,"num_object_copies":112200,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37400,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":37400,"num_write_kb":2393600,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"61'37327","reported_seq":37685,"reported_epoch":61,"state":"active+clean+scrubbing","last_fresh":"2026-02-24T07:50:20.727821+0000","last_change":"2026-02-24T07:50:17.422930+0000","last_active":"2026-02-24T07:50:20.727821+0000","last_peered":"2026-02-24T07:50:20.727821+0000","last_clean":"2026-02-24T07:50:20.727821+0000","last_became_active":"2026-02-24T07:48:57.903584+0000","last_became_peered":"2026-02-24T07:48:57.903584+0000","last_unstale":"2026-02-24T07:50:20.727821+0000","last_undegraded":"2026-02-24T07:50:20.727821+0000","last_fullsized":"2026-02-24T07:50:20.727821+0000","mapping_epoch":55,"log_start":"61'37324","ondisk_log_start":"61'37324","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:48:56.890768+0000","objects_scrubbed":33099,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"scrubbing for 4s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2446262272,"num_objects":37327,"num_object_clones":0,"num_object_copies":111981,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37327,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":37327,"num_write_kb":2388928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.d","version":"61'37130","reported_seq":37472,"reported_epoch":61,"state":"active+clean","last_fresh":"2026-02-24T07:50:20.742238+0000","last_change":"2026-02-24T07:50:06.701973+0000","last_active":"2026-02-24T07:50:20.742238+0000","last_peered":"2026-02-24T07:50:20.742238+0000","last_clean":"2026-02-24T07:50:20.742238+0000","last_became_active":"2026-02-24T07:48:57.905031+0000","last_became_peered":"2026-02-24T07:48:57.905031+0000","last_unstale":"2026-02-24T07:50:20.742238+0000","last_undegraded":"2026-02-24T07:50:20.742238+0000","last_fullsized":"2026-02-24T07:50:20.742238+0000","mapping_epoch":55,"log_start":"61'37127","ondisk_log_start":"61'37127","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"61'31648","last_scrub_stamp":"2026-02-24T07:50:06.701883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:50:06.701883+0000","objects_scrubbed":30990,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:51:19.224966+0000","scrub_duration":3493,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2433351680,"num_objects":37130,"num_object_clones":0,"num_object_copies":111390,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37130,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":37130,"num_write_kb":2376320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"61'36989","reported_seq":37011,"reported_epoch":61,"state":"active+clean","last_fresh":"2026-02-24T07:50:20.738423+0000","last_change":"2026-02-24T07:48:57.905290+0000","last_active":"2026-02-24T07:50:20.738423+0000","last_peered":"2026-02-24T07:50:20.738423+0000","last_clean":"2026-02-24T07:50:20.738423+0000","last_became_active":"2026-02-24T07:48:57.904683+0000","last_became_peered":"2026-02-24T07:48:57.904683+0000","last_unstale":"2026-02-24T07:50:20.738423+0000","last_undegraded":"2026-02-24T07:50:20.738423+0000","last_fullsized":"2026-02-24T07:50:20.738423+0000","mapping_epoch":55,"log_start":"61'36986","ondisk_log_start":"61'36986","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:48:56.890768+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2424111104,"num_objects":36989,"num_object_clones":0,"num_object_copies":110967,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36989,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":36989,"num_write_kb":2367296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.b","version":"61'36990","reported_seq":37012,"reported_epoch":61,"state":"active+clean","last_fresh":"2026-02-24T07:50:20.739530+0000","last_change":"2026-02-24T07:48:57.902025+0000","last_active":"2026-02-24T07:50:20.739530+0000","last_peered":"2026-02-24T07:50:20.739530+0000","last_clean":"2026-02-24T07:50:20.739530+0000","last_became_active":"2026-02-24T07:48:57.901700+0000","last_became_peered":"2026-02-24T07:48:57.901700+0000","last_unstale":"2026-02-24T07:50:20.739530+0000","last_undegraded":"2026-02-24T07:50:20.739530+0000","last_fullsized":"2026-02-24T07:50:20.739530+0000","mapping_epoch":55,"log_start":"61'36987","ondisk_log_start":"61'36987","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:48:56.890768+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:50:23.370319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2424176640,"num_objects":36990,"num_object_clones":0,"num_object_copies":110970,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36990,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":36990,"num_write_kb":2367360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.a","version":"61'37246","reported_seq":37268,"reported_epoch":61,"state":"active+clean","last_fresh":"2026-02-24T07:50:20.725357+0000","last_change":"2026-02-24T07:48:57.903698+0000","last_active":"2026-02-24T07:50:20.725357+0000","last_peered":"2026-02-24T07:50:20.725357+0000","last_clean":"2026-02-24T07:50:20.725357+0000","last_became_active":"2026-02-24T07:48:57.903228+0000","last_became_peered":"2026-02-24T07:48:57.903228+0000","last_unstale":"2026-02-24T07:50:20.725357+0000","last_undegraded":"2026-02-24T07:50:20.725357+0000","last_fullsized":"2026-02-24T07:50:20.725357+0000","mapping_epoch":55,"log_start":"61'37243","ondisk_log_start":"61'37243","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:48:56.890768+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:50:26.780637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2440953856,"num_objects":37246,"num_object_clones":0,"num_object_copies":111738,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37246,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":37246,"num_write_kb":2383744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"61'37179","reported_seq":37544,"reported_epoch":61,"state":"active+clean","last_fresh":"2026-02-24T07:50:20.724533+0000","last_change":"2026-02-24T07:50:13.165492+0000","last_active":"2026-02-24T07:50:20.724533+0000","last_peered":"2026-02-24T07:50:20.724533+0000","last_clean":"2026-02-24T07:50:20.724533+0000","last_became_active":"2026-02-24T07:48:57.901730+0000","last_became_peered":"2026-02-24T07:48:57.901730+0000","last_unstale":"2026-02-24T07:50:20.724533+0000","last_undegraded":"2026-02-24T07:50:20.724533+0000","last_fullsized":"2026-02-24T07:50:20.724533+0000","mapping_epoch":55,"log_start":"61'37176","ondisk_log_start":"61'37176","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"61'33943","last_scrub_stamp":"2026-02-24T07:50:13.165459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:50:13.165459+0000","objects_scrubbed":33253,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:51:33.820708+0000","scrub_duration":3776,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2436562944,"num_objects":37179,"num_object_clones":0,"num_object_copies":111537,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37179,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":37179,"num_write_kb":2379456,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"61'37208","reported_seq":37537,"reported_epoch":61,"state":"active+clean","last_fresh":"2026-02-24T07:50:20.743122+0000","last_change":"2026-02-24T07:50:03.153499+0000","last_active":"2026-02-24T07:50:20.743122+0000","last_peered":"2026-02-24T07:50:20.743122+0000","last_clean":"2026-02-24T07:50:20.743122+0000","last_became_active":"2026-02-24T07:48:57.904376+0000","last_became_peered":"2026-02-24T07:48:57.904376+0000","last_unstale":"2026-02-24T07:50:20.743122+0000","last_undegraded":"2026-02-24T07:50:20.743122+0000","last_fullsized":"2026-02-24T07:50:20.743122+0000","mapping_epoch":55,"log_start":"61'37205","ondisk_log_start":"61'37205","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"61'30163","last_scrub_stamp":"2026-02-24T07:50:03.153454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:50:03.153454+0000","objects_scrubbed":29681,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:51:10.266136+0000","scrub_duration":2916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2438463488,"num_objects":37208,"num_object_clones":0,"num_object_copies":111624,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37208,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":37208,"num_write_kb":2381312,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.7","version":"61'37004","reported_seq":37360,"reported_epoch":61,"state":"active+clean","last_fresh":"2026-02-24T07:50:20.832719+0000","last_change":"2026-02-24T07:50:11.227608+0000","last_active":"2026-02-24T07:50:20.832719+0000","last_peered":"2026-02-24T07:50:20.832719+0000","last_clean":"2026-02-24T07:50:20.832719+0000","last_became_active":"2026-02-24T07:48:57.902484+0000","last_became_peered":"2026-02-24T07:48:57.902484+0000","last_unstale":"2026-02-24T07:50:20.832719+0000","last_undegraded":"2026-02-24T07:50:20.832719+0000","last_fullsized":"2026-02-24T07:50:20.832719+0000","mapping_epoch":55,"log_start":"61'37001","ondisk_log_start":"61'37001","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"61'32990","last_scrub_stamp":"2026-02-24T07:50:11.227573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:50:11.227573+0000","objects_scrubbed":32460,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:51:29.781611+0000","scrub_duration":3701,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2425094144,"num_objects":37004,"num_object_clones":0,"num_object_copies":111012,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37004,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":37004,"num_write_kb":2368256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.6","version":"61'37008","reported_seq":37375,"reported_epoch":61,"state":"active+clean","last_fresh":"2026-02-24T07:50:20.739925+0000","last_change":"2026-02-24T07:50:14.386679+0000","last_active":"2026-02-24T07:50:20.739925+0000","last_peered":"2026-02-24T07:50:20.739925+0000","last_clean":"2026-02-24T07:50:20.739925+0000","last_became_active":"2026-02-24T07:48:57.901565+0000","last_became_peered":"2026-02-24T07:48:57.901565+0000","last_unstale":"2026-02-24T07:50:20.739925+0000","last_undegraded":"2026-02-24T07:50:20.739925+0000","last_fullsized":"2026-02-24T07:50:20.739925+0000","mapping_epoch":55,"log_start":"61'37005","ondisk_log_start":"61'37005","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"61'34244","last_scrub_stamp":"2026-02-24T07:50:14.386625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:50:14.386625+0000","objects_scrubbed":33505,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:51:16.739697+0000","scrub_duration":3790,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2425356288,"num_objects":37008,"num_object_clones":0,"num_object_copies":111024,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37008,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":37008,"num_write_kb":2368512,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"61'37410","reported_seq":37791,"reported_epoch":61,"state":"active+clean","last_fresh":"2026-02-24T07:50:20.831699+0000","last_change":"2026-02-24T07:50:16.399238+0000","last_active":"2026-02-24T07:50:20.831699+0000","last_peered":"2026-02-24T07:50:20.831699+0000","last_clean":"2026-02-24T07:50:20.831699+0000","last_became_active":"2026-02-24T07:48:57.901824+0000","last_became_peered":"2026-02-24T07:48:57.901824+0000","last_unstale":"2026-02-24T07:50:20.831699+0000","last_undegraded":"2026-02-24T07:50:20.831699+0000","last_fullsized":"2026-02-24T07:50:20.831699+0000","mapping_epoch":55,"log_start":"61'37407","ondisk_log_start":"61'37407","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"61'35510","last_scrub_stamp":"2026-02-24T07:50:16.399194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:50:16.399194+0000","objects_scrubbed":34844,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:51:44.770701+0000","scrub_duration":2971,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2451701760,"num_objects":37410,"num_object_clones":0,"num_object_copies":112230,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37410,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":37410,"num_write_kb":2394240,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[1,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"61'36971","reported_seq":37323,"reported_epoch":61,"state":"active+clean","last_fresh":"2026-02-24T07:50:20.742909+0000","last_change":"2026-02-24T07:50:10.124042+0000","last_active":"2026-02-24T07:50:20.742909+0000","last_peered":"2026-02-24T07:50:20.742909+0000","last_clean":"2026-02-24T07:50:20.742909+0000","last_became_active":"2026-02-24T07:48:57.902224+0000","last_became_peered":"2026-02-24T07:48:57.902224+0000","last_unstale":"2026-02-24T07:50:20.742909+0000","last_undegraded":"2026-02-24T07:50:20.742909+0000","last_fullsized":"2026-02-24T07:50:20.742909+0000","mapping_epoch":55,"log_start":"61'36967","ondisk_log_start":"61'36967","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"61'32620","last_scrub_stamp":"2026-02-24T07:50:10.124005+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:50:10.124005+0000","objects_scrubbed":32078,"log_size":4,"log_dups_size":3000,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:51:12.877892+0000","scrub_duration":3559,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2422931456,"num_objects":36971,"num_object_clones":0,"num_object_copies":110913,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36971,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":36971,"num_write_kb":2366144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"61'37386","reported_seq":37711,"reported_epoch":61,"state":"active+clean","last_fresh":"2026-02-24T07:50:20.826782+0000","last_change":"2026-02-24T07:50:01.398116+0000","last_active":"2026-02-24T07:50:20.826782+0000","last_peered":"2026-02-24T07:50:20.826782+0000","last_clean":"2026-02-24T07:50:20.826782+0000","last_became_active":"2026-02-24T07:48:57.902125+0000","last_became_peered":"2026-02-24T07:48:57.902125+0000","last_unstale":"2026-02-24T07:50:20.826782+0000","last_undegraded":"2026-02-24T07:50:20.826782+0000","last_fullsized":"2026-02-24T07:50:20.826782+0000","mapping_epoch":55,"log_start":"61'37383","ondisk_log_start":"61'37383","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"61'29707","last_scrub_stamp":"2026-02-24T07:50:01.398090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:50:01.398090+0000","objects_scrubbed":29171,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:51:10.566529+0000","scrub_duration":2880,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2450128896,"num_objects":37386,"num_object_clones":0,"num_object_copies":112158,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37386,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":37386,"num_write_kb":2392704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1","version":"61'37349","reported_seq":37371,"reported_epoch":61,"state":"active+clean","last_fresh":"2026-02-24T07:50:20.721551+0000","last_change":"2026-02-24T07:48:57.903745+0000","last_active":"2026-02-24T07:50:20.721551+0000","last_peered":"2026-02-24T07:50:20.721551+0000","last_clean":"2026-02-24T07:50:20.721551+0000","last_became_active":"2026-02-24T07:48:57.903439+0000","last_became_peered":"2026-02-24T07:48:57.903439+0000","last_unstale":"2026-02-24T07:50:20.721551+0000","last_undegraded":"2026-02-24T07:50:20.721551+0000","last_fullsized":"2026-02-24T07:50:20.721551+0000","mapping_epoch":55,"log_start":"61'37346","ondisk_log_start":"61'37346","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:48:56.890768+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:50:24.058110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2447704064,"num_objects":37349,"num_object_clones":0,"num_object_copies":112047,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37349,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":37349,"num_write_kb":2390336,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.0","version":"61'37167","reported_seq":37189,"reported_epoch":61,"state":"active+clean","last_fresh":"2026-02-24T07:50:20.732703+0000","last_change":"2026-02-24T07:48:57.902838+0000","last_active":"2026-02-24T07:50:20.732703+0000","last_peered":"2026-02-24T07:50:20.732703+0000","last_clean":"2026-02-24T07:50:20.732703+0000","last_became_active":"2026-02-24T07:48:57.902565+0000","last_became_peered":"2026-02-24T07:48:57.902565+0000","last_unstale":"2026-02-24T07:50:20.732703+0000","last_undegraded":"2026-02-24T07:50:20.732703+0000","last_fullsized":"2026-02-24T07:50:20.732703+0000","mapping_epoch":55,"log_start":"61'37164","ondisk_log_start":"61'37164","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:48:56.890768+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:50:25.079534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2435776512,"num_objects":37167,"num_object_clones":0,"num_object_copies":111501,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37167,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":37167,"num_write_kb":2378688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,3],"acting":[0,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.5","version":"61'37250","reported_seq":37585,"reported_epoch":61,"state":"active+clean","last_fresh":"2026-02-24T07:50:20.738810+0000","last_change":"2026-02-24T07:50:04.821914+0000","last_active":"2026-02-24T07:50:20.738810+0000","last_peered":"2026-02-24T07:50:20.738810+0000","last_clean":"2026-02-24T07:50:20.738810+0000","last_became_active":"2026-02-24T07:48:57.905127+0000","last_became_peered":"2026-02-24T07:48:57.905127+0000","last_unstale":"2026-02-24T07:50:20.738810+0000","last_undegraded":"2026-02-24T07:50:20.738810+0000","last_fullsized":"2026-02-24T07:50:20.738810+0000","mapping_epoch":55,"log_start":"61'37247","ondisk_log_start":"61'37247","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"61'30833","last_scrub_stamp":"2026-02-24T07:50:04.821861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:50:04.821861+0000","objects_scrubbed":30132,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:51:15.928207+0000","scrub_duration":3578,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2441216000,"num_objects":37250,"num_object_clones":0,"num_object_copies":111750,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37250,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":37250,"num_write_kb":2384000,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"29'99","reported_seq":285,"reported_epoch":61,"state":"active+clean","last_fresh":"2026-02-24T07:49:41.888452+0000","last_change":"2026-02-24T07:49:06.478572+0000","last_active":"2026-02-24T07:49:41.888452+0000","last_peered":"2026-02-24T07:49:41.888452+0000","last_clean":"2026-02-24T07:49:41.888452+0000","last_became_active":"2026-02-24T07:42:54.186719+0000","last_became_peered":"2026-02-24T07:42:54.186719+0000","last_unstale":"2026-02-24T07:49:41.888452+0000","last_undegraded":"2026-02-24T07:49:41.888452+0000","last_fullsized":"2026-02-24T07:49:41.888452+0000","mapping_epoch":27,"log_start":"29'96","ondisk_log_start":"29'96","created":20,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'99","last_scrub_stamp":"2026-02-24T07:49:06.478534+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:42:43.640776+0000","last_clean_scrub_stamp":"2026-02-24T07:49:06.478534+0000","objects_scrubbed":2,"log_size":3,"log_dups_size":96,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:50:26.893586+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":240,"num_write":135,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":16,"stat_sum":{"num_bytes":38994837504,"num_objects":595014,"num_object_clones":0,"num_object_copies":1785042,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":595014,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":595014,"num_write_kb":38080896,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":116982284288,"data_stored":116982284288,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":49,"ondisk_log_size":49,"up":48,"acting":48,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":240,"num_write":135,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":3,"up_from":25,"seq":107374182492,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":30348616,"kb_used_data":28421940,"kb_used_omap":12,"kb_used_meta":1926643,"kb_avail":703654584,"statfs":{"total":751619276800,"available":720542294016,"internally_reserved":0,"allocated":29104066560,"data_stored":29103653076,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12605,"internal_metadata":1972883139},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":19,"seq":81604378718,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":35289224,"kb_used_data":33171444,"kb_used_omap":13,"kb_used_meta":2117746,"kb_avail":698713976,"statfs":{"total":751619276800,"available":715483111424,"internally_reserved":0,"allocated":33967558656,"data_stored":33967145172,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13640,"internal_metadata":2168572600},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542239,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":25562052,"kb_used_data":23721776,"kb_used_omap":14,"kb_used_meta":1840241,"kb_avail":708441148,"statfs":{"total":751619276800,"available":725443735552,"internally_reserved":0,"allocated":24291098624,"data_stored":24290688692,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":15140,"internal_metadata":1884407004},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,1],"upper_bound":4},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705762,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":30419720,"kb_used_data":28486708,"kb_used_omap":10,"kb_used_meta":1932981,"kb_avail":703583480,"statfs":{"total":751619276800,"available":720469483520,"internally_reserved":0,"allocated":29170388992,"data_stored":29169975508,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":11186,"internal_metadata":1979372622},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,2,1,1],"upper_bound":16},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":29225320448,"data_stored":29225320448,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":24394661888,"data_stored":24394661888,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":34083569664,"data_stored":34083569664,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":29278732288,"data_stored":29278732288,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-24T07:50:27.726 INFO:tasks.cephadm.ceph_manager.ceph:increase pool size by 10 2026-02-24T07:50:27.726 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph osd pool set unique_pool_2 pg_num 56 2026-02-24T07:50:27.854 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2026-02-24T07:50:27.871 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:50:27.936 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:50:27 trial035 ceph-mon[39284]: 4.b scrub ok 2026-02-24T07:50:27.936 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:50:27 trial035 ceph-mon[39284]: from='client.15001 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2026-02-24T07:50:27.936 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:50:27 trial035 ceph-mon[39284]: pgmap v259: 17 pgs: 3 active+clean+scrubbing, 14 active+clean; 38 GiB data, 122 GiB used, 2.6 TiB / 2.7 TiB avail; 517 MiB/s wr, 8.28k op/s 2026-02-24T07:50:28.001 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 89 16 641067 641051 450.135 375.625 0.00114303 0.00222034 2026-02-24T07:50:28.002 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:50:28.008 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:50:28.006+0000 7f16bc919640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:50:28.008 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:50:28.006+0000 7f16bc919640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:50:28.131 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:50:28.129+0000 7fcd20578640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:50:28.132 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:50:28.129+0000 7fcd20578640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:50:28.150 INFO:teuthology.orchestra.run.trial035.stdout:osd.0: {} 2026-02-24T07:50:28.151 INFO:teuthology.orchestra.run.trial035.stderr:osd.0: osd_enable_op_tracker = '' 2026-02-24T07:50:28.158 INFO:teuthology.orchestra.run.trial035.stderr:Error EPERM: crimson_allow_pg_split is false; pg_num increase denied for crimson pool 2026-02-24T07:50:28.164 INFO:teuthology.orchestra.run.trial035.stdout:osd.1: {} 2026-02-24T07:50:28.165 INFO:teuthology.orchestra.run.trial035.stderr:osd.1: osd_enable_op_tracker = '' 2026-02-24T07:50:28.177 INFO:teuthology.orchestra.run.trial035.stdout:osd.2: {} 2026-02-24T07:50:28.178 INFO:teuthology.orchestra.run.trial035.stderr:osd.2: osd_enable_op_tracker = '' 2026-02-24T07:50:28.191 INFO:teuthology.orchestra.run.trial035.stdout:osd.3: {} 2026-02-24T07:50:28.191 INFO:teuthology.orchestra.run.trial035.stderr:osd.3: osd_enable_op_tracker = '' 2026-02-24T07:50:28.196 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-24T07:50:29.002 INFO:tasks.radosbench.radosbench.0.trial035.stdout:Total time run: 90.0022 2026-02-24T07:50:29.002 INFO:tasks.radosbench.radosbench.0.trial035.stdout:Total writes made: 647367 2026-02-24T07:50:29.002 INFO:tasks.radosbench.radosbench.0.trial035.stdout:Write size: 65536 2026-02-24T07:50:29.003 INFO:tasks.radosbench.radosbench.0.trial035.stdout:Object size: 65536 2026-02-24T07:50:29.003 INFO:tasks.radosbench.radosbench.0.trial035.stdout:Bandwidth (MB/sec): 449.549 2026-02-24T07:50:29.003 INFO:tasks.radosbench.radosbench.0.trial035.stdout:Stddev Bandwidth: 68.264 2026-02-24T07:50:29.003 INFO:tasks.radosbench.radosbench.0.trial035.stdout:Max bandwidth (MB/sec): 537.438 2026-02-24T07:50:29.003 INFO:tasks.radosbench.radosbench.0.trial035.stdout:Min bandwidth (MB/sec): 274.625 2026-02-24T07:50:29.003 INFO:tasks.radosbench.radosbench.0.trial035.stdout:Average IOPS: 7192 2026-02-24T07:50:29.003 INFO:tasks.radosbench.radosbench.0.trial035.stdout:Stddev IOPS: 1092.22 2026-02-24T07:50:29.003 INFO:tasks.radosbench.radosbench.0.trial035.stdout:Max IOPS: 8599 2026-02-24T07:50:29.003 INFO:tasks.radosbench.radosbench.0.trial035.stdout:Min IOPS: 4394 2026-02-24T07:50:29.003 INFO:tasks.radosbench.radosbench.0.trial035.stdout:Average Latency(s): 0.00222325 2026-02-24T07:50:29.003 INFO:tasks.radosbench.radosbench.0.trial035.stdout:Stddev Latency(s): 0.00216435 2026-02-24T07:50:29.003 INFO:tasks.radosbench.radosbench.0.trial035.stdout:Max latency(s): 0.0946675 2026-02-24T07:50:29.004 INFO:tasks.radosbench.radosbench.0.trial035.stdout:Min latency(s): 0.00066633 2026-02-24T07:50:29.004 INFO:tasks.radosbench.radosbench.0.trial035.stdout:Cleaning up (deleting benchmark objects) 2026-02-24T07:50:29.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:50:28 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/4191126145' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2026-02-24T07:50:29.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:50:28 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/3285620613' entity='client.admin' cmd={"prefix": "osd pool set", "pool": "unique_pool_2", "var": "pg_num", "val": "56"} : dispatch 2026-02-24T07:50:30.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:50:29 trial035 ceph-mon[39284]: 4.a scrub starts 2026-02-24T07:50:30.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:50:29 trial035 ceph-mon[39284]: pgmap v260: 17 pgs: 3 active+clean+scrubbing, 14 active+clean; 38 GiB data, 122 GiB used, 2.6 TiB / 2.7 TiB avail; 354 MiB/s wr, 5.66k op/s 2026-02-24T07:50:30.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:50:29 trial035 ceph-mon[39284]: from='mgr.14284 10.20.193.35:0/3681373222' entity='mgr.x' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2026-02-24T07:50:30.235 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2026-02-24T07:50:30.372 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:50:30.499 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:50:30.497+0000 7fd59cafd640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:50:30.499 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:50:30.498+0000 7fd59cafd640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:50:30.515 INFO:teuthology.orchestra.run.trial035.stdout:osd.0: {} 2026-02-24T07:50:30.516 INFO:teuthology.orchestra.run.trial035.stderr:osd.0: osd_enable_op_tracker = '' 2026-02-24T07:50:30.529 INFO:teuthology.orchestra.run.trial035.stdout:osd.1: {} 2026-02-24T07:50:30.529 INFO:teuthology.orchestra.run.trial035.stderr:osd.1: osd_enable_op_tracker = '' 2026-02-24T07:50:30.541 INFO:teuthology.orchestra.run.trial035.stdout:osd.2: {} 2026-02-24T07:50:30.541 INFO:teuthology.orchestra.run.trial035.stderr:osd.2: osd_enable_op_tracker = '' 2026-02-24T07:50:30.553 INFO:teuthology.orchestra.run.trial035.stdout:osd.3: {} 2026-02-24T07:50:30.554 INFO:teuthology.orchestra.run.trial035.stderr:osd.3: osd_enable_op_tracker = '' 2026-02-24T07:50:31.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:50:30 trial035 ceph-mon[39284]: 4.0 scrub ok 2026-02-24T07:50:31.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:50:30 trial035 ceph-mon[39284]: 4.1 scrub ok 2026-02-24T07:50:31.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:50:30 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/4218692712' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2026-02-24T07:50:32.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:50:31 trial035 ceph-mon[39284]: 1.0 scrub starts 2026-02-24T07:50:32.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:50:31 trial035 ceph-mon[39284]: 1.0 scrub ok 2026-02-24T07:50:32.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:50:31 trial035 ceph-mon[39284]: pgmap v261: 17 pgs: 1 active+clean+scrubbing, 16 active+clean; 39 GiB data, 123 GiB used, 2.6 TiB / 2.7 TiB avail; 85 B/s rd, 454 MiB/s wr, 8.43k op/s 2026-02-24T07:50:32.596 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2026-02-24T07:50:32.744 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:50:33.101 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:50:33.079+0000 7fc1231d5640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:50:33.102 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:50:33.080+0000 7fc1231d5640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:50:33.102 INFO:teuthology.orchestra.run.trial035.stderr:osd.0: osd_enable_op_tracker = '' 2026-02-24T07:50:33.102 INFO:teuthology.orchestra.run.trial035.stdout:osd.0: {} 2026-02-24T07:50:33.116 INFO:teuthology.orchestra.run.trial035.stdout:osd.1: {} 2026-02-24T07:50:33.117 INFO:teuthology.orchestra.run.trial035.stderr:osd.1: osd_enable_op_tracker = '' 2026-02-24T07:50:33.128 INFO:teuthology.orchestra.run.trial035.stdout:osd.2: {} 2026-02-24T07:50:33.129 INFO:teuthology.orchestra.run.trial035.stderr:osd.2: osd_enable_op_tracker = '' 2026-02-24T07:50:33.141 INFO:teuthology.orchestra.run.trial035.stdout:osd.3: {} 2026-02-24T07:50:33.141 INFO:teuthology.orchestra.run.trial035.stderr:osd.3: osd_enable_op_tracker = '' 2026-02-24T07:50:33.198 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2026-02-24T07:50:33.198 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2026-02-24T07:50:33.198 INFO:tasks.thrashosds.thrasher:Growing pool .mgr 2026-02-24T07:50:33.198 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph pg dump --format=json 2026-02-24T07:50:33.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:50:33 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/89496578' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2026-02-24T07:50:33.865 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:50:34.065 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:50:34.062+0000 7f52c2f4d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:50:34.065 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:50:34.062+0000 7f52c2f4d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:50:34.205 INFO:teuthology.orchestra.run.trial035.stdout: 2026-02-24T07:50:34.206 INFO:teuthology.orchestra.run.trial035.stderr:dumped all 2026-02-24T07:50:34.248 INFO:teuthology.orchestra.run.trial035.stdout:{"pg_ready":true,"pg_map":{"version":262,"stamp":"2026-02-24T07:50:33.718657+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":41511420472,"num_objects":633408,"num_object_clones":0,"num_object_copies":1900224,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":633408,"num_whiteouts":0,"num_read":137,"num_read_kb":241,"num_write":661465,"num_write_kb":41433639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":53,"ondisk_log_size":53,"up":51,"acting":51,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":51,"num_osds":4,"num_per_pool_osds":4,"num_per_pool_omap_osds":4,"kb":2936012800,"kb_used":129180200,"kb_used_data":122093528,"kb_used_omap":54,"kb_used_meta":7086537,"kb_avail":2806832600,"statfs":{"total":3006477107200,"available":2874196582400,"internally_reserved":0,"allocated":125023772672,"data_stored":125022110072,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":55812,"internal_metadata":7256614396},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[17,3,3,2],"upper_bound":16},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2515992600,"num_objects":38392,"num_object_clones":0,"num_object_copies":115176,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38392,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":66316,"num_write_kb":3350593,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001654"},"pg_stats":[{"pgid":"4.f","version":"61'41567","reported_seq":41900,"reported_epoch":61,"state":"active+clean","last_fresh":"2026-02-24T07:50:30.829676+0000","last_change":"2026-02-24T07:50:03.071459+0000","last_active":"2026-02-24T07:50:30.829676+0000","last_peered":"2026-02-24T07:50:30.829676+0000","last_clean":"2026-02-24T07:50:30.829676+0000","last_became_active":"2026-02-24T07:48:57.902313+0000","last_became_peered":"2026-02-24T07:48:57.902313+0000","last_unstale":"2026-02-24T07:50:30.829676+0000","last_undegraded":"2026-02-24T07:50:30.829676+0000","last_fullsized":"2026-02-24T07:50:30.829676+0000","mapping_epoch":55,"log_start":"61'41564","ondisk_log_start":"61'41564","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"61'30419","last_scrub_stamp":"2026-02-24T07:50:03.071380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:50:03.071380+0000","objects_scrubbed":29975,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:51:16.123453+0000","scrub_duration":2568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2606563328,"num_objects":39773,"num_object_clones":0,"num_object_copies":119319,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39773,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":41567,"num_write_kb":2602880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"61'41462","reported_seq":41862,"reported_epoch":61,"state":"active+clean","last_fresh":"2026-02-24T07:50:30.729171+0000","last_change":"2026-02-24T07:50:21.065676+0000","last_active":"2026-02-24T07:50:30.729171+0000","last_peered":"2026-02-24T07:50:30.729171+0000","last_clean":"2026-02-24T07:50:30.729171+0000","last_became_active":"2026-02-24T07:48:57.903584+0000","last_became_peered":"2026-02-24T07:48:57.903584+0000","last_unstale":"2026-02-24T07:50:30.729171+0000","last_undegraded":"2026-02-24T07:50:30.729171+0000","last_fullsized":"2026-02-24T07:50:30.729171+0000","mapping_epoch":55,"log_start":"61'41459","ondisk_log_start":"61'41459","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"61'37487","last_scrub_stamp":"2026-02-24T07:50:21.065638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:50:21.065638+0000","objects_scrubbed":36748,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:51:31.991874+0000","scrub_duration":3643,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2602827776,"num_objects":39716,"num_object_clones":0,"num_object_copies":119148,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39716,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":41462,"num_write_kb":2597696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.d","version":"61'41268","reported_seq":41611,"reported_epoch":61,"state":"active+clean","last_fresh":"2026-02-24T07:50:30.743382+0000","last_change":"2026-02-24T07:50:06.701973+0000","last_active":"2026-02-24T07:50:30.743382+0000","last_peered":"2026-02-24T07:50:30.743382+0000","last_clean":"2026-02-24T07:50:30.743382+0000","last_became_active":"2026-02-24T07:48:57.905031+0000","last_became_peered":"2026-02-24T07:48:57.905031+0000","last_unstale":"2026-02-24T07:50:30.743382+0000","last_undegraded":"2026-02-24T07:50:30.743382+0000","last_fullsized":"2026-02-24T07:50:30.743382+0000","mapping_epoch":55,"log_start":"61'41265","ondisk_log_start":"61'41265","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"61'31648","last_scrub_stamp":"2026-02-24T07:50:06.701883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:50:06.701883+0000","objects_scrubbed":30990,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:51:19.224966+0000","scrub_duration":3493,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2590179352,"num_objects":39524,"num_object_clones":0,"num_object_copies":118572,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39524,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":41268,"num_write_kb":2585281,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"61'41100","reported_seq":41509,"reported_epoch":61,"state":"active+clean","last_fresh":"2026-02-24T07:50:30.747676+0000","last_change":"2026-02-24T07:50:24.798674+0000","last_active":"2026-02-24T07:50:30.747676+0000","last_peered":"2026-02-24T07:50:30.747676+0000","last_clean":"2026-02-24T07:50:30.747676+0000","last_became_active":"2026-02-24T07:48:57.904683+0000","last_became_peered":"2026-02-24T07:48:57.904683+0000","last_unstale":"2026-02-24T07:50:30.747676+0000","last_undegraded":"2026-02-24T07:50:30.747676+0000","last_fullsized":"2026-02-24T07:50:30.747676+0000","mapping_epoch":55,"log_start":"61'41095","ondisk_log_start":"61'41095","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"61'38696","last_scrub_stamp":"2026-02-24T07:50:24.798629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:50:24.798629+0000","objects_scrubbed":37979,"log_size":5,"log_dups_size":3000,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:51:26.521338+0000","scrub_duration":3612,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2581594112,"num_objects":39392,"num_object_clones":0,"num_object_copies":118176,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39392,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":41100,"num_write_kb":2575744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.b","version":"61'41207","reported_seq":41629,"reported_epoch":61,"state":"active+clean","last_fresh":"2026-02-24T07:50:30.740725+0000","last_change":"2026-02-24T07:50:27.267008+0000","last_active":"2026-02-24T07:50:30.740725+0000","last_peered":"2026-02-24T07:50:30.740725+0000","last_clean":"2026-02-24T07:50:30.740725+0000","last_became_active":"2026-02-24T07:48:57.901700+0000","last_became_peered":"2026-02-24T07:48:57.901700+0000","last_unstale":"2026-02-24T07:50:30.740725+0000","last_undegraded":"2026-02-24T07:50:30.740725+0000","last_fullsized":"2026-02-24T07:50:30.740725+0000","mapping_epoch":55,"log_start":"61'41204","ondisk_log_start":"61'41204","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"61'39639","last_scrub_stamp":"2026-02-24T07:50:27.266943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:50:27.266943+0000","objects_scrubbed":38982,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:51:34.346751+0000","scrub_duration":3744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2585198592,"num_objects":39447,"num_object_clones":0,"num_object_copies":118341,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39447,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":41207,"num_write_kb":2580928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.a","version":"61'41318","reported_seq":41462,"reported_epoch":61,"state":"active+clean+scrubbing","last_fresh":"2026-02-24T07:50:30.731136+0000","last_change":"2026-02-24T07:50:29.496830+0000","last_active":"2026-02-24T07:50:30.731136+0000","last_peered":"2026-02-24T07:50:30.731136+0000","last_clean":"2026-02-24T07:50:30.731136+0000","last_became_active":"2026-02-24T07:48:57.903228+0000","last_became_peered":"2026-02-24T07:48:57.903228+0000","last_unstale":"2026-02-24T07:50:30.731136+0000","last_undegraded":"2026-02-24T07:50:30.731136+0000","last_fullsized":"2026-02-24T07:50:30.731136+0000","mapping_epoch":55,"log_start":"61'41315","ondisk_log_start":"61'41315","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:48:56.890768+0000","objects_scrubbed":11899,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"scrubbing for 2s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2593128448,"num_objects":39568,"num_object_clones":0,"num_object_copies":118704,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39568,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":41318,"num_write_kb":2588352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"61'41286","reported_seq":41651,"reported_epoch":61,"state":"active+clean","last_fresh":"2026-02-24T07:50:30.728312+0000","last_change":"2026-02-24T07:50:13.165492+0000","last_active":"2026-02-24T07:50:30.728312+0000","last_peered":"2026-02-24T07:50:30.728312+0000","last_clean":"2026-02-24T07:50:30.728312+0000","last_became_active":"2026-02-24T07:48:57.901730+0000","last_became_peered":"2026-02-24T07:48:57.901730+0000","last_unstale":"2026-02-24T07:50:30.728312+0000","last_undegraded":"2026-02-24T07:50:30.728312+0000","last_fullsized":"2026-02-24T07:50:30.728312+0000","mapping_epoch":55,"log_start":"61'41283","ondisk_log_start":"61'41283","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"61'33943","last_scrub_stamp":"2026-02-24T07:50:13.165459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:50:13.165459+0000","objects_scrubbed":33253,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:51:33.820708+0000","scrub_duration":3776,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2594177024,"num_objects":39584,"num_object_clones":0,"num_object_copies":118752,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39584,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":41286,"num_write_kb":2587840,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"61'41281","reported_seq":41609,"reported_epoch":61,"state":"active+clean","last_fresh":"2026-02-24T07:50:30.745704+0000","last_change":"2026-02-24T07:50:03.153499+0000","last_active":"2026-02-24T07:50:30.745704+0000","last_peered":"2026-02-24T07:50:30.745704+0000","last_clean":"2026-02-24T07:50:30.745704+0000","last_became_active":"2026-02-24T07:48:57.904376+0000","last_became_peered":"2026-02-24T07:48:57.904376+0000","last_unstale":"2026-02-24T07:50:30.745704+0000","last_undegraded":"2026-02-24T07:50:30.745704+0000","last_fullsized":"2026-02-24T07:50:30.745704+0000","mapping_epoch":55,"log_start":"61'41278","ondisk_log_start":"61'41278","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"61'30163","last_scrub_stamp":"2026-02-24T07:50:03.153454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:50:03.153454+0000","objects_scrubbed":29681,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:51:10.266136+0000","scrub_duration":2916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2593062912,"num_objects":39567,"num_object_clones":0,"num_object_copies":118701,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39567,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":41281,"num_write_kb":2587136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.7","version":"61'41243","reported_seq":41599,"reported_epoch":61,"state":"active+clean","last_fresh":"2026-02-24T07:50:30.832990+0000","last_change":"2026-02-24T07:50:11.227608+0000","last_active":"2026-02-24T07:50:30.832990+0000","last_peered":"2026-02-24T07:50:30.832990+0000","last_clean":"2026-02-24T07:50:30.832990+0000","last_became_active":"2026-02-24T07:48:57.902484+0000","last_became_peered":"2026-02-24T07:48:57.902484+0000","last_unstale":"2026-02-24T07:50:30.832990+0000","last_undegraded":"2026-02-24T07:50:30.832990+0000","last_fullsized":"2026-02-24T07:50:30.832990+0000","mapping_epoch":55,"log_start":"61'41240","ondisk_log_start":"61'41240","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"61'32990","last_scrub_stamp":"2026-02-24T07:50:11.227573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:50:11.227573+0000","objects_scrubbed":32460,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:51:29.781611+0000","scrub_duration":3701,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2582315008,"num_objects":39403,"num_object_clones":0,"num_object_copies":118209,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39403,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":41243,"num_write_kb":2580672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.6","version":"61'41105","reported_seq":41472,"reported_epoch":61,"state":"active+clean","last_fresh":"2026-02-24T07:50:30.743223+0000","last_change":"2026-02-24T07:50:14.386679+0000","last_active":"2026-02-24T07:50:30.743223+0000","last_peered":"2026-02-24T07:50:30.743223+0000","last_clean":"2026-02-24T07:50:30.743223+0000","last_became_active":"2026-02-24T07:48:57.901565+0000","last_became_peered":"2026-02-24T07:48:57.901565+0000","last_unstale":"2026-02-24T07:50:30.743223+0000","last_undegraded":"2026-02-24T07:50:30.743223+0000","last_fullsized":"2026-02-24T07:50:30.743223+0000","mapping_epoch":55,"log_start":"61'41102","ondisk_log_start":"61'41102","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"61'34244","last_scrub_stamp":"2026-02-24T07:50:14.386625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:50:14.386625+0000","objects_scrubbed":33505,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:51:16.739697+0000","scrub_duration":3790,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2585722880,"num_objects":39455,"num_object_clones":0,"num_object_copies":118365,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39455,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":41105,"num_write_kb":2577920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"61'41580","reported_seq":41961,"reported_epoch":61,"state":"active+clean","last_fresh":"2026-02-24T07:50:30.835182+0000","last_change":"2026-02-24T07:50:16.399238+0000","last_active":"2026-02-24T07:50:30.835182+0000","last_peered":"2026-02-24T07:50:30.835182+0000","last_clean":"2026-02-24T07:50:30.835182+0000","last_became_active":"2026-02-24T07:48:57.901824+0000","last_became_peered":"2026-02-24T07:48:57.901824+0000","last_unstale":"2026-02-24T07:50:30.835182+0000","last_undegraded":"2026-02-24T07:50:30.835182+0000","last_fullsized":"2026-02-24T07:50:30.835182+0000","mapping_epoch":55,"log_start":"61'41577","ondisk_log_start":"61'41577","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"61'35510","last_scrub_stamp":"2026-02-24T07:50:16.399194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:50:16.399194+0000","objects_scrubbed":34844,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:51:44.770701+0000","scrub_duration":2971,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2600206336,"num_objects":39676,"num_object_clones":0,"num_object_copies":119028,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39676,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":41580,"num_write_kb":2600192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[1,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"61'41042","reported_seq":41395,"reported_epoch":61,"state":"active+clean","last_fresh":"2026-02-24T07:50:30.744170+0000","last_change":"2026-02-24T07:50:10.124042+0000","last_active":"2026-02-24T07:50:30.744170+0000","last_peered":"2026-02-24T07:50:30.744170+0000","last_clean":"2026-02-24T07:50:30.744170+0000","last_became_active":"2026-02-24T07:48:57.902224+0000","last_became_peered":"2026-02-24T07:48:57.902224+0000","last_unstale":"2026-02-24T07:50:30.744170+0000","last_undegraded":"2026-02-24T07:50:30.744170+0000","last_fullsized":"2026-02-24T07:50:30.744170+0000","mapping_epoch":55,"log_start":"61'41039","ondisk_log_start":"61'41039","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"61'32620","last_scrub_stamp":"2026-02-24T07:50:10.124005+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:50:10.124005+0000","objects_scrubbed":32078,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:51:12.877892+0000","scrub_duration":3559,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2577924096,"num_objects":39336,"num_object_clones":0,"num_object_copies":118008,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39336,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":41042,"num_write_kb":2572096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"61'41481","reported_seq":41806,"reported_epoch":61,"state":"active+clean","last_fresh":"2026-02-24T07:50:30.834178+0000","last_change":"2026-02-24T07:50:01.398116+0000","last_active":"2026-02-24T07:50:30.834178+0000","last_peered":"2026-02-24T07:50:30.834178+0000","last_clean":"2026-02-24T07:50:30.834178+0000","last_became_active":"2026-02-24T07:48:57.902125+0000","last_became_peered":"2026-02-24T07:48:57.902125+0000","last_unstale":"2026-02-24T07:50:30.834178+0000","last_undegraded":"2026-02-24T07:50:30.834178+0000","last_fullsized":"2026-02-24T07:50:30.834178+0000","mapping_epoch":55,"log_start":"61'41478","ondisk_log_start":"61'41478","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"61'29707","last_scrub_stamp":"2026-02-24T07:50:01.398090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:50:01.398090+0000","objects_scrubbed":29171,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:51:10.566529+0000","scrub_duration":2880,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2602369024,"num_objects":39709,"num_object_clones":0,"num_object_copies":119127,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39709,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":41481,"num_write_kb":2598080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1","version":"61'41554","reported_seq":41986,"reported_epoch":61,"state":"active+clean","last_fresh":"2026-02-24T07:50:30.728613+0000","last_change":"2026-02-24T07:50:29.889880+0000","last_active":"2026-02-24T07:50:30.728613+0000","last_peered":"2026-02-24T07:50:30.728613+0000","last_clean":"2026-02-24T07:50:30.728613+0000","last_became_active":"2026-02-24T07:48:57.903439+0000","last_became_peered":"2026-02-24T07:48:57.903439+0000","last_unstale":"2026-02-24T07:50:30.728613+0000","last_undegraded":"2026-02-24T07:50:30.728613+0000","last_fullsized":"2026-02-24T07:50:30.728613+0000","mapping_epoch":55,"log_start":"61'41551","ondisk_log_start":"61'41551","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"61'41105","last_scrub_stamp":"2026-02-24T07:50:29.889816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:50:29.889816+0000","objects_scrubbed":40127,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:51:50.788414+0000","scrub_duration":4409,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2612002816,"num_objects":39856,"num_object_clones":0,"num_object_copies":119568,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39856,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":41554,"num_write_kb":2605120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.0","version":"61'41448","reported_seq":41880,"reported_epoch":61,"state":"active+clean","last_fresh":"2026-02-24T07:50:30.739173+0000","last_change":"2026-02-24T07:50:29.387323+0000","last_active":"2026-02-24T07:50:30.739173+0000","last_peered":"2026-02-24T07:50:30.739173+0000","last_clean":"2026-02-24T07:50:30.739173+0000","last_became_active":"2026-02-24T07:48:57.902565+0000","last_became_peered":"2026-02-24T07:48:57.902565+0000","last_unstale":"2026-02-24T07:50:30.739173+0000","last_undegraded":"2026-02-24T07:50:30.739173+0000","last_fullsized":"2026-02-24T07:50:30.739173+0000","mapping_epoch":55,"log_start":"61'41445","ondisk_log_start":"61'41445","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"61'40736","last_scrub_stamp":"2026-02-24T07:50:29.387206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:50:29.387206+0000","objects_scrubbed":39946,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:51:59.173732+0000","scrub_duration":3834,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2601385984,"num_objects":39694,"num_object_clones":0,"num_object_copies":119082,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39694,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":41448,"num_write_kb":2596544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,3],"acting":[0,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.5","version":"61'41388","reported_seq":41723,"reported_epoch":61,"state":"active+clean","last_fresh":"2026-02-24T07:50:30.747305+0000","last_change":"2026-02-24T07:50:04.821914+0000","last_active":"2026-02-24T07:50:30.747305+0000","last_peered":"2026-02-24T07:50:30.747305+0000","last_clean":"2026-02-24T07:50:30.747305+0000","last_became_active":"2026-02-24T07:48:57.905127+0000","last_became_peered":"2026-02-24T07:48:57.905127+0000","last_unstale":"2026-02-24T07:50:30.747305+0000","last_undegraded":"2026-02-24T07:50:30.747305+0000","last_fullsized":"2026-02-24T07:50:30.747305+0000","mapping_epoch":55,"log_start":"61'41385","ondisk_log_start":"61'41385","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"61'30833","last_scrub_stamp":"2026-02-24T07:50:04.821861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:50:04.821861+0000","objects_scrubbed":30132,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:51:15.928207+0000","scrub_duration":3578,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2602172416,"num_objects":39706,"num_object_clones":0,"num_object_copies":119118,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39706,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":41388,"num_write_kb":2595008,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"29'99","reported_seq":293,"reported_epoch":61,"state":"active+clean","last_fresh":"2026-02-24T07:50:30.401963+0000","last_change":"2026-02-24T07:50:30.401963+0000","last_active":"2026-02-24T07:50:30.401963+0000","last_peered":"2026-02-24T07:50:30.401963+0000","last_clean":"2026-02-24T07:50:30.401963+0000","last_became_active":"2026-02-24T07:42:54.186719+0000","last_became_peered":"2026-02-24T07:42:54.186719+0000","last_unstale":"2026-02-24T07:50:30.401963+0000","last_undegraded":"2026-02-24T07:50:30.401963+0000","last_fullsized":"2026-02-24T07:50:30.401963+0000","mapping_epoch":27,"log_start":"29'96","ondisk_log_start":"29'96","created":20,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'99","last_scrub_stamp":"2026-02-24T07:50:30.401929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:42:43.640776+0000","last_clean_scrub_stamp":"2026-02-24T07:50:30.401929+0000","objects_scrubbed":2,"log_size":3,"log_dups_size":96,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:51:52.990727+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":240,"num_write":135,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":16,"stat_sum":{"num_bytes":41510830104,"num_objects":633406,"num_object_clones":0,"num_object_copies":1900218,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":633406,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":661330,"num_write_kb":41431489,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":124537810944,"data_stored":124537798728,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":50,"ondisk_log_size":50,"up":48,"acting":48,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":240,"num_write":135,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":948,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":3,"up_from":25,"seq":107374182494,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":32667084,"kb_used_data":30590904,"kb_used_omap":13,"kb_used_meta":2076146,"kb_avail":701336116,"statfs":{"total":751619276800,"available":718168182784,"internally_reserved":0,"allocated":31325085696,"data_stored":31324668140,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13685,"internal_metadata":2125974155},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[4,1,1,2],"upper_bound":16},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":19,"seq":81604378720,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":37010828,"kb_used_data":35593208,"kb_used_omap":14,"kb_used_meta":1417585,"kb_avail":696992372,"statfs":{"total":751619276800,"available":713720188928,"internally_reserved":0,"allocated":36447444992,"data_stored":36447027436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":14721,"internal_metadata":1451607679},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,1,2],"upper_bound":8},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542241,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":27399300,"kb_used_data":25414640,"kb_used_omap":14,"kb_used_meta":1984625,"kb_avail":706603900,"statfs":{"total":751619276800,"available":723562393600,"internally_reserved":0,"allocated":26024591360,"data_stored":26024181428,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":15140,"internal_metadata":2032256220},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705764,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":32102988,"kb_used_data":30494776,"kb_used_omap":11,"kb_used_meta":1608180,"kb_avail":701900212,"statfs":{"total":751619276800,"available":718745817088,"internally_reserved":0,"allocated":31226650624,"data_stored":31226233068,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12266,"internal_metadata":1646776342},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[7,1],"upper_bound":4},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":948,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":31135567872,"data_stored":31135563800,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":25911820288,"data_stored":25911820288,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":36320186368,"data_stored":36320182296,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":31170236416,"data_stored":31170232344,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-24T07:50:34.250 INFO:tasks.cephadm.ceph_manager.ceph:increase pool size by 10 2026-02-24T07:50:34.250 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph osd pool set .mgr pg_num 251 2026-02-24T07:50:34.394 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:50:34.420 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:50:34 trial035 ceph-mon[39284]: 4.a scrub ok 2026-02-24T07:50:34.420 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:50:34 trial035 ceph-mon[39284]: pgmap v262: 17 pgs: 1 active+clean+scrubbing, 16 active+clean; 39 GiB data, 123 GiB used, 2.6 TiB / 2.7 TiB avail; 85 B/s rd, 273 MiB/s wr, 5.53k op/s 2026-02-24T07:50:34.514 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:50:34.511+0000 7f77bbe11640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:50:34.514 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:50:34.511+0000 7f77bbe11640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:50:34.663 INFO:teuthology.orchestra.run.trial035.stderr:Error EPERM: crimson_allow_pg_split is false; pg_num increase denied for crimson pool 2026-02-24T07:50:34.701 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-24T07:50:35.454 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:50:35 trial035 ceph-mon[39284]: from='client.15011 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2026-02-24T07:50:35.454 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:50:35 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/1268036778' entity='client.admin' cmd={"prefix": "osd pool set", "pool": ".mgr", "var": "pg_num", "val": "251"} : dispatch 2026-02-24T07:50:35.807 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2026-02-24T07:50:35.948 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:50:36.068 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:50:36.066+0000 7f8a2b08d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:50:36.068 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:50:36.066+0000 7f8a2b08d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:50:36.084 INFO:teuthology.orchestra.run.trial035.stdout:osd.0: {} 2026-02-24T07:50:36.085 INFO:teuthology.orchestra.run.trial035.stderr:osd.0: osd_enable_op_tracker = '' 2026-02-24T07:50:36.096 INFO:teuthology.orchestra.run.trial035.stdout:osd.1: {} 2026-02-24T07:50:36.097 INFO:teuthology.orchestra.run.trial035.stderr:osd.1: osd_enable_op_tracker = '' 2026-02-24T07:50:36.110 INFO:teuthology.orchestra.run.trial035.stdout:osd.2: {} 2026-02-24T07:50:36.110 INFO:teuthology.orchestra.run.trial035.stderr:osd.2: osd_enable_op_tracker = '' 2026-02-24T07:50:36.126 INFO:teuthology.orchestra.run.trial035.stdout:osd.3: {} 2026-02-24T07:50:36.127 INFO:teuthology.orchestra.run.trial035.stderr:osd.3: osd_enable_op_tracker = '' 2026-02-24T07:50:36.454 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:50:36 trial035 ceph-mon[39284]: pgmap v263: 17 pgs: 1 active+clean+scrubbing, 16 active+clean; 39 GiB data, 123 GiB used, 2.6 TiB / 2.7 TiB avail; 85 B/s rd, 273 MiB/s wr, 5.53k op/s 2026-02-24T07:50:36.454 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:50:36 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/373973721' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2026-02-24T07:50:37.356 INFO:journalctl@ceph.prometheus.a.trial091.stdout:Feb 24 07:50:36 trial091 ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-prometheus-a[66007]: time=2026-02-24T07:50:36.973Z level=ERROR source=scrape.go:1630 msg="Failed to determine correct type of scrape target." component="scrape manager" scrape_pool=ceph target=http://trial035:9283/metrics content_type="text/html;charset=utf-8" fallback_media_type="" err="received unsupported Content-Type \"text/html;charset=utf-8\" and no fallback_scrape_protocol specified for target" 2026-02-24T07:50:37.454 INFO:journalctl@ceph.mgr.x.trial035.stdout:Feb 24 07:50:36 trial035 ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-mgr-x[39621]: ::ffff:10.20.193.91 - - [24/Feb/2026:07:50:36] "GET /metrics HTTP/1.1" 200 4216 "" "Prometheus/3.6.0" 2026-02-24T07:50:38.170 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2026-02-24T07:50:38.309 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:50:38.427 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:50:38.426+0000 7fd9fa76e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:50:38.428 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:50:38.426+0000 7fd9fa76e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:50:38.442 INFO:teuthology.orchestra.run.trial035.stdout:osd.0: {} 2026-02-24T07:50:38.443 INFO:teuthology.orchestra.run.trial035.stderr:osd.0: osd_enable_op_tracker = '' 2026-02-24T07:50:38.455 INFO:teuthology.orchestra.run.trial035.stdout:osd.1: {} 2026-02-24T07:50:38.456 INFO:teuthology.orchestra.run.trial035.stderr:osd.1: osd_enable_op_tracker = '' 2026-02-24T07:50:38.470 INFO:teuthology.orchestra.run.trial035.stdout:osd.2: {} 2026-02-24T07:50:38.471 INFO:teuthology.orchestra.run.trial035.stderr:osd.2: osd_enable_op_tracker = '' 2026-02-24T07:50:38.483 INFO:teuthology.orchestra.run.trial035.stdout:osd.3: {} 2026-02-24T07:50:38.483 INFO:teuthology.orchestra.run.trial035.stderr:osd.3: osd_enable_op_tracker = '' 2026-02-24T07:50:39.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:50:38 trial035 ceph-mon[39284]: pgmap v264: 17 pgs: 17 active+clean; 36 GiB data, 115 GiB used, 2.6 TiB / 2.7 TiB avail; 85 B/s rd, 273 MiB/s wr, 9.10k op/s 2026-02-24T07:50:39.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:50:38 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/2788923969' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2026-02-24T07:50:39.703 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2026-02-24T07:50:39.703 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2026-02-24T07:50:39.703 INFO:tasks.thrashosds.thrasher:Growing pool unique_pool_2 2026-02-24T07:50:39.704 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph pg dump --format=json 2026-02-24T07:50:39.847 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:50:39.969 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:50:39.967+0000 7f437597d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:50:39.969 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:50:39.967+0000 7f437597d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:50:40.107 INFO:teuthology.orchestra.run.trial035.stdout: 2026-02-24T07:50:40.108 INFO:teuthology.orchestra.run.trial035.stderr:dumped all 2026-02-24T07:50:40.150 INFO:teuthology.orchestra.run.trial035.stdout:{"pg_ready":true,"pg_map":{"version":265,"stamp":"2026-02-24T07:50:39.719473+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":38703596088,"num_objects":590564,"num_object_clones":0,"num_object_copies":1771692,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":590564,"num_whiteouts":0,"num_read":137,"num_read_kb":241,"num_write":704309,"num_write_kb":41433639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":51,"ondisk_log_size":51,"up":51,"acting":51,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":51,"num_osds":4,"num_per_pool_osds":4,"num_per_pool_omap_osds":4,"kb":2936012800,"kb_used":120969832,"kb_used_data":113810712,"kb_used_omap":55,"kb_used_meta":7158792,"kb_avail":2815042968,"statfs":{"total":3006477107200,"available":2882603999232,"internally_reserved":0,"allocated":116542169088,"data_stored":116540506488,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":56855,"internal_metadata":7330603497},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[7,2],"upper_bound":4},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-2457731048,"num_objects":-37501,"num_object_clones":0,"num_object_copies":-112503,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-37501,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":76111,"num_write_kb":1235457,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001614"},"pg_stats":[{"pgid":"4.f","version":"61'44287","reported_seq":44619,"reported_epoch":61,"state":"active+clean","last_fresh":"2026-02-24T07:50:35.833176+0000","last_change":"2026-02-24T07:50:03.071459+0000","last_active":"2026-02-24T07:50:35.833176+0000","last_peered":"2026-02-24T07:50:35.833176+0000","last_clean":"2026-02-24T07:50:35.833176+0000","last_became_active":"2026-02-24T07:48:57.902313+0000","last_became_peered":"2026-02-24T07:48:57.902313+0000","last_unstale":"2026-02-24T07:50:35.833176+0000","last_undegraded":"2026-02-24T07:50:35.833176+0000","last_fullsized":"2026-02-24T07:50:35.833176+0000","mapping_epoch":55,"log_start":"61'44284","ondisk_log_start":"61'44284","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"61'30419","last_scrub_stamp":"2026-02-24T07:50:03.071380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:50:03.071380+0000","objects_scrubbed":29975,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:51:16.123453+0000","scrub_duration":2568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2428305408,"num_objects":37053,"num_object_clones":0,"num_object_copies":111159,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37053,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44287,"num_write_kb":2602880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"61'44155","reported_seq":44555,"reported_epoch":61,"state":"active+clean","last_fresh":"2026-02-24T07:50:35.731248+0000","last_change":"2026-02-24T07:50:21.065676+0000","last_active":"2026-02-24T07:50:35.731248+0000","last_peered":"2026-02-24T07:50:35.731248+0000","last_clean":"2026-02-24T07:50:35.731248+0000","last_became_active":"2026-02-24T07:48:57.903584+0000","last_became_peered":"2026-02-24T07:48:57.903584+0000","last_unstale":"2026-02-24T07:50:35.731248+0000","last_undegraded":"2026-02-24T07:50:35.731248+0000","last_fullsized":"2026-02-24T07:50:35.731248+0000","mapping_epoch":55,"log_start":"61'44152","ondisk_log_start":"61'44152","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"61'37487","last_scrub_stamp":"2026-02-24T07:50:21.065638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:50:21.065638+0000","objects_scrubbed":36748,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:51:31.991874+0000","scrub_duration":3643,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2426339328,"num_objects":37023,"num_object_clones":0,"num_object_copies":111069,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37023,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44155,"num_write_kb":2597696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.d","version":"61'43925","reported_seq":44267,"reported_epoch":61,"state":"active+clean","last_fresh":"2026-02-24T07:50:35.746925+0000","last_change":"2026-02-24T07:50:06.701973+0000","last_active":"2026-02-24T07:50:35.746925+0000","last_peered":"2026-02-24T07:50:35.746925+0000","last_clean":"2026-02-24T07:50:35.746925+0000","last_became_active":"2026-02-24T07:48:57.905031+0000","last_became_peered":"2026-02-24T07:48:57.905031+0000","last_unstale":"2026-02-24T07:50:35.746925+0000","last_undegraded":"2026-02-24T07:50:35.746925+0000","last_fullsized":"2026-02-24T07:50:35.746925+0000","mapping_epoch":55,"log_start":"61'43922","ondisk_log_start":"61'43922","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"61'31648","last_scrub_stamp":"2026-02-24T07:50:06.701883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:50:06.701883+0000","objects_scrubbed":30990,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:51:19.224966+0000","scrub_duration":3493,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2416050200,"num_objects":36867,"num_object_clones":0,"num_object_copies":110601,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36867,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":43925,"num_write_kb":2585281,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"61'43788","reported_seq":44199,"reported_epoch":61,"state":"active+clean","last_fresh":"2026-02-24T07:50:35.744878+0000","last_change":"2026-02-24T07:50:24.798674+0000","last_active":"2026-02-24T07:50:35.744878+0000","last_peered":"2026-02-24T07:50:35.744878+0000","last_clean":"2026-02-24T07:50:35.744878+0000","last_became_active":"2026-02-24T07:48:57.904683+0000","last_became_peered":"2026-02-24T07:48:57.904683+0000","last_unstale":"2026-02-24T07:50:35.744878+0000","last_undegraded":"2026-02-24T07:50:35.744878+0000","last_fullsized":"2026-02-24T07:50:35.744878+0000","mapping_epoch":55,"log_start":"61'43785","ondisk_log_start":"61'43785","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"61'38696","last_scrub_stamp":"2026-02-24T07:50:24.798629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:50:24.798629+0000","objects_scrubbed":37979,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:51:26.521338+0000","scrub_duration":3612,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2405433344,"num_objects":36704,"num_object_clones":0,"num_object_copies":110112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36704,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":43788,"num_write_kb":2575744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.b","version":"61'43879","reported_seq":44301,"reported_epoch":61,"state":"active+clean","last_fresh":"2026-02-24T07:50:35.743076+0000","last_change":"2026-02-24T07:50:27.267008+0000","last_active":"2026-02-24T07:50:35.743076+0000","last_peered":"2026-02-24T07:50:35.743076+0000","last_clean":"2026-02-24T07:50:35.743076+0000","last_became_active":"2026-02-24T07:48:57.901700+0000","last_became_peered":"2026-02-24T07:48:57.901700+0000","last_unstale":"2026-02-24T07:50:35.743076+0000","last_undegraded":"2026-02-24T07:50:35.743076+0000","last_fullsized":"2026-02-24T07:50:35.743076+0000","mapping_epoch":55,"log_start":"61'43876","ondisk_log_start":"61'43876","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"61'39639","last_scrub_stamp":"2026-02-24T07:50:27.266943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:50:27.266943+0000","objects_scrubbed":38982,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:51:34.346751+0000","scrub_duration":3744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2410086400,"num_objects":36775,"num_object_clones":0,"num_object_copies":110325,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36775,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":43879,"num_write_kb":2580928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.a","version":"61'44012","reported_seq":44437,"reported_epoch":61,"state":"active+clean","last_fresh":"2026-02-24T07:50:35.730415+0000","last_change":"2026-02-24T07:50:33.043786+0000","last_active":"2026-02-24T07:50:35.730415+0000","last_peered":"2026-02-24T07:50:35.730415+0000","last_clean":"2026-02-24T07:50:35.730415+0000","last_became_active":"2026-02-24T07:48:57.903228+0000","last_became_peered":"2026-02-24T07:48:57.903228+0000","last_unstale":"2026-02-24T07:50:35.730415+0000","last_undegraded":"2026-02-24T07:50:35.730415+0000","last_fullsized":"2026-02-24T07:50:35.730415+0000","mapping_epoch":55,"log_start":"61'44009","ondisk_log_start":"61'44009","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"61'42539","last_scrub_stamp":"2026-02-24T07:50:33.043726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:50:33.043726+0000","objects_scrubbed":39246,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:51:51.239665+0000","scrub_duration":3547,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2416574464,"num_objects":36874,"num_object_clones":0,"num_object_copies":110622,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36874,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44012,"num_write_kb":2588352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"61'43901","reported_seq":44266,"reported_epoch":61,"state":"active+clean","last_fresh":"2026-02-24T07:50:35.730387+0000","last_change":"2026-02-24T07:50:13.165492+0000","last_active":"2026-02-24T07:50:35.730387+0000","last_peered":"2026-02-24T07:50:35.730387+0000","last_clean":"2026-02-24T07:50:35.730387+0000","last_became_active":"2026-02-24T07:48:57.901730+0000","last_became_peered":"2026-02-24T07:48:57.901730+0000","last_unstale":"2026-02-24T07:50:35.730387+0000","last_undegraded":"2026-02-24T07:50:35.730387+0000","last_fullsized":"2026-02-24T07:50:35.730387+0000","mapping_epoch":55,"log_start":"61'43898","ondisk_log_start":"61'43898","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"61'33943","last_scrub_stamp":"2026-02-24T07:50:13.165459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:50:13.165459+0000","objects_scrubbed":33253,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:51:33.820708+0000","scrub_duration":3776,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2422800384,"num_objects":36969,"num_object_clones":0,"num_object_copies":110907,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36969,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":43901,"num_write_kb":2587840,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"61'43932","reported_seq":44261,"reported_epoch":61,"state":"active+clean","last_fresh":"2026-02-24T07:50:35.743133+0000","last_change":"2026-02-24T07:50:03.153499+0000","last_active":"2026-02-24T07:50:35.743133+0000","last_peered":"2026-02-24T07:50:35.743133+0000","last_clean":"2026-02-24T07:50:35.743133+0000","last_became_active":"2026-02-24T07:48:57.904376+0000","last_became_peered":"2026-02-24T07:48:57.904376+0000","last_unstale":"2026-02-24T07:50:35.743133+0000","last_undegraded":"2026-02-24T07:50:35.743133+0000","last_fullsized":"2026-02-24T07:50:35.743133+0000","mapping_epoch":55,"log_start":"61'43929","ondisk_log_start":"61'43929","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"61'30163","last_scrub_stamp":"2026-02-24T07:50:03.153454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:50:03.153454+0000","objects_scrubbed":29681,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:51:10.266136+0000","scrub_duration":2916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2419326976,"num_objects":36916,"num_object_clones":0,"num_object_copies":110748,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36916,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":43932,"num_write_kb":2587136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.7","version":"61'43963","reported_seq":44318,"reported_epoch":61,"state":"active+clean","last_fresh":"2026-02-24T07:50:35.837106+0000","last_change":"2026-02-24T07:50:11.227608+0000","last_active":"2026-02-24T07:50:35.837106+0000","last_peered":"2026-02-24T07:50:35.837106+0000","last_clean":"2026-02-24T07:50:35.837106+0000","last_became_active":"2026-02-24T07:48:57.902484+0000","last_became_peered":"2026-02-24T07:48:57.902484+0000","last_unstale":"2026-02-24T07:50:35.837106+0000","last_undegraded":"2026-02-24T07:50:35.837106+0000","last_fullsized":"2026-02-24T07:50:35.837106+0000","mapping_epoch":55,"log_start":"61'43960","ondisk_log_start":"61'43960","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"61'32990","last_scrub_stamp":"2026-02-24T07:50:11.227573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:50:11.227573+0000","objects_scrubbed":32460,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:51:29.781611+0000","scrub_duration":3701,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2404057088,"num_objects":36683,"num_object_clones":0,"num_object_copies":110049,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36683,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":43963,"num_write_kb":2580672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.6","version":"61'43821","reported_seq":44188,"reported_epoch":61,"state":"active+clean","last_fresh":"2026-02-24T07:50:35.743152+0000","last_change":"2026-02-24T07:50:14.386679+0000","last_active":"2026-02-24T07:50:35.743152+0000","last_peered":"2026-02-24T07:50:35.743152+0000","last_clean":"2026-02-24T07:50:35.743152+0000","last_became_active":"2026-02-24T07:48:57.901565+0000","last_became_peered":"2026-02-24T07:48:57.901565+0000","last_unstale":"2026-02-24T07:50:35.743152+0000","last_undegraded":"2026-02-24T07:50:35.743152+0000","last_fullsized":"2026-02-24T07:50:35.743152+0000","mapping_epoch":55,"log_start":"61'43818","ondisk_log_start":"61'43818","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"61'34244","last_scrub_stamp":"2026-02-24T07:50:14.386625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:50:14.386625+0000","objects_scrubbed":33505,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:51:16.739697+0000","scrub_duration":3790,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2407727104,"num_objects":36739,"num_object_clones":0,"num_object_copies":110217,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36739,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":43821,"num_write_kb":2577920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"61'44309","reported_seq":44689,"reported_epoch":61,"state":"active+clean","last_fresh":"2026-02-24T07:50:35.834920+0000","last_change":"2026-02-24T07:50:16.399238+0000","last_active":"2026-02-24T07:50:35.834920+0000","last_peered":"2026-02-24T07:50:35.834920+0000","last_clean":"2026-02-24T07:50:35.834920+0000","last_became_active":"2026-02-24T07:48:57.901824+0000","last_became_peered":"2026-02-24T07:48:57.901824+0000","last_unstale":"2026-02-24T07:50:35.834920+0000","last_undegraded":"2026-02-24T07:50:35.834920+0000","last_fullsized":"2026-02-24T07:50:35.834920+0000","mapping_epoch":55,"log_start":"61'44306","ondisk_log_start":"61'44306","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"61'35510","last_scrub_stamp":"2026-02-24T07:50:16.399194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:50:16.399194+0000","objects_scrubbed":34844,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:51:44.770701+0000","scrub_duration":2971,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2421358592,"num_objects":36947,"num_object_clones":0,"num_object_copies":110841,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36947,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44309,"num_write_kb":2600192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[1,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"61'43676","reported_seq":44027,"reported_epoch":61,"state":"active+clean","last_fresh":"2026-02-24T07:50:35.744411+0000","last_change":"2026-02-24T07:50:10.124042+0000","last_active":"2026-02-24T07:50:35.744411+0000","last_peered":"2026-02-24T07:50:35.744411+0000","last_clean":"2026-02-24T07:50:35.744411+0000","last_became_active":"2026-02-24T07:48:57.902224+0000","last_became_peered":"2026-02-24T07:48:57.902224+0000","last_unstale":"2026-02-24T07:50:35.744411+0000","last_undegraded":"2026-02-24T07:50:35.744411+0000","last_fullsized":"2026-02-24T07:50:35.744411+0000","mapping_epoch":55,"log_start":"61'43673","ondisk_log_start":"61'43673","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"61'32620","last_scrub_stamp":"2026-02-24T07:50:10.124005+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:50:10.124005+0000","objects_scrubbed":32078,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:51:12.877892+0000","scrub_duration":3559,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2405302272,"num_objects":36702,"num_object_clones":0,"num_object_copies":110106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36702,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":43676,"num_write_kb":2572096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"61'44185","reported_seq":44510,"reported_epoch":61,"state":"active+clean","last_fresh":"2026-02-24T07:50:35.835954+0000","last_change":"2026-02-24T07:50:01.398116+0000","last_active":"2026-02-24T07:50:35.835954+0000","last_peered":"2026-02-24T07:50:35.835954+0000","last_clean":"2026-02-24T07:50:35.835954+0000","last_became_active":"2026-02-24T07:48:57.902125+0000","last_became_peered":"2026-02-24T07:48:57.902125+0000","last_unstale":"2026-02-24T07:50:35.835954+0000","last_undegraded":"2026-02-24T07:50:35.835954+0000","last_fullsized":"2026-02-24T07:50:35.835954+0000","mapping_epoch":55,"log_start":"61'44182","ondisk_log_start":"61'44182","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"61'29707","last_scrub_stamp":"2026-02-24T07:50:01.398090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:50:01.398090+0000","objects_scrubbed":29171,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:51:10.566529+0000","scrub_duration":2880,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2425159680,"num_objects":37005,"num_object_clones":0,"num_object_copies":111015,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37005,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44185,"num_write_kb":2598080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1","version":"61'44239","reported_seq":44671,"reported_epoch":61,"state":"active+clean","last_fresh":"2026-02-24T07:50:35.729979+0000","last_change":"2026-02-24T07:50:29.889880+0000","last_active":"2026-02-24T07:50:35.729979+0000","last_peered":"2026-02-24T07:50:35.729979+0000","last_clean":"2026-02-24T07:50:35.729979+0000","last_became_active":"2026-02-24T07:48:57.903439+0000","last_became_peered":"2026-02-24T07:48:57.903439+0000","last_unstale":"2026-02-24T07:50:35.729979+0000","last_undegraded":"2026-02-24T07:50:35.729979+0000","last_fullsized":"2026-02-24T07:50:35.729979+0000","mapping_epoch":55,"log_start":"61'44236","ondisk_log_start":"61'44236","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"61'41105","last_scrub_stamp":"2026-02-24T07:50:29.889816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:50:29.889816+0000","objects_scrubbed":40127,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:51:50.788414+0000","scrub_duration":4409,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2436038656,"num_objects":37171,"num_object_clones":0,"num_object_copies":111513,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37171,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44239,"num_write_kb":2605120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.0","version":"61'44088","reported_seq":44520,"reported_epoch":61,"state":"active+clean","last_fresh":"2026-02-24T07:50:35.740174+0000","last_change":"2026-02-24T07:50:29.387323+0000","last_active":"2026-02-24T07:50:35.740174+0000","last_peered":"2026-02-24T07:50:35.740174+0000","last_clean":"2026-02-24T07:50:35.740174+0000","last_became_active":"2026-02-24T07:48:57.902565+0000","last_became_peered":"2026-02-24T07:48:57.902565+0000","last_unstale":"2026-02-24T07:50:35.740174+0000","last_undegraded":"2026-02-24T07:50:35.740174+0000","last_fullsized":"2026-02-24T07:50:35.740174+0000","mapping_epoch":55,"log_start":"61'44085","ondisk_log_start":"61'44085","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"61'40736","last_scrub_stamp":"2026-02-24T07:50:29.387206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:50:29.387206+0000","objects_scrubbed":39946,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:51:59.173732+0000","scrub_duration":3834,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2428370944,"num_objects":37054,"num_object_clones":0,"num_object_copies":111162,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37054,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44088,"num_write_kb":2596544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,3],"acting":[0,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.5","version":"61'44014","reported_seq":44349,"reported_epoch":61,"state":"active+clean","last_fresh":"2026-02-24T07:50:35.745214+0000","last_change":"2026-02-24T07:50:04.821914+0000","last_active":"2026-02-24T07:50:35.745214+0000","last_peered":"2026-02-24T07:50:35.745214+0000","last_clean":"2026-02-24T07:50:35.745214+0000","last_became_active":"2026-02-24T07:48:57.905127+0000","last_became_peered":"2026-02-24T07:48:57.905127+0000","last_unstale":"2026-02-24T07:50:35.745214+0000","last_undegraded":"2026-02-24T07:50:35.745214+0000","last_fullsized":"2026-02-24T07:50:35.745214+0000","mapping_epoch":55,"log_start":"61'44011","ondisk_log_start":"61'44011","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"61'30833","last_scrub_stamp":"2026-02-24T07:50:04.821861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:50:04.821861+0000","objects_scrubbed":30132,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:51:15.928207+0000","scrub_duration":3578,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2430074880,"num_objects":37080,"num_object_clones":0,"num_object_copies":111240,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37080,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44014,"num_write_kb":2595008,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"29'99","reported_seq":293,"reported_epoch":61,"state":"active+clean","last_fresh":"2026-02-24T07:50:30.401963+0000","last_change":"2026-02-24T07:50:30.401963+0000","last_active":"2026-02-24T07:50:30.401963+0000","last_peered":"2026-02-24T07:50:30.401963+0000","last_clean":"2026-02-24T07:50:30.401963+0000","last_became_active":"2026-02-24T07:42:54.186719+0000","last_became_peered":"2026-02-24T07:42:54.186719+0000","last_unstale":"2026-02-24T07:50:30.401963+0000","last_undegraded":"2026-02-24T07:50:30.401963+0000","last_fullsized":"2026-02-24T07:50:30.401963+0000","mapping_epoch":27,"log_start":"29'96","ondisk_log_start":"29'96","created":20,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'99","last_scrub_stamp":"2026-02-24T07:50:30.401929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:42:43.640776+0000","last_clean_scrub_stamp":"2026-02-24T07:50:30.401929+0000","objects_scrubbed":2,"log_size":3,"log_dups_size":96,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:51:52.990727+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":240,"num_write":135,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":16,"stat_sum":{"num_bytes":38703005720,"num_objects":590562,"num_object_clones":0,"num_object_copies":1771686,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":590562,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":704174,"num_write_kb":41431489,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":116120760320,"data_stored":116120748104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":48,"ondisk_log_size":48,"up":48,"acting":48,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":240,"num_write":135,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":948,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":3,"up_from":25,"seq":107374182495,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":30674892,"kb_used_data":28544504,"kb_used_omap":13,"kb_used_meta":2130290,"kb_avail":703328308,"statfs":{"total":751619276800,"available":720208187392,"internally_reserved":0,"allocated":29229572096,"data_stored":29229154540,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13743,"internal_metadata":2181417553},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1],"upper_bound":4},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":19,"seq":81604378721,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":34658444,"kb_used_data":33168632,"kb_used_omap":14,"kb_used_meta":1489777,"kb_avail":699344756,"statfs":{"total":751619276800,"available":716129030144,"internally_reserved":0,"allocated":33964679168,"data_stored":33964261612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":14721,"internal_metadata":1525532287},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542242,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":25492804,"kb_used_data":23670384,"kb_used_omap":15,"kb_used_meta":1822256,"kb_avail":708510396,"statfs":{"total":751619276800,"available":725514645504,"internally_reserved":0,"allocated":24238473216,"data_stored":24238063284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16125,"internal_metadata":1865990403},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705765,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":30143692,"kb_used_data":28427192,"kb_used_omap":11,"kb_used_meta":1716468,"kb_avail":703859508,"statfs":{"total":751619276800,"available":720752136192,"internally_reserved":0,"allocated":29109444608,"data_stored":29109027052,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12266,"internal_metadata":1757663254},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[4,1],"upper_bound":4},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":948,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":29036187648,"data_stored":29036183576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":24146739200,"data_stored":24146739200,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":33869991936,"data_stored":33869987864,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":29067841536,"data_stored":29067837464,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-24T07:50:40.152 INFO:tasks.cephadm.ceph_manager.ceph:increase pool size by 10 2026-02-24T07:50:40.153 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph osd pool set unique_pool_2 pg_num 66 2026-02-24T07:50:40.297 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:50:40.425 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:50:40.423+0000 7f1017615640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:50:40.425 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:50:40.423+0000 7f1017615640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:50:40.525 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2026-02-24T07:50:40.573 INFO:teuthology.orchestra.run.trial035.stderr:Error EPERM: crimson_allow_pg_split is false; pg_num increase denied for crimson pool 2026-02-24T07:50:40.616 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-24T07:50:40.665 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:50:40.778 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:50:40.777+0000 7fc6eff44640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:50:40.778 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:50:40.778+0000 7fc6eff44640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:50:40.797 INFO:teuthology.orchestra.run.trial035.stdout:osd.0: {} 2026-02-24T07:50:40.798 INFO:teuthology.orchestra.run.trial035.stderr:osd.0: osd_enable_op_tracker = '' 2026-02-24T07:50:40.811 INFO:teuthology.orchestra.run.trial035.stdout:osd.1: {} 2026-02-24T07:50:40.812 INFO:teuthology.orchestra.run.trial035.stderr:osd.1: osd_enable_op_tracker = '' 2026-02-24T07:50:40.827 INFO:teuthology.orchestra.run.trial035.stdout:osd.2: {} 2026-02-24T07:50:40.827 INFO:teuthology.orchestra.run.trial035.stderr:osd.2: osd_enable_op_tracker = '' 2026-02-24T07:50:40.840 INFO:teuthology.orchestra.run.trial035.stdout:osd.3: {} 2026-02-24T07:50:40.840 INFO:teuthology.orchestra.run.trial035.stderr:osd.3: osd_enable_op_tracker = '' 2026-02-24T07:50:41.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:50:40 trial035 ceph-mon[39284]: pgmap v265: 17 pgs: 17 active+clean; 36 GiB data, 115 GiB used, 2.6 TiB / 2.7 TiB avail; 85 B/s rd, 101 MiB/s wr, 6.34k op/s 2026-02-24T07:50:41.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:50:40 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/1956267846' entity='client.admin' cmd={"prefix": "osd pool set", "pool": "unique_pool_2", "var": "pg_num", "val": "66"} : dispatch 2026-02-24T07:50:41.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:50:40 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/4001290129' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2026-02-24T07:50:42.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:50:41 trial035 ceph-mon[39284]: from='client.15019 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2026-02-24T07:50:42.881 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2026-02-24T07:50:43.025 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:50:43.142 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:50:42 trial035 ceph-mon[39284]: pgmap v266: 17 pgs: 17 active+clean; 33 GiB data, 106 GiB used, 2.6 TiB / 2.7 TiB avail; 85 B/s rd, 101 MiB/s wr, 10.19k op/s 2026-02-24T07:50:43.144 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:50:43.143+0000 7f8deb178640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:50:43.144 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:50:43.143+0000 7f8deb178640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:50:43.161 INFO:teuthology.orchestra.run.trial035.stdout:osd.0: {} 2026-02-24T07:50:43.162 INFO:teuthology.orchestra.run.trial035.stderr:osd.0: osd_enable_op_tracker = '' 2026-02-24T07:50:43.174 INFO:teuthology.orchestra.run.trial035.stdout:osd.1: {} 2026-02-24T07:50:43.175 INFO:teuthology.orchestra.run.trial035.stderr:osd.1: osd_enable_op_tracker = '' 2026-02-24T07:50:43.190 INFO:teuthology.orchestra.run.trial035.stdout:osd.2: {} 2026-02-24T07:50:43.190 INFO:teuthology.orchestra.run.trial035.stderr:osd.2: osd_enable_op_tracker = '' 2026-02-24T07:50:43.204 INFO:teuthology.orchestra.run.trial035.stdout:osd.3: {} 2026-02-24T07:50:43.204 INFO:teuthology.orchestra.run.trial035.stderr:osd.3: osd_enable_op_tracker = '' 2026-02-24T07:50:44.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:50:43 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/516459763' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2026-02-24T07:50:45.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:50:44 trial035 ceph-mon[39284]: pgmap v267: 17 pgs: 17 active+clean; 33 GiB data, 106 GiB used, 2.6 TiB / 2.7 TiB avail; 0 B/s wr, 7.42k op/s 2026-02-24T07:50:45.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:50:44 trial035 ceph-mon[39284]: from='mgr.14284 10.20.193.35:0/3681373222' entity='mgr.x' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2026-02-24T07:50:45.247 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2026-02-24T07:50:45.385 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:50:45.512 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:50:45.510+0000 7fb823334640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:50:45.512 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:50:45.511+0000 7fb823334640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:50:45.526 INFO:teuthology.orchestra.run.trial035.stdout:osd.0: {} 2026-02-24T07:50:45.527 INFO:teuthology.orchestra.run.trial035.stderr:osd.0: osd_enable_op_tracker = '' 2026-02-24T07:50:45.543 INFO:teuthology.orchestra.run.trial035.stdout:osd.1: {} 2026-02-24T07:50:45.544 INFO:teuthology.orchestra.run.trial035.stderr:osd.1: osd_enable_op_tracker = '' 2026-02-24T07:50:45.557 INFO:teuthology.orchestra.run.trial035.stdout:osd.2: {} 2026-02-24T07:50:45.557 INFO:teuthology.orchestra.run.trial035.stderr:osd.2: osd_enable_op_tracker = '' 2026-02-24T07:50:45.570 INFO:teuthology.orchestra.run.trial035.stdout:osd.3: {} 2026-02-24T07:50:45.570 INFO:teuthology.orchestra.run.trial035.stderr:osd.3: osd_enable_op_tracker = '' 2026-02-24T07:50:45.618 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2026-02-24T07:50:45.618 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2026-02-24T07:50:45.618 INFO:tasks.thrashosds.thrasher:Growing pool .mgr 2026-02-24T07:50:45.619 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph pg dump --format=json 2026-02-24T07:50:45.761 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:50:45.888 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:50:45.886+0000 7feb0eecf640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:50:45.888 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:50:45.887+0000 7feb0eecf640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:50:46.023 INFO:teuthology.orchestra.run.trial035.stdout: 2026-02-24T07:50:46.024 INFO:teuthology.orchestra.run.trial035.stderr:dumped all 2026-02-24T07:50:46.043 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:50:45 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/431640877' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2026-02-24T07:50:46.067 INFO:teuthology.orchestra.run.trial035.stdout:{"pg_ready":true,"pg_map":{"version":268,"stamp":"2026-02-24T07:50:45.720271+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":35675636280,"num_objects":544361,"num_object_clones":0,"num_object_copies":1633083,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":544361,"num_whiteouts":0,"num_read":137,"num_read_kb":241,"num_write":750512,"num_write_kb":41433639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":51,"ondisk_log_size":51,"up":51,"acting":51,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":51,"num_osds":4,"num_per_pool_osds":4,"num_per_pool_omap_osds":4,"kb":2936012800,"kb_used":111558312,"kb_used_data":104976728,"kb_used_omap":55,"kb_used_meta":6581320,"kb_avail":2824454488,"statfs":{"total":3006477107200,"available":2892241395712,"internally_reserved":0,"allocated":107496169472,"data_stored":107494506872,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":56855,"internal_metadata":6739272169},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[7,1],"upper_bound":4},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-5835784192,"num_objects":-89047,"num_object_clones":0,"num_object_copies":-267141,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-89047,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":89047,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001614"},"pg_stats":[{"pgid":"4.f","version":"61'47169","reported_seq":47502,"reported_epoch":61,"state":"active+clean","last_fresh":"2026-02-24T07:50:40.833111+0000","last_change":"2026-02-24T07:50:03.071459+0000","last_active":"2026-02-24T07:50:40.833111+0000","last_peered":"2026-02-24T07:50:40.833111+0000","last_clean":"2026-02-24T07:50:40.833111+0000","last_became_active":"2026-02-24T07:48:57.902313+0000","last_became_peered":"2026-02-24T07:48:57.902313+0000","last_unstale":"2026-02-24T07:50:40.833111+0000","last_undegraded":"2026-02-24T07:50:40.833111+0000","last_fullsized":"2026-02-24T07:50:40.833111+0000","mapping_epoch":55,"log_start":"61'47166","ondisk_log_start":"61'47166","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"61'30419","last_scrub_stamp":"2026-02-24T07:50:03.071380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:50:03.071380+0000","objects_scrubbed":29975,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:51:16.123453+0000","scrub_duration":2568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2239430656,"num_objects":34171,"num_object_clones":0,"num_object_copies":102513,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34171,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":47169,"num_write_kb":2602880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"61'47044","reported_seq":47444,"reported_epoch":61,"state":"active+clean","last_fresh":"2026-02-24T07:50:40.731670+0000","last_change":"2026-02-24T07:50:21.065676+0000","last_active":"2026-02-24T07:50:40.731670+0000","last_peered":"2026-02-24T07:50:40.731670+0000","last_clean":"2026-02-24T07:50:40.731670+0000","last_became_active":"2026-02-24T07:48:57.903584+0000","last_became_peered":"2026-02-24T07:48:57.903584+0000","last_unstale":"2026-02-24T07:50:40.731670+0000","last_undegraded":"2026-02-24T07:50:40.731670+0000","last_fullsized":"2026-02-24T07:50:40.731670+0000","mapping_epoch":55,"log_start":"61'47041","ondisk_log_start":"61'47041","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"61'37487","last_scrub_stamp":"2026-02-24T07:50:21.065638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:50:21.065638+0000","objects_scrubbed":36748,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:51:31.991874+0000","scrub_duration":3643,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2237005824,"num_objects":34134,"num_object_clones":0,"num_object_copies":102402,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34134,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":47044,"num_write_kb":2597696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.d","version":"61'46826","reported_seq":47169,"reported_epoch":61,"state":"active+clean","last_fresh":"2026-02-24T07:50:40.746805+0000","last_change":"2026-02-24T07:50:06.701973+0000","last_active":"2026-02-24T07:50:40.746805+0000","last_peered":"2026-02-24T07:50:40.746805+0000","last_clean":"2026-02-24T07:50:40.746805+0000","last_became_active":"2026-02-24T07:48:57.905031+0000","last_became_peered":"2026-02-24T07:48:57.905031+0000","last_unstale":"2026-02-24T07:50:40.746805+0000","last_undegraded":"2026-02-24T07:50:40.746805+0000","last_fullsized":"2026-02-24T07:50:40.746805+0000","mapping_epoch":55,"log_start":"61'46823","ondisk_log_start":"61'46823","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"61'31648","last_scrub_stamp":"2026-02-24T07:50:06.701883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:50:06.701883+0000","objects_scrubbed":30990,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:51:19.224966+0000","scrub_duration":3493,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2225930264,"num_objects":33966,"num_object_clones":0,"num_object_copies":101898,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33966,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":46826,"num_write_kb":2585281,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"61'46583","reported_seq":46994,"reported_epoch":61,"state":"active+clean","last_fresh":"2026-02-24T07:50:40.744469+0000","last_change":"2026-02-24T07:50:24.798674+0000","last_active":"2026-02-24T07:50:40.744469+0000","last_peered":"2026-02-24T07:50:40.744469+0000","last_clean":"2026-02-24T07:50:40.744469+0000","last_became_active":"2026-02-24T07:48:57.904683+0000","last_became_peered":"2026-02-24T07:48:57.904683+0000","last_unstale":"2026-02-24T07:50:40.744469+0000","last_undegraded":"2026-02-24T07:50:40.744469+0000","last_fullsized":"2026-02-24T07:50:40.744469+0000","mapping_epoch":55,"log_start":"61'46580","ondisk_log_start":"61'46580","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"61'38696","last_scrub_stamp":"2026-02-24T07:50:24.798629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:50:24.798629+0000","objects_scrubbed":37979,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:51:26.521338+0000","scrub_duration":3612,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2222260224,"num_objects":33909,"num_object_clones":0,"num_object_copies":101727,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33909,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":46583,"num_write_kb":2575744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.b","version":"61'46779","reported_seq":47199,"reported_epoch":61,"state":"active+clean","last_fresh":"2026-02-24T07:50:40.745282+0000","last_change":"2026-02-24T07:50:27.267008+0000","last_active":"2026-02-24T07:50:40.745282+0000","last_peered":"2026-02-24T07:50:40.745282+0000","last_clean":"2026-02-24T07:50:40.745282+0000","last_became_active":"2026-02-24T07:48:57.901700+0000","last_became_peered":"2026-02-24T07:48:57.901700+0000","last_unstale":"2026-02-24T07:50:40.745282+0000","last_undegraded":"2026-02-24T07:50:40.745282+0000","last_fullsized":"2026-02-24T07:50:40.745282+0000","mapping_epoch":55,"log_start":"61'46776","ondisk_log_start":"61'46776","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"61'39639","last_scrub_stamp":"2026-02-24T07:50:27.266943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:50:27.266943+0000","objects_scrubbed":38982,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:51:34.346751+0000","scrub_duration":3744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2220032000,"num_objects":33875,"num_object_clones":0,"num_object_copies":101625,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33875,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":46779,"num_write_kb":2580928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.a","version":"61'46899","reported_seq":47323,"reported_epoch":61,"state":"active+clean","last_fresh":"2026-02-24T07:50:40.732623+0000","last_change":"2026-02-24T07:50:33.043786+0000","last_active":"2026-02-24T07:50:40.732623+0000","last_peered":"2026-02-24T07:50:40.732623+0000","last_clean":"2026-02-24T07:50:40.732623+0000","last_became_active":"2026-02-24T07:48:57.903228+0000","last_became_peered":"2026-02-24T07:48:57.903228+0000","last_unstale":"2026-02-24T07:50:40.732623+0000","last_undegraded":"2026-02-24T07:50:40.732623+0000","last_fullsized":"2026-02-24T07:50:40.732623+0000","mapping_epoch":55,"log_start":"61'46896","ondisk_log_start":"61'46896","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"61'42539","last_scrub_stamp":"2026-02-24T07:50:33.043726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:50:33.043726+0000","objects_scrubbed":39246,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:51:51.239665+0000","scrub_duration":3547,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2227372032,"num_objects":33987,"num_object_clones":0,"num_object_copies":101961,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33987,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":46899,"num_write_kb":2588352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"61'46835","reported_seq":47200,"reported_epoch":61,"state":"active+clean","last_fresh":"2026-02-24T07:50:40.731508+0000","last_change":"2026-02-24T07:50:13.165492+0000","last_active":"2026-02-24T07:50:40.731508+0000","last_peered":"2026-02-24T07:50:40.731508+0000","last_clean":"2026-02-24T07:50:40.731508+0000","last_became_active":"2026-02-24T07:48:57.901730+0000","last_became_peered":"2026-02-24T07:48:57.901730+0000","last_unstale":"2026-02-24T07:50:40.731508+0000","last_undegraded":"2026-02-24T07:50:40.731508+0000","last_fullsized":"2026-02-24T07:50:40.731508+0000","mapping_epoch":55,"log_start":"61'46832","ondisk_log_start":"61'46832","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"61'33943","last_scrub_stamp":"2026-02-24T07:50:13.165459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:50:13.165459+0000","objects_scrubbed":33253,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:51:33.820708+0000","scrub_duration":3776,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2230517760,"num_objects":34035,"num_object_clones":0,"num_object_copies":102105,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34035,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":46835,"num_write_kb":2587840,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"61'46855","reported_seq":47183,"reported_epoch":61,"state":"active+clean","last_fresh":"2026-02-24T07:50:40.743089+0000","last_change":"2026-02-24T07:50:03.153499+0000","last_active":"2026-02-24T07:50:40.743089+0000","last_peered":"2026-02-24T07:50:40.743089+0000","last_clean":"2026-02-24T07:50:40.743089+0000","last_became_active":"2026-02-24T07:48:57.904376+0000","last_became_peered":"2026-02-24T07:48:57.904376+0000","last_unstale":"2026-02-24T07:50:40.743089+0000","last_undegraded":"2026-02-24T07:50:40.743089+0000","last_fullsized":"2026-02-24T07:50:40.743089+0000","mapping_epoch":55,"log_start":"61'46852","ondisk_log_start":"61'46852","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"61'30163","last_scrub_stamp":"2026-02-24T07:50:03.153454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:50:03.153454+0000","objects_scrubbed":29681,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:51:10.266136+0000","scrub_duration":2916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2227765248,"num_objects":33993,"num_object_clones":0,"num_object_copies":101979,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33993,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":46855,"num_write_kb":2587136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.7","version":"61'46897","reported_seq":47253,"reported_epoch":61,"state":"active+clean","last_fresh":"2026-02-24T07:50:40.836334+0000","last_change":"2026-02-24T07:50:11.227608+0000","last_active":"2026-02-24T07:50:40.836334+0000","last_peered":"2026-02-24T07:50:40.836334+0000","last_clean":"2026-02-24T07:50:40.836334+0000","last_became_active":"2026-02-24T07:48:57.902484+0000","last_became_peered":"2026-02-24T07:48:57.902484+0000","last_unstale":"2026-02-24T07:50:40.836334+0000","last_undegraded":"2026-02-24T07:50:40.836334+0000","last_fullsized":"2026-02-24T07:50:40.836334+0000","mapping_epoch":55,"log_start":"61'46894","ondisk_log_start":"61'46894","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"61'32990","last_scrub_stamp":"2026-02-24T07:50:11.227573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:50:11.227573+0000","objects_scrubbed":32460,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:51:29.781611+0000","scrub_duration":3701,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2211774464,"num_objects":33749,"num_object_clones":0,"num_object_copies":101247,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33749,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":46897,"num_write_kb":2580672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.6","version":"61'46670","reported_seq":47037,"reported_epoch":61,"state":"active+clean","last_fresh":"2026-02-24T07:50:40.745218+0000","last_change":"2026-02-24T07:50:14.386679+0000","last_active":"2026-02-24T07:50:40.745218+0000","last_peered":"2026-02-24T07:50:40.745218+0000","last_clean":"2026-02-24T07:50:40.745218+0000","last_became_active":"2026-02-24T07:48:57.901565+0000","last_became_peered":"2026-02-24T07:48:57.901565+0000","last_unstale":"2026-02-24T07:50:40.745218+0000","last_undegraded":"2026-02-24T07:50:40.745218+0000","last_fullsized":"2026-02-24T07:50:40.745218+0000","mapping_epoch":55,"log_start":"61'46667","ondisk_log_start":"61'46667","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"61'34244","last_scrub_stamp":"2026-02-24T07:50:14.386625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:50:14.386625+0000","objects_scrubbed":33505,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:51:16.739697+0000","scrub_duration":3790,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2221015040,"num_objects":33890,"num_object_clones":0,"num_object_copies":101670,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33890,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":46670,"num_write_kb":2577920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"61'47204","reported_seq":47585,"reported_epoch":61,"state":"active+clean","last_fresh":"2026-02-24T07:50:40.836403+0000","last_change":"2026-02-24T07:50:16.399238+0000","last_active":"2026-02-24T07:50:40.836403+0000","last_peered":"2026-02-24T07:50:40.836403+0000","last_clean":"2026-02-24T07:50:40.836403+0000","last_became_active":"2026-02-24T07:48:57.901824+0000","last_became_peered":"2026-02-24T07:48:57.901824+0000","last_unstale":"2026-02-24T07:50:40.836403+0000","last_undegraded":"2026-02-24T07:50:40.836403+0000","last_fullsized":"2026-02-24T07:50:40.836403+0000","mapping_epoch":55,"log_start":"61'47201","ondisk_log_start":"61'47201","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"61'35510","last_scrub_stamp":"2026-02-24T07:50:16.399194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:50:16.399194+0000","objects_scrubbed":34844,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:51:44.770701+0000","scrub_duration":2971,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2231631872,"num_objects":34052,"num_object_clones":0,"num_object_copies":102156,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34052,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":47204,"num_write_kb":2600192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[1,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"61'46554","reported_seq":46907,"reported_epoch":61,"state":"active+clean","last_fresh":"2026-02-24T07:50:40.739929+0000","last_change":"2026-02-24T07:50:10.124042+0000","last_active":"2026-02-24T07:50:40.739929+0000","last_peered":"2026-02-24T07:50:40.739929+0000","last_clean":"2026-02-24T07:50:40.739929+0000","last_became_active":"2026-02-24T07:48:57.902224+0000","last_became_peered":"2026-02-24T07:48:57.902224+0000","last_unstale":"2026-02-24T07:50:40.739929+0000","last_undegraded":"2026-02-24T07:50:40.739929+0000","last_fullsized":"2026-02-24T07:50:40.739929+0000","mapping_epoch":55,"log_start":"61'46551","ondisk_log_start":"61'46551","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"61'32620","last_scrub_stamp":"2026-02-24T07:50:10.124005+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:50:10.124005+0000","objects_scrubbed":32078,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:51:12.877892+0000","scrub_duration":3559,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2216689664,"num_objects":33824,"num_object_clones":0,"num_object_copies":101472,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33824,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":46554,"num_write_kb":2572096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"61'47137","reported_seq":47462,"reported_epoch":61,"state":"active+clean","last_fresh":"2026-02-24T07:50:40.833751+0000","last_change":"2026-02-24T07:50:01.398116+0000","last_active":"2026-02-24T07:50:40.833751+0000","last_peered":"2026-02-24T07:50:40.833751+0000","last_clean":"2026-02-24T07:50:40.833751+0000","last_became_active":"2026-02-24T07:48:57.902125+0000","last_became_peered":"2026-02-24T07:48:57.902125+0000","last_unstale":"2026-02-24T07:50:40.833751+0000","last_undegraded":"2026-02-24T07:50:40.833751+0000","last_fullsized":"2026-02-24T07:50:40.833751+0000","mapping_epoch":55,"log_start":"61'47134","ondisk_log_start":"61'47134","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"61'29707","last_scrub_stamp":"2026-02-24T07:50:01.398090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:50:01.398090+0000","objects_scrubbed":29171,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:51:10.566529+0000","scrub_duration":2880,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2231697408,"num_objects":34053,"num_object_clones":0,"num_object_copies":102159,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34053,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":47137,"num_write_kb":2598080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1","version":"61'47142","reported_seq":47574,"reported_epoch":61,"state":"active+clean","last_fresh":"2026-02-24T07:50:40.731123+0000","last_change":"2026-02-24T07:50:29.889880+0000","last_active":"2026-02-24T07:50:40.731123+0000","last_peered":"2026-02-24T07:50:40.731123+0000","last_clean":"2026-02-24T07:50:40.731123+0000","last_became_active":"2026-02-24T07:48:57.903439+0000","last_became_peered":"2026-02-24T07:48:57.903439+0000","last_unstale":"2026-02-24T07:50:40.731123+0000","last_undegraded":"2026-02-24T07:50:40.731123+0000","last_fullsized":"2026-02-24T07:50:40.731123+0000","mapping_epoch":55,"log_start":"61'47139","ondisk_log_start":"61'47139","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"61'41105","last_scrub_stamp":"2026-02-24T07:50:29.889816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:50:29.889816+0000","objects_scrubbed":40127,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:51:50.788414+0000","scrub_duration":4409,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2245787648,"num_objects":34268,"num_object_clones":0,"num_object_copies":102804,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34268,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":47142,"num_write_kb":2605120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.0","version":"61'46960","reported_seq":47392,"reported_epoch":61,"state":"active+clean","last_fresh":"2026-02-24T07:50:40.744780+0000","last_change":"2026-02-24T07:50:29.387323+0000","last_active":"2026-02-24T07:50:40.744780+0000","last_peered":"2026-02-24T07:50:40.744780+0000","last_clean":"2026-02-24T07:50:40.744780+0000","last_became_active":"2026-02-24T07:48:57.902565+0000","last_became_peered":"2026-02-24T07:48:57.902565+0000","last_unstale":"2026-02-24T07:50:40.744780+0000","last_undegraded":"2026-02-24T07:50:40.744780+0000","last_fullsized":"2026-02-24T07:50:40.744780+0000","mapping_epoch":55,"log_start":"61'46957","ondisk_log_start":"61'46957","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"61'40736","last_scrub_stamp":"2026-02-24T07:50:29.387206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:50:29.387206+0000","objects_scrubbed":39946,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:51:59.173732+0000","scrub_duration":3834,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2240151552,"num_objects":34182,"num_object_clones":0,"num_object_copies":102546,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34182,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":46960,"num_write_kb":2596544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,3],"acting":[0,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.5","version":"61'46823","reported_seq":47158,"reported_epoch":61,"state":"active+clean","last_fresh":"2026-02-24T07:50:40.746083+0000","last_change":"2026-02-24T07:50:04.821914+0000","last_active":"2026-02-24T07:50:40.746083+0000","last_peered":"2026-02-24T07:50:40.746083+0000","last_clean":"2026-02-24T07:50:40.746083+0000","last_became_active":"2026-02-24T07:48:57.905127+0000","last_became_peered":"2026-02-24T07:48:57.905127+0000","last_unstale":"2026-02-24T07:50:40.746083+0000","last_undegraded":"2026-02-24T07:50:40.746083+0000","last_fullsized":"2026-02-24T07:50:40.746083+0000","mapping_epoch":55,"log_start":"61'46820","ondisk_log_start":"61'46820","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"61'30833","last_scrub_stamp":"2026-02-24T07:50:04.821861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:50:04.821861+0000","objects_scrubbed":30132,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:51:15.928207+0000","scrub_duration":3578,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2245984256,"num_objects":34271,"num_object_clones":0,"num_object_copies":102813,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34271,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":46823,"num_write_kb":2595008,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"29'99","reported_seq":293,"reported_epoch":61,"state":"active+clean","last_fresh":"2026-02-24T07:50:30.401963+0000","last_change":"2026-02-24T07:50:30.401963+0000","last_active":"2026-02-24T07:50:30.401963+0000","last_peered":"2026-02-24T07:50:30.401963+0000","last_clean":"2026-02-24T07:50:30.401963+0000","last_became_active":"2026-02-24T07:42:54.186719+0000","last_became_peered":"2026-02-24T07:42:54.186719+0000","last_unstale":"2026-02-24T07:50:30.401963+0000","last_undegraded":"2026-02-24T07:50:30.401963+0000","last_fullsized":"2026-02-24T07:50:30.401963+0000","mapping_epoch":27,"log_start":"29'96","ondisk_log_start":"29'96","created":20,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'99","last_scrub_stamp":"2026-02-24T07:50:30.401929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:42:43.640776+0000","last_clean_scrub_stamp":"2026-02-24T07:50:30.401929+0000","objects_scrubbed":2,"log_size":3,"log_dups_size":96,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:51:52.990727+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":240,"num_write":135,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":16,"stat_sum":{"num_bytes":35675045912,"num_objects":544359,"num_object_clones":0,"num_object_copies":1633077,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":544359,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":750377,"num_write_kb":41431489,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":107033145344,"data_stored":107033133128,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":48,"ondisk_log_size":48,"up":48,"acting":48,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":240,"num_write":135,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":948,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":3,"up_from":25,"seq":107374182496,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":27723276,"kb_used_data":26368952,"kb_used_omap":13,"kb_used_meta":1354290,"kb_avail":706279924,"statfs":{"total":751619276800,"available":723230642176,"internally_reserved":0,"allocated":27001806848,"data_stored":27001389292,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13743,"internal_metadata":1386793553},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":19,"seq":81604378722,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":32112204,"kb_used_data":30550136,"kb_used_omap":14,"kb_used_meta":1561969,"kb_avail":701890996,"statfs":{"total":751619276800,"available":718736379904,"internally_reserved":0,"allocated":31283339264,"data_stored":31282921708,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":14721,"internal_metadata":1599456895},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542243,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":23720580,"kb_used_data":21844080,"kb_used_omap":15,"kb_used_meta":1876400,"kb_avail":710282620,"statfs":{"total":751619276800,"available":727329402880,"internally_reserved":0,"allocated":22368337920,"data_stored":22367927988,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16125,"internal_metadata":1921433859},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705766,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":28002252,"kb_used_data":26213560,"kb_used_omap":11,"kb_used_meta":1788660,"kb_avail":706000948,"statfs":{"total":751619276800,"available":722944970752,"internally_reserved":0,"allocated":26842685440,"data_stored":26842267884,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12266,"internal_metadata":1831587862},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[4,1],"upper_bound":4},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":948,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":26757828608,"data_stored":26757824536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":22250913792,"data_stored":22250913792,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":31224696832,"data_stored":31224692760,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":26799706112,"data_stored":26799702040,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-24T07:50:46.069 INFO:tasks.cephadm.ceph_manager.ceph:increase pool size by 10 2026-02-24T07:50:46.069 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph osd pool set .mgr pg_num 261 2026-02-24T07:50:46.210 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:50:46.333 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:50:46.332+0000 7f56058d2640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:50:46.333 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:50:46.332+0000 7f56058d2640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:50:46.472 INFO:teuthology.orchestra.run.trial035.stderr:Error EPERM: crimson_allow_pg_split is false; pg_num increase denied for crimson pool 2026-02-24T07:50:46.510 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-24T07:50:47.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:50:46 trial035 ceph-mon[39284]: pgmap v268: 17 pgs: 17 active+clean; 33 GiB data, 106 GiB used, 2.6 TiB / 2.7 TiB avail; 0 B/s wr, 7.42k op/s 2026-02-24T07:50:47.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:50:46 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/3955960932' entity='client.admin' cmd={"prefix": "osd pool set", "pool": ".mgr", "var": "pg_num", "val": "261"} : dispatch 2026-02-24T07:50:47.205 INFO:journalctl@ceph.mgr.x.trial035.stdout:Feb 24 07:50:46 trial035 ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-mgr-x[39621]: ::ffff:10.20.193.91 - - [24/Feb/2026:07:50:46] "GET /metrics HTTP/1.1" 200 4230 "" "Prometheus/3.6.0" 2026-02-24T07:50:47.356 INFO:journalctl@ceph.prometheus.a.trial091.stdout:Feb 24 07:50:46 trial091 ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-prometheus-a[66007]: time=2026-02-24T07:50:46.973Z level=ERROR source=scrape.go:1630 msg="Failed to determine correct type of scrape target." component="scrape manager" scrape_pool=ceph target=http://trial035:9283/metrics content_type="text/html;charset=utf-8" fallback_media_type="" err="received unsupported Content-Type \"text/html;charset=utf-8\" and no fallback_scrape_protocol specified for target" 2026-02-24T07:50:47.637 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2026-02-24T07:50:47.776 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:50:47.898 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:50:47.897+0000 7f955c2ae640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:50:47.898 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:50:47.897+0000 7f955c2ae640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:50:47.914 INFO:teuthology.orchestra.run.trial035.stdout:osd.0: {} 2026-02-24T07:50:47.914 INFO:teuthology.orchestra.run.trial035.stderr:osd.0: osd_enable_op_tracker = '' 2026-02-24T07:50:47.915 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:50:47 trial035 ceph-mon[39284]: from='client.15029 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2026-02-24T07:50:47.929 INFO:teuthology.orchestra.run.trial035.stdout:osd.1: {} 2026-02-24T07:50:47.930 INFO:teuthology.orchestra.run.trial035.stderr:osd.1: osd_enable_op_tracker = '' 2026-02-24T07:50:47.946 INFO:teuthology.orchestra.run.trial035.stdout:osd.2: {} 2026-02-24T07:50:47.946 INFO:teuthology.orchestra.run.trial035.stderr:osd.2: osd_enable_op_tracker = '' 2026-02-24T07:50:47.958 INFO:teuthology.orchestra.run.trial035.stdout:osd.3: {} 2026-02-24T07:50:47.958 INFO:teuthology.orchestra.run.trial035.stderr:osd.3: osd_enable_op_tracker = '' 2026-02-24T07:50:49.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:50:48 trial035 ceph-mon[39284]: pgmap v269: 17 pgs: 17 active+clean; 30 GiB data, 98 GiB used, 2.6 TiB / 2.7 TiB avail; 0 B/s wr, 11.39k op/s 2026-02-24T07:50:49.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:50:48 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/1506109540' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2026-02-24T07:50:50.000 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2026-02-24T07:50:50.143 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:50:50.630 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:50:50.611+0000 7f9b1b2fe640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:50:50.630 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:50:50.611+0000 7f9b1b2fe640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:50:50.643 INFO:teuthology.orchestra.run.trial035.stdout:osd.0: {} 2026-02-24T07:50:50.644 INFO:teuthology.orchestra.run.trial035.stderr:osd.0: osd_enable_op_tracker = '' 2026-02-24T07:50:50.658 INFO:teuthology.orchestra.run.trial035.stdout:osd.1: {} 2026-02-24T07:50:50.658 INFO:teuthology.orchestra.run.trial035.stderr:osd.1: osd_enable_op_tracker = '' 2026-02-24T07:50:50.670 INFO:teuthology.orchestra.run.trial035.stdout:osd.2: {} 2026-02-24T07:50:50.670 INFO:teuthology.orchestra.run.trial035.stderr:osd.2: osd_enable_op_tracker = '' 2026-02-24T07:50:50.684 INFO:teuthology.orchestra.run.trial035.stdout:osd.3: {} 2026-02-24T07:50:50.684 INFO:teuthology.orchestra.run.trial035.stderr:osd.3: osd_enable_op_tracker = '' 2026-02-24T07:50:51.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:50:50 trial035 ceph-mon[39284]: pgmap v270: 17 pgs: 17 active+clean; 30 GiB data, 98 GiB used, 2.6 TiB / 2.7 TiB avail; 0 B/s wr, 7.82k op/s 2026-02-24T07:50:51.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:50:50 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/3525479835' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2026-02-24T07:50:51.512 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2026-02-24T07:50:51.512 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph osd reweight 0 1 2026-02-24T07:50:51.654 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:50:52.109 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:50:52.064+0000 7ff70c29d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:50:52.109 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:50:52.064+0000 7ff70c29d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:50:52.885 INFO:teuthology.orchestra.run.trial035.stderr:reweighted osd.0 to 1 (10000) 2026-02-24T07:50:52.929 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph osd reweight 1 1 2026-02-24T07:50:53.075 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:50:53.136 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:50:52 trial035 ceph-mon[39284]: pgmap v271: 17 pgs: 17 active+clean; 28 GiB data, 90 GiB used, 2.6 TiB / 2.7 TiB avail; 0 B/s wr, 11.62k op/s 2026-02-24T07:50:53.137 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:50:52 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/3693330441' entity='client.admin' cmd={"prefix": "osd reweight", "id": 0, "weight": 1.0} : dispatch 2026-02-24T07:50:53.204 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:50:53.202+0000 7f947bc03640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:50:53.204 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:50:53.202+0000 7f947bc03640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:50:53.531 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2026-02-24T07:50:53.672 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:50:53.792 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:50:53.790+0000 7f069c337640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:50:53.792 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:50:53.790+0000 7f069c337640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:50:53.806 INFO:teuthology.orchestra.run.trial035.stdout:osd.0: {} 2026-02-24T07:50:53.807 INFO:teuthology.orchestra.run.trial035.stderr:osd.0: osd_enable_op_tracker = '' 2026-02-24T07:50:53.820 INFO:teuthology.orchestra.run.trial035.stdout:osd.1: {} 2026-02-24T07:50:53.820 INFO:teuthology.orchestra.run.trial035.stderr:osd.1: osd_enable_op_tracker = '' 2026-02-24T07:50:53.836 INFO:teuthology.orchestra.run.trial035.stdout:osd.2: {} 2026-02-24T07:50:53.837 INFO:teuthology.orchestra.run.trial035.stderr:osd.2: osd_enable_op_tracker = '' 2026-02-24T07:50:53.849 INFO:teuthology.orchestra.run.trial035.stdout:osd.3: {} 2026-02-24T07:50:53.849 INFO:teuthology.orchestra.run.trial035.stderr:osd.3: osd_enable_op_tracker = '' 2026-02-24T07:50:53.889 INFO:teuthology.orchestra.run.trial035.stderr:reweighted osd.1 to 1 (10000) 2026-02-24T07:50:53.931 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph osd reweight 2 1 2026-02-24T07:50:54.067 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:50:54.094 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:50:53 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/3693330441' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 0, "weight": 1.0}]': finished 2026-02-24T07:50:54.094 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:50:53 trial035 ceph-mon[39284]: osdmap e62: 4 total, 4 up, 4 in 2026-02-24T07:50:54.095 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:50:53 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/807226715' entity='client.admin' cmd={"prefix": "osd reweight", "id": 1, "weight": 1.0} : dispatch 2026-02-24T07:50:54.095 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:50:53 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/2671204656' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2026-02-24T07:50:54.194 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:50:54.191+0000 7f73b1fae640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:50:54.194 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:50:54.191+0000 7f73b1fae640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:50:54.890 INFO:teuthology.orchestra.run.trial035.stderr:reweighted osd.2 to 1 (10000) 2026-02-24T07:50:54.935 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph osd reweight 3 1 2026-02-24T07:50:55.075 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:50:55.142 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:50:54 trial035 ceph-mon[39284]: pgmap v273: 17 pgs: 17 active+clean; 28 GiB data, 90 GiB used, 2.6 TiB / 2.7 TiB avail; 0 B/s wr, 9.32k op/s 2026-02-24T07:50:55.142 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:50:54 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/807226715' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 1, "weight": 1.0}]': finished 2026-02-24T07:50:55.142 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:50:54 trial035 ceph-mon[39284]: osdmap e63: 4 total, 4 up, 4 in 2026-02-24T07:50:55.142 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:50:54 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/4220027942' entity='client.admin' cmd={"prefix": "osd reweight", "id": 2, "weight": 1.0} : dispatch 2026-02-24T07:50:55.193 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:50:55.191+0000 7feb091bd640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:50:55.193 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:50:55.191+0000 7feb091bd640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:50:55.892 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2026-02-24T07:50:55.895 INFO:teuthology.orchestra.run.trial035.stderr:reweighted osd.3 to 1 (10000) 2026-02-24T07:50:55.938 INFO:tasks.cephadm.ceph_manager.ceph:waiting for recovery to complete 2026-02-24T07:50:55.939 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph pg dump --format=json 2026-02-24T07:50:56.038 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:50:56.084 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:50:56.145 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:50:55 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/4220027942' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 2, "weight": 1.0}]': finished 2026-02-24T07:50:56.145 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:50:55 trial035 ceph-mon[39284]: osdmap e64: 4 total, 4 up, 4 in 2026-02-24T07:50:56.145 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:50:55 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/4063880605' entity='client.admin' cmd={"prefix": "osd reweight", "id": 3, "weight": 1.0} : dispatch 2026-02-24T07:50:56.162 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:50:56.160+0000 7fdd2614b640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:50:56.163 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:50:56.161+0000 7fdd2614b640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:50:56.178 INFO:teuthology.orchestra.run.trial035.stdout:osd.0: {} 2026-02-24T07:50:56.179 INFO:teuthology.orchestra.run.trial035.stderr:osd.0: osd_enable_op_tracker = '' 2026-02-24T07:50:56.196 INFO:teuthology.orchestra.run.trial035.stdout:osd.1: {} 2026-02-24T07:50:56.197 INFO:teuthology.orchestra.run.trial035.stderr:osd.1: osd_enable_op_tracker = '' 2026-02-24T07:50:56.203 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:50:56.201+0000 7f09cb213640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:50:56.204 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:50:56.202+0000 7f09cb213640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:50:56.210 INFO:teuthology.orchestra.run.trial035.stdout:osd.2: {} 2026-02-24T07:50:56.210 INFO:teuthology.orchestra.run.trial035.stderr:osd.2: osd_enable_op_tracker = '' 2026-02-24T07:50:56.222 INFO:teuthology.orchestra.run.trial035.stdout:osd.3: {} 2026-02-24T07:50:56.223 INFO:teuthology.orchestra.run.trial035.stderr:osd.3: osd_enable_op_tracker = '' 2026-02-24T07:50:56.359 INFO:teuthology.orchestra.run.trial035.stdout: 2026-02-24T07:50:56.360 INFO:teuthology.orchestra.run.trial035.stderr:dumped all 2026-02-24T07:50:56.403 INFO:teuthology.orchestra.run.trial035.stdout:{"pg_ready":true,"pg_map":{"version":277,"stamp":"2026-02-24T07:50:55.894350+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":26647527992,"num_objects":406603,"num_object_clones":0,"num_object_copies":1219809,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":406603,"num_whiteouts":0,"num_read":137,"num_read_kb":241,"num_write":888270,"num_write_kb":41433639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":52,"ondisk_log_size":52,"up":51,"acting":51,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":51,"num_osds":4,"num_per_pool_osds":4,"num_per_pool_omap_osds":4,"kb":2936012800,"kb_used":86073208,"kb_used_data":78750824,"kb_used_omap":63,"kb_used_meta":7322112,"kb_avail":2849939592,"statfs":{"total":3006477107200,"available":2918338142208,"internally_reserved":0,"allocated":80640843776,"data_stored":80639086628,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":64675,"internal_metadata":7497843549},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[9,1],"upper_bound":4},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-2918907904,"num_objects":-44539,"num_object_clones":0,"num_object_copies":-133617,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-44539,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44539,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.173298"},"pg_stats":[{"pgid":"4.f","version":"64'55795","reported_seq":56134,"reported_epoch":64,"state":"active+clean","last_fresh":"2026-02-24T07:50:55.837275+0000","last_change":"2026-02-24T07:50:03.071459+0000","last_active":"2026-02-24T07:50:55.837275+0000","last_peered":"2026-02-24T07:50:55.837275+0000","last_clean":"2026-02-24T07:50:55.837275+0000","last_became_active":"2026-02-24T07:48:57.902313+0000","last_became_peered":"2026-02-24T07:48:57.902313+0000","last_unstale":"2026-02-24T07:50:55.837275+0000","last_undegraded":"2026-02-24T07:50:55.837275+0000","last_fullsized":"2026-02-24T07:50:55.837275+0000","mapping_epoch":55,"log_start":"64'55792","ondisk_log_start":"64'55792","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"61'30419","last_scrub_stamp":"2026-02-24T07:50:03.071380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:50:03.071380+0000","objects_scrubbed":29975,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:51:16.123453+0000","scrub_duration":2568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1674117120,"num_objects":25545,"num_object_clones":0,"num_object_copies":76635,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25545,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":55795,"num_write_kb":2602880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"64'55662","reported_seq":56068,"reported_epoch":64,"state":"active+clean","last_fresh":"2026-02-24T07:50:55.728974+0000","last_change":"2026-02-24T07:50:21.065676+0000","last_active":"2026-02-24T07:50:55.728974+0000","last_peered":"2026-02-24T07:50:55.728974+0000","last_clean":"2026-02-24T07:50:55.728974+0000","last_became_active":"2026-02-24T07:48:57.903584+0000","last_became_peered":"2026-02-24T07:48:57.903584+0000","last_unstale":"2026-02-24T07:50:55.728974+0000","last_undegraded":"2026-02-24T07:50:55.728974+0000","last_fullsized":"2026-02-24T07:50:55.728974+0000","mapping_epoch":55,"log_start":"64'55659","ondisk_log_start":"64'55659","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"61'37487","last_scrub_stamp":"2026-02-24T07:50:21.065638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:50:21.065638+0000","objects_scrubbed":36748,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:51:31.991874+0000","scrub_duration":3643,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1672216576,"num_objects":25516,"num_object_clones":0,"num_object_copies":76548,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25516,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":55662,"num_write_kb":2597696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.d","version":"64'55289","reported_seq":55637,"reported_epoch":64,"state":"active+clean","last_fresh":"2026-02-24T07:50:55.748203+0000","last_change":"2026-02-24T07:50:06.701973+0000","last_active":"2026-02-24T07:50:55.748203+0000","last_peered":"2026-02-24T07:50:55.748203+0000","last_clean":"2026-02-24T07:50:55.748203+0000","last_became_active":"2026-02-24T07:48:57.905031+0000","last_became_peered":"2026-02-24T07:48:57.905031+0000","last_unstale":"2026-02-24T07:50:55.748203+0000","last_undegraded":"2026-02-24T07:50:55.748203+0000","last_fullsized":"2026-02-24T07:50:55.748203+0000","mapping_epoch":55,"log_start":"64'55286","ondisk_log_start":"64'55286","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"61'31648","last_scrub_stamp":"2026-02-24T07:50:06.701883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:50:06.701883+0000","objects_scrubbed":30990,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:51:19.224966+0000","scrub_duration":3493,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1671299096,"num_objects":25503,"num_object_clones":0,"num_object_copies":76509,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25503,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":55289,"num_write_kb":2585281,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"64'55197","reported_seq":55614,"reported_epoch":64,"state":"active+clean","last_fresh":"2026-02-24T07:50:55.747366+0000","last_change":"2026-02-24T07:50:24.798674+0000","last_active":"2026-02-24T07:50:55.747366+0000","last_peered":"2026-02-24T07:50:55.747366+0000","last_clean":"2026-02-24T07:50:55.747366+0000","last_became_active":"2026-02-24T07:48:57.904683+0000","last_became_peered":"2026-02-24T07:48:57.904683+0000","last_unstale":"2026-02-24T07:50:55.747366+0000","last_undegraded":"2026-02-24T07:50:55.747366+0000","last_fullsized":"2026-02-24T07:50:55.747366+0000","mapping_epoch":55,"log_start":"64'55194","ondisk_log_start":"64'55194","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"61'38696","last_scrub_stamp":"2026-02-24T07:50:24.798629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:50:24.798629+0000","objects_scrubbed":37979,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:51:26.521338+0000","scrub_duration":3612,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1657733120,"num_objects":25295,"num_object_clones":0,"num_object_copies":75885,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25295,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":55197,"num_write_kb":2575744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.b","version":"64'55231","reported_seq":55659,"reported_epoch":64,"state":"active+clean","last_fresh":"2026-02-24T07:50:55.743933+0000","last_change":"2026-02-24T07:50:27.267008+0000","last_active":"2026-02-24T07:50:55.743933+0000","last_peered":"2026-02-24T07:50:55.743933+0000","last_clean":"2026-02-24T07:50:55.743933+0000","last_became_active":"2026-02-24T07:48:57.901700+0000","last_became_peered":"2026-02-24T07:48:57.901700+0000","last_unstale":"2026-02-24T07:50:55.743933+0000","last_undegraded":"2026-02-24T07:50:55.743933+0000","last_fullsized":"2026-02-24T07:50:55.743933+0000","mapping_epoch":55,"log_start":"64'55228","ondisk_log_start":"64'55228","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"61'39639","last_scrub_stamp":"2026-02-24T07:50:27.266943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:50:27.266943+0000","objects_scrubbed":38982,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:51:34.346751+0000","scrub_duration":3744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1666121728,"num_objects":25423,"num_object_clones":0,"num_object_copies":76269,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25423,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":55231,"num_write_kb":2580928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.a","version":"64'55703","reported_seq":56132,"reported_epoch":64,"state":"active+clean","last_fresh":"2026-02-24T07:50:55.733614+0000","last_change":"2026-02-24T07:50:33.043786+0000","last_active":"2026-02-24T07:50:55.733614+0000","last_peered":"2026-02-24T07:50:55.733614+0000","last_clean":"2026-02-24T07:50:55.733614+0000","last_became_active":"2026-02-24T07:48:57.903228+0000","last_became_peered":"2026-02-24T07:48:57.903228+0000","last_unstale":"2026-02-24T07:50:55.733614+0000","last_undegraded":"2026-02-24T07:50:55.733614+0000","last_fullsized":"2026-02-24T07:50:55.733614+0000","mapping_epoch":55,"log_start":"64'55700","ondisk_log_start":"64'55700","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"61'42539","last_scrub_stamp":"2026-02-24T07:50:33.043726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:50:33.043726+0000","objects_scrubbed":39246,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:51:51.239665+0000","scrub_duration":3547,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1650393088,"num_objects":25183,"num_object_clones":0,"num_object_copies":75549,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25183,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":55703,"num_write_kb":2588352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"64'55479","reported_seq":55850,"reported_epoch":64,"state":"active+clean","last_fresh":"2026-02-24T07:50:55.732663+0000","last_change":"2026-02-24T07:50:13.165492+0000","last_active":"2026-02-24T07:50:55.732663+0000","last_peered":"2026-02-24T07:50:55.732663+0000","last_clean":"2026-02-24T07:50:55.732663+0000","last_became_active":"2026-02-24T07:48:57.901730+0000","last_became_peered":"2026-02-24T07:48:57.901730+0000","last_unstale":"2026-02-24T07:50:55.732663+0000","last_undegraded":"2026-02-24T07:50:55.732663+0000","last_fullsized":"2026-02-24T07:50:55.732663+0000","mapping_epoch":55,"log_start":"64'55476","ondisk_log_start":"64'55476","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"61'33943","last_scrub_stamp":"2026-02-24T07:50:13.165459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:50:13.165459+0000","objects_scrubbed":33253,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:51:33.820708+0000","scrub_duration":3776,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1664024576,"num_objects":25391,"num_object_clones":0,"num_object_copies":76173,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25391,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":55479,"num_write_kb":2587840,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"64'55433","reported_seq":55768,"reported_epoch":64,"state":"active+clean","last_fresh":"2026-02-24T07:50:55.748914+0000","last_change":"2026-02-24T07:50:03.153499+0000","last_active":"2026-02-24T07:50:55.748914+0000","last_peered":"2026-02-24T07:50:55.748914+0000","last_clean":"2026-02-24T07:50:55.748914+0000","last_became_active":"2026-02-24T07:48:57.904376+0000","last_became_peered":"2026-02-24T07:48:57.904376+0000","last_unstale":"2026-02-24T07:50:55.748914+0000","last_undegraded":"2026-02-24T07:50:55.748914+0000","last_fullsized":"2026-02-24T07:50:55.748914+0000","mapping_epoch":55,"log_start":"64'55430","ondisk_log_start":"64'55430","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"61'30163","last_scrub_stamp":"2026-02-24T07:50:03.153454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:50:03.153454+0000","objects_scrubbed":29681,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:51:10.266136+0000","scrub_duration":2916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1665597440,"num_objects":25415,"num_object_clones":0,"num_object_copies":76245,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25415,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":55433,"num_write_kb":2587136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.7","version":"64'55546","reported_seq":55908,"reported_epoch":64,"state":"active+clean","last_fresh":"2026-02-24T07:50:55.837300+0000","last_change":"2026-02-24T07:50:11.227608+0000","last_active":"2026-02-24T07:50:55.837300+0000","last_peered":"2026-02-24T07:50:55.837300+0000","last_clean":"2026-02-24T07:50:55.837300+0000","last_became_active":"2026-02-24T07:48:57.902484+0000","last_became_peered":"2026-02-24T07:48:57.902484+0000","last_unstale":"2026-02-24T07:50:55.837300+0000","last_undegraded":"2026-02-24T07:50:55.837300+0000","last_fullsized":"2026-02-24T07:50:55.837300+0000","mapping_epoch":55,"log_start":"64'55543","ondisk_log_start":"64'55543","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"61'32990","last_scrub_stamp":"2026-02-24T07:50:11.227573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:50:11.227573+0000","objects_scrubbed":32460,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:51:29.781611+0000","scrub_duration":3701,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1644953600,"num_objects":25100,"num_object_clones":0,"num_object_copies":75300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25100,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":55546,"num_write_kb":2580672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.6","version":"64'55064","reported_seq":55437,"reported_epoch":64,"state":"active+clean","last_fresh":"2026-02-24T07:50:55.747341+0000","last_change":"2026-02-24T07:50:14.386679+0000","last_active":"2026-02-24T07:50:55.747341+0000","last_peered":"2026-02-24T07:50:55.747341+0000","last_clean":"2026-02-24T07:50:55.747341+0000","last_became_active":"2026-02-24T07:48:57.901565+0000","last_became_peered":"2026-02-24T07:48:57.901565+0000","last_unstale":"2026-02-24T07:50:55.747341+0000","last_undegraded":"2026-02-24T07:50:55.747341+0000","last_fullsized":"2026-02-24T07:50:55.747341+0000","mapping_epoch":55,"log_start":"64'55061","ondisk_log_start":"64'55061","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"61'34244","last_scrub_stamp":"2026-02-24T07:50:14.386625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:50:14.386625+0000","objects_scrubbed":33505,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:51:16.739697+0000","scrub_duration":3790,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1670905856,"num_objects":25496,"num_object_clones":0,"num_object_copies":76488,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25496,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":55064,"num_write_kb":2577920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"64'55913","reported_seq":56300,"reported_epoch":64,"state":"active+clean","last_fresh":"2026-02-24T07:50:55.837898+0000","last_change":"2026-02-24T07:50:16.399238+0000","last_active":"2026-02-24T07:50:55.837898+0000","last_peered":"2026-02-24T07:50:55.837898+0000","last_clean":"2026-02-24T07:50:55.837898+0000","last_became_active":"2026-02-24T07:48:57.901824+0000","last_became_peered":"2026-02-24T07:48:57.901824+0000","last_unstale":"2026-02-24T07:50:55.837898+0000","last_undegraded":"2026-02-24T07:50:55.837898+0000","last_fullsized":"2026-02-24T07:50:55.837898+0000","mapping_epoch":55,"log_start":"64'55910","ondisk_log_start":"64'55910","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"61'35510","last_scrub_stamp":"2026-02-24T07:50:16.399194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:50:16.399194+0000","objects_scrubbed":34844,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:51:44.770701+0000","scrub_duration":2971,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1660878848,"num_objects":25343,"num_object_clones":0,"num_object_copies":76029,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25343,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":55913,"num_write_kb":2600192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[1,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"64'55163","reported_seq":55522,"reported_epoch":64,"state":"active+clean","last_fresh":"2026-02-24T07:50:55.744322+0000","last_change":"2026-02-24T07:50:10.124042+0000","last_active":"2026-02-24T07:50:55.744322+0000","last_peered":"2026-02-24T07:50:55.744322+0000","last_clean":"2026-02-24T07:50:55.744322+0000","last_became_active":"2026-02-24T07:48:57.902224+0000","last_became_peered":"2026-02-24T07:48:57.902224+0000","last_unstale":"2026-02-24T07:50:55.744322+0000","last_undegraded":"2026-02-24T07:50:55.744322+0000","last_fullsized":"2026-02-24T07:50:55.744322+0000","mapping_epoch":55,"log_start":"64'55159","ondisk_log_start":"64'55159","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"61'32620","last_scrub_stamp":"2026-02-24T07:50:10.124005+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:50:10.124005+0000","objects_scrubbed":32078,"log_size":4,"log_dups_size":3000,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:51:12.877892+0000","scrub_duration":3559,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1652490240,"num_objects":25215,"num_object_clones":0,"num_object_copies":75645,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25215,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":55163,"num_write_kb":2572096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"64'55730","reported_seq":56060,"reported_epoch":64,"state":"active+clean","last_fresh":"2026-02-24T07:50:55.839244+0000","last_change":"2026-02-24T07:50:01.398116+0000","last_active":"2026-02-24T07:50:55.839244+0000","last_peered":"2026-02-24T07:50:55.839244+0000","last_clean":"2026-02-24T07:50:55.839244+0000","last_became_active":"2026-02-24T07:48:57.902125+0000","last_became_peered":"2026-02-24T07:48:57.902125+0000","last_unstale":"2026-02-24T07:50:55.839244+0000","last_undegraded":"2026-02-24T07:50:55.839244+0000","last_fullsized":"2026-02-24T07:50:55.839244+0000","mapping_epoch":55,"log_start":"64'55727","ondisk_log_start":"64'55727","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"61'29707","last_scrub_stamp":"2026-02-24T07:50:01.398090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:50:01.398090+0000","objects_scrubbed":29171,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:51:10.566529+0000","scrub_duration":2880,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1668546560,"num_objects":25460,"num_object_clones":0,"num_object_copies":76380,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25460,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":55730,"num_write_kb":2598080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1","version":"64'55776","reported_seq":56213,"reported_epoch":64,"state":"active+clean","last_fresh":"2026-02-24T07:50:55.733095+0000","last_change":"2026-02-24T07:50:29.889880+0000","last_active":"2026-02-24T07:50:55.733095+0000","last_peered":"2026-02-24T07:50:55.733095+0000","last_clean":"2026-02-24T07:50:55.733095+0000","last_became_active":"2026-02-24T07:48:57.903439+0000","last_became_peered":"2026-02-24T07:48:57.903439+0000","last_unstale":"2026-02-24T07:50:55.733095+0000","last_undegraded":"2026-02-24T07:50:55.733095+0000","last_fullsized":"2026-02-24T07:50:55.733095+0000","mapping_epoch":55,"log_start":"64'55773","ondisk_log_start":"64'55773","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"61'41105","last_scrub_stamp":"2026-02-24T07:50:29.889816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:50:29.889816+0000","objects_scrubbed":40127,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:51:50.788414+0000","scrub_duration":4409,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1679949824,"num_objects":25634,"num_object_clones":0,"num_object_copies":76902,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25634,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":55776,"num_write_kb":2605120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.0","version":"64'55646","reported_seq":56084,"reported_epoch":64,"state":"active+clean","last_fresh":"2026-02-24T07:50:55.746289+0000","last_change":"2026-02-24T07:50:29.387323+0000","last_active":"2026-02-24T07:50:55.746289+0000","last_peered":"2026-02-24T07:50:55.746289+0000","last_clean":"2026-02-24T07:50:55.746289+0000","last_became_active":"2026-02-24T07:48:57.902565+0000","last_became_peered":"2026-02-24T07:48:57.902565+0000","last_unstale":"2026-02-24T07:50:55.746289+0000","last_undegraded":"2026-02-24T07:50:55.746289+0000","last_fullsized":"2026-02-24T07:50:55.746289+0000","mapping_epoch":55,"log_start":"64'55643","ondisk_log_start":"64'55643","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"61'40736","last_scrub_stamp":"2026-02-24T07:50:29.387206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:50:29.387206+0000","objects_scrubbed":39946,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:51:59.173732+0000","scrub_duration":3834,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1670905856,"num_objects":25496,"num_object_clones":0,"num_object_copies":76488,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25496,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":55646,"num_write_kb":2596544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,3],"acting":[0,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.5","version":"64'55508","reported_seq":55849,"reported_epoch":64,"state":"active+clean","last_fresh":"2026-02-24T07:50:55.746054+0000","last_change":"2026-02-24T07:50:04.821914+0000","last_active":"2026-02-24T07:50:55.746054+0000","last_peered":"2026-02-24T07:50:55.746054+0000","last_clean":"2026-02-24T07:50:55.746054+0000","last_became_active":"2026-02-24T07:48:57.905127+0000","last_became_peered":"2026-02-24T07:48:57.905127+0000","last_unstale":"2026-02-24T07:50:55.746054+0000","last_undegraded":"2026-02-24T07:50:55.746054+0000","last_fullsized":"2026-02-24T07:50:55.746054+0000","mapping_epoch":55,"log_start":"64'55505","ondisk_log_start":"64'55505","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"61'30833","last_scrub_stamp":"2026-02-24T07:50:04.821861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:50:04.821861+0000","objects_scrubbed":30132,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:51:15.928207+0000","scrub_duration":3578,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1676804096,"num_objects":25586,"num_object_clones":0,"num_object_copies":76758,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25586,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":55508,"num_write_kb":2595008,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"29'99","reported_seq":300,"reported_epoch":64,"state":"active+clean","last_fresh":"2026-02-24T07:50:54.891383+0000","last_change":"2026-02-24T07:50:30.401963+0000","last_active":"2026-02-24T07:50:54.891383+0000","last_peered":"2026-02-24T07:50:54.891383+0000","last_clean":"2026-02-24T07:50:54.891383+0000","last_became_active":"2026-02-24T07:42:54.186719+0000","last_became_peered":"2026-02-24T07:42:54.186719+0000","last_unstale":"2026-02-24T07:50:54.891383+0000","last_undegraded":"2026-02-24T07:50:54.891383+0000","last_fullsized":"2026-02-24T07:50:54.891383+0000","mapping_epoch":27,"log_start":"29'96","ondisk_log_start":"29'96","created":20,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'99","last_scrub_stamp":"2026-02-24T07:50:30.401929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:42:43.640776+0000","last_clean_scrub_stamp":"2026-02-24T07:50:30.401929+0000","objects_scrubbed":2,"log_size":3,"log_dups_size":96,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:51:52.990727+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":240,"num_write":135,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":16,"stat_sum":{"num_bytes":26646937624,"num_objects":406601,"num_object_clones":0,"num_object_copies":1219803,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":406601,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":888135,"num_write_kb":41431489,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":79955636224,"data_stored":79955624008,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":49,"ondisk_log_size":49,"up":48,"acting":48,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":240,"num_write":135,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1054,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":3,"up_from":25,"seq":107374182499,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":21348208,"kb_used_data":19813404,"kb_used_omap":15,"kb_used_meta":1534768,"kb_avail":712654992,"statfs":{"total":751619276800,"available":729758711808,"internally_reserved":0,"allocated":20288925696,"data_stored":20288484503,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":15695,"internal_metadata":1571603121},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[4,1],"upper_bound":4},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":19,"seq":81604378725,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":24658992,"kb_used_data":22879324,"kb_used_omap":16,"kb_used_meta":1779503,"kb_avail":709344208,"statfs":{"total":751619276800,"available":726368468992,"internally_reserved":0,"allocated":23428427776,"data_stored":23427986583,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16677,"internal_metadata":1822211803},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542246,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":18460008,"kb_used_data":16421268,"kb_used_omap":17,"kb_used_meta":2038702,"kb_avail":715543192,"statfs":{"total":751619276800,"available":732716228608,"internally_reserved":0,"allocated":16815378432,"data_stored":16814944863,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":18081,"internal_metadata":2087631199},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[4],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705769,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":21606000,"kb_used_data":19636828,"kb_used_omap":13,"kb_used_meta":1969138,"kb_avail":712397200,"statfs":{"total":751619276800,"available":729494732800,"internally_reserved":0,"allocated":20108111872,"data_stored":20107670679,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":14222,"internal_metadata":2016397426},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1054,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":20003033088,"data_stored":20003029016,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16608329728,"data_stored":16608329728,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":23332851712,"data_stored":23332847640,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":20011421696,"data_stored":20011417624,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-24T07:50:56.405 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph pg dump --format=json 2026-02-24T07:50:56.549 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:50:56.674 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:50:56.671+0000 7f6bfd22a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:50:56.674 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:50:56.672+0000 7f6bfd22a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:50:56.818 INFO:teuthology.orchestra.run.trial035.stdout: 2026-02-24T07:50:56.819 INFO:teuthology.orchestra.run.trial035.stderr:dumped all 2026-02-24T07:50:56.864 INFO:teuthology.orchestra.run.trial035.stdout:{"pg_ready":true,"pg_map":{"version":277,"stamp":"2026-02-24T07:50:55.894350+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":26647527992,"num_objects":406603,"num_object_clones":0,"num_object_copies":1219809,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":406603,"num_whiteouts":0,"num_read":137,"num_read_kb":241,"num_write":888270,"num_write_kb":41433639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":52,"ondisk_log_size":52,"up":51,"acting":51,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":51,"num_osds":4,"num_per_pool_osds":4,"num_per_pool_omap_osds":4,"kb":2936012800,"kb_used":86073208,"kb_used_data":78750824,"kb_used_omap":63,"kb_used_meta":7322112,"kb_avail":2849939592,"statfs":{"total":3006477107200,"available":2918338142208,"internally_reserved":0,"allocated":80640843776,"data_stored":80639086628,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":64675,"internal_metadata":7497843549},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[9,1],"upper_bound":4},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-2918907904,"num_objects":-44539,"num_object_clones":0,"num_object_copies":-133617,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-44539,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44539,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.173298"},"pg_stats":[{"pgid":"4.f","version":"64'55795","reported_seq":56134,"reported_epoch":64,"state":"active+clean","last_fresh":"2026-02-24T07:50:55.837275+0000","last_change":"2026-02-24T07:50:03.071459+0000","last_active":"2026-02-24T07:50:55.837275+0000","last_peered":"2026-02-24T07:50:55.837275+0000","last_clean":"2026-02-24T07:50:55.837275+0000","last_became_active":"2026-02-24T07:48:57.902313+0000","last_became_peered":"2026-02-24T07:48:57.902313+0000","last_unstale":"2026-02-24T07:50:55.837275+0000","last_undegraded":"2026-02-24T07:50:55.837275+0000","last_fullsized":"2026-02-24T07:50:55.837275+0000","mapping_epoch":55,"log_start":"64'55792","ondisk_log_start":"64'55792","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"61'30419","last_scrub_stamp":"2026-02-24T07:50:03.071380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:50:03.071380+0000","objects_scrubbed":29975,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:51:16.123453+0000","scrub_duration":2568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1674117120,"num_objects":25545,"num_object_clones":0,"num_object_copies":76635,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25545,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":55795,"num_write_kb":2602880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"64'55662","reported_seq":56068,"reported_epoch":64,"state":"active+clean","last_fresh":"2026-02-24T07:50:55.728974+0000","last_change":"2026-02-24T07:50:21.065676+0000","last_active":"2026-02-24T07:50:55.728974+0000","last_peered":"2026-02-24T07:50:55.728974+0000","last_clean":"2026-02-24T07:50:55.728974+0000","last_became_active":"2026-02-24T07:48:57.903584+0000","last_became_peered":"2026-02-24T07:48:57.903584+0000","last_unstale":"2026-02-24T07:50:55.728974+0000","last_undegraded":"2026-02-24T07:50:55.728974+0000","last_fullsized":"2026-02-24T07:50:55.728974+0000","mapping_epoch":55,"log_start":"64'55659","ondisk_log_start":"64'55659","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"61'37487","last_scrub_stamp":"2026-02-24T07:50:21.065638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:50:21.065638+0000","objects_scrubbed":36748,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:51:31.991874+0000","scrub_duration":3643,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1672216576,"num_objects":25516,"num_object_clones":0,"num_object_copies":76548,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25516,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":55662,"num_write_kb":2597696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.d","version":"64'55289","reported_seq":55637,"reported_epoch":64,"state":"active+clean","last_fresh":"2026-02-24T07:50:55.748203+0000","last_change":"2026-02-24T07:50:06.701973+0000","last_active":"2026-02-24T07:50:55.748203+0000","last_peered":"2026-02-24T07:50:55.748203+0000","last_clean":"2026-02-24T07:50:55.748203+0000","last_became_active":"2026-02-24T07:48:57.905031+0000","last_became_peered":"2026-02-24T07:48:57.905031+0000","last_unstale":"2026-02-24T07:50:55.748203+0000","last_undegraded":"2026-02-24T07:50:55.748203+0000","last_fullsized":"2026-02-24T07:50:55.748203+0000","mapping_epoch":55,"log_start":"64'55286","ondisk_log_start":"64'55286","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"61'31648","last_scrub_stamp":"2026-02-24T07:50:06.701883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:50:06.701883+0000","objects_scrubbed":30990,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:51:19.224966+0000","scrub_duration":3493,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1671299096,"num_objects":25503,"num_object_clones":0,"num_object_copies":76509,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25503,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":55289,"num_write_kb":2585281,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"64'55197","reported_seq":55614,"reported_epoch":64,"state":"active+clean","last_fresh":"2026-02-24T07:50:55.747366+0000","last_change":"2026-02-24T07:50:24.798674+0000","last_active":"2026-02-24T07:50:55.747366+0000","last_peered":"2026-02-24T07:50:55.747366+0000","last_clean":"2026-02-24T07:50:55.747366+0000","last_became_active":"2026-02-24T07:48:57.904683+0000","last_became_peered":"2026-02-24T07:48:57.904683+0000","last_unstale":"2026-02-24T07:50:55.747366+0000","last_undegraded":"2026-02-24T07:50:55.747366+0000","last_fullsized":"2026-02-24T07:50:55.747366+0000","mapping_epoch":55,"log_start":"64'55194","ondisk_log_start":"64'55194","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"61'38696","last_scrub_stamp":"2026-02-24T07:50:24.798629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:50:24.798629+0000","objects_scrubbed":37979,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:51:26.521338+0000","scrub_duration":3612,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1657733120,"num_objects":25295,"num_object_clones":0,"num_object_copies":75885,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25295,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":55197,"num_write_kb":2575744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.b","version":"64'55231","reported_seq":55659,"reported_epoch":64,"state":"active+clean","last_fresh":"2026-02-24T07:50:55.743933+0000","last_change":"2026-02-24T07:50:27.267008+0000","last_active":"2026-02-24T07:50:55.743933+0000","last_peered":"2026-02-24T07:50:55.743933+0000","last_clean":"2026-02-24T07:50:55.743933+0000","last_became_active":"2026-02-24T07:48:57.901700+0000","last_became_peered":"2026-02-24T07:48:57.901700+0000","last_unstale":"2026-02-24T07:50:55.743933+0000","last_undegraded":"2026-02-24T07:50:55.743933+0000","last_fullsized":"2026-02-24T07:50:55.743933+0000","mapping_epoch":55,"log_start":"64'55228","ondisk_log_start":"64'55228","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"61'39639","last_scrub_stamp":"2026-02-24T07:50:27.266943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:50:27.266943+0000","objects_scrubbed":38982,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:51:34.346751+0000","scrub_duration":3744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1666121728,"num_objects":25423,"num_object_clones":0,"num_object_copies":76269,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25423,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":55231,"num_write_kb":2580928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.a","version":"64'55703","reported_seq":56132,"reported_epoch":64,"state":"active+clean","last_fresh":"2026-02-24T07:50:55.733614+0000","last_change":"2026-02-24T07:50:33.043786+0000","last_active":"2026-02-24T07:50:55.733614+0000","last_peered":"2026-02-24T07:50:55.733614+0000","last_clean":"2026-02-24T07:50:55.733614+0000","last_became_active":"2026-02-24T07:48:57.903228+0000","last_became_peered":"2026-02-24T07:48:57.903228+0000","last_unstale":"2026-02-24T07:50:55.733614+0000","last_undegraded":"2026-02-24T07:50:55.733614+0000","last_fullsized":"2026-02-24T07:50:55.733614+0000","mapping_epoch":55,"log_start":"64'55700","ondisk_log_start":"64'55700","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"61'42539","last_scrub_stamp":"2026-02-24T07:50:33.043726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:50:33.043726+0000","objects_scrubbed":39246,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:51:51.239665+0000","scrub_duration":3547,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1650393088,"num_objects":25183,"num_object_clones":0,"num_object_copies":75549,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25183,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":55703,"num_write_kb":2588352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"64'55479","reported_seq":55850,"reported_epoch":64,"state":"active+clean","last_fresh":"2026-02-24T07:50:55.732663+0000","last_change":"2026-02-24T07:50:13.165492+0000","last_active":"2026-02-24T07:50:55.732663+0000","last_peered":"2026-02-24T07:50:55.732663+0000","last_clean":"2026-02-24T07:50:55.732663+0000","last_became_active":"2026-02-24T07:48:57.901730+0000","last_became_peered":"2026-02-24T07:48:57.901730+0000","last_unstale":"2026-02-24T07:50:55.732663+0000","last_undegraded":"2026-02-24T07:50:55.732663+0000","last_fullsized":"2026-02-24T07:50:55.732663+0000","mapping_epoch":55,"log_start":"64'55476","ondisk_log_start":"64'55476","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"61'33943","last_scrub_stamp":"2026-02-24T07:50:13.165459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:50:13.165459+0000","objects_scrubbed":33253,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:51:33.820708+0000","scrub_duration":3776,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1664024576,"num_objects":25391,"num_object_clones":0,"num_object_copies":76173,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25391,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":55479,"num_write_kb":2587840,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"64'55433","reported_seq":55768,"reported_epoch":64,"state":"active+clean","last_fresh":"2026-02-24T07:50:55.748914+0000","last_change":"2026-02-24T07:50:03.153499+0000","last_active":"2026-02-24T07:50:55.748914+0000","last_peered":"2026-02-24T07:50:55.748914+0000","last_clean":"2026-02-24T07:50:55.748914+0000","last_became_active":"2026-02-24T07:48:57.904376+0000","last_became_peered":"2026-02-24T07:48:57.904376+0000","last_unstale":"2026-02-24T07:50:55.748914+0000","last_undegraded":"2026-02-24T07:50:55.748914+0000","last_fullsized":"2026-02-24T07:50:55.748914+0000","mapping_epoch":55,"log_start":"64'55430","ondisk_log_start":"64'55430","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"61'30163","last_scrub_stamp":"2026-02-24T07:50:03.153454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:50:03.153454+0000","objects_scrubbed":29681,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:51:10.266136+0000","scrub_duration":2916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1665597440,"num_objects":25415,"num_object_clones":0,"num_object_copies":76245,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25415,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":55433,"num_write_kb":2587136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.7","version":"64'55546","reported_seq":55908,"reported_epoch":64,"state":"active+clean","last_fresh":"2026-02-24T07:50:55.837300+0000","last_change":"2026-02-24T07:50:11.227608+0000","last_active":"2026-02-24T07:50:55.837300+0000","last_peered":"2026-02-24T07:50:55.837300+0000","last_clean":"2026-02-24T07:50:55.837300+0000","last_became_active":"2026-02-24T07:48:57.902484+0000","last_became_peered":"2026-02-24T07:48:57.902484+0000","last_unstale":"2026-02-24T07:50:55.837300+0000","last_undegraded":"2026-02-24T07:50:55.837300+0000","last_fullsized":"2026-02-24T07:50:55.837300+0000","mapping_epoch":55,"log_start":"64'55543","ondisk_log_start":"64'55543","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"61'32990","last_scrub_stamp":"2026-02-24T07:50:11.227573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:50:11.227573+0000","objects_scrubbed":32460,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:51:29.781611+0000","scrub_duration":3701,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1644953600,"num_objects":25100,"num_object_clones":0,"num_object_copies":75300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25100,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":55546,"num_write_kb":2580672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.6","version":"64'55064","reported_seq":55437,"reported_epoch":64,"state":"active+clean","last_fresh":"2026-02-24T07:50:55.747341+0000","last_change":"2026-02-24T07:50:14.386679+0000","last_active":"2026-02-24T07:50:55.747341+0000","last_peered":"2026-02-24T07:50:55.747341+0000","last_clean":"2026-02-24T07:50:55.747341+0000","last_became_active":"2026-02-24T07:48:57.901565+0000","last_became_peered":"2026-02-24T07:48:57.901565+0000","last_unstale":"2026-02-24T07:50:55.747341+0000","last_undegraded":"2026-02-24T07:50:55.747341+0000","last_fullsized":"2026-02-24T07:50:55.747341+0000","mapping_epoch":55,"log_start":"64'55061","ondisk_log_start":"64'55061","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"61'34244","last_scrub_stamp":"2026-02-24T07:50:14.386625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:50:14.386625+0000","objects_scrubbed":33505,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:51:16.739697+0000","scrub_duration":3790,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1670905856,"num_objects":25496,"num_object_clones":0,"num_object_copies":76488,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25496,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":55064,"num_write_kb":2577920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"64'55913","reported_seq":56300,"reported_epoch":64,"state":"active+clean","last_fresh":"2026-02-24T07:50:55.837898+0000","last_change":"2026-02-24T07:50:16.399238+0000","last_active":"2026-02-24T07:50:55.837898+0000","last_peered":"2026-02-24T07:50:55.837898+0000","last_clean":"2026-02-24T07:50:55.837898+0000","last_became_active":"2026-02-24T07:48:57.901824+0000","last_became_peered":"2026-02-24T07:48:57.901824+0000","last_unstale":"2026-02-24T07:50:55.837898+0000","last_undegraded":"2026-02-24T07:50:55.837898+0000","last_fullsized":"2026-02-24T07:50:55.837898+0000","mapping_epoch":55,"log_start":"64'55910","ondisk_log_start":"64'55910","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"61'35510","last_scrub_stamp":"2026-02-24T07:50:16.399194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:50:16.399194+0000","objects_scrubbed":34844,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:51:44.770701+0000","scrub_duration":2971,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1660878848,"num_objects":25343,"num_object_clones":0,"num_object_copies":76029,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25343,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":55913,"num_write_kb":2600192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[1,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"64'55163","reported_seq":55522,"reported_epoch":64,"state":"active+clean","last_fresh":"2026-02-24T07:50:55.744322+0000","last_change":"2026-02-24T07:50:10.124042+0000","last_active":"2026-02-24T07:50:55.744322+0000","last_peered":"2026-02-24T07:50:55.744322+0000","last_clean":"2026-02-24T07:50:55.744322+0000","last_became_active":"2026-02-24T07:48:57.902224+0000","last_became_peered":"2026-02-24T07:48:57.902224+0000","last_unstale":"2026-02-24T07:50:55.744322+0000","last_undegraded":"2026-02-24T07:50:55.744322+0000","last_fullsized":"2026-02-24T07:50:55.744322+0000","mapping_epoch":55,"log_start":"64'55159","ondisk_log_start":"64'55159","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"61'32620","last_scrub_stamp":"2026-02-24T07:50:10.124005+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:50:10.124005+0000","objects_scrubbed":32078,"log_size":4,"log_dups_size":3000,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:51:12.877892+0000","scrub_duration":3559,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1652490240,"num_objects":25215,"num_object_clones":0,"num_object_copies":75645,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25215,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":55163,"num_write_kb":2572096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"64'55730","reported_seq":56060,"reported_epoch":64,"state":"active+clean","last_fresh":"2026-02-24T07:50:55.839244+0000","last_change":"2026-02-24T07:50:01.398116+0000","last_active":"2026-02-24T07:50:55.839244+0000","last_peered":"2026-02-24T07:50:55.839244+0000","last_clean":"2026-02-24T07:50:55.839244+0000","last_became_active":"2026-02-24T07:48:57.902125+0000","last_became_peered":"2026-02-24T07:48:57.902125+0000","last_unstale":"2026-02-24T07:50:55.839244+0000","last_undegraded":"2026-02-24T07:50:55.839244+0000","last_fullsized":"2026-02-24T07:50:55.839244+0000","mapping_epoch":55,"log_start":"64'55727","ondisk_log_start":"64'55727","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"61'29707","last_scrub_stamp":"2026-02-24T07:50:01.398090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:50:01.398090+0000","objects_scrubbed":29171,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:51:10.566529+0000","scrub_duration":2880,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1668546560,"num_objects":25460,"num_object_clones":0,"num_object_copies":76380,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25460,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":55730,"num_write_kb":2598080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1","version":"64'55776","reported_seq":56213,"reported_epoch":64,"state":"active+clean","last_fresh":"2026-02-24T07:50:55.733095+0000","last_change":"2026-02-24T07:50:29.889880+0000","last_active":"2026-02-24T07:50:55.733095+0000","last_peered":"2026-02-24T07:50:55.733095+0000","last_clean":"2026-02-24T07:50:55.733095+0000","last_became_active":"2026-02-24T07:48:57.903439+0000","last_became_peered":"2026-02-24T07:48:57.903439+0000","last_unstale":"2026-02-24T07:50:55.733095+0000","last_undegraded":"2026-02-24T07:50:55.733095+0000","last_fullsized":"2026-02-24T07:50:55.733095+0000","mapping_epoch":55,"log_start":"64'55773","ondisk_log_start":"64'55773","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"61'41105","last_scrub_stamp":"2026-02-24T07:50:29.889816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:50:29.889816+0000","objects_scrubbed":40127,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:51:50.788414+0000","scrub_duration":4409,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1679949824,"num_objects":25634,"num_object_clones":0,"num_object_copies":76902,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25634,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":55776,"num_write_kb":2605120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.0","version":"64'55646","reported_seq":56084,"reported_epoch":64,"state":"active+clean","last_fresh":"2026-02-24T07:50:55.746289+0000","last_change":"2026-02-24T07:50:29.387323+0000","last_active":"2026-02-24T07:50:55.746289+0000","last_peered":"2026-02-24T07:50:55.746289+0000","last_clean":"2026-02-24T07:50:55.746289+0000","last_became_active":"2026-02-24T07:48:57.902565+0000","last_became_peered":"2026-02-24T07:48:57.902565+0000","last_unstale":"2026-02-24T07:50:55.746289+0000","last_undegraded":"2026-02-24T07:50:55.746289+0000","last_fullsized":"2026-02-24T07:50:55.746289+0000","mapping_epoch":55,"log_start":"64'55643","ondisk_log_start":"64'55643","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"61'40736","last_scrub_stamp":"2026-02-24T07:50:29.387206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:50:29.387206+0000","objects_scrubbed":39946,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:51:59.173732+0000","scrub_duration":3834,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1670905856,"num_objects":25496,"num_object_clones":0,"num_object_copies":76488,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25496,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":55646,"num_write_kb":2596544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,3],"acting":[0,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.5","version":"64'55508","reported_seq":55849,"reported_epoch":64,"state":"active+clean","last_fresh":"2026-02-24T07:50:55.746054+0000","last_change":"2026-02-24T07:50:04.821914+0000","last_active":"2026-02-24T07:50:55.746054+0000","last_peered":"2026-02-24T07:50:55.746054+0000","last_clean":"2026-02-24T07:50:55.746054+0000","last_became_active":"2026-02-24T07:48:57.905127+0000","last_became_peered":"2026-02-24T07:48:57.905127+0000","last_unstale":"2026-02-24T07:50:55.746054+0000","last_undegraded":"2026-02-24T07:50:55.746054+0000","last_fullsized":"2026-02-24T07:50:55.746054+0000","mapping_epoch":55,"log_start":"64'55505","ondisk_log_start":"64'55505","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"61'30833","last_scrub_stamp":"2026-02-24T07:50:04.821861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:50:04.821861+0000","objects_scrubbed":30132,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:51:15.928207+0000","scrub_duration":3578,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1676804096,"num_objects":25586,"num_object_clones":0,"num_object_copies":76758,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25586,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":55508,"num_write_kb":2595008,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"29'99","reported_seq":300,"reported_epoch":64,"state":"active+clean","last_fresh":"2026-02-24T07:50:54.891383+0000","last_change":"2026-02-24T07:50:30.401963+0000","last_active":"2026-02-24T07:50:54.891383+0000","last_peered":"2026-02-24T07:50:54.891383+0000","last_clean":"2026-02-24T07:50:54.891383+0000","last_became_active":"2026-02-24T07:42:54.186719+0000","last_became_peered":"2026-02-24T07:42:54.186719+0000","last_unstale":"2026-02-24T07:50:54.891383+0000","last_undegraded":"2026-02-24T07:50:54.891383+0000","last_fullsized":"2026-02-24T07:50:54.891383+0000","mapping_epoch":27,"log_start":"29'96","ondisk_log_start":"29'96","created":20,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'99","last_scrub_stamp":"2026-02-24T07:50:30.401929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:42:43.640776+0000","last_clean_scrub_stamp":"2026-02-24T07:50:30.401929+0000","objects_scrubbed":2,"log_size":3,"log_dups_size":96,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:51:52.990727+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":240,"num_write":135,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":16,"stat_sum":{"num_bytes":26646937624,"num_objects":406601,"num_object_clones":0,"num_object_copies":1219803,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":406601,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":888135,"num_write_kb":41431489,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":79955636224,"data_stored":79955624008,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":49,"ondisk_log_size":49,"up":48,"acting":48,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":240,"num_write":135,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1054,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":3,"up_from":25,"seq":107374182499,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":21348208,"kb_used_data":19813404,"kb_used_omap":15,"kb_used_meta":1534768,"kb_avail":712654992,"statfs":{"total":751619276800,"available":729758711808,"internally_reserved":0,"allocated":20288925696,"data_stored":20288484503,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":15695,"internal_metadata":1571603121},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[4,1],"upper_bound":4},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":19,"seq":81604378725,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":24658992,"kb_used_data":22879324,"kb_used_omap":16,"kb_used_meta":1779503,"kb_avail":709344208,"statfs":{"total":751619276800,"available":726368468992,"internally_reserved":0,"allocated":23428427776,"data_stored":23427986583,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16677,"internal_metadata":1822211803},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542246,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":18460008,"kb_used_data":16421268,"kb_used_omap":17,"kb_used_meta":2038702,"kb_avail":715543192,"statfs":{"total":751619276800,"available":732716228608,"internally_reserved":0,"allocated":16815378432,"data_stored":16814944863,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":18081,"internal_metadata":2087631199},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[4],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705769,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":21606000,"kb_used_data":19636828,"kb_used_omap":13,"kb_used_meta":1969138,"kb_avail":712397200,"statfs":{"total":751619276800,"available":729494732800,"internally_reserved":0,"allocated":20108111872,"data_stored":20107670679,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":14222,"internal_metadata":2016397426},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1054,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":20003033088,"data_stored":20003029016,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16608329728,"data_stored":16608329728,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":23332851712,"data_stored":23332847640,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":20011421696,"data_stored":20011417624,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-24T07:50:56.866 INFO:tasks.cephadm.ceph_manager.ceph:recovered! 2026-02-24T07:50:56.866 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2026-02-24T07:50:56.866 INFO:tasks.thrashosds.thrasher:fixing pg num pool .mgr 2026-02-24T07:50:56.866 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph pg dump --format=json 2026-02-24T07:50:56.954 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:50:56 trial035 ceph-mon[39284]: pgmap v276: 17 pgs: 17 active+clean; 28 GiB data, 90 GiB used, 2.6 TiB / 2.7 TiB avail; 0 B/s wr, 7.59k op/s 2026-02-24T07:50:56.954 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:50:56 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/4063880605' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 3, "weight": 1.0}]': finished 2026-02-24T07:50:56.954 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:50:56 trial035 ceph-mon[39284]: osdmap e65: 4 total, 4 up, 4 in 2026-02-24T07:50:56.954 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:50:56 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/2926936953' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2026-02-24T07:50:57.008 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:50:57.133 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:50:57.131+0000 7fa12a13b640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:50:57.134 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:50:57.131+0000 7fa12a13b640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:50:57.280 INFO:teuthology.orchestra.run.trial035.stdout: 2026-02-24T07:50:57.281 INFO:teuthology.orchestra.run.trial035.stderr:dumped all 2026-02-24T07:50:57.281 INFO:journalctl@ceph.mgr.x.trial035.stdout:Feb 24 07:50:56 trial035 ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-mgr-x[39621]: ::ffff:10.20.193.91 - - [24/Feb/2026:07:50:56] "GET /metrics HTTP/1.1" 200 4230 "" "Prometheus/3.6.0" 2026-02-24T07:50:57.324 INFO:teuthology.orchestra.run.trial035.stdout:{"pg_ready":true,"pg_map":{"version":277,"stamp":"2026-02-24T07:50:55.894350+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":26647527992,"num_objects":406603,"num_object_clones":0,"num_object_copies":1219809,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":406603,"num_whiteouts":0,"num_read":137,"num_read_kb":241,"num_write":888270,"num_write_kb":41433639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":52,"ondisk_log_size":52,"up":51,"acting":51,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":51,"num_osds":4,"num_per_pool_osds":4,"num_per_pool_omap_osds":4,"kb":2936012800,"kb_used":86073208,"kb_used_data":78750824,"kb_used_omap":63,"kb_used_meta":7322112,"kb_avail":2849939592,"statfs":{"total":3006477107200,"available":2918338142208,"internally_reserved":0,"allocated":80640843776,"data_stored":80639086628,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":64675,"internal_metadata":7497843549},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[9,1],"upper_bound":4},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-2918907904,"num_objects":-44539,"num_object_clones":0,"num_object_copies":-133617,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-44539,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44539,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.173298"},"pg_stats":[{"pgid":"4.f","version":"64'55795","reported_seq":56134,"reported_epoch":64,"state":"active+clean","last_fresh":"2026-02-24T07:50:55.837275+0000","last_change":"2026-02-24T07:50:03.071459+0000","last_active":"2026-02-24T07:50:55.837275+0000","last_peered":"2026-02-24T07:50:55.837275+0000","last_clean":"2026-02-24T07:50:55.837275+0000","last_became_active":"2026-02-24T07:48:57.902313+0000","last_became_peered":"2026-02-24T07:48:57.902313+0000","last_unstale":"2026-02-24T07:50:55.837275+0000","last_undegraded":"2026-02-24T07:50:55.837275+0000","last_fullsized":"2026-02-24T07:50:55.837275+0000","mapping_epoch":55,"log_start":"64'55792","ondisk_log_start":"64'55792","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"61'30419","last_scrub_stamp":"2026-02-24T07:50:03.071380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:50:03.071380+0000","objects_scrubbed":29975,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:51:16.123453+0000","scrub_duration":2568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1674117120,"num_objects":25545,"num_object_clones":0,"num_object_copies":76635,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25545,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":55795,"num_write_kb":2602880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"64'55662","reported_seq":56068,"reported_epoch":64,"state":"active+clean","last_fresh":"2026-02-24T07:50:55.728974+0000","last_change":"2026-02-24T07:50:21.065676+0000","last_active":"2026-02-24T07:50:55.728974+0000","last_peered":"2026-02-24T07:50:55.728974+0000","last_clean":"2026-02-24T07:50:55.728974+0000","last_became_active":"2026-02-24T07:48:57.903584+0000","last_became_peered":"2026-02-24T07:48:57.903584+0000","last_unstale":"2026-02-24T07:50:55.728974+0000","last_undegraded":"2026-02-24T07:50:55.728974+0000","last_fullsized":"2026-02-24T07:50:55.728974+0000","mapping_epoch":55,"log_start":"64'55659","ondisk_log_start":"64'55659","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"61'37487","last_scrub_stamp":"2026-02-24T07:50:21.065638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:50:21.065638+0000","objects_scrubbed":36748,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:51:31.991874+0000","scrub_duration":3643,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1672216576,"num_objects":25516,"num_object_clones":0,"num_object_copies":76548,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25516,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":55662,"num_write_kb":2597696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.d","version":"64'55289","reported_seq":55637,"reported_epoch":64,"state":"active+clean","last_fresh":"2026-02-24T07:50:55.748203+0000","last_change":"2026-02-24T07:50:06.701973+0000","last_active":"2026-02-24T07:50:55.748203+0000","last_peered":"2026-02-24T07:50:55.748203+0000","last_clean":"2026-02-24T07:50:55.748203+0000","last_became_active":"2026-02-24T07:48:57.905031+0000","last_became_peered":"2026-02-24T07:48:57.905031+0000","last_unstale":"2026-02-24T07:50:55.748203+0000","last_undegraded":"2026-02-24T07:50:55.748203+0000","last_fullsized":"2026-02-24T07:50:55.748203+0000","mapping_epoch":55,"log_start":"64'55286","ondisk_log_start":"64'55286","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"61'31648","last_scrub_stamp":"2026-02-24T07:50:06.701883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:50:06.701883+0000","objects_scrubbed":30990,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:51:19.224966+0000","scrub_duration":3493,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1671299096,"num_objects":25503,"num_object_clones":0,"num_object_copies":76509,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25503,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":55289,"num_write_kb":2585281,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"64'55197","reported_seq":55614,"reported_epoch":64,"state":"active+clean","last_fresh":"2026-02-24T07:50:55.747366+0000","last_change":"2026-02-24T07:50:24.798674+0000","last_active":"2026-02-24T07:50:55.747366+0000","last_peered":"2026-02-24T07:50:55.747366+0000","last_clean":"2026-02-24T07:50:55.747366+0000","last_became_active":"2026-02-24T07:48:57.904683+0000","last_became_peered":"2026-02-24T07:48:57.904683+0000","last_unstale":"2026-02-24T07:50:55.747366+0000","last_undegraded":"2026-02-24T07:50:55.747366+0000","last_fullsized":"2026-02-24T07:50:55.747366+0000","mapping_epoch":55,"log_start":"64'55194","ondisk_log_start":"64'55194","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"61'38696","last_scrub_stamp":"2026-02-24T07:50:24.798629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:50:24.798629+0000","objects_scrubbed":37979,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:51:26.521338+0000","scrub_duration":3612,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1657733120,"num_objects":25295,"num_object_clones":0,"num_object_copies":75885,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25295,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":55197,"num_write_kb":2575744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.b","version":"64'55231","reported_seq":55659,"reported_epoch":64,"state":"active+clean","last_fresh":"2026-02-24T07:50:55.743933+0000","last_change":"2026-02-24T07:50:27.267008+0000","last_active":"2026-02-24T07:50:55.743933+0000","last_peered":"2026-02-24T07:50:55.743933+0000","last_clean":"2026-02-24T07:50:55.743933+0000","last_became_active":"2026-02-24T07:48:57.901700+0000","last_became_peered":"2026-02-24T07:48:57.901700+0000","last_unstale":"2026-02-24T07:50:55.743933+0000","last_undegraded":"2026-02-24T07:50:55.743933+0000","last_fullsized":"2026-02-24T07:50:55.743933+0000","mapping_epoch":55,"log_start":"64'55228","ondisk_log_start":"64'55228","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"61'39639","last_scrub_stamp":"2026-02-24T07:50:27.266943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:50:27.266943+0000","objects_scrubbed":38982,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:51:34.346751+0000","scrub_duration":3744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1666121728,"num_objects":25423,"num_object_clones":0,"num_object_copies":76269,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25423,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":55231,"num_write_kb":2580928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.a","version":"64'55703","reported_seq":56132,"reported_epoch":64,"state":"active+clean","last_fresh":"2026-02-24T07:50:55.733614+0000","last_change":"2026-02-24T07:50:33.043786+0000","last_active":"2026-02-24T07:50:55.733614+0000","last_peered":"2026-02-24T07:50:55.733614+0000","last_clean":"2026-02-24T07:50:55.733614+0000","last_became_active":"2026-02-24T07:48:57.903228+0000","last_became_peered":"2026-02-24T07:48:57.903228+0000","last_unstale":"2026-02-24T07:50:55.733614+0000","last_undegraded":"2026-02-24T07:50:55.733614+0000","last_fullsized":"2026-02-24T07:50:55.733614+0000","mapping_epoch":55,"log_start":"64'55700","ondisk_log_start":"64'55700","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"61'42539","last_scrub_stamp":"2026-02-24T07:50:33.043726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:50:33.043726+0000","objects_scrubbed":39246,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:51:51.239665+0000","scrub_duration":3547,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1650393088,"num_objects":25183,"num_object_clones":0,"num_object_copies":75549,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25183,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":55703,"num_write_kb":2588352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"64'55479","reported_seq":55850,"reported_epoch":64,"state":"active+clean","last_fresh":"2026-02-24T07:50:55.732663+0000","last_change":"2026-02-24T07:50:13.165492+0000","last_active":"2026-02-24T07:50:55.732663+0000","last_peered":"2026-02-24T07:50:55.732663+0000","last_clean":"2026-02-24T07:50:55.732663+0000","last_became_active":"2026-02-24T07:48:57.901730+0000","last_became_peered":"2026-02-24T07:48:57.901730+0000","last_unstale":"2026-02-24T07:50:55.732663+0000","last_undegraded":"2026-02-24T07:50:55.732663+0000","last_fullsized":"2026-02-24T07:50:55.732663+0000","mapping_epoch":55,"log_start":"64'55476","ondisk_log_start":"64'55476","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"61'33943","last_scrub_stamp":"2026-02-24T07:50:13.165459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:50:13.165459+0000","objects_scrubbed":33253,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:51:33.820708+0000","scrub_duration":3776,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1664024576,"num_objects":25391,"num_object_clones":0,"num_object_copies":76173,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25391,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":55479,"num_write_kb":2587840,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"64'55433","reported_seq":55768,"reported_epoch":64,"state":"active+clean","last_fresh":"2026-02-24T07:50:55.748914+0000","last_change":"2026-02-24T07:50:03.153499+0000","last_active":"2026-02-24T07:50:55.748914+0000","last_peered":"2026-02-24T07:50:55.748914+0000","last_clean":"2026-02-24T07:50:55.748914+0000","last_became_active":"2026-02-24T07:48:57.904376+0000","last_became_peered":"2026-02-24T07:48:57.904376+0000","last_unstale":"2026-02-24T07:50:55.748914+0000","last_undegraded":"2026-02-24T07:50:55.748914+0000","last_fullsized":"2026-02-24T07:50:55.748914+0000","mapping_epoch":55,"log_start":"64'55430","ondisk_log_start":"64'55430","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"61'30163","last_scrub_stamp":"2026-02-24T07:50:03.153454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:50:03.153454+0000","objects_scrubbed":29681,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:51:10.266136+0000","scrub_duration":2916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1665597440,"num_objects":25415,"num_object_clones":0,"num_object_copies":76245,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25415,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":55433,"num_write_kb":2587136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.7","version":"64'55546","reported_seq":55908,"reported_epoch":64,"state":"active+clean","last_fresh":"2026-02-24T07:50:55.837300+0000","last_change":"2026-02-24T07:50:11.227608+0000","last_active":"2026-02-24T07:50:55.837300+0000","last_peered":"2026-02-24T07:50:55.837300+0000","last_clean":"2026-02-24T07:50:55.837300+0000","last_became_active":"2026-02-24T07:48:57.902484+0000","last_became_peered":"2026-02-24T07:48:57.902484+0000","last_unstale":"2026-02-24T07:50:55.837300+0000","last_undegraded":"2026-02-24T07:50:55.837300+0000","last_fullsized":"2026-02-24T07:50:55.837300+0000","mapping_epoch":55,"log_start":"64'55543","ondisk_log_start":"64'55543","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"61'32990","last_scrub_stamp":"2026-02-24T07:50:11.227573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:50:11.227573+0000","objects_scrubbed":32460,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:51:29.781611+0000","scrub_duration":3701,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1644953600,"num_objects":25100,"num_object_clones":0,"num_object_copies":75300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25100,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":55546,"num_write_kb":2580672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.6","version":"64'55064","reported_seq":55437,"reported_epoch":64,"state":"active+clean","last_fresh":"2026-02-24T07:50:55.747341+0000","last_change":"2026-02-24T07:50:14.386679+0000","last_active":"2026-02-24T07:50:55.747341+0000","last_peered":"2026-02-24T07:50:55.747341+0000","last_clean":"2026-02-24T07:50:55.747341+0000","last_became_active":"2026-02-24T07:48:57.901565+0000","last_became_peered":"2026-02-24T07:48:57.901565+0000","last_unstale":"2026-02-24T07:50:55.747341+0000","last_undegraded":"2026-02-24T07:50:55.747341+0000","last_fullsized":"2026-02-24T07:50:55.747341+0000","mapping_epoch":55,"log_start":"64'55061","ondisk_log_start":"64'55061","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"61'34244","last_scrub_stamp":"2026-02-24T07:50:14.386625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:50:14.386625+0000","objects_scrubbed":33505,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:51:16.739697+0000","scrub_duration":3790,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1670905856,"num_objects":25496,"num_object_clones":0,"num_object_copies":76488,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25496,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":55064,"num_write_kb":2577920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"64'55913","reported_seq":56300,"reported_epoch":64,"state":"active+clean","last_fresh":"2026-02-24T07:50:55.837898+0000","last_change":"2026-02-24T07:50:16.399238+0000","last_active":"2026-02-24T07:50:55.837898+0000","last_peered":"2026-02-24T07:50:55.837898+0000","last_clean":"2026-02-24T07:50:55.837898+0000","last_became_active":"2026-02-24T07:48:57.901824+0000","last_became_peered":"2026-02-24T07:48:57.901824+0000","last_unstale":"2026-02-24T07:50:55.837898+0000","last_undegraded":"2026-02-24T07:50:55.837898+0000","last_fullsized":"2026-02-24T07:50:55.837898+0000","mapping_epoch":55,"log_start":"64'55910","ondisk_log_start":"64'55910","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"61'35510","last_scrub_stamp":"2026-02-24T07:50:16.399194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:50:16.399194+0000","objects_scrubbed":34844,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:51:44.770701+0000","scrub_duration":2971,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1660878848,"num_objects":25343,"num_object_clones":0,"num_object_copies":76029,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25343,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":55913,"num_write_kb":2600192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[1,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"64'55163","reported_seq":55522,"reported_epoch":64,"state":"active+clean","last_fresh":"2026-02-24T07:50:55.744322+0000","last_change":"2026-02-24T07:50:10.124042+0000","last_active":"2026-02-24T07:50:55.744322+0000","last_peered":"2026-02-24T07:50:55.744322+0000","last_clean":"2026-02-24T07:50:55.744322+0000","last_became_active":"2026-02-24T07:48:57.902224+0000","last_became_peered":"2026-02-24T07:48:57.902224+0000","last_unstale":"2026-02-24T07:50:55.744322+0000","last_undegraded":"2026-02-24T07:50:55.744322+0000","last_fullsized":"2026-02-24T07:50:55.744322+0000","mapping_epoch":55,"log_start":"64'55159","ondisk_log_start":"64'55159","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"61'32620","last_scrub_stamp":"2026-02-24T07:50:10.124005+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:50:10.124005+0000","objects_scrubbed":32078,"log_size":4,"log_dups_size":3000,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:51:12.877892+0000","scrub_duration":3559,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1652490240,"num_objects":25215,"num_object_clones":0,"num_object_copies":75645,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25215,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":55163,"num_write_kb":2572096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"64'55730","reported_seq":56060,"reported_epoch":64,"state":"active+clean","last_fresh":"2026-02-24T07:50:55.839244+0000","last_change":"2026-02-24T07:50:01.398116+0000","last_active":"2026-02-24T07:50:55.839244+0000","last_peered":"2026-02-24T07:50:55.839244+0000","last_clean":"2026-02-24T07:50:55.839244+0000","last_became_active":"2026-02-24T07:48:57.902125+0000","last_became_peered":"2026-02-24T07:48:57.902125+0000","last_unstale":"2026-02-24T07:50:55.839244+0000","last_undegraded":"2026-02-24T07:50:55.839244+0000","last_fullsized":"2026-02-24T07:50:55.839244+0000","mapping_epoch":55,"log_start":"64'55727","ondisk_log_start":"64'55727","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"61'29707","last_scrub_stamp":"2026-02-24T07:50:01.398090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:50:01.398090+0000","objects_scrubbed":29171,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:51:10.566529+0000","scrub_duration":2880,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1668546560,"num_objects":25460,"num_object_clones":0,"num_object_copies":76380,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25460,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":55730,"num_write_kb":2598080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1","version":"64'55776","reported_seq":56213,"reported_epoch":64,"state":"active+clean","last_fresh":"2026-02-24T07:50:55.733095+0000","last_change":"2026-02-24T07:50:29.889880+0000","last_active":"2026-02-24T07:50:55.733095+0000","last_peered":"2026-02-24T07:50:55.733095+0000","last_clean":"2026-02-24T07:50:55.733095+0000","last_became_active":"2026-02-24T07:48:57.903439+0000","last_became_peered":"2026-02-24T07:48:57.903439+0000","last_unstale":"2026-02-24T07:50:55.733095+0000","last_undegraded":"2026-02-24T07:50:55.733095+0000","last_fullsized":"2026-02-24T07:50:55.733095+0000","mapping_epoch":55,"log_start":"64'55773","ondisk_log_start":"64'55773","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"61'41105","last_scrub_stamp":"2026-02-24T07:50:29.889816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:50:29.889816+0000","objects_scrubbed":40127,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:51:50.788414+0000","scrub_duration":4409,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1679949824,"num_objects":25634,"num_object_clones":0,"num_object_copies":76902,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25634,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":55776,"num_write_kb":2605120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.0","version":"64'55646","reported_seq":56084,"reported_epoch":64,"state":"active+clean","last_fresh":"2026-02-24T07:50:55.746289+0000","last_change":"2026-02-24T07:50:29.387323+0000","last_active":"2026-02-24T07:50:55.746289+0000","last_peered":"2026-02-24T07:50:55.746289+0000","last_clean":"2026-02-24T07:50:55.746289+0000","last_became_active":"2026-02-24T07:48:57.902565+0000","last_became_peered":"2026-02-24T07:48:57.902565+0000","last_unstale":"2026-02-24T07:50:55.746289+0000","last_undegraded":"2026-02-24T07:50:55.746289+0000","last_fullsized":"2026-02-24T07:50:55.746289+0000","mapping_epoch":55,"log_start":"64'55643","ondisk_log_start":"64'55643","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"61'40736","last_scrub_stamp":"2026-02-24T07:50:29.387206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:50:29.387206+0000","objects_scrubbed":39946,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:51:59.173732+0000","scrub_duration":3834,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1670905856,"num_objects":25496,"num_object_clones":0,"num_object_copies":76488,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25496,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":55646,"num_write_kb":2596544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,3],"acting":[0,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.5","version":"64'55508","reported_seq":55849,"reported_epoch":64,"state":"active+clean","last_fresh":"2026-02-24T07:50:55.746054+0000","last_change":"2026-02-24T07:50:04.821914+0000","last_active":"2026-02-24T07:50:55.746054+0000","last_peered":"2026-02-24T07:50:55.746054+0000","last_clean":"2026-02-24T07:50:55.746054+0000","last_became_active":"2026-02-24T07:48:57.905127+0000","last_became_peered":"2026-02-24T07:48:57.905127+0000","last_unstale":"2026-02-24T07:50:55.746054+0000","last_undegraded":"2026-02-24T07:50:55.746054+0000","last_fullsized":"2026-02-24T07:50:55.746054+0000","mapping_epoch":55,"log_start":"64'55505","ondisk_log_start":"64'55505","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"61'30833","last_scrub_stamp":"2026-02-24T07:50:04.821861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:50:04.821861+0000","objects_scrubbed":30132,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:51:15.928207+0000","scrub_duration":3578,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1676804096,"num_objects":25586,"num_object_clones":0,"num_object_copies":76758,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25586,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":55508,"num_write_kb":2595008,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"29'99","reported_seq":300,"reported_epoch":64,"state":"active+clean","last_fresh":"2026-02-24T07:50:54.891383+0000","last_change":"2026-02-24T07:50:30.401963+0000","last_active":"2026-02-24T07:50:54.891383+0000","last_peered":"2026-02-24T07:50:54.891383+0000","last_clean":"2026-02-24T07:50:54.891383+0000","last_became_active":"2026-02-24T07:42:54.186719+0000","last_became_peered":"2026-02-24T07:42:54.186719+0000","last_unstale":"2026-02-24T07:50:54.891383+0000","last_undegraded":"2026-02-24T07:50:54.891383+0000","last_fullsized":"2026-02-24T07:50:54.891383+0000","mapping_epoch":27,"log_start":"29'96","ondisk_log_start":"29'96","created":20,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'99","last_scrub_stamp":"2026-02-24T07:50:30.401929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:42:43.640776+0000","last_clean_scrub_stamp":"2026-02-24T07:50:30.401929+0000","objects_scrubbed":2,"log_size":3,"log_dups_size":96,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:51:52.990727+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":240,"num_write":135,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":16,"stat_sum":{"num_bytes":26646937624,"num_objects":406601,"num_object_clones":0,"num_object_copies":1219803,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":406601,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":888135,"num_write_kb":41431489,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":79955636224,"data_stored":79955624008,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":49,"ondisk_log_size":49,"up":48,"acting":48,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":240,"num_write":135,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1054,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":3,"up_from":25,"seq":107374182499,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":21348208,"kb_used_data":19813404,"kb_used_omap":15,"kb_used_meta":1534768,"kb_avail":712654992,"statfs":{"total":751619276800,"available":729758711808,"internally_reserved":0,"allocated":20288925696,"data_stored":20288484503,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":15695,"internal_metadata":1571603121},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[4,1],"upper_bound":4},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":19,"seq":81604378725,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":24658992,"kb_used_data":22879324,"kb_used_omap":16,"kb_used_meta":1779503,"kb_avail":709344208,"statfs":{"total":751619276800,"available":726368468992,"internally_reserved":0,"allocated":23428427776,"data_stored":23427986583,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16677,"internal_metadata":1822211803},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542246,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":18460008,"kb_used_data":16421268,"kb_used_omap":17,"kb_used_meta":2038702,"kb_avail":715543192,"statfs":{"total":751619276800,"available":732716228608,"internally_reserved":0,"allocated":16815378432,"data_stored":16814944863,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":18081,"internal_metadata":2087631199},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[4],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705769,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":21606000,"kb_used_data":19636828,"kb_used_omap":13,"kb_used_meta":1969138,"kb_avail":712397200,"statfs":{"total":751619276800,"available":729494732800,"internally_reserved":0,"allocated":20108111872,"data_stored":20107670679,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":14222,"internal_metadata":2016397426},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1054,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":20003033088,"data_stored":20003029016,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16608329728,"data_stored":16608329728,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":23332851712,"data_stored":23332847640,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":20011421696,"data_stored":20011417624,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-24T07:50:57.327 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph osd pool set .mgr pgp_num 261 2026-02-24T07:50:57.356 INFO:journalctl@ceph.prometheus.a.trial091.stdout:Feb 24 07:50:56 trial091 ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-prometheus-a[66007]: time=2026-02-24T07:50:56.972Z level=ERROR source=scrape.go:1630 msg="Failed to determine correct type of scrape target." component="scrape manager" scrape_pool=ceph target=http://trial035:9283/metrics content_type="text/html;charset=utf-8" fallback_media_type="" err="received unsupported Content-Type \"text/html;charset=utf-8\" and no fallback_scrape_protocol specified for target" 2026-02-24T07:50:57.459 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:50:57.939 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:50:57.874+0000 7fee01cde640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:50:57.939 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:50:57.874+0000 7fee01cde640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:50:58.018 INFO:teuthology.orchestra.run.trial035.stderr:Error EPERM: crimson_allow_pg_split is false; pgp_num increase denied 2026-02-24T07:50:58.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:50:57 trial035 ceph-mon[39284]: from='client.15049 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2026-02-24T07:50:58.205 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:50:57 trial035 ceph-mon[39284]: from='client.15051 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2026-02-24T07:50:58.265 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2026-02-24T07:50:58.464 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:50:58.538 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-24T07:50:58.951 INFO:teuthology.orchestra.run.trial035.stdout:osd.0: {} 2026-02-24T07:50:58.951 INFO:teuthology.orchestra.run.trial035.stdout:osd.1: {} 2026-02-24T07:50:58.951 INFO:teuthology.orchestra.run.trial035.stdout:osd.2: {} 2026-02-24T07:50:58.951 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:50:58.899+0000 7f0cb9642640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:50:58.951 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:50:58.899+0000 7f0cb9642640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:50:58.951 INFO:teuthology.orchestra.run.trial035.stderr:osd.0: osd_enable_op_tracker = '' 2026-02-24T07:50:58.951 INFO:teuthology.orchestra.run.trial035.stderr:osd.1: osd_enable_op_tracker = '' 2026-02-24T07:50:58.951 INFO:teuthology.orchestra.run.trial035.stderr:osd.2: osd_enable_op_tracker = '' 2026-02-24T07:50:58.960 INFO:teuthology.orchestra.run.trial035.stdout:osd.3: {} 2026-02-24T07:50:58.960 INFO:teuthology.orchestra.run.trial035.stderr:osd.3: osd_enable_op_tracker = '' 2026-02-24T07:50:59.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:50:58 trial035 ceph-mon[39284]: from='client.15053 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2026-02-24T07:50:59.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:50:58 trial035 ceph-mon[39284]: pgmap v278: 17 pgs: 17 active+clean; 25 GiB data, 82 GiB used, 2.7 TiB / 2.7 TiB avail; 0 B/s wr, 9.21k op/s 2026-02-24T07:50:59.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:50:58 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/3915246195' entity='client.admin' cmd={"prefix": "osd pool set", "pool": ".mgr", "var": "pgp_num", "val": "261"} : dispatch 2026-02-24T07:51:00.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:50:59 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/1169350920' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2026-02-24T07:51:00.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:50:59 trial035 ceph-mon[39284]: from='mgr.14284 10.20.193.35:0/3681373222' entity='mgr.x' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2026-02-24T07:51:01.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:51:00 trial035 ceph-mon[39284]: pgmap v279: 17 pgs: 17 active+clean; 25 GiB data, 82 GiB used, 2.7 TiB / 2.7 TiB avail; 0 B/s wr, 7.42k op/s 2026-02-24T07:51:01.543 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2026-02-24T07:51:01.686 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:51:01.815 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:51:01.812+0000 7ff7fc8fb640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:51:01.815 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:51:01.813+0000 7ff7fc8fb640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:51:01.830 INFO:teuthology.orchestra.run.trial035.stdout:osd.0: {} 2026-02-24T07:51:01.830 INFO:teuthology.orchestra.run.trial035.stderr:osd.0: osd_enable_op_tracker = '' 2026-02-24T07:51:01.843 INFO:teuthology.orchestra.run.trial035.stdout:osd.1: {} 2026-02-24T07:51:01.844 INFO:teuthology.orchestra.run.trial035.stderr:osd.1: osd_enable_op_tracker = '' 2026-02-24T07:51:01.856 INFO:teuthology.orchestra.run.trial035.stdout:osd.2: {} 2026-02-24T07:51:01.856 INFO:teuthology.orchestra.run.trial035.stderr:osd.2: osd_enable_op_tracker = '' 2026-02-24T07:51:01.868 INFO:teuthology.orchestra.run.trial035.stdout:osd.3: {} 2026-02-24T07:51:01.868 INFO:teuthology.orchestra.run.trial035.stderr:osd.3: osd_enable_op_tracker = '' 2026-02-24T07:51:02.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:51:01 trial035 ceph-mon[39284]: pgmap v280: 17 pgs: 17 active+clean; 22 GiB data, 74 GiB used, 2.7 TiB / 2.7 TiB avail; 0 B/s wr, 10.77k op/s 2026-02-24T07:51:02.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:51:01 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/3477711265' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2026-02-24T07:51:03.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:51:02 trial035 ceph-mon[39284]: purged_snaps scrub starts 2026-02-24T07:51:03.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:51:02 trial035 ceph-mon[39284]: purged_snaps scrub ok 2026-02-24T07:51:03.539 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2026-02-24T07:51:03.539 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2026-02-24T07:51:03.539 INFO:tasks.thrashosds.thrasher:Growing pool .mgr 2026-02-24T07:51:03.539 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph pg dump --format=json 2026-02-24T07:51:03.681 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:51:03.907 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2026-02-24T07:51:04.050 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:51:04.034+0000 7ff6c8b6f640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:51:04.050 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:51:04.035+0000 7ff6c8b6f640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:51:04.060 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:51:04.171 INFO:teuthology.orchestra.run.trial035.stdout: 2026-02-24T07:51:04.172 INFO:teuthology.orchestra.run.trial035.stderr:dumped all 2026-02-24T07:51:04.172 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:51:03 trial035 ceph-mon[39284]: pgmap v281: 17 pgs: 17 active+clean; 22 GiB data, 74 GiB used, 2.7 TiB / 2.7 TiB avail; 0 B/s wr, 9.55k op/s 2026-02-24T07:51:04.870 INFO:teuthology.orchestra.run.trial035.stdout:{"pg_ready":true,"pg_map":{"version":281,"stamp":"2026-02-24T07:51:03.722740+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":24036639288,"num_objects":366764,"num_object_clones":0,"num_object_copies":1100292,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":366764,"num_whiteouts":0,"num_read":137,"num_read_kb":241,"num_write":928109,"num_write_kb":41433639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":52,"ondisk_log_size":52,"up":51,"acting":51,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":51,"num_osds":4,"num_per_pool_osds":4,"num_per_pool_omap_osds":4,"kb":2936012800,"kb_used":77531048,"kb_used_data":71172696,"kb_used_omap":65,"kb_used_meta":6358142,"kb_avail":2858481752,"statfs":{"total":3006477107200,"available":2927085314048,"internally_reserved":0,"allocated":72880840704,"data_stored":72879052040,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":67290,"internal_metadata":6510737702},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-5529796608,"num_objects":-84378,"num_object_clones":0,"num_object_copies":-253134,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-84378,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":84378,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.832876"},"pg_stats":[{"pgid":"4.f","version":"65'58304","reported_seq":58645,"reported_epoch":65,"state":"active+clean","last_fresh":"2026-02-24T07:51:00.835046+0000","last_change":"2026-02-24T07:50:03.071459+0000","last_active":"2026-02-24T07:51:00.835046+0000","last_peered":"2026-02-24T07:51:00.835046+0000","last_clean":"2026-02-24T07:51:00.835046+0000","last_became_active":"2026-02-24T07:48:57.902313+0000","last_became_peered":"2026-02-24T07:48:57.902313+0000","last_unstale":"2026-02-24T07:51:00.835046+0000","last_undegraded":"2026-02-24T07:51:00.835046+0000","last_fullsized":"2026-02-24T07:51:00.835046+0000","mapping_epoch":55,"log_start":"65'58301","ondisk_log_start":"65'58301","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"61'30419","last_scrub_stamp":"2026-02-24T07:50:03.071380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:50:03.071380+0000","objects_scrubbed":29975,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:51:16.123453+0000","scrub_duration":2568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1509687296,"num_objects":23036,"num_object_clones":0,"num_object_copies":69108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23036,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":58304,"num_write_kb":2602880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"65'58224","reported_seq":58631,"reported_epoch":65,"state":"active+clean","last_fresh":"2026-02-24T07:51:00.734715+0000","last_change":"2026-02-24T07:50:21.065676+0000","last_active":"2026-02-24T07:51:00.734715+0000","last_peered":"2026-02-24T07:51:00.734715+0000","last_clean":"2026-02-24T07:51:00.734715+0000","last_became_active":"2026-02-24T07:48:57.903584+0000","last_became_peered":"2026-02-24T07:48:57.903584+0000","last_unstale":"2026-02-24T07:51:00.734715+0000","last_undegraded":"2026-02-24T07:51:00.734715+0000","last_fullsized":"2026-02-24T07:51:00.734715+0000","mapping_epoch":55,"log_start":"65'58221","ondisk_log_start":"65'58221","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"61'37487","last_scrub_stamp":"2026-02-24T07:50:21.065638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:50:21.065638+0000","objects_scrubbed":36748,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:51:31.991874+0000","scrub_duration":3643,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1504313344,"num_objects":22954,"num_object_clones":0,"num_object_copies":68862,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22954,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":58224,"num_write_kb":2597696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.d","version":"65'57735","reported_seq":58086,"reported_epoch":65,"state":"active+clean","last_fresh":"2026-02-24T07:51:00.745197+0000","last_change":"2026-02-24T07:50:06.701973+0000","last_active":"2026-02-24T07:51:00.745197+0000","last_peered":"2026-02-24T07:51:00.745197+0000","last_clean":"2026-02-24T07:51:00.745197+0000","last_became_active":"2026-02-24T07:48:57.905031+0000","last_became_peered":"2026-02-24T07:48:57.905031+0000","last_unstale":"2026-02-24T07:51:00.745197+0000","last_undegraded":"2026-02-24T07:51:00.745197+0000","last_fullsized":"2026-02-24T07:51:00.745197+0000","mapping_epoch":55,"log_start":"65'57732","ondisk_log_start":"65'57732","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"61'31648","last_scrub_stamp":"2026-02-24T07:50:06.701883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:50:06.701883+0000","objects_scrubbed":30990,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:51:19.224966+0000","scrub_duration":3493,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1510998040,"num_objects":23057,"num_object_clones":0,"num_object_copies":69171,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23057,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":57735,"num_write_kb":2585281,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"65'57685","reported_seq":58104,"reported_epoch":65,"state":"active+clean","last_fresh":"2026-02-24T07:51:00.747742+0000","last_change":"2026-02-24T07:50:24.798674+0000","last_active":"2026-02-24T07:51:00.747742+0000","last_peered":"2026-02-24T07:51:00.747742+0000","last_clean":"2026-02-24T07:51:00.747742+0000","last_became_active":"2026-02-24T07:48:57.904683+0000","last_became_peered":"2026-02-24T07:48:57.904683+0000","last_unstale":"2026-02-24T07:51:00.747742+0000","last_undegraded":"2026-02-24T07:51:00.747742+0000","last_fullsized":"2026-02-24T07:51:00.747742+0000","mapping_epoch":55,"log_start":"65'57682","ondisk_log_start":"65'57682","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"61'38696","last_scrub_stamp":"2026-02-24T07:50:24.798629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:50:24.798629+0000","objects_scrubbed":37979,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:51:26.521338+0000","scrub_duration":3612,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1494679552,"num_objects":22807,"num_object_clones":0,"num_object_copies":68421,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22807,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":57685,"num_write_kb":2575744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.b","version":"65'57684","reported_seq":58114,"reported_epoch":65,"state":"active+clean","last_fresh":"2026-02-24T07:51:00.749053+0000","last_change":"2026-02-24T07:50:27.267008+0000","last_active":"2026-02-24T07:51:00.749053+0000","last_peered":"2026-02-24T07:51:00.749053+0000","last_clean":"2026-02-24T07:51:00.749053+0000","last_became_active":"2026-02-24T07:48:57.901700+0000","last_became_peered":"2026-02-24T07:48:57.901700+0000","last_unstale":"2026-02-24T07:51:00.749053+0000","last_undegraded":"2026-02-24T07:51:00.749053+0000","last_fullsized":"2026-02-24T07:51:00.749053+0000","mapping_epoch":55,"log_start":"65'57681","ondisk_log_start":"65'57681","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"61'39639","last_scrub_stamp":"2026-02-24T07:50:27.266943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:50:27.266943+0000","objects_scrubbed":38982,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:51:34.346751+0000","scrub_duration":3744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1505361920,"num_objects":22970,"num_object_clones":0,"num_object_copies":68910,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22970,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":57684,"num_write_kb":2580928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.a","version":"65'58177","reported_seq":58610,"reported_epoch":65,"state":"active+clean","last_fresh":"2026-02-24T07:51:00.728590+0000","last_change":"2026-02-24T07:50:33.043786+0000","last_active":"2026-02-24T07:51:00.728590+0000","last_peered":"2026-02-24T07:51:00.728590+0000","last_clean":"2026-02-24T07:51:00.728590+0000","last_became_active":"2026-02-24T07:48:57.903228+0000","last_became_peered":"2026-02-24T07:48:57.903228+0000","last_unstale":"2026-02-24T07:51:00.728590+0000","last_undegraded":"2026-02-24T07:51:00.728590+0000","last_fullsized":"2026-02-24T07:51:00.728590+0000","mapping_epoch":55,"log_start":"65'58174","ondisk_log_start":"65'58174","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"61'42539","last_scrub_stamp":"2026-02-24T07:50:33.043726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:50:33.043726+0000","objects_scrubbed":39246,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:51:51.239665+0000","scrub_duration":3547,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1488257024,"num_objects":22709,"num_object_clones":0,"num_object_copies":68127,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22709,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":58177,"num_write_kb":2588352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"65'57981","reported_seq":58353,"reported_epoch":65,"state":"active+clean","last_fresh":"2026-02-24T07:51:00.732345+0000","last_change":"2026-02-24T07:50:13.165492+0000","last_active":"2026-02-24T07:51:00.732345+0000","last_peered":"2026-02-24T07:51:00.732345+0000","last_clean":"2026-02-24T07:51:00.732345+0000","last_became_active":"2026-02-24T07:48:57.901730+0000","last_became_peered":"2026-02-24T07:48:57.901730+0000","last_unstale":"2026-02-24T07:51:00.732345+0000","last_undegraded":"2026-02-24T07:51:00.732345+0000","last_fullsized":"2026-02-24T07:51:00.732345+0000","mapping_epoch":55,"log_start":"65'57978","ondisk_log_start":"65'57978","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"61'33943","last_scrub_stamp":"2026-02-24T07:50:13.165459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:50:13.165459+0000","objects_scrubbed":33253,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:51:33.820708+0000","scrub_duration":3776,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1500053504,"num_objects":22889,"num_object_clones":0,"num_object_copies":68667,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22889,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":57981,"num_write_kb":2587840,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"65'57912","reported_seq":58249,"reported_epoch":65,"state":"active+clean","last_fresh":"2026-02-24T07:51:00.746844+0000","last_change":"2026-02-24T07:50:03.153499+0000","last_active":"2026-02-24T07:51:00.746844+0000","last_peered":"2026-02-24T07:51:00.746844+0000","last_clean":"2026-02-24T07:51:00.746844+0000","last_became_active":"2026-02-24T07:48:57.904376+0000","last_became_peered":"2026-02-24T07:48:57.904376+0000","last_unstale":"2026-02-24T07:51:00.746844+0000","last_undegraded":"2026-02-24T07:51:00.746844+0000","last_fullsized":"2026-02-24T07:51:00.746844+0000","mapping_epoch":55,"log_start":"65'57909","ondisk_log_start":"65'57909","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"61'30163","last_scrub_stamp":"2026-02-24T07:50:03.153454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:50:03.153454+0000","objects_scrubbed":29681,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:51:10.266136+0000","scrub_duration":2916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1503133696,"num_objects":22936,"num_object_clones":0,"num_object_copies":68808,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22936,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":57912,"num_write_kb":2587136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.7","version":"65'58023","reported_seq":58387,"reported_epoch":65,"state":"active+clean","last_fresh":"2026-02-24T07:51:00.834217+0000","last_change":"2026-02-24T07:50:11.227608+0000","last_active":"2026-02-24T07:51:00.834217+0000","last_peered":"2026-02-24T07:51:00.834217+0000","last_clean":"2026-02-24T07:51:00.834217+0000","last_became_active":"2026-02-24T07:48:57.902484+0000","last_became_peered":"2026-02-24T07:48:57.902484+0000","last_unstale":"2026-02-24T07:51:00.834217+0000","last_undegraded":"2026-02-24T07:51:00.834217+0000","last_fullsized":"2026-02-24T07:51:00.834217+0000","mapping_epoch":55,"log_start":"65'58020","ondisk_log_start":"65'58020","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"61'32990","last_scrub_stamp":"2026-02-24T07:50:11.227573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:50:11.227573+0000","objects_scrubbed":32460,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:51:29.781611+0000","scrub_duration":3701,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1482620928,"num_objects":22623,"num_object_clones":0,"num_object_copies":67869,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22623,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":58023,"num_write_kb":2580672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.6","version":"65'57482","reported_seq":57857,"reported_epoch":65,"state":"active+clean","last_fresh":"2026-02-24T07:51:00.743374+0000","last_change":"2026-02-24T07:50:14.386679+0000","last_active":"2026-02-24T07:51:00.743374+0000","last_peered":"2026-02-24T07:51:00.743374+0000","last_clean":"2026-02-24T07:51:00.743374+0000","last_became_active":"2026-02-24T07:48:57.901565+0000","last_became_peered":"2026-02-24T07:48:57.901565+0000","last_unstale":"2026-02-24T07:51:00.743374+0000","last_undegraded":"2026-02-24T07:51:00.743374+0000","last_fullsized":"2026-02-24T07:51:00.743374+0000","mapping_epoch":55,"log_start":"65'57479","ondisk_log_start":"65'57479","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"61'34244","last_scrub_stamp":"2026-02-24T07:50:14.386625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:50:14.386625+0000","objects_scrubbed":33505,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:51:16.739697+0000","scrub_duration":3790,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1512439808,"num_objects":23078,"num_object_clones":0,"num_object_copies":69234,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23078,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":57482,"num_write_kb":2577920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"65'58446","reported_seq":58835,"reported_epoch":65,"state":"active+clean","last_fresh":"2026-02-24T07:51:00.836036+0000","last_change":"2026-02-24T07:50:16.399238+0000","last_active":"2026-02-24T07:51:00.836036+0000","last_peered":"2026-02-24T07:51:00.836036+0000","last_clean":"2026-02-24T07:51:00.836036+0000","last_became_active":"2026-02-24T07:48:57.901824+0000","last_became_peered":"2026-02-24T07:48:57.901824+0000","last_unstale":"2026-02-24T07:51:00.836036+0000","last_undegraded":"2026-02-24T07:51:00.836036+0000","last_fullsized":"2026-02-24T07:51:00.836036+0000","mapping_epoch":55,"log_start":"65'58443","ondisk_log_start":"65'58443","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"61'35510","last_scrub_stamp":"2026-02-24T07:50:16.399194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:50:16.399194+0000","objects_scrubbed":34844,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:51:44.770701+0000","scrub_duration":2971,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1494876160,"num_objects":22810,"num_object_clones":0,"num_object_copies":68430,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22810,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":58446,"num_write_kb":2600192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[1,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"65'57688","reported_seq":58048,"reported_epoch":65,"state":"active+clean","last_fresh":"2026-02-24T07:51:00.749312+0000","last_change":"2026-02-24T07:50:10.124042+0000","last_active":"2026-02-24T07:51:00.749312+0000","last_peered":"2026-02-24T07:51:00.749312+0000","last_clean":"2026-02-24T07:51:00.749312+0000","last_became_active":"2026-02-24T07:48:57.902224+0000","last_became_peered":"2026-02-24T07:48:57.902224+0000","last_unstale":"2026-02-24T07:51:00.749312+0000","last_undegraded":"2026-02-24T07:51:00.749312+0000","last_fullsized":"2026-02-24T07:51:00.749312+0000","mapping_epoch":55,"log_start":"65'57685","ondisk_log_start":"65'57685","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"61'32620","last_scrub_stamp":"2026-02-24T07:50:10.124005+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:50:10.124005+0000","objects_scrubbed":32078,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:51:12.877892+0000","scrub_duration":3559,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1487011840,"num_objects":22690,"num_object_clones":0,"num_object_copies":68070,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22690,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":57688,"num_write_kb":2572096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"65'58224","reported_seq":58557,"reported_epoch":65,"state":"active+clean","last_fresh":"2026-02-24T07:51:00.836235+0000","last_change":"2026-02-24T07:50:01.398116+0000","last_active":"2026-02-24T07:51:00.836235+0000","last_peered":"2026-02-24T07:51:00.836235+0000","last_clean":"2026-02-24T07:51:00.836235+0000","last_became_active":"2026-02-24T07:48:57.902125+0000","last_became_peered":"2026-02-24T07:48:57.902125+0000","last_unstale":"2026-02-24T07:51:00.836235+0000","last_undegraded":"2026-02-24T07:51:00.836235+0000","last_fullsized":"2026-02-24T07:51:00.836235+0000","mapping_epoch":55,"log_start":"65'58221","ondisk_log_start":"65'58221","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"61'29707","last_scrub_stamp":"2026-02-24T07:50:01.398090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:50:01.398090+0000","objects_scrubbed":29171,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:51:10.566529+0000","scrub_duration":2880,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1505099776,"num_objects":22966,"num_object_clones":0,"num_object_copies":68898,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22966,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":58224,"num_write_kb":2598080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1","version":"65'58254","reported_seq":58694,"reported_epoch":65,"state":"active+clean","last_fresh":"2026-02-24T07:51:00.726507+0000","last_change":"2026-02-24T07:50:29.889880+0000","last_active":"2026-02-24T07:51:00.726507+0000","last_peered":"2026-02-24T07:51:00.726507+0000","last_clean":"2026-02-24T07:51:00.726507+0000","last_became_active":"2026-02-24T07:48:57.903439+0000","last_became_peered":"2026-02-24T07:48:57.903439+0000","last_unstale":"2026-02-24T07:51:00.726507+0000","last_undegraded":"2026-02-24T07:51:00.726507+0000","last_fullsized":"2026-02-24T07:51:00.726507+0000","mapping_epoch":55,"log_start":"65'58251","ondisk_log_start":"65'58251","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"61'41105","last_scrub_stamp":"2026-02-24T07:50:29.889816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:50:29.889816+0000","objects_scrubbed":40127,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:51:50.788414+0000","scrub_duration":4409,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1517551616,"num_objects":23156,"num_object_clones":0,"num_object_copies":69468,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23156,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":58254,"num_write_kb":2605120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.0","version":"65'58137","reported_seq":58574,"reported_epoch":65,"state":"active+clean","last_fresh":"2026-02-24T07:51:00.747851+0000","last_change":"2026-02-24T07:50:29.387323+0000","last_active":"2026-02-24T07:51:00.747851+0000","last_peered":"2026-02-24T07:51:00.747851+0000","last_clean":"2026-02-24T07:51:00.747851+0000","last_became_active":"2026-02-24T07:48:57.902565+0000","last_became_peered":"2026-02-24T07:48:57.902565+0000","last_unstale":"2026-02-24T07:51:00.747851+0000","last_undegraded":"2026-02-24T07:51:00.747851+0000","last_fullsized":"2026-02-24T07:51:00.747851+0000","mapping_epoch":55,"log_start":"65'58133","ondisk_log_start":"65'58133","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"61'40736","last_scrub_stamp":"2026-02-24T07:50:29.387206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:50:29.387206+0000","objects_scrubbed":39946,"log_size":4,"log_dups_size":3000,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:51:59.173732+0000","scrub_duration":3834,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1507655680,"num_objects":23005,"num_object_clones":0,"num_object_copies":69015,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23005,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":58137,"num_write_kb":2596544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,3],"acting":[0,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.5","version":"65'58018","reported_seq":58361,"reported_epoch":65,"state":"active+clean","last_fresh":"2026-02-24T07:51:00.741574+0000","last_change":"2026-02-24T07:50:04.821914+0000","last_active":"2026-02-24T07:51:00.741574+0000","last_peered":"2026-02-24T07:51:00.741574+0000","last_clean":"2026-02-24T07:51:00.741574+0000","last_became_active":"2026-02-24T07:48:57.905127+0000","last_became_peered":"2026-02-24T07:48:57.905127+0000","last_unstale":"2026-02-24T07:51:00.741574+0000","last_undegraded":"2026-02-24T07:51:00.741574+0000","last_fullsized":"2026-02-24T07:51:00.741574+0000","mapping_epoch":55,"log_start":"65'58015","ondisk_log_start":"65'58015","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"61'30833","last_scrub_stamp":"2026-02-24T07:50:04.821861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:50:04.821861+0000","objects_scrubbed":30132,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:51:15.928207+0000","scrub_duration":3578,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1512308736,"num_objects":23076,"num_object_clones":0,"num_object_copies":69228,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23076,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":58018,"num_write_kb":2595008,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"29'99","reported_seq":302,"reported_epoch":65,"state":"active+clean","last_fresh":"2026-02-24T07:50:55.895043+0000","last_change":"2026-02-24T07:50:30.401963+0000","last_active":"2026-02-24T07:50:55.895043+0000","last_peered":"2026-02-24T07:50:55.895043+0000","last_clean":"2026-02-24T07:50:55.895043+0000","last_became_active":"2026-02-24T07:42:54.186719+0000","last_became_peered":"2026-02-24T07:42:54.186719+0000","last_unstale":"2026-02-24T07:50:55.895043+0000","last_undegraded":"2026-02-24T07:50:55.895043+0000","last_fullsized":"2026-02-24T07:50:55.895043+0000","mapping_epoch":27,"log_start":"29'96","ondisk_log_start":"29'96","created":20,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'99","last_scrub_stamp":"2026-02-24T07:50:30.401929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:42:43.640776+0000","last_clean_scrub_stamp":"2026-02-24T07:50:30.401929+0000","objects_scrubbed":2,"log_size":3,"log_dups_size":96,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:51:52.990727+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":240,"num_write":135,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":16,"stat_sum":{"num_bytes":24036048920,"num_objects":366762,"num_object_clones":0,"num_object_copies":1100286,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":366762,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":927974,"num_write_kb":41431489,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":72116350976,"data_stored":72116338760,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":49,"ondisk_log_size":49,"up":48,"acting":48,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":240,"num_write":135,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1078,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":3,"up_from":25,"seq":107374182500,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":19514044,"kb_used_data":17925096,"kb_used_omap":15,"kb_used_meta":1588912,"kb_avail":714489156,"statfs":{"total":751619276800,"available":731636895744,"internally_reserved":0,"allocated":18355298304,"data_stored":18354849232,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16354,"internal_metadata":1627045918},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":19,"seq":81604378726,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":22515388,"kb_used_data":20681640,"kb_used_omap":16,"kb_used_meta":1833647,"kb_avail":711487812,"statfs":{"total":751619276800,"available":728563519488,"internally_reserved":0,"allocated":21177999360,"data_stored":21177550288,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":17329,"internal_metadata":1877654607},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542247,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":16165428,"kb_used_data":14812448,"kb_used_omap":18,"kb_used_meta":1352941,"kb_avail":717837772,"statfs":{"total":751619276800,"available":735065878528,"internally_reserved":0,"allocated":15167946752,"data_stored":15167505304,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":18733,"internal_metadata":1385412307},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705770,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":19336188,"kb_used_data":17753512,"kb_used_omap":14,"kb_used_meta":1582641,"kb_avail":714667012,"statfs":{"total":751619276800,"available":731819020288,"internally_reserved":0,"allocated":18179596288,"data_stored":18179147216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":14874,"internal_metadata":1620624870},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1078,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":18048421888,"data_stored":18048417816,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":14976614400,"data_stored":14976614400,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":21047349248,"data_stored":21047345176,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":18043965440,"data_stored":18043961368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-24T07:51:04.871 INFO:tasks.cephadm.ceph_manager.ceph:increase pool size by 10 2026-02-24T07:51:04.871 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph osd pool set .mgr pg_num 271 2026-02-24T07:51:04.991 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:51:04.988+0000 7f7eb4260640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:51:04.991 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:51:04.988+0000 7f7eb4260640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:51:05.005 INFO:teuthology.orchestra.run.trial035.stdout:osd.0: {} 2026-02-24T07:51:05.006 INFO:teuthology.orchestra.run.trial035.stderr:osd.0: osd_enable_op_tracker = '' 2026-02-24T07:51:05.006 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:51:05.020 INFO:teuthology.orchestra.run.trial035.stdout:osd.1: {} 2026-02-24T07:51:05.020 INFO:teuthology.orchestra.run.trial035.stderr:osd.1: osd_enable_op_tracker = '' 2026-02-24T07:51:05.032 INFO:teuthology.orchestra.run.trial035.stdout:osd.2: {} 2026-02-24T07:51:05.032 INFO:teuthology.orchestra.run.trial035.stderr:osd.2: osd_enable_op_tracker = '' 2026-02-24T07:51:05.045 INFO:teuthology.orchestra.run.trial035.stdout:osd.3: {} 2026-02-24T07:51:05.045 INFO:teuthology.orchestra.run.trial035.stderr:osd.3: osd_enable_op_tracker = '' 2026-02-24T07:51:05.123 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:51:04 trial035 ceph-mon[39284]: from='client.15061 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2026-02-24T07:51:05.416 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:51:05.414+0000 7ff84fba7640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:51:05.416 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:51:05.414+0000 7ff84fba7640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:51:05.558 INFO:teuthology.orchestra.run.trial035.stderr:Error EPERM: crimson_allow_pg_split is false; pg_num increase denied for crimson pool 2026-02-24T07:51:05.698 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-24T07:51:05.954 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:51:05 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/1337102961' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2026-02-24T07:51:05.954 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:51:05 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/1424002813' entity='client.admin' cmd={"prefix": "osd pool set", "pool": ".mgr", "var": "pg_num", "val": "271"} : dispatch 2026-02-24T07:51:05.954 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:51:05 trial035 ceph-mon[39284]: pgmap v282: 17 pgs: 17 active+clean; 22 GiB data, 74 GiB used, 2.7 TiB / 2.7 TiB avail; 0 B/s wr, 8.44k op/s 2026-02-24T07:51:07.356 INFO:journalctl@ceph.prometheus.a.trial091.stdout:Feb 24 07:51:06 trial091 ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-prometheus-a[66007]: time=2026-02-24T07:51:06.973Z level=ERROR source=scrape.go:1630 msg="Failed to determine correct type of scrape target." component="scrape manager" scrape_pool=ceph target=http://trial035:9283/metrics content_type="text/html;charset=utf-8" fallback_media_type="" err="received unsupported Content-Type \"text/html;charset=utf-8\" and no fallback_scrape_protocol specified for target" 2026-02-24T07:51:07.454 INFO:journalctl@ceph.mgr.x.trial035.stdout:Feb 24 07:51:06 trial035 ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-mgr-x[39621]: ::ffff:10.20.193.91 - - [24/Feb/2026:07:51:06] "GET /metrics HTTP/1.1" 200 4227 "" "Prometheus/3.6.0" 2026-02-24T07:51:07.689 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2026-02-24T07:51:07.830 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:51:07.946 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:51:07.944+0000 7f3821bfe640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:51:07.947 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:51:07.944+0000 7f3821bfe640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:51:07.962 INFO:teuthology.orchestra.run.trial035.stdout:osd.0: {} 2026-02-24T07:51:07.962 INFO:teuthology.orchestra.run.trial035.stderr:osd.0: osd_enable_op_tracker = '' 2026-02-24T07:51:07.976 INFO:teuthology.orchestra.run.trial035.stdout:osd.1: {} 2026-02-24T07:51:07.976 INFO:teuthology.orchestra.run.trial035.stderr:osd.1: osd_enable_op_tracker = '' 2026-02-24T07:51:07.992 INFO:teuthology.orchestra.run.trial035.stdout:osd.2: {} 2026-02-24T07:51:07.993 INFO:teuthology.orchestra.run.trial035.stderr:osd.2: osd_enable_op_tracker = '' 2026-02-24T07:51:08.005 INFO:teuthology.orchestra.run.trial035.stdout:osd.3: {} 2026-02-24T07:51:08.005 INFO:teuthology.orchestra.run.trial035.stderr:osd.3: osd_enable_op_tracker = '' 2026-02-24T07:51:09.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:51:08 trial035 ceph-mon[39284]: pgmap v283: 17 pgs: 17 active+clean; 20 GiB data, 66 GiB used, 2.7 TiB / 2.7 TiB avail; 0 B/s wr, 6.82k op/s 2026-02-24T07:51:09.205 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:51:08 trial035 ceph-mon[39284]: from='mgr.14284 10.20.193.35:0/3681373222' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2026-02-24T07:51:09.205 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:51:08 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/1137001401' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2026-02-24T07:51:09.205 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:51:08 trial035 ceph-mon[39284]: from='mgr.14284 10.20.193.35:0/3681373222' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-24T07:51:09.205 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:51:08 trial035 ceph-mon[39284]: from='mgr.14284 10.20.193.35:0/3681373222' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2026-02-24T07:51:09.205 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:51:08 trial035 ceph-mon[39284]: from='mgr.14284 10.20.193.35:0/3681373222' entity='mgr.x' 2026-02-24T07:51:09.205 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:51:08 trial035 ceph-mon[39284]: from='mgr.14284 10.20.193.35:0/3681373222' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2026-02-24T07:51:10.053 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2026-02-24T07:51:10.201 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:51:10.616 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:51:10.553+0000 7f919db2c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:51:10.616 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:51:10.554+0000 7f919db2c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:51:10.616 INFO:teuthology.orchestra.run.trial035.stderr:osd.0: osd_enable_op_tracker = '' 2026-02-24T07:51:10.616 INFO:teuthology.orchestra.run.trial035.stderr:osd.1: osd_enable_op_tracker = '' 2026-02-24T07:51:10.617 INFO:teuthology.orchestra.run.trial035.stdout:osd.0: {} 2026-02-24T07:51:10.617 INFO:teuthology.orchestra.run.trial035.stdout:osd.1: {} 2026-02-24T07:51:10.617 INFO:teuthology.orchestra.run.trial035.stdout:osd.2: {} 2026-02-24T07:51:10.617 INFO:teuthology.orchestra.run.trial035.stdout:osd.3: {} 2026-02-24T07:51:10.617 INFO:teuthology.orchestra.run.trial035.stderr:osd.2: osd_enable_op_tracker = '' 2026-02-24T07:51:10.618 INFO:teuthology.orchestra.run.trial035.stderr:osd.3: osd_enable_op_tracker = '' 2026-02-24T07:51:10.699 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2026-02-24T07:51:10.699 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2026-02-24T07:51:10.700 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph pg dump --format=json 2026-02-24T07:51:10.954 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:51:10 trial035 ceph-mon[39284]: pgmap v284: 17 pgs: 17 active+clean; 20 GiB data, 66 GiB used, 2.7 TiB / 2.7 TiB avail; 0 B/s wr, 6.72k op/s 2026-02-24T07:51:10.954 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:51:10 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/2574141785' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2026-02-24T07:51:11.156 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:51:11.589 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:51:11.581+0000 7f356f392640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:51:11.589 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:51:11.581+0000 7f356f392640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:51:11.725 INFO:teuthology.orchestra.run.trial035.stdout: 2026-02-24T07:51:11.726 INFO:teuthology.orchestra.run.trial035.stderr:dumped all 2026-02-24T07:51:12.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:51:11 trial035 ceph-mon[39284]: 4.8 scrub starts 2026-02-24T07:51:12.340 INFO:teuthology.orchestra.run.trial035.stdout:{"pg_ready":true,"pg_map":{"version":284,"stamp":"2026-02-24T07:51:09.723548+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":21360673336,"num_objects":325932,"num_object_clones":0,"num_object_copies":977796,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":325932,"num_whiteouts":0,"num_read":137,"num_read_kb":241,"num_write":968941,"num_write_kb":41433639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":51,"ondisk_log_size":51,"up":51,"acting":51,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":51,"num_osds":4,"num_per_pool_osds":4,"num_per_pool_omap_osds":4,"kb":2936012800,"kb_used":68740712,"kb_used_data":63212504,"kb_used_omap":62,"kb_used_meta":5528001,"kb_avail":2867272088,"statfs":{"total":3006477107200,"available":2936086618112,"internally_reserved":0,"allocated":64729604096,"data_stored":64727815432,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":63750,"internal_metadata":5660673786},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[4,1],"upper_bound":4},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-5286854656,"num_objects":-80671,"num_object_clones":0,"num_object_copies":-242013,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-80671,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":80671,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001665"},"pg_stats":[{"pgid":"4.f","version":"65'60828","reported_seq":61169,"reported_epoch":65,"state":"active+clean","last_fresh":"2026-02-24T07:51:05.839737+0000","last_change":"2026-02-24T07:50:03.071459+0000","last_active":"2026-02-24T07:51:05.839737+0000","last_peered":"2026-02-24T07:51:05.839737+0000","last_clean":"2026-02-24T07:51:05.839737+0000","last_became_active":"2026-02-24T07:48:57.902313+0000","last_became_peered":"2026-02-24T07:48:57.902313+0000","last_unstale":"2026-02-24T07:51:05.839737+0000","last_undegraded":"2026-02-24T07:51:05.839737+0000","last_fullsized":"2026-02-24T07:51:05.839737+0000","mapping_epoch":55,"log_start":"65'60825","ondisk_log_start":"65'60825","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"61'30419","last_scrub_stamp":"2026-02-24T07:50:03.071380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:50:03.071380+0000","objects_scrubbed":29975,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:51:16.123453+0000","scrub_duration":2568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1344274432,"num_objects":20512,"num_object_clones":0,"num_object_copies":61536,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20512,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":60828,"num_write_kb":2602880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"65'60778","reported_seq":61186,"reported_epoch":65,"state":"active+clean","last_fresh":"2026-02-24T07:51:05.730766+0000","last_change":"2026-02-24T07:50:21.065676+0000","last_active":"2026-02-24T07:51:05.730766+0000","last_peered":"2026-02-24T07:51:05.730766+0000","last_clean":"2026-02-24T07:51:05.730766+0000","last_became_active":"2026-02-24T07:48:57.903584+0000","last_became_peered":"2026-02-24T07:48:57.903584+0000","last_unstale":"2026-02-24T07:51:05.730766+0000","last_undegraded":"2026-02-24T07:51:05.730766+0000","last_fullsized":"2026-02-24T07:51:05.730766+0000","mapping_epoch":55,"log_start":"65'60775","ondisk_log_start":"65'60775","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"61'37487","last_scrub_stamp":"2026-02-24T07:50:21.065638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:50:21.065638+0000","objects_scrubbed":36748,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:51:31.991874+0000","scrub_duration":3643,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1336934400,"num_objects":20400,"num_object_clones":0,"num_object_copies":61200,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20400,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":60778,"num_write_kb":2597696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.d","version":"65'60334","reported_seq":60685,"reported_epoch":65,"state":"active+clean","last_fresh":"2026-02-24T07:51:05.744623+0000","last_change":"2026-02-24T07:50:06.701973+0000","last_active":"2026-02-24T07:51:05.744623+0000","last_peered":"2026-02-24T07:51:05.744623+0000","last_clean":"2026-02-24T07:51:05.744623+0000","last_became_active":"2026-02-24T07:48:57.905031+0000","last_became_peered":"2026-02-24T07:48:57.905031+0000","last_unstale":"2026-02-24T07:51:05.744623+0000","last_undegraded":"2026-02-24T07:51:05.744623+0000","last_fullsized":"2026-02-24T07:51:05.744623+0000","mapping_epoch":55,"log_start":"65'60331","ondisk_log_start":"65'60331","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"61'31648","last_scrub_stamp":"2026-02-24T07:50:06.701883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:50:06.701883+0000","objects_scrubbed":30990,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:51:19.224966+0000","scrub_duration":3493,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1340669976,"num_objects":20458,"num_object_clones":0,"num_object_copies":61374,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20458,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":60334,"num_write_kb":2585281,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"65'60167","reported_seq":60586,"reported_epoch":65,"state":"active+clean","last_fresh":"2026-02-24T07:51:05.744477+0000","last_change":"2026-02-24T07:50:24.798674+0000","last_active":"2026-02-24T07:51:05.744477+0000","last_peered":"2026-02-24T07:51:05.744477+0000","last_clean":"2026-02-24T07:51:05.744477+0000","last_became_active":"2026-02-24T07:48:57.904683+0000","last_became_peered":"2026-02-24T07:48:57.904683+0000","last_unstale":"2026-02-24T07:51:05.744477+0000","last_undegraded":"2026-02-24T07:51:05.744477+0000","last_fullsized":"2026-02-24T07:51:05.744477+0000","mapping_epoch":55,"log_start":"65'60164","ondisk_log_start":"65'60164","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"61'38696","last_scrub_stamp":"2026-02-24T07:50:24.798629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:50:24.798629+0000","objects_scrubbed":37979,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:51:26.521338+0000","scrub_duration":3612,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1332019200,"num_objects":20325,"num_object_clones":0,"num_object_copies":60975,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20325,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":60167,"num_write_kb":2575744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.b","version":"65'60262","reported_seq":60691,"reported_epoch":65,"state":"active+clean","last_fresh":"2026-02-24T07:51:05.747798+0000","last_change":"2026-02-24T07:50:27.267008+0000","last_active":"2026-02-24T07:51:05.747798+0000","last_peered":"2026-02-24T07:51:05.747798+0000","last_clean":"2026-02-24T07:51:05.747798+0000","last_became_active":"2026-02-24T07:48:57.901700+0000","last_became_peered":"2026-02-24T07:48:57.901700+0000","last_unstale":"2026-02-24T07:51:05.747798+0000","last_undegraded":"2026-02-24T07:51:05.747798+0000","last_fullsized":"2026-02-24T07:51:05.747798+0000","mapping_epoch":55,"log_start":"65'60259","ondisk_log_start":"65'60259","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"61'39639","last_scrub_stamp":"2026-02-24T07:50:27.266943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:50:27.266943+0000","objects_scrubbed":38982,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:51:34.346751+0000","scrub_duration":3744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1336410112,"num_objects":20392,"num_object_clones":0,"num_object_copies":61176,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20392,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":60262,"num_write_kb":2580928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.a","version":"65'60700","reported_seq":61133,"reported_epoch":65,"state":"active+clean","last_fresh":"2026-02-24T07:51:05.732677+0000","last_change":"2026-02-24T07:50:33.043786+0000","last_active":"2026-02-24T07:51:05.732677+0000","last_peered":"2026-02-24T07:51:05.732677+0000","last_clean":"2026-02-24T07:51:05.732677+0000","last_became_active":"2026-02-24T07:48:57.903228+0000","last_became_peered":"2026-02-24T07:48:57.903228+0000","last_unstale":"2026-02-24T07:51:05.732677+0000","last_undegraded":"2026-02-24T07:51:05.732677+0000","last_fullsized":"2026-02-24T07:51:05.732677+0000","mapping_epoch":55,"log_start":"65'60697","ondisk_log_start":"65'60697","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"61'42539","last_scrub_stamp":"2026-02-24T07:50:33.043726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:50:33.043726+0000","objects_scrubbed":39246,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:51:51.239665+0000","scrub_duration":3547,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1322909696,"num_objects":20186,"num_object_clones":0,"num_object_copies":60558,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20186,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":60700,"num_write_kb":2588352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"65'60586","reported_seq":60959,"reported_epoch":65,"state":"active+clean","last_fresh":"2026-02-24T07:51:05.735288+0000","last_change":"2026-02-24T07:50:13.165492+0000","last_active":"2026-02-24T07:51:05.735288+0000","last_peered":"2026-02-24T07:51:05.735288+0000","last_clean":"2026-02-24T07:51:05.735288+0000","last_became_active":"2026-02-24T07:48:57.901730+0000","last_became_peered":"2026-02-24T07:48:57.901730+0000","last_unstale":"2026-02-24T07:51:05.735288+0000","last_undegraded":"2026-02-24T07:51:05.735288+0000","last_fullsized":"2026-02-24T07:51:05.735288+0000","mapping_epoch":55,"log_start":"65'60583","ondisk_log_start":"65'60583","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"61'33943","last_scrub_stamp":"2026-02-24T07:50:13.165459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:50:13.165459+0000","objects_scrubbed":33253,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:51:33.820708+0000","scrub_duration":3776,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1329332224,"num_objects":20284,"num_object_clones":0,"num_object_copies":60852,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20284,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":60586,"num_write_kb":2587840,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"65'60493","reported_seq":60830,"reported_epoch":65,"state":"active+clean","last_fresh":"2026-02-24T07:51:05.744329+0000","last_change":"2026-02-24T07:50:03.153499+0000","last_active":"2026-02-24T07:51:05.744329+0000","last_peered":"2026-02-24T07:51:05.744329+0000","last_clean":"2026-02-24T07:51:05.744329+0000","last_became_active":"2026-02-24T07:48:57.904376+0000","last_became_peered":"2026-02-24T07:48:57.904376+0000","last_unstale":"2026-02-24T07:51:05.744329+0000","last_undegraded":"2026-02-24T07:51:05.744329+0000","last_fullsized":"2026-02-24T07:51:05.744329+0000","mapping_epoch":55,"log_start":"65'60490","ondisk_log_start":"65'60490","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"61'30163","last_scrub_stamp":"2026-02-24T07:50:03.153454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:50:03.153454+0000","objects_scrubbed":29681,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:51:10.266136+0000","scrub_duration":2916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1333985280,"num_objects":20355,"num_object_clones":0,"num_object_copies":61065,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20355,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":60493,"num_write_kb":2587136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.7","version":"65'60525","reported_seq":60889,"reported_epoch":65,"state":"active+clean","last_fresh":"2026-02-24T07:51:05.838519+0000","last_change":"2026-02-24T07:50:11.227608+0000","last_active":"2026-02-24T07:51:05.838519+0000","last_peered":"2026-02-24T07:51:05.838519+0000","last_clean":"2026-02-24T07:51:05.838519+0000","last_became_active":"2026-02-24T07:48:57.902484+0000","last_became_peered":"2026-02-24T07:48:57.902484+0000","last_unstale":"2026-02-24T07:51:05.838519+0000","last_undegraded":"2026-02-24T07:51:05.838519+0000","last_fullsized":"2026-02-24T07:51:05.838519+0000","mapping_epoch":55,"log_start":"65'60522","ondisk_log_start":"65'60522","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"61'32990","last_scrub_stamp":"2026-02-24T07:50:11.227573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:50:11.227573+0000","objects_scrubbed":32460,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:51:29.781611+0000","scrub_duration":3701,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1318649856,"num_objects":20121,"num_object_clones":0,"num_object_copies":60363,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20121,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":60525,"num_write_kb":2580672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.6","version":"65'60099","reported_seq":60474,"reported_epoch":65,"state":"active+clean","last_fresh":"2026-02-24T07:51:05.749799+0000","last_change":"2026-02-24T07:50:14.386679+0000","last_active":"2026-02-24T07:51:05.749799+0000","last_peered":"2026-02-24T07:51:05.749799+0000","last_clean":"2026-02-24T07:51:05.749799+0000","last_became_active":"2026-02-24T07:48:57.901565+0000","last_became_peered":"2026-02-24T07:48:57.901565+0000","last_unstale":"2026-02-24T07:51:05.749799+0000","last_undegraded":"2026-02-24T07:51:05.749799+0000","last_fullsized":"2026-02-24T07:51:05.749799+0000","mapping_epoch":55,"log_start":"65'60096","ondisk_log_start":"65'60096","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"61'34244","last_scrub_stamp":"2026-02-24T07:50:14.386625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:50:14.386625+0000","objects_scrubbed":33505,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:51:16.739697+0000","scrub_duration":3790,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1340932096,"num_objects":20461,"num_object_clones":0,"num_object_copies":61383,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20461,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":60099,"num_write_kb":2577920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"65'60917","reported_seq":61306,"reported_epoch":65,"state":"active+clean","last_fresh":"2026-02-24T07:51:05.832570+0000","last_change":"2026-02-24T07:50:16.399238+0000","last_active":"2026-02-24T07:51:05.832570+0000","last_peered":"2026-02-24T07:51:05.832570+0000","last_clean":"2026-02-24T07:51:05.832570+0000","last_became_active":"2026-02-24T07:48:57.901824+0000","last_became_peered":"2026-02-24T07:48:57.901824+0000","last_unstale":"2026-02-24T07:51:05.832570+0000","last_undegraded":"2026-02-24T07:51:05.832570+0000","last_fullsized":"2026-02-24T07:51:05.832570+0000","mapping_epoch":55,"log_start":"65'60914","ondisk_log_start":"65'60914","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"61'35510","last_scrub_stamp":"2026-02-24T07:50:16.399194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:50:16.399194+0000","objects_scrubbed":34844,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:51:44.770701+0000","scrub_duration":2971,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1332936704,"num_objects":20339,"num_object_clones":0,"num_object_copies":61017,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20339,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":60917,"num_write_kb":2600192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[1,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"65'60234","reported_seq":60595,"reported_epoch":65,"state":"active+clean","last_fresh":"2026-02-24T07:51:05.749282+0000","last_change":"2026-02-24T07:50:10.124042+0000","last_active":"2026-02-24T07:51:05.749282+0000","last_peered":"2026-02-24T07:51:05.749282+0000","last_clean":"2026-02-24T07:51:05.749282+0000","last_became_active":"2026-02-24T07:48:57.902224+0000","last_became_peered":"2026-02-24T07:48:57.902224+0000","last_unstale":"2026-02-24T07:51:05.749282+0000","last_undegraded":"2026-02-24T07:51:05.749282+0000","last_fullsized":"2026-02-24T07:51:05.749282+0000","mapping_epoch":55,"log_start":"65'60231","ondisk_log_start":"65'60231","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"61'32620","last_scrub_stamp":"2026-02-24T07:50:10.124005+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:50:10.124005+0000","objects_scrubbed":32078,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:51:12.877892+0000","scrub_duration":3559,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1320157184,"num_objects":20144,"num_object_clones":0,"num_object_copies":60432,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20144,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":60234,"num_write_kb":2572096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"65'60719","reported_seq":61052,"reported_epoch":65,"state":"active+clean","last_fresh":"2026-02-24T07:51:05.834223+0000","last_change":"2026-02-24T07:50:01.398116+0000","last_active":"2026-02-24T07:51:05.834223+0000","last_peered":"2026-02-24T07:51:05.834223+0000","last_clean":"2026-02-24T07:51:05.834223+0000","last_became_active":"2026-02-24T07:48:57.902125+0000","last_became_peered":"2026-02-24T07:48:57.902125+0000","last_unstale":"2026-02-24T07:51:05.834223+0000","last_undegraded":"2026-02-24T07:51:05.834223+0000","last_fullsized":"2026-02-24T07:51:05.834223+0000","mapping_epoch":55,"log_start":"65'60716","ondisk_log_start":"65'60716","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"61'29707","last_scrub_stamp":"2026-02-24T07:50:01.398090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:50:01.398090+0000","objects_scrubbed":29171,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:51:10.566529+0000","scrub_duration":2880,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1341587456,"num_objects":20471,"num_object_clones":0,"num_object_copies":61413,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20471,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":60719,"num_write_kb":2598080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1","version":"65'60865","reported_seq":61305,"reported_epoch":65,"state":"active+clean","last_fresh":"2026-02-24T07:51:05.734406+0000","last_change":"2026-02-24T07:50:29.889880+0000","last_active":"2026-02-24T07:51:05.734406+0000","last_peered":"2026-02-24T07:51:05.734406+0000","last_clean":"2026-02-24T07:51:05.734406+0000","last_became_active":"2026-02-24T07:48:57.903439+0000","last_became_peered":"2026-02-24T07:48:57.903439+0000","last_unstale":"2026-02-24T07:51:05.734406+0000","last_undegraded":"2026-02-24T07:51:05.734406+0000","last_fullsized":"2026-02-24T07:51:05.734406+0000","mapping_epoch":55,"log_start":"65'60862","ondisk_log_start":"65'60862","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"61'41105","last_scrub_stamp":"2026-02-24T07:50:29.889816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:50:29.889816+0000","objects_scrubbed":40127,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:51:50.788414+0000","scrub_duration":4409,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1346437120,"num_objects":20545,"num_object_clones":0,"num_object_copies":61635,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20545,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":60865,"num_write_kb":2605120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.0","version":"65'60666","reported_seq":61104,"reported_epoch":65,"state":"active+clean","last_fresh":"2026-02-24T07:51:05.748006+0000","last_change":"2026-02-24T07:50:29.387323+0000","last_active":"2026-02-24T07:51:05.748006+0000","last_peered":"2026-02-24T07:51:05.748006+0000","last_clean":"2026-02-24T07:51:05.748006+0000","last_became_active":"2026-02-24T07:48:57.902565+0000","last_became_peered":"2026-02-24T07:48:57.902565+0000","last_unstale":"2026-02-24T07:51:05.748006+0000","last_undegraded":"2026-02-24T07:51:05.748006+0000","last_fullsized":"2026-02-24T07:51:05.748006+0000","mapping_epoch":55,"log_start":"65'60663","ondisk_log_start":"65'60663","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"61'40736","last_scrub_stamp":"2026-02-24T07:50:29.387206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:50:29.387206+0000","objects_scrubbed":39946,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:51:59.173732+0000","scrub_duration":3834,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1341915136,"num_objects":20476,"num_object_clones":0,"num_object_copies":61428,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20476,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":60666,"num_write_kb":2596544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,3],"acting":[0,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.5","version":"65'60633","reported_seq":60977,"reported_epoch":65,"state":"active+clean","last_fresh":"2026-02-24T07:51:05.749839+0000","last_change":"2026-02-24T07:50:04.821914+0000","last_active":"2026-02-24T07:51:05.749839+0000","last_peered":"2026-02-24T07:51:05.749839+0000","last_clean":"2026-02-24T07:51:05.749839+0000","last_became_active":"2026-02-24T07:48:57.905127+0000","last_became_peered":"2026-02-24T07:48:57.905127+0000","last_unstale":"2026-02-24T07:51:05.749839+0000","last_undegraded":"2026-02-24T07:51:05.749839+0000","last_fullsized":"2026-02-24T07:51:05.749839+0000","mapping_epoch":55,"log_start":"65'60630","ondisk_log_start":"65'60630","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"61'30833","last_scrub_stamp":"2026-02-24T07:50:04.821861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:50:04.821861+0000","objects_scrubbed":30132,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:51:15.928207+0000","scrub_duration":3578,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1340932096,"num_objects":20461,"num_object_clones":0,"num_object_copies":61383,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20461,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":60633,"num_write_kb":2595008,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"29'99","reported_seq":302,"reported_epoch":65,"state":"active+clean","last_fresh":"2026-02-24T07:50:55.895043+0000","last_change":"2026-02-24T07:50:30.401963+0000","last_active":"2026-02-24T07:50:55.895043+0000","last_peered":"2026-02-24T07:50:55.895043+0000","last_clean":"2026-02-24T07:50:55.895043+0000","last_became_active":"2026-02-24T07:42:54.186719+0000","last_became_peered":"2026-02-24T07:42:54.186719+0000","last_unstale":"2026-02-24T07:50:55.895043+0000","last_undegraded":"2026-02-24T07:50:55.895043+0000","last_fullsized":"2026-02-24T07:50:55.895043+0000","mapping_epoch":27,"log_start":"29'96","ondisk_log_start":"29'96","created":20,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'99","last_scrub_stamp":"2026-02-24T07:50:30.401929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:42:43.640776+0000","last_clean_scrub_stamp":"2026-02-24T07:50:30.401929+0000","objects_scrubbed":2,"log_size":3,"log_dups_size":96,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:51:52.990727+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":240,"num_write":135,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":16,"stat_sum":{"num_bytes":21360082968,"num_objects":325930,"num_object_clones":0,"num_object_copies":977790,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":325930,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":968806,"num_write_kb":41431489,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":64088584192,"data_stored":64088571976,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":48,"ondisk_log_size":48,"up":48,"acting":48,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":240,"num_write":135,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1078,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":3,"up_from":25,"seq":107374182501,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":17557756,"kb_used_data":15914664,"kb_used_omap":15,"kb_used_meta":1643056,"kb_avail":716445444,"statfs":{"total":751619276800,"available":733640134656,"internally_reserved":0,"allocated":16296615936,"data_stored":16296166864,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16354,"internal_metadata":1682489374},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":19,"seq":81604378727,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":19157500,"kb_used_data":18370472,"kb_used_omap":12,"kb_used_meta":786931,"kb_avail":714845700,"statfs":{"total":751619276800,"available":732001996800,"internally_reserved":0,"allocated":18811363328,"data_stored":18810914256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13137,"internal_metadata":805817519},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,1],"upper_bound":4},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542248,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":14605236,"kb_used_data":13143968,"kb_used_omap":18,"kb_used_meta":1461229,"kb_avail":719397964,"statfs":{"total":751619276800,"available":736663515136,"internally_reserved":0,"allocated":13459423232,"data_stored":13458981784,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":18733,"internal_metadata":1496299219},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705771,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":17420220,"kb_used_data":15783400,"kb_used_omap":15,"kb_used_meta":1636784,"kb_avail":716582980,"statfs":{"total":751619276800,"available":733780971520,"internally_reserved":0,"allocated":16162201600,"data_stored":16161752528,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":15526,"internal_metadata":1676067674},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1078,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16034107392,"data_stored":16034103320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":13312589824,"data_stored":13312589824,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":18697818112,"data_stored":18697814040,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16044068864,"data_stored":16044064792,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-24T07:51:13.159 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2026-02-24T07:51:13.183 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:51:12 trial035 ceph-mon[39284]: 4.2 scrub starts 2026-02-24T07:51:13.183 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:51:12 trial035 ceph-mon[39284]: from='client.15071 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2026-02-24T07:51:13.183 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:51:12 trial035 ceph-mon[39284]: pgmap v285: 17 pgs: 17 active+clean; 17 GiB data, 57 GiB used, 2.7 TiB / 2.7 TiB avail; 0 B/s wr, 10.48k op/s 2026-02-24T07:51:13.300 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:51:13.428 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:51:13.427+0000 7f53e1c41640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:51:13.428 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:51:13.427+0000 7f53e1c41640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:51:13.443 INFO:teuthology.orchestra.run.trial035.stdout:osd.0: {} 2026-02-24T07:51:13.443 INFO:teuthology.orchestra.run.trial035.stderr:osd.0: osd_enable_op_tracker = '' 2026-02-24T07:51:13.460 INFO:teuthology.orchestra.run.trial035.stdout:osd.1: {} 2026-02-24T07:51:13.460 INFO:teuthology.orchestra.run.trial035.stderr:osd.1: osd_enable_op_tracker = '' 2026-02-24T07:51:13.474 INFO:teuthology.orchestra.run.trial035.stdout:osd.2: {} 2026-02-24T07:51:13.475 INFO:teuthology.orchestra.run.trial035.stderr:osd.2: osd_enable_op_tracker = '' 2026-02-24T07:51:13.487 INFO:teuthology.orchestra.run.trial035.stdout:osd.3: {} 2026-02-24T07:51:13.487 INFO:teuthology.orchestra.run.trial035.stderr:osd.3: osd_enable_op_tracker = '' 2026-02-24T07:51:14.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:51:13 trial035 ceph-mon[39284]: 4.8 scrub ok 2026-02-24T07:51:14.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:51:13 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/2111298201' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2026-02-24T07:51:15.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:51:14 trial035 ceph-mon[39284]: 4.2 scrub ok 2026-02-24T07:51:15.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:51:14 trial035 ceph-mon[39284]: 4.3 scrub starts 2026-02-24T07:51:15.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:51:14 trial035 ceph-mon[39284]: pgmap v286: 17 pgs: 17 active+clean; 17 GiB data, 57 GiB used, 2.7 TiB / 2.7 TiB avail; 0 B/s wr, 7.16k op/s 2026-02-24T07:51:15.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:51:14 trial035 ceph-mon[39284]: purged_snaps scrub starts 2026-02-24T07:51:15.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:51:14 trial035 ceph-mon[39284]: purged_snaps scrub ok 2026-02-24T07:51:15.529 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2026-02-24T07:51:15.684 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:51:15.811 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:51:15 trial035 ceph-mon[39284]: from='mgr.14284 10.20.193.35:0/3681373222' entity='mgr.x' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2026-02-24T07:51:15.811 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:51:15 trial035 ceph-mon[39284]: 4.3 scrub ok 2026-02-24T07:51:15.876 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:51:15.874+0000 7fcd1c348640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:51:15.876 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:51:15.874+0000 7fcd1c348640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:51:15.889 INFO:teuthology.orchestra.run.trial035.stdout:osd.0: {} 2026-02-24T07:51:15.890 INFO:teuthology.orchestra.run.trial035.stderr:osd.0: osd_enable_op_tracker = '' 2026-02-24T07:51:15.902 INFO:teuthology.orchestra.run.trial035.stdout:osd.1: {} 2026-02-24T07:51:15.903 INFO:teuthology.orchestra.run.trial035.stderr:osd.1: osd_enable_op_tracker = '' 2026-02-24T07:51:15.915 INFO:teuthology.orchestra.run.trial035.stdout:osd.2: {} 2026-02-24T07:51:15.916 INFO:teuthology.orchestra.run.trial035.stderr:osd.2: osd_enable_op_tracker = '' 2026-02-24T07:51:15.928 INFO:teuthology.orchestra.run.trial035.stdout:osd.3: {} 2026-02-24T07:51:15.928 INFO:teuthology.orchestra.run.trial035.stderr:osd.3: osd_enable_op_tracker = '' 2026-02-24T07:51:17.204 INFO:journalctl@ceph.mgr.x.trial035.stdout:Feb 24 07:51:16 trial035 ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-mgr-x[39621]: ::ffff:10.20.193.91 - - [24/Feb/2026:07:51:16] "GET /metrics HTTP/1.1" 200 4229 "" "Prometheus/3.6.0" 2026-02-24T07:51:17.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:51:16 trial035 ceph-mon[39284]: purged_snaps scrub starts 2026-02-24T07:51:17.205 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:51:16 trial035 ceph-mon[39284]: purged_snaps scrub ok 2026-02-24T07:51:17.205 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:51:16 trial035 ceph-mon[39284]: pgmap v287: 17 pgs: 17 active+clean; 17 GiB data, 57 GiB used, 2.7 TiB / 2.7 TiB avail; 0 B/s wr, 7.16k op/s 2026-02-24T07:51:17.205 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:51:16 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/1071270476' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2026-02-24T07:51:17.264 INFO:journalctl@ceph.prometheus.a.trial091.stdout:Feb 24 07:51:16 trial091 ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-prometheus-a[66007]: time=2026-02-24T07:51:16.973Z level=ERROR source=scrape.go:1630 msg="Failed to determine correct type of scrape target." component="scrape manager" scrape_pool=ceph target=http://trial035:9283/metrics content_type="text/html;charset=utf-8" fallback_media_type="" err="received unsupported Content-Type \"text/html;charset=utf-8\" and no fallback_scrape_protocol specified for target" 2026-02-24T07:51:17.343 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2026-02-24T07:51:17.343 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2026-02-24T07:51:17.344 INFO:tasks.thrashosds.thrasher:fixing pg num pool unique_pool_2 2026-02-24T07:51:17.344 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph pg dump --format=json 2026-02-24T07:51:17.483 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:51:17.602 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:51:17.601+0000 7fb597d02640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:51:17.602 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:51:17.601+0000 7fb597d02640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:51:17.757 INFO:teuthology.orchestra.run.trial035.stdout: 2026-02-24T07:51:17.758 INFO:teuthology.orchestra.run.trial035.stderr:dumped all 2026-02-24T07:51:17.804 INFO:teuthology.orchestra.run.trial035.stdout:{"pg_ready":true,"pg_map":{"version":288,"stamp":"2026-02-24T07:51:17.724812+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":15939994168,"num_objects":243219,"num_object_clones":0,"num_object_copies":729657,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":243219,"num_whiteouts":0,"num_read":137,"num_read_kb":241,"num_write":1051654,"num_write_kb":41433639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":52,"ondisk_log_size":52,"up":51,"acting":51,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":51,"num_osds":4,"num_per_pool_osds":4,"num_per_pool_omap_osds":4,"kb":2936012800,"kb_used":53460840,"kb_used_data":47481432,"kb_used_omap":63,"kb_used_meta":5979200,"kb_avail":2882551960,"statfs":{"total":3006477107200,"available":2951733207040,"internally_reserved":0,"allocated":48620986368,"data_stored":48619197704,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":65319,"internal_metadata":6122701017},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-8096645120,"num_objects":-123545,"num_object_clones":0,"num_object_copies":-370635,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-123545,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":123545,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001837"},"pg_stats":[{"pgid":"4.f","version":"65'66219","reported_seq":66559,"reported_epoch":65,"state":"active+clean","last_fresh":"2026-02-24T07:51:15.840674+0000","last_change":"2026-02-24T07:50:03.071459+0000","last_active":"2026-02-24T07:51:15.840674+0000","last_peered":"2026-02-24T07:51:15.840674+0000","last_clean":"2026-02-24T07:51:15.840674+0000","last_became_active":"2026-02-24T07:48:57.902313+0000","last_became_peered":"2026-02-24T07:48:57.902313+0000","last_unstale":"2026-02-24T07:51:15.840674+0000","last_undegraded":"2026-02-24T07:51:15.840674+0000","last_fullsized":"2026-02-24T07:51:15.840674+0000","mapping_epoch":55,"log_start":"65'66216","ondisk_log_start":"65'66216","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"61'30419","last_scrub_stamp":"2026-02-24T07:50:03.071380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:50:03.071380+0000","objects_scrubbed":29975,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:51:16.123453+0000","scrub_duration":2568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":990969856,"num_objects":15121,"num_object_clones":0,"num_object_copies":45363,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15121,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":66219,"num_write_kb":2602880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"65'65969","reported_seq":66377,"reported_epoch":65,"state":"active+clean","last_fresh":"2026-02-24T07:51:15.735106+0000","last_change":"2026-02-24T07:50:21.065676+0000","last_active":"2026-02-24T07:51:15.735106+0000","last_peered":"2026-02-24T07:51:15.735106+0000","last_clean":"2026-02-24T07:51:15.735106+0000","last_became_active":"2026-02-24T07:48:57.903584+0000","last_became_peered":"2026-02-24T07:48:57.903584+0000","last_unstale":"2026-02-24T07:51:15.735106+0000","last_undegraded":"2026-02-24T07:51:15.735106+0000","last_fullsized":"2026-02-24T07:51:15.735106+0000","mapping_epoch":55,"log_start":"65'65966","ondisk_log_start":"65'65966","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"61'37487","last_scrub_stamp":"2026-02-24T07:50:21.065638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:50:21.065638+0000","objects_scrubbed":36748,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:51:31.991874+0000","scrub_duration":3643,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":996737024,"num_objects":15209,"num_object_clones":0,"num_object_copies":45627,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15209,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":65969,"num_write_kb":2597696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.d","version":"65'65609","reported_seq":65960,"reported_epoch":65,"state":"active+clean","last_fresh":"2026-02-24T07:51:15.747841+0000","last_change":"2026-02-24T07:50:06.701973+0000","last_active":"2026-02-24T07:51:15.747841+0000","last_peered":"2026-02-24T07:51:15.747841+0000","last_clean":"2026-02-24T07:51:15.747841+0000","last_became_active":"2026-02-24T07:48:57.905031+0000","last_became_peered":"2026-02-24T07:48:57.905031+0000","last_unstale":"2026-02-24T07:51:15.747841+0000","last_undegraded":"2026-02-24T07:51:15.747841+0000","last_fullsized":"2026-02-24T07:51:15.747841+0000","mapping_epoch":55,"log_start":"65'65606","ondisk_log_start":"65'65606","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"61'31648","last_scrub_stamp":"2026-02-24T07:50:06.701883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:50:06.701883+0000","objects_scrubbed":30990,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:51:19.224966+0000","scrub_duration":3493,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":994967576,"num_objects":15183,"num_object_clones":0,"num_object_copies":45549,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15183,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":65609,"num_write_kb":2585281,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"65'65260","reported_seq":65679,"reported_epoch":65,"state":"active+clean","last_fresh":"2026-02-24T07:51:15.750960+0000","last_change":"2026-02-24T07:50:24.798674+0000","last_active":"2026-02-24T07:51:15.750960+0000","last_peered":"2026-02-24T07:51:15.750960+0000","last_clean":"2026-02-24T07:51:15.750960+0000","last_became_active":"2026-02-24T07:48:57.904683+0000","last_became_peered":"2026-02-24T07:48:57.904683+0000","last_unstale":"2026-02-24T07:51:15.750960+0000","last_undegraded":"2026-02-24T07:51:15.750960+0000","last_fullsized":"2026-02-24T07:51:15.750960+0000","mapping_epoch":55,"log_start":"65'65257","ondisk_log_start":"65'65257","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"61'38696","last_scrub_stamp":"2026-02-24T07:50:24.798629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:50:24.798629+0000","objects_scrubbed":37979,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:51:26.521338+0000","scrub_duration":3612,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":998244352,"num_objects":15232,"num_object_clones":0,"num_object_copies":45696,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15232,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":65260,"num_write_kb":2575744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.b","version":"65'65369","reported_seq":65799,"reported_epoch":65,"state":"active+clean","last_fresh":"2026-02-24T07:51:15.749103+0000","last_change":"2026-02-24T07:50:27.267008+0000","last_active":"2026-02-24T07:51:15.749103+0000","last_peered":"2026-02-24T07:51:15.749103+0000","last_clean":"2026-02-24T07:51:15.749103+0000","last_became_active":"2026-02-24T07:48:57.901700+0000","last_became_peered":"2026-02-24T07:48:57.901700+0000","last_unstale":"2026-02-24T07:51:15.749103+0000","last_undegraded":"2026-02-24T07:51:15.749103+0000","last_fullsized":"2026-02-24T07:51:15.749103+0000","mapping_epoch":55,"log_start":"65'65366","ondisk_log_start":"65'65366","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"61'39639","last_scrub_stamp":"2026-02-24T07:50:27.266943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:50:27.266943+0000","objects_scrubbed":38982,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:51:34.346751+0000","scrub_duration":3744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1001717760,"num_objects":15285,"num_object_clones":0,"num_object_copies":45855,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15285,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":65369,"num_write_kb":2580928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.a","version":"65'65888","reported_seq":66321,"reported_epoch":65,"state":"active+clean","last_fresh":"2026-02-24T07:51:15.733506+0000","last_change":"2026-02-24T07:50:33.043786+0000","last_active":"2026-02-24T07:51:15.733506+0000","last_peered":"2026-02-24T07:51:15.733506+0000","last_clean":"2026-02-24T07:51:15.733506+0000","last_became_active":"2026-02-24T07:48:57.903228+0000","last_became_peered":"2026-02-24T07:48:57.903228+0000","last_unstale":"2026-02-24T07:51:15.733506+0000","last_undegraded":"2026-02-24T07:51:15.733506+0000","last_fullsized":"2026-02-24T07:51:15.733506+0000","mapping_epoch":55,"log_start":"65'65885","ondisk_log_start":"65'65885","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"61'42539","last_scrub_stamp":"2026-02-24T07:50:33.043726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:50:33.043726+0000","objects_scrubbed":39246,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:51:51.239665+0000","scrub_duration":3547,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":982908928,"num_objects":14998,"num_object_clones":0,"num_object_copies":44994,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14998,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":65888,"num_write_kb":2588352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"65'65792","reported_seq":66165,"reported_epoch":65,"state":"active+clean","last_fresh":"2026-02-24T07:51:15.735702+0000","last_change":"2026-02-24T07:50:13.165492+0000","last_active":"2026-02-24T07:51:15.735702+0000","last_peered":"2026-02-24T07:51:15.735702+0000","last_clean":"2026-02-24T07:51:15.735702+0000","last_became_active":"2026-02-24T07:48:57.901730+0000","last_became_peered":"2026-02-24T07:48:57.901730+0000","last_unstale":"2026-02-24T07:51:15.735702+0000","last_undegraded":"2026-02-24T07:51:15.735702+0000","last_fullsized":"2026-02-24T07:51:15.735702+0000","mapping_epoch":55,"log_start":"65'65789","ondisk_log_start":"65'65789","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"61'33943","last_scrub_stamp":"2026-02-24T07:50:13.165459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:50:13.165459+0000","objects_scrubbed":33253,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:51:33.820708+0000","scrub_duration":3776,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":988151808,"num_objects":15078,"num_object_clones":0,"num_object_copies":45234,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15078,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":65792,"num_write_kb":2587840,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"65'65610","reported_seq":66125,"reported_epoch":65,"state":"active+clean","last_fresh":"2026-02-24T07:51:15.749320+0000","last_change":"2026-02-24T07:51:12.758378+0000","last_active":"2026-02-24T07:51:15.749320+0000","last_peered":"2026-02-24T07:51:15.749320+0000","last_clean":"2026-02-24T07:51:15.749320+0000","last_became_active":"2026-02-24T07:48:57.904376+0000","last_became_peered":"2026-02-24T07:48:57.904376+0000","last_unstale":"2026-02-24T07:51:15.749320+0000","last_undegraded":"2026-02-24T07:51:15.749320+0000","last_fullsized":"2026-02-24T07:51:15.749320+0000","mapping_epoch":55,"log_start":"65'65606","ondisk_log_start":"65'65606","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"65'64020","last_scrub_stamp":"2026-02-24T07:51:12.758336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:51:12.758336+0000","objects_scrubbed":17129,"log_size":4,"log_dups_size":3000,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:52:28.496810+0000","scrub_duration":1688,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":998637568,"num_objects":15238,"num_object_clones":0,"num_object_copies":45714,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15238,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":65610,"num_write_kb":2587136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.7","version":"65'65608","reported_seq":65972,"reported_epoch":65,"state":"active+clean","last_fresh":"2026-02-24T07:51:15.835248+0000","last_change":"2026-02-24T07:50:11.227608+0000","last_active":"2026-02-24T07:51:15.835248+0000","last_peered":"2026-02-24T07:51:15.835248+0000","last_clean":"2026-02-24T07:51:15.835248+0000","last_became_active":"2026-02-24T07:48:57.902484+0000","last_became_peered":"2026-02-24T07:48:57.902484+0000","last_unstale":"2026-02-24T07:51:15.835248+0000","last_undegraded":"2026-02-24T07:51:15.835248+0000","last_fullsized":"2026-02-24T07:51:15.835248+0000","mapping_epoch":55,"log_start":"65'65605","ondisk_log_start":"65'65605","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"61'32990","last_scrub_stamp":"2026-02-24T07:50:11.227573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:50:11.227573+0000","objects_scrubbed":32460,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:51:29.781611+0000","scrub_duration":3701,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":985530368,"num_objects":15038,"num_object_clones":0,"num_object_copies":45114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15038,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":65608,"num_write_kb":2580672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.6","version":"65'65366","reported_seq":65741,"reported_epoch":65,"state":"active+clean","last_fresh":"2026-02-24T07:51:15.746324+0000","last_change":"2026-02-24T07:50:14.386679+0000","last_active":"2026-02-24T07:51:15.746324+0000","last_peered":"2026-02-24T07:51:15.746324+0000","last_clean":"2026-02-24T07:51:15.746324+0000","last_became_active":"2026-02-24T07:48:57.901565+0000","last_became_peered":"2026-02-24T07:48:57.901565+0000","last_unstale":"2026-02-24T07:51:15.746324+0000","last_undegraded":"2026-02-24T07:51:15.746324+0000","last_fullsized":"2026-02-24T07:51:15.746324+0000","mapping_epoch":55,"log_start":"65'65363","ondisk_log_start":"65'65363","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"61'34244","last_scrub_stamp":"2026-02-24T07:50:14.386625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:50:14.386625+0000","objects_scrubbed":33505,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:51:16.739697+0000","scrub_duration":3790,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":995753984,"num_objects":15194,"num_object_clones":0,"num_object_copies":45582,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15194,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":65366,"num_write_kb":2577920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"65'66109","reported_seq":66496,"reported_epoch":65,"state":"active+clean","last_fresh":"2026-02-24T07:51:15.839493+0000","last_change":"2026-02-24T07:50:16.399238+0000","last_active":"2026-02-24T07:51:15.839493+0000","last_peered":"2026-02-24T07:51:15.839493+0000","last_clean":"2026-02-24T07:51:15.839493+0000","last_became_active":"2026-02-24T07:48:57.901824+0000","last_became_peered":"2026-02-24T07:48:57.901824+0000","last_unstale":"2026-02-24T07:51:15.839493+0000","last_undegraded":"2026-02-24T07:51:15.839493+0000","last_fullsized":"2026-02-24T07:51:15.839493+0000","mapping_epoch":55,"log_start":"65'66106","ondisk_log_start":"65'66106","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"61'35510","last_scrub_stamp":"2026-02-24T07:50:16.399194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:50:16.399194+0000","objects_scrubbed":34844,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:51:44.770701+0000","scrub_duration":2971,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":992673792,"num_objects":15147,"num_object_clones":0,"num_object_copies":45441,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15147,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":66109,"num_write_kb":2600192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[1,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"65'65282","reported_seq":65813,"reported_epoch":65,"state":"active+clean","last_fresh":"2026-02-24T07:51:15.744380+0000","last_change":"2026-02-24T07:51:15.067567+0000","last_active":"2026-02-24T07:51:15.744380+0000","last_peered":"2026-02-24T07:51:15.744380+0000","last_clean":"2026-02-24T07:51:15.744380+0000","last_became_active":"2026-02-24T07:48:57.902224+0000","last_became_peered":"2026-02-24T07:48:57.902224+0000","last_unstale":"2026-02-24T07:51:15.744380+0000","last_undegraded":"2026-02-24T07:51:15.744380+0000","last_fullsized":"2026-02-24T07:51:15.744380+0000","mapping_epoch":55,"log_start":"65'65279","ondisk_log_start":"65'65279","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"65'64892","last_scrub_stamp":"2026-02-24T07:51:15.067417+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:51:15.067417+0000","objects_scrubbed":15892,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:52:16.892433+0000","scrub_duration":1747,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":989331456,"num_objects":15096,"num_object_clones":0,"num_object_copies":45288,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15096,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":65282,"num_write_kb":2572096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"65'65892","reported_seq":66405,"reported_epoch":65,"state":"active+clean","last_fresh":"2026-02-24T07:51:15.839443+0000","last_change":"2026-02-24T07:51:13.251872+0000","last_active":"2026-02-24T07:51:15.839443+0000","last_peered":"2026-02-24T07:51:15.839443+0000","last_clean":"2026-02-24T07:51:15.839443+0000","last_became_active":"2026-02-24T07:48:57.902125+0000","last_became_peered":"2026-02-24T07:48:57.902125+0000","last_unstale":"2026-02-24T07:51:15.839443+0000","last_undegraded":"2026-02-24T07:51:15.839443+0000","last_fullsized":"2026-02-24T07:51:15.839443+0000","mapping_epoch":55,"log_start":"65'65889","ondisk_log_start":"65'65889","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"65'64550","last_scrub_stamp":"2026-02-24T07:51:13.251826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:51:13.251826+0000","objects_scrubbed":17042,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:52:21.793363+0000","scrub_duration":1966,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1002569728,"num_objects":15298,"num_object_clones":0,"num_object_copies":45894,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15298,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":65892,"num_write_kb":2598080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1","version":"65'66049","reported_seq":66489,"reported_epoch":65,"state":"active+clean","last_fresh":"2026-02-24T07:51:15.732754+0000","last_change":"2026-02-24T07:50:29.889880+0000","last_active":"2026-02-24T07:51:15.732754+0000","last_peered":"2026-02-24T07:51:15.732754+0000","last_clean":"2026-02-24T07:51:15.732754+0000","last_became_active":"2026-02-24T07:48:57.903439+0000","last_became_peered":"2026-02-24T07:48:57.903439+0000","last_unstale":"2026-02-24T07:51:15.732754+0000","last_undegraded":"2026-02-24T07:51:15.732754+0000","last_fullsized":"2026-02-24T07:51:15.732754+0000","mapping_epoch":55,"log_start":"65'66046","ondisk_log_start":"65'66046","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"61'41105","last_scrub_stamp":"2026-02-24T07:50:29.889816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:50:29.889816+0000","objects_scrubbed":40127,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:51:50.788414+0000","scrub_duration":4409,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1006698496,"num_objects":15361,"num_object_clones":0,"num_object_copies":46083,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15361,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":66049,"num_write_kb":2605120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.0","version":"65'65771","reported_seq":66211,"reported_epoch":65,"state":"active+clean","last_fresh":"2026-02-24T07:51:15.747366+0000","last_change":"2026-02-24T07:50:29.387323+0000","last_active":"2026-02-24T07:51:15.747366+0000","last_peered":"2026-02-24T07:51:15.747366+0000","last_clean":"2026-02-24T07:51:15.747366+0000","last_became_active":"2026-02-24T07:48:57.902565+0000","last_became_peered":"2026-02-24T07:48:57.902565+0000","last_unstale":"2026-02-24T07:51:15.747366+0000","last_undegraded":"2026-02-24T07:51:15.747366+0000","last_fullsized":"2026-02-24T07:51:15.747366+0000","mapping_epoch":55,"log_start":"65'65768","ondisk_log_start":"65'65768","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"61'40736","last_scrub_stamp":"2026-02-24T07:50:29.387206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:50:29.387206+0000","objects_scrubbed":39946,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:51:59.173732+0000","scrub_duration":3834,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1007353856,"num_objects":15371,"num_object_clones":0,"num_object_copies":46113,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15371,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":65771,"num_write_kb":2596544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,3],"acting":[0,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.5","version":"65'65726","reported_seq":66070,"reported_epoch":65,"state":"active+clean","last_fresh":"2026-02-24T07:51:15.749312+0000","last_change":"2026-02-24T07:50:04.821914+0000","last_active":"2026-02-24T07:51:15.749312+0000","last_peered":"2026-02-24T07:51:15.749312+0000","last_clean":"2026-02-24T07:51:15.749312+0000","last_became_active":"2026-02-24T07:48:57.905127+0000","last_became_peered":"2026-02-24T07:48:57.905127+0000","last_unstale":"2026-02-24T07:51:15.749312+0000","last_undegraded":"2026-02-24T07:51:15.749312+0000","last_fullsized":"2026-02-24T07:51:15.749312+0000","mapping_epoch":55,"log_start":"65'65723","ondisk_log_start":"65'65723","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"61'30833","last_scrub_stamp":"2026-02-24T07:50:04.821861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:50:04.821861+0000","objects_scrubbed":30132,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:51:15.928207+0000","scrub_duration":3578,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1007157248,"num_objects":15368,"num_object_clones":0,"num_object_copies":46104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15368,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":65726,"num_write_kb":2595008,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"29'99","reported_seq":302,"reported_epoch":65,"state":"active+clean","last_fresh":"2026-02-24T07:50:55.895043+0000","last_change":"2026-02-24T07:50:30.401963+0000","last_active":"2026-02-24T07:50:55.895043+0000","last_peered":"2026-02-24T07:50:55.895043+0000","last_clean":"2026-02-24T07:50:55.895043+0000","last_became_active":"2026-02-24T07:42:54.186719+0000","last_became_peered":"2026-02-24T07:42:54.186719+0000","last_unstale":"2026-02-24T07:50:55.895043+0000","last_undegraded":"2026-02-24T07:50:55.895043+0000","last_fullsized":"2026-02-24T07:50:55.895043+0000","mapping_epoch":27,"log_start":"29'96","ondisk_log_start":"29'96","created":20,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'99","last_scrub_stamp":"2026-02-24T07:50:30.401929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:42:43.640776+0000","last_clean_scrub_stamp":"2026-02-24T07:50:30.401929+0000","objects_scrubbed":2,"log_size":3,"log_dups_size":96,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:51:52.990727+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":240,"num_write":135,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":16,"stat_sum":{"num_bytes":15939403800,"num_objects":243217,"num_object_clones":0,"num_object_copies":729651,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":243217,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1051519,"num_write_kb":41431489,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":47826677760,"data_stored":47826665544,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1710,"internal_metadata":0},"log_size":49,"ondisk_log_size":49,"up":48,"acting":48,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":240,"num_write":135,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1096,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":3,"up_from":25,"seq":107374182503,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":13745788,"kb_used_data":11994344,"kb_used_omap":16,"kb_used_meta":1751343,"kb_avail":720257412,"statfs":{"total":751619276800,"available":737543589888,"internally_reserved":0,"allocated":12282208256,"data_stored":12281759184,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":17094,"internal_metadata":1793375546},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":19,"seq":81604378729,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":14650812,"kb_used_data":13719464,"kb_used_omap":13,"kb_used_meta":931314,"kb_avail":719352388,"statfs":{"total":751619276800,"available":736616845312,"internally_reserved":0,"allocated":14048731136,"data_stored":14048282064,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13789,"internal_metadata":953666083},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542250,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":11441332,"kb_used_data":9889824,"kb_used_omap":18,"kb_used_meta":1551469,"kb_avail":722561868,"statfs":{"total":751619276800,"available":739903352832,"internally_reserved":0,"allocated":10127179776,"data_stored":10126738328,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":18826,"internal_metadata":1588704886},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705773,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":13622908,"kb_used_data":11877800,"kb_used_omap":15,"kb_used_meta":1745072,"kb_avail":720380292,"statfs":{"total":751619276800,"available":737669419008,"internally_reserved":0,"allocated":12162867200,"data_stored":12162418128,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":15610,"internal_metadata":1786954502},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1096,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":11979264000,"data_stored":11979259928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1524,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":9914548224,"data_stored":9914548224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":93,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":13955174400,"data_stored":13955170328,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":11977691136,"data_stored":11977687064,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":93,"internal_metadata":0}]}} 2026-02-24T07:51:17.806 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph osd pool set unique_pool_2 pgp_num 66 2026-02-24T07:51:17.944 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:51:17.971 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2026-02-24T07:51:18.076 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:51:18.075+0000 7f9177930640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:51:18.076 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:51:18.075+0000 7f9177930640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:51:18.125 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:51:18.229 INFO:teuthology.orchestra.run.trial035.stderr:Error EPERM: crimson_allow_pg_split is false; pgp_num increase denied 2026-02-24T07:51:18.244 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:51:18.243+0000 7f2090083640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:51:18.244 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:51:18.243+0000 7f2090083640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:51:18.260 INFO:teuthology.orchestra.run.trial035.stdout:osd.0: {} 2026-02-24T07:51:18.261 INFO:teuthology.orchestra.run.trial035.stderr:osd.0: osd_enable_op_tracker = '' 2026-02-24T07:51:18.269 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-24T07:51:18.275 INFO:teuthology.orchestra.run.trial035.stdout:osd.1: {} 2026-02-24T07:51:18.275 INFO:teuthology.orchestra.run.trial035.stderr:osd.1: osd_enable_op_tracker = '' 2026-02-24T07:51:18.289 INFO:teuthology.orchestra.run.trial035.stdout:osd.2: {} 2026-02-24T07:51:18.289 INFO:teuthology.orchestra.run.trial035.stderr:osd.2: osd_enable_op_tracker = '' 2026-02-24T07:51:18.301 INFO:teuthology.orchestra.run.trial035.stdout:osd.3: {} 2026-02-24T07:51:18.302 INFO:teuthology.orchestra.run.trial035.stderr:osd.3: osd_enable_op_tracker = '' 2026-02-24T07:51:19.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:51:18 trial035 ceph-mon[39284]: pgmap v288: 17 pgs: 17 active+clean; 15 GiB data, 51 GiB used, 2.7 TiB / 2.7 TiB avail; 0 B/s wr, 10.29k op/s 2026-02-24T07:51:19.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:51:18 trial035 ceph-mon[39284]: from='client.15077 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2026-02-24T07:51:19.205 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:51:18 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/3851284846' entity='client.admin' cmd={"prefix": "osd pool set", "pool": "unique_pool_2", "var": "pgp_num", "val": "66"} : dispatch 2026-02-24T07:51:19.205 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:51:18 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/1762658816' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2026-02-24T07:51:20.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:51:19 trial035 ceph-mon[39284]: 4.6 scrub starts 2026-02-24T07:51:20.346 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2026-02-24T07:51:20.480 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:51:20.843 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:51:20.809+0000 7efcbc600640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:51:20.843 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:51:20.809+0000 7efcbc600640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:51:20.843 INFO:teuthology.orchestra.run.trial035.stderr:osd.0: osd_enable_op_tracker = '' 2026-02-24T07:51:20.843 INFO:teuthology.orchestra.run.trial035.stdout:osd.0: {} 2026-02-24T07:51:20.844 INFO:teuthology.orchestra.run.trial035.stdout:osd.1: {} 2026-02-24T07:51:20.844 INFO:teuthology.orchestra.run.trial035.stderr:osd.1: osd_enable_op_tracker = '' 2026-02-24T07:51:20.858 INFO:teuthology.orchestra.run.trial035.stdout:osd.2: {} 2026-02-24T07:51:20.858 INFO:teuthology.orchestra.run.trial035.stderr:osd.2: osd_enable_op_tracker = '' 2026-02-24T07:51:20.870 INFO:teuthology.orchestra.run.trial035.stdout:osd.3: {} 2026-02-24T07:51:20.871 INFO:teuthology.orchestra.run.trial035.stderr:osd.3: osd_enable_op_tracker = '' 2026-02-24T07:51:20.954 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:51:20 trial035 ceph-mon[39284]: pgmap v289: 17 pgs: 17 active+clean; 15 GiB data, 51 GiB used, 2.7 TiB / 2.7 TiB avail; 0 B/s wr, 6.89k op/s 2026-02-24T07:51:20.954 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:51:20 trial035 ceph-mon[39284]: 4.6 scrub ok 2026-02-24T07:51:20.954 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:51:20 trial035 ceph-mon[39284]: 4.5 scrub starts 2026-02-24T07:51:20.954 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:51:20 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/1843095718' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2026-02-24T07:51:23.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:51:22 trial035 ceph-mon[39284]: 4.f scrub starts 2026-02-24T07:51:23.205 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:51:22 trial035 ceph-mon[39284]: 4.5 scrub ok 2026-02-24T07:51:23.205 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:51:22 trial035 ceph-mon[39284]: pgmap v290: 17 pgs: 1 active+clean+scrubbing, 16 active+clean; 12 GiB data, 42 GiB used, 2.7 TiB / 2.7 TiB avail; 0 B/s wr, 10.53k op/s 2026-02-24T07:51:23.270 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2026-02-24T07:51:23.271 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2026-02-24T07:51:23.271 INFO:tasks.thrashosds.thrasher:Growing pool .mgr 2026-02-24T07:51:23.271 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph pg dump --format=json 2026-02-24T07:51:23.415 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:51:23.536 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:51:23.534+0000 7fd089b4d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:51:23.537 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:51:23.534+0000 7fd089b4d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:51:23.629 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2026-02-24T07:51:23.679 INFO:teuthology.orchestra.run.trial035.stdout: 2026-02-24T07:51:23.680 INFO:teuthology.orchestra.run.trial035.stderr:dumped all 2026-02-24T07:51:23.723 INFO:teuthology.orchestra.run.trial035.stdout:{"pg_ready":true,"pg_map":{"version":290,"stamp":"2026-02-24T07:51:21.725356+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":13075022392,"num_objects":199503,"num_object_clones":0,"num_object_copies":598509,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":199503,"num_whiteouts":0,"num_read":137,"num_read_kb":241,"num_write":1095370,"num_write_kb":41433639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":57,"ondisk_log_size":57,"up":51,"acting":51,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":51,"num_osds":4,"num_per_pool_osds":4,"num_per_pool_omap_osds":4,"kb":2936012800,"kb_used":43973800,"kb_used_data":38842520,"kb_used_omap":59,"kb_used_meta":5131012,"kb_avail":2892039000,"statfs":{"total":3006477107200,"available":2961447936000,"internally_reserved":0,"allocated":39774740480,"data_stored":39772951816,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":60518,"internal_metadata":5254157210},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,2],"upper_bound":4},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-8285650944,"num_objects":-126429,"num_object_clones":0,"num_object_copies":-379287,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-126429,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":126429,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001808"},"pg_stats":[{"pgid":"4.f","version":"65'68876","reported_seq":69216,"reported_epoch":65,"state":"active+clean","last_fresh":"2026-02-24T07:51:20.836334+0000","last_change":"2026-02-24T07:50:03.071459+0000","last_active":"2026-02-24T07:51:20.836334+0000","last_peered":"2026-02-24T07:51:20.836334+0000","last_clean":"2026-02-24T07:51:20.836334+0000","last_became_active":"2026-02-24T07:48:57.902313+0000","last_became_peered":"2026-02-24T07:48:57.902313+0000","last_unstale":"2026-02-24T07:51:20.836334+0000","last_undegraded":"2026-02-24T07:51:20.836334+0000","last_fullsized":"2026-02-24T07:51:20.836334+0000","mapping_epoch":55,"log_start":"65'68873","ondisk_log_start":"65'68873","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"61'30419","last_scrub_stamp":"2026-02-24T07:50:03.071380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:50:03.071380+0000","objects_scrubbed":29975,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"queued for scrub","scrub_duration":2568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":816840704,"num_objects":12464,"num_object_clones":0,"num_object_copies":37392,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12464,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":68876,"num_write_kb":2602880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"65'68734","reported_seq":69142,"reported_epoch":65,"state":"active+clean","last_fresh":"2026-02-24T07:51:20.723382+0000","last_change":"2026-02-24T07:50:21.065676+0000","last_active":"2026-02-24T07:51:20.723382+0000","last_peered":"2026-02-24T07:51:20.723382+0000","last_clean":"2026-02-24T07:51:20.723382+0000","last_became_active":"2026-02-24T07:48:57.903584+0000","last_became_peered":"2026-02-24T07:48:57.903584+0000","last_unstale":"2026-02-24T07:51:20.723382+0000","last_undegraded":"2026-02-24T07:51:20.723382+0000","last_fullsized":"2026-02-24T07:51:20.723382+0000","mapping_epoch":55,"log_start":"65'68731","ondisk_log_start":"65'68731","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"61'37487","last_scrub_stamp":"2026-02-24T07:50:21.065638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:50:21.065638+0000","objects_scrubbed":36748,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:51:31.991874+0000","scrub_duration":3643,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":815529984,"num_objects":12444,"num_object_clones":0,"num_object_copies":37332,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12444,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":68734,"num_write_kb":2597696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.d","version":"65'68329","reported_seq":68680,"reported_epoch":65,"state":"active+clean","last_fresh":"2026-02-24T07:51:20.720375+0000","last_change":"2026-02-24T07:50:06.701973+0000","last_active":"2026-02-24T07:51:20.720375+0000","last_peered":"2026-02-24T07:51:20.720375+0000","last_clean":"2026-02-24T07:51:20.720375+0000","last_became_active":"2026-02-24T07:48:57.905031+0000","last_became_peered":"2026-02-24T07:48:57.905031+0000","last_unstale":"2026-02-24T07:51:20.720375+0000","last_undegraded":"2026-02-24T07:51:20.720375+0000","last_fullsized":"2026-02-24T07:51:20.720375+0000","mapping_epoch":55,"log_start":"65'68326","ondisk_log_start":"65'68326","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"61'31648","last_scrub_stamp":"2026-02-24T07:50:06.701883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:50:06.701883+0000","objects_scrubbed":30990,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"queued for scrub","scrub_duration":3493,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":816709656,"num_objects":12463,"num_object_clones":0,"num_object_copies":37389,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12463,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":68329,"num_write_kb":2585281,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"65'68045","reported_seq":68465,"reported_epoch":65,"state":"active+clean","last_fresh":"2026-02-24T07:51:20.722747+0000","last_change":"2026-02-24T07:50:24.798674+0000","last_active":"2026-02-24T07:51:20.722747+0000","last_peered":"2026-02-24T07:51:20.722747+0000","last_clean":"2026-02-24T07:51:20.722747+0000","last_became_active":"2026-02-24T07:48:57.904683+0000","last_became_peered":"2026-02-24T07:48:57.904683+0000","last_unstale":"2026-02-24T07:51:20.722747+0000","last_undegraded":"2026-02-24T07:51:20.722747+0000","last_fullsized":"2026-02-24T07:51:20.722747+0000","mapping_epoch":55,"log_start":"65'68042","ondisk_log_start":"65'68042","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"61'38696","last_scrub_stamp":"2026-02-24T07:50:24.798629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:50:24.798629+0000","objects_scrubbed":37979,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:51:26.521338+0000","scrub_duration":3612,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":815726592,"num_objects":12447,"num_object_clones":0,"num_object_copies":37341,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12447,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":68045,"num_write_kb":2575744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.b","version":"65'68118","reported_seq":68548,"reported_epoch":65,"state":"active+clean","last_fresh":"2026-02-24T07:51:20.725977+0000","last_change":"2026-02-24T07:50:27.267008+0000","last_active":"2026-02-24T07:51:20.725977+0000","last_peered":"2026-02-24T07:51:20.725977+0000","last_clean":"2026-02-24T07:51:20.725977+0000","last_became_active":"2026-02-24T07:48:57.901700+0000","last_became_peered":"2026-02-24T07:48:57.901700+0000","last_unstale":"2026-02-24T07:51:20.725977+0000","last_undegraded":"2026-02-24T07:51:20.725977+0000","last_fullsized":"2026-02-24T07:51:20.725977+0000","mapping_epoch":55,"log_start":"65'68115","ondisk_log_start":"65'68115","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"61'39639","last_scrub_stamp":"2026-02-24T07:50:27.266943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:50:27.266943+0000","objects_scrubbed":38982,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:51:34.346751+0000","scrub_duration":3744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":821559296,"num_objects":12536,"num_object_clones":0,"num_object_copies":37608,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12536,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":68118,"num_write_kb":2580928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.a","version":"65'68637","reported_seq":69070,"reported_epoch":65,"state":"active+clean","last_fresh":"2026-02-24T07:51:20.724635+0000","last_change":"2026-02-24T07:50:33.043786+0000","last_active":"2026-02-24T07:51:20.724635+0000","last_peered":"2026-02-24T07:51:20.724635+0000","last_clean":"2026-02-24T07:51:20.724635+0000","last_became_active":"2026-02-24T07:48:57.903228+0000","last_became_peered":"2026-02-24T07:48:57.903228+0000","last_unstale":"2026-02-24T07:51:20.724635+0000","last_undegraded":"2026-02-24T07:51:20.724635+0000","last_fullsized":"2026-02-24T07:51:20.724635+0000","mapping_epoch":55,"log_start":"65'68634","ondisk_log_start":"65'68634","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"61'42539","last_scrub_stamp":"2026-02-24T07:50:33.043726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:50:33.043726+0000","objects_scrubbed":39246,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:51:51.239665+0000","scrub_duration":3547,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":802750464,"num_objects":12249,"num_object_clones":0,"num_object_copies":36747,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12249,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":68637,"num_write_kb":2588352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"65'68498","reported_seq":68871,"reported_epoch":65,"state":"active+clean","last_fresh":"2026-02-24T07:51:20.723197+0000","last_change":"2026-02-24T07:50:13.165492+0000","last_active":"2026-02-24T07:51:20.723197+0000","last_peered":"2026-02-24T07:51:20.723197+0000","last_clean":"2026-02-24T07:51:20.723197+0000","last_became_active":"2026-02-24T07:48:57.901730+0000","last_became_peered":"2026-02-24T07:48:57.901730+0000","last_unstale":"2026-02-24T07:51:20.723197+0000","last_undegraded":"2026-02-24T07:51:20.723197+0000","last_fullsized":"2026-02-24T07:51:20.723197+0000","mapping_epoch":55,"log_start":"65'68495","ondisk_log_start":"65'68495","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"61'33943","last_scrub_stamp":"2026-02-24T07:50:13.165459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:50:13.165459+0000","objects_scrubbed":33253,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:51:33.820708+0000","scrub_duration":3776,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":810811392,"num_objects":12372,"num_object_clones":0,"num_object_copies":37116,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12372,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":68498,"num_write_kb":2587840,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"65'68319","reported_seq":68837,"reported_epoch":65,"state":"active+clean","last_fresh":"2026-02-24T07:51:20.724217+0000","last_change":"2026-02-24T07:51:12.758378+0000","last_active":"2026-02-24T07:51:20.724217+0000","last_peered":"2026-02-24T07:51:20.724217+0000","last_clean":"2026-02-24T07:51:20.724217+0000","last_became_active":"2026-02-24T07:48:57.904376+0000","last_became_peered":"2026-02-24T07:48:57.904376+0000","last_unstale":"2026-02-24T07:51:20.724217+0000","last_undegraded":"2026-02-24T07:51:20.724217+0000","last_fullsized":"2026-02-24T07:51:20.724217+0000","mapping_epoch":55,"log_start":"65'68316","ondisk_log_start":"65'68316","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"65'64020","last_scrub_stamp":"2026-02-24T07:51:12.758336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:51:12.758336+0000","objects_scrubbed":17129,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:52:28.496810+0000","scrub_duration":1688,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":821100544,"num_objects":12529,"num_object_clones":0,"num_object_copies":37587,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12529,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":68319,"num_write_kb":2587136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.7","version":"65'68289","reported_seq":68652,"reported_epoch":65,"state":"active+clean","last_fresh":"2026-02-24T07:51:20.840866+0000","last_change":"2026-02-24T07:50:11.227608+0000","last_active":"2026-02-24T07:51:20.840866+0000","last_peered":"2026-02-24T07:51:20.840866+0000","last_clean":"2026-02-24T07:51:20.840866+0000","last_became_active":"2026-02-24T07:48:57.902484+0000","last_became_peered":"2026-02-24T07:48:57.902484+0000","last_unstale":"2026-02-24T07:51:20.840866+0000","last_undegraded":"2026-02-24T07:51:20.840866+0000","last_fullsized":"2026-02-24T07:51:20.840866+0000","mapping_epoch":55,"log_start":"65'68286","ondisk_log_start":"65'68286","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"61'32990","last_scrub_stamp":"2026-02-24T07:50:11.227573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:50:11.227573+0000","objects_scrubbed":32460,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:51:29.781611+0000","scrub_duration":3701,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":809828352,"num_objects":12357,"num_object_clones":0,"num_object_copies":37071,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12357,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":68289,"num_write_kb":2580672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.6","version":"65'68045","reported_seq":68564,"reported_epoch":65,"state":"active+clean","last_fresh":"2026-02-24T07:51:20.724125+0000","last_change":"2026-02-24T07:51:19.852133+0000","last_active":"2026-02-24T07:51:20.724125+0000","last_peered":"2026-02-24T07:51:20.724125+0000","last_clean":"2026-02-24T07:51:20.724125+0000","last_became_active":"2026-02-24T07:48:57.901565+0000","last_became_peered":"2026-02-24T07:48:57.901565+0000","last_unstale":"2026-02-24T07:51:20.724125+0000","last_undegraded":"2026-02-24T07:51:20.724125+0000","last_fullsized":"2026-02-24T07:51:20.724125+0000","mapping_epoch":55,"log_start":"65'68042","ondisk_log_start":"65'68042","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"65'67636","last_scrub_stamp":"2026-02-24T07:51:19.852077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:51:19.852077+0000","objects_scrubbed":13301,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:52:23.813957+0000","scrub_duration":1519,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":820183040,"num_objects":12515,"num_object_clones":0,"num_object_copies":37545,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12515,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":68045,"num_write_kb":2577920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"65'68808","reported_seq":69196,"reported_epoch":65,"state":"active+clean","last_fresh":"2026-02-24T07:51:20.834047+0000","last_change":"2026-02-24T07:50:16.399238+0000","last_active":"2026-02-24T07:51:20.834047+0000","last_peered":"2026-02-24T07:51:20.834047+0000","last_clean":"2026-02-24T07:51:20.834047+0000","last_became_active":"2026-02-24T07:48:57.901824+0000","last_became_peered":"2026-02-24T07:48:57.901824+0000","last_unstale":"2026-02-24T07:51:20.834047+0000","last_undegraded":"2026-02-24T07:51:20.834047+0000","last_fullsized":"2026-02-24T07:51:20.834047+0000","mapping_epoch":55,"log_start":"65'68805","ondisk_log_start":"65'68805","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"61'35510","last_scrub_stamp":"2026-02-24T07:50:16.399194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:50:16.399194+0000","objects_scrubbed":34844,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:51:44.770701+0000","scrub_duration":2971,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":815792128,"num_objects":12448,"num_object_clones":0,"num_object_copies":37344,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12448,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":68808,"num_write_kb":2600192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[1,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"65'68018","reported_seq":68549,"reported_epoch":65,"state":"active+clean","last_fresh":"2026-02-24T07:51:20.725350+0000","last_change":"2026-02-24T07:51:15.067567+0000","last_active":"2026-02-24T07:51:20.725350+0000","last_peered":"2026-02-24T07:51:20.725350+0000","last_clean":"2026-02-24T07:51:20.725350+0000","last_became_active":"2026-02-24T07:48:57.902224+0000","last_became_peered":"2026-02-24T07:48:57.902224+0000","last_unstale":"2026-02-24T07:51:20.725350+0000","last_undegraded":"2026-02-24T07:51:20.725350+0000","last_fullsized":"2026-02-24T07:51:20.725350+0000","mapping_epoch":55,"log_start":"65'68015","ondisk_log_start":"65'68015","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"65'64892","last_scrub_stamp":"2026-02-24T07:51:15.067417+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:51:15.067417+0000","objects_scrubbed":15892,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:52:16.892433+0000","scrub_duration":1747,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":810024960,"num_objects":12360,"num_object_clones":0,"num_object_copies":37080,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12360,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":68018,"num_write_kb":2572096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"65'68661","reported_seq":69174,"reported_epoch":65,"state":"active+clean","last_fresh":"2026-02-24T07:51:20.834846+0000","last_change":"2026-02-24T07:51:13.251872+0000","last_active":"2026-02-24T07:51:20.834846+0000","last_peered":"2026-02-24T07:51:20.834846+0000","last_clean":"2026-02-24T07:51:20.834846+0000","last_became_active":"2026-02-24T07:48:57.902125+0000","last_became_peered":"2026-02-24T07:48:57.902125+0000","last_unstale":"2026-02-24T07:51:20.834846+0000","last_undegraded":"2026-02-24T07:51:20.834846+0000","last_fullsized":"2026-02-24T07:51:20.834846+0000","mapping_epoch":55,"log_start":"65'68658","ondisk_log_start":"65'68658","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"65'64550","last_scrub_stamp":"2026-02-24T07:51:13.251826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:51:13.251826+0000","objects_scrubbed":17042,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:52:21.793363+0000","scrub_duration":1966,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":821100544,"num_objects":12529,"num_object_clones":0,"num_object_copies":37587,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12529,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":68661,"num_write_kb":2598080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1","version":"65'68882","reported_seq":69322,"reported_epoch":65,"state":"active+clean","last_fresh":"2026-02-24T07:51:20.721713+0000","last_change":"2026-02-24T07:50:29.889880+0000","last_active":"2026-02-24T07:51:20.721713+0000","last_peered":"2026-02-24T07:51:20.721713+0000","last_clean":"2026-02-24T07:51:20.721713+0000","last_became_active":"2026-02-24T07:48:57.903439+0000","last_became_peered":"2026-02-24T07:48:57.903439+0000","last_unstale":"2026-02-24T07:51:20.721713+0000","last_undegraded":"2026-02-24T07:51:20.721713+0000","last_fullsized":"2026-02-24T07:51:20.721713+0000","mapping_epoch":55,"log_start":"65'68879","ondisk_log_start":"65'68879","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"61'41105","last_scrub_stamp":"2026-02-24T07:50:29.889816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:50:29.889816+0000","objects_scrubbed":40127,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:51:50.788414+0000","scrub_duration":4409,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":821035008,"num_objects":12528,"num_object_clones":0,"num_object_copies":37584,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12528,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":68882,"num_write_kb":2605120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.0","version":"65'68483","reported_seq":68921,"reported_epoch":65,"state":"active+clean","last_fresh":"2026-02-24T07:51:20.724462+0000","last_change":"2026-02-24T07:50:29.387323+0000","last_active":"2026-02-24T07:51:20.724462+0000","last_peered":"2026-02-24T07:51:20.724462+0000","last_clean":"2026-02-24T07:51:20.724462+0000","last_became_active":"2026-02-24T07:48:57.902565+0000","last_became_peered":"2026-02-24T07:48:57.902565+0000","last_unstale":"2026-02-24T07:51:20.724462+0000","last_undegraded":"2026-02-24T07:51:20.724462+0000","last_fullsized":"2026-02-24T07:51:20.724462+0000","mapping_epoch":55,"log_start":"65'68480","ondisk_log_start":"65'68480","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"61'40736","last_scrub_stamp":"2026-02-24T07:50:29.387206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:50:29.387206+0000","objects_scrubbed":39946,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:51:59.173732+0000","scrub_duration":3834,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":829620224,"num_objects":12659,"num_object_clones":0,"num_object_copies":37977,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12659,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":68483,"num_write_kb":2596544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,3],"acting":[0,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.5","version":"65'68493","reported_seq":68894,"reported_epoch":65,"state":"active+clean+scrubbing","last_fresh":"2026-02-24T07:51:20.727218+0000","last_change":"2026-02-24T07:51:20.021651+0000","last_active":"2026-02-24T07:51:20.727218+0000","last_peered":"2026-02-24T07:51:20.727218+0000","last_clean":"2026-02-24T07:51:20.727218+0000","last_became_active":"2026-02-24T07:48:57.905127+0000","last_became_peered":"2026-02-24T07:48:57.905127+0000","last_unstale":"2026-02-24T07:51:20.727218+0000","last_undegraded":"2026-02-24T07:51:20.727218+0000","last_fullsized":"2026-02-24T07:51:20.727218+0000","mapping_epoch":55,"log_start":"65'68484","ondisk_log_start":"65'68484","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"61'30833","last_scrub_stamp":"2026-02-24T07:50:04.821861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:50:04.821861+0000","objects_scrubbed":6199,"log_size":9,"log_dups_size":3000,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"scrubbing for 1s","scrub_duration":3578,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":825819136,"num_objects":12601,"num_object_clones":0,"num_object_copies":37803,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12601,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":68493,"num_write_kb":2595008,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"29'99","reported_seq":302,"reported_epoch":65,"state":"active+clean","last_fresh":"2026-02-24T07:50:55.895043+0000","last_change":"2026-02-24T07:50:30.401963+0000","last_active":"2026-02-24T07:50:55.895043+0000","last_peered":"2026-02-24T07:50:55.895043+0000","last_clean":"2026-02-24T07:50:55.895043+0000","last_became_active":"2026-02-24T07:42:54.186719+0000","last_became_peered":"2026-02-24T07:42:54.186719+0000","last_unstale":"2026-02-24T07:50:55.895043+0000","last_undegraded":"2026-02-24T07:50:55.895043+0000","last_fullsized":"2026-02-24T07:50:55.895043+0000","mapping_epoch":27,"log_start":"29'96","ondisk_log_start":"29'96","created":20,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'99","last_scrub_stamp":"2026-02-24T07:50:30.401929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:42:43.640776+0000","last_clean_scrub_stamp":"2026-02-24T07:50:30.401929+0000","objects_scrubbed":2,"log_size":3,"log_dups_size":96,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:51:52.990727+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":240,"num_write":135,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":16,"stat_sum":{"num_bytes":13074432024,"num_objects":199501,"num_object_clones":0,"num_object_copies":598503,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":199501,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1095235,"num_write_kb":41431489,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":39226454016,"data_stored":39226441800,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1896,"internal_metadata":0},"log_size":54,"ondisk_log_size":54,"up":48,"acting":48,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":240,"num_write":135,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1682,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":3,"up_from":25,"seq":107374182504,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":10569468,"kb_used_data":9828584,"kb_used_omap":11,"kb_used_meta":740724,"kb_avail":723433732,"statfs":{"total":751619276800,"available":740796141568,"internally_reserved":0,"allocated":10064470016,"data_stored":10064020944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12200,"internal_metadata":758501464},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":19,"seq":81604378730,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":12191292,"kb_used_data":11205800,"kb_used_omap":13,"kb_used_meta":985458,"kb_avail":721811908,"statfs":{"total":751619276800,"available":739135393792,"internally_reserved":0,"allocated":11474739200,"data_stored":11474290128,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13789,"internal_metadata":1009109539},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542251,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":9668916,"kb_used_data":8081312,"kb_used_omap":18,"kb_used_meta":1587565,"kb_avail":724334284,"statfs":{"total":751619276800,"available":741718306816,"internally_reserved":0,"allocated":8275263488,"data_stored":8274822040,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":18826,"internal_metadata":1625667190},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705774,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":11544124,"kb_used_data":9726824,"kb_used_omap":15,"kb_used_meta":1817264,"kb_avail":722459076,"statfs":{"total":751619276800,"available":739798093824,"internally_reserved":0,"allocated":9960267776,"data_stored":9959818704,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":15703,"internal_metadata":1860879017},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,2],"upper_bound":4},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1682,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":9829224448,"data_stored":9829220376,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1617,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8139702272,"data_stored":8139702272,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":93,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":11440361472,"data_stored":11440357400,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":9817165824,"data_stored":9817161752,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":186,"internal_metadata":0}]}} 2026-02-24T07:51:23.726 INFO:tasks.cephadm.ceph_manager.ceph:increase pool size by 10 2026-02-24T07:51:23.726 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph osd pool set .mgr pg_num 281 2026-02-24T07:51:23.773 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:51:23.868 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:51:23.889 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:51:23.887+0000 7fb3e7dc8640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:51:23.889 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:51:23.887+0000 7fb3e7dc8640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:51:23.904 INFO:teuthology.orchestra.run.trial035.stdout:osd.0: {} 2026-02-24T07:51:23.905 INFO:teuthology.orchestra.run.trial035.stderr:osd.0: osd_enable_op_tracker = '' 2026-02-24T07:51:23.918 INFO:teuthology.orchestra.run.trial035.stdout:osd.1: {} 2026-02-24T07:51:23.919 INFO:teuthology.orchestra.run.trial035.stderr:osd.1: osd_enable_op_tracker = '' 2026-02-24T07:51:23.919 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:51:23 trial035 ceph-mon[39284]: 4.f scrub ok 2026-02-24T07:51:23.919 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:51:23 trial035 ceph-mon[39284]: 4.d scrub starts 2026-02-24T07:51:23.934 INFO:teuthology.orchestra.run.trial035.stdout:osd.2: {} 2026-02-24T07:51:23.934 INFO:teuthology.orchestra.run.trial035.stderr:osd.2: osd_enable_op_tracker = '' 2026-02-24T07:51:23.947 INFO:teuthology.orchestra.run.trial035.stdout:osd.3: {} 2026-02-24T07:51:23.947 INFO:teuthology.orchestra.run.trial035.stderr:osd.3: osd_enable_op_tracker = '' 2026-02-24T07:51:23.996 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:51:23.994+0000 7f5b4b93b640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:51:23.996 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:51:23.994+0000 7f5b4b93b640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:51:24.127 INFO:teuthology.orchestra.run.trial035.stderr:Error EPERM: crimson_allow_pg_split is false; pg_num increase denied for crimson pool 2026-02-24T07:51:24.165 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-24T07:51:25.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:51:24 trial035 ceph-mon[39284]: from='client.15085 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2026-02-24T07:51:25.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:51:24 trial035 ceph-mon[39284]: pgmap v291: 17 pgs: 1 active+clean+scrubbing, 16 active+clean; 12 GiB data, 42 GiB used, 2.7 TiB / 2.7 TiB avail; 0 B/s wr, 6.78k op/s 2026-02-24T07:51:25.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:51:24 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/692619118' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2026-02-24T07:51:25.205 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:51:24 trial035 ceph-mon[39284]: 4.d scrub ok 2026-02-24T07:51:25.205 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:51:24 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/534654603' entity='client.admin' cmd={"prefix": "osd pool set", "pool": ".mgr", "var": "pg_num", "val": "281"} : dispatch 2026-02-24T07:51:25.990 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2026-02-24T07:51:26.129 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:51:26.256 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:51:26.254+0000 7f115c2b3640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:51:26.256 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:51:26.254+0000 7f115c2b3640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:51:26.271 INFO:teuthology.orchestra.run.trial035.stdout:osd.0: {} 2026-02-24T07:51:26.272 INFO:teuthology.orchestra.run.trial035.stderr:osd.0: osd_enable_op_tracker = '' 2026-02-24T07:51:26.285 INFO:teuthology.orchestra.run.trial035.stdout:osd.1: {} 2026-02-24T07:51:26.286 INFO:teuthology.orchestra.run.trial035.stderr:osd.1: osd_enable_op_tracker = '' 2026-02-24T07:51:26.300 INFO:teuthology.orchestra.run.trial035.stdout:osd.2: {} 2026-02-24T07:51:26.300 INFO:teuthology.orchestra.run.trial035.stderr:osd.2: osd_enable_op_tracker = '' 2026-02-24T07:51:26.312 INFO:teuthology.orchestra.run.trial035.stdout:osd.3: {} 2026-02-24T07:51:26.312 INFO:teuthology.orchestra.run.trial035.stderr:osd.3: osd_enable_op_tracker = '' 2026-02-24T07:51:27.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:51:26 trial035 ceph-mon[39284]: pgmap v292: 17 pgs: 1 active+clean+scrubbing, 16 active+clean; 12 GiB data, 42 GiB used, 2.7 TiB / 2.7 TiB avail; 0 B/s wr, 6.78k op/s 2026-02-24T07:51:27.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:51:26 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/2701892065' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2026-02-24T07:51:27.205 INFO:journalctl@ceph.mgr.x.trial035.stdout:Feb 24 07:51:26 trial035 ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-mgr-x[39621]: ::ffff:10.20.193.91 - - [24/Feb/2026:07:51:26] "GET /metrics HTTP/1.1" 200 4229 "" "Prometheus/3.6.0" 2026-02-24T07:51:27.356 INFO:journalctl@ceph.prometheus.a.trial091.stdout:Feb 24 07:51:26 trial091 ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-prometheus-a[66007]: time=2026-02-24T07:51:26.973Z level=ERROR source=scrape.go:1630 msg="Failed to determine correct type of scrape target." component="scrape manager" scrape_pool=ceph target=http://trial035:9283/metrics content_type="text/html;charset=utf-8" fallback_media_type="" err="received unsupported Content-Type \"text/html;charset=utf-8\" and no fallback_scrape_protocol specified for target" 2026-02-24T07:51:28.352 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2026-02-24T07:51:28.489 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:51:28.609 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:51:28.606+0000 7fe3138b6640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:51:28.609 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:51:28.606+0000 7fe3138b6640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:51:28.625 INFO:teuthology.orchestra.run.trial035.stdout:osd.0: {} 2026-02-24T07:51:28.625 INFO:teuthology.orchestra.run.trial035.stderr:osd.0: osd_enable_op_tracker = '' 2026-02-24T07:51:28.641 INFO:teuthology.orchestra.run.trial035.stdout:osd.1: {} 2026-02-24T07:51:28.641 INFO:teuthology.orchestra.run.trial035.stderr:osd.1: osd_enable_op_tracker = '' 2026-02-24T07:51:28.653 INFO:teuthology.orchestra.run.trial035.stdout:osd.2: {} 2026-02-24T07:51:28.654 INFO:teuthology.orchestra.run.trial035.stderr:osd.2: osd_enable_op_tracker = '' 2026-02-24T07:51:28.665 INFO:teuthology.orchestra.run.trial035.stdout:osd.3: {} 2026-02-24T07:51:28.666 INFO:teuthology.orchestra.run.trial035.stderr:osd.3: osd_enable_op_tracker = '' 2026-02-24T07:51:28.954 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:51:28 trial035 ceph-mon[39284]: pgmap v293: 17 pgs: 17 active+clean; 9.7 GiB data, 34 GiB used, 2.7 TiB / 2.7 TiB avail; 0 B/s wr, 10.20k op/s 2026-02-24T07:51:28.954 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:51:28 trial035 ceph-mon[39284]: 4.c scrub starts 2026-02-24T07:51:28.954 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:51:28 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/1587004994' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2026-02-24T07:51:29.166 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2026-02-24T07:51:29.166 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph osd reweight 0 1 2026-02-24T07:51:29.307 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:51:29.429 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:51:29.427+0000 7f84f7bfe640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:51:29.429 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:51:29.428+0000 7f84f7bfe640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:51:29.888 INFO:teuthology.orchestra.run.trial035.stderr:reweighted osd.0 to 1 (10000) 2026-02-24T07:51:29.930 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph osd reweight 1 1 2026-02-24T07:51:30.064 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:51:30.179 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:51:29 trial035 ceph-mon[39284]: 4.c scrub ok 2026-02-24T07:51:30.180 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:51:29 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/2417896768' entity='client.admin' cmd={"prefix": "osd reweight", "id": 0, "weight": 1.0} : dispatch 2026-02-24T07:51:30.180 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:51:29 trial035 ceph-mon[39284]: from='mgr.14284 10.20.193.35:0/3681373222' entity='mgr.x' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2026-02-24T07:51:30.190 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:51:30.188+0000 7fda4b882640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:51:30.190 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:51:30.188+0000 7fda4b882640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:51:30.709 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2026-02-24T07:51:30.842 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:51:30.888 INFO:teuthology.orchestra.run.trial035.stderr:reweighted osd.1 to 1 (10000) 2026-02-24T07:51:30.933 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph osd reweight 2 1 2026-02-24T07:51:30.962 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:51:30.960+0000 7f6d76118640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:51:30.963 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:51:30.960+0000 7f6d76118640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:51:30.981 INFO:teuthology.orchestra.run.trial035.stdout:osd.0: {} 2026-02-24T07:51:30.981 INFO:teuthology.orchestra.run.trial035.stderr:osd.0: osd_enable_op_tracker = '' 2026-02-24T07:51:30.995 INFO:teuthology.orchestra.run.trial035.stdout:osd.1: {} 2026-02-24T07:51:30.996 INFO:teuthology.orchestra.run.trial035.stderr:osd.1: osd_enable_op_tracker = '' 2026-02-24T07:51:30.996 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:51:30 trial035 ceph-mon[39284]: pgmap v294: 17 pgs: 17 active+clean; 9.7 GiB data, 34 GiB used, 2.7 TiB / 2.7 TiB avail; 0 B/s wr, 7.06k op/s 2026-02-24T07:51:30.996 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:51:30 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/2417896768' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 0, "weight": 1.0}]': finished 2026-02-24T07:51:30.996 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:51:30 trial035 ceph-mon[39284]: osdmap e66: 4 total, 4 up, 4 in 2026-02-24T07:51:30.996 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:51:30 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/1266194467' entity='client.admin' cmd={"prefix": "osd reweight", "id": 1, "weight": 1.0} : dispatch 2026-02-24T07:51:31.008 INFO:teuthology.orchestra.run.trial035.stdout:osd.2: {} 2026-02-24T07:51:31.008 INFO:teuthology.orchestra.run.trial035.stderr:osd.2: osd_enable_op_tracker = '' 2026-02-24T07:51:31.024 INFO:teuthology.orchestra.run.trial035.stdout:osd.3: {} 2026-02-24T07:51:31.024 INFO:teuthology.orchestra.run.trial035.stderr:osd.3: osd_enable_op_tracker = '' 2026-02-24T07:51:31.077 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:51:31.456 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:51:31.387+0000 7f8ce618a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:51:31.456 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:51:31.387+0000 7f8ce618a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:51:31.894 INFO:teuthology.orchestra.run.trial035.stderr:reweighted osd.2 to 1 (10000) 2026-02-24T07:51:32.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:51:31 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/1266194467' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 1, "weight": 1.0}]': finished 2026-02-24T07:51:32.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:51:31 trial035 ceph-mon[39284]: osdmap e67: 4 total, 4 up, 4 in 2026-02-24T07:51:32.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:51:31 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/3153953946' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2026-02-24T07:51:32.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:51:31 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/986766644' entity='client.admin' cmd={"prefix": "osd reweight", "id": 2, "weight": 1.0} : dispatch 2026-02-24T07:51:32.552 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph osd reweight 3 1 2026-02-24T07:51:32.739 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:51:33.072 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2026-02-24T07:51:33.138 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:51:32 trial035 ceph-mon[39284]: pgmap v297: 17 pgs: 17 active+clean; 6.8 GiB data, 25 GiB used, 2.7 TiB / 2.7 TiB avail; 0 B/s wr, 10.95k op/s 2026-02-24T07:51:33.139 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:51:32 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/986766644' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 2, "weight": 1.0}]': finished 2026-02-24T07:51:33.139 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:51:32 trial035 ceph-mon[39284]: osdmap e68: 4 total, 4 up, 4 in 2026-02-24T07:51:33.139 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:51:32 trial035 ceph-mon[39284]: 4.e scrub starts 2026-02-24T07:51:33.203 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:51:33.136+0000 7f04e9fda640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:51:33.204 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:51:33.136+0000 7f04e9fda640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:51:33.213 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:51:33.721 INFO:teuthology.orchestra.run.trial035.stdout:osd.0: {} 2026-02-24T07:51:33.721 INFO:teuthology.orchestra.run.trial035.stdout:osd.1: {} 2026-02-24T07:51:33.722 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:51:33.671+0000 7f44ab55a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:51:33.722 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:51:33.671+0000 7f44ab55a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:51:33.722 INFO:teuthology.orchestra.run.trial035.stderr:osd.0: osd_enable_op_tracker = '' 2026-02-24T07:51:33.722 INFO:teuthology.orchestra.run.trial035.stderr:osd.1: osd_enable_op_tracker = '' 2026-02-24T07:51:33.731 INFO:teuthology.orchestra.run.trial035.stdout:osd.2: {} 2026-02-24T07:51:33.731 INFO:teuthology.orchestra.run.trial035.stderr:osd.2: osd_enable_op_tracker = '' 2026-02-24T07:51:33.744 INFO:teuthology.orchestra.run.trial035.stdout:osd.3: {} 2026-02-24T07:51:33.744 INFO:teuthology.orchestra.run.trial035.stderr:osd.3: osd_enable_op_tracker = '' 2026-02-24T07:51:33.879 INFO:teuthology.orchestra.run.trial035.stderr:reweighted osd.3 to 1 (10000) 2026-02-24T07:51:33.954 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:51:33 trial035 ceph-mon[39284]: 4.7 scrub starts 2026-02-24T07:51:33.954 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:51:33 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/3047221139' entity='client.admin' cmd={"prefix": "osd reweight", "id": 3, "weight": 1.0} : dispatch 2026-02-24T07:51:33.954 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:51:33 trial035 ceph-mon[39284]: 4.e scrub ok 2026-02-24T07:51:33.954 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:51:33 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/3795255425' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2026-02-24T07:51:33.954 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:51:33 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/3047221139' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 3, "weight": 1.0}]': finished 2026-02-24T07:51:33.954 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:51:33 trial035 ceph-mon[39284]: osdmap e69: 4 total, 4 up, 4 in 2026-02-24T07:51:34.594 INFO:tasks.cephadm.ceph_manager.ceph:waiting for recovery to complete 2026-02-24T07:51:34.594 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph pg dump --format=json 2026-02-24T07:51:34.736 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:51:34.858 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:51:34.856+0000 7fb08ab2f640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:51:34.858 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:51:34.856+0000 7fb08ab2f640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:51:35.004 INFO:teuthology.orchestra.run.trial035.stdout: 2026-02-24T07:51:35.005 INFO:teuthology.orchestra.run.trial035.stderr:dumped all 2026-02-24T07:51:35.047 INFO:teuthology.orchestra.run.trial035.stdout:{"pg_ready":true,"pg_map":{"version":300,"stamp":"2026-02-24T07:51:33.878933+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":7332364856,"num_objects":111877,"num_object_clones":0,"num_object_copies":335631,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":111877,"num_whiteouts":0,"num_read":137,"num_read_kb":241,"num_write":1182996,"num_write_kb":41433639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":52,"ondisk_log_size":52,"up":51,"acting":51,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":51,"num_osds":4,"num_per_pool_osds":4,"num_per_pool_omap_osds":4,"kb":2936012800,"kb_used":26343832,"kb_used_data":22150920,"kb_used_omap":57,"kb_used_meta":4192582,"kb_avail":2909668968,"statfs":{"total":3006477107200,"available":2979501023232,"internally_reserved":0,"allocated":22682542080,"data_stored":22680721900,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":58527,"internal_metadata":4293204833},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[4,1],"upper_bound":4},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-3052077056,"num_objects":-46571,"num_object_clones":0,"num_object_copies":-139713,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-46571,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":46571,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.152523"},"pg_stats":[{"pgid":"4.f","version":"66'74362","reported_seq":74834,"reported_epoch":66,"state":"active+clean","last_fresh":"2026-02-24T07:51:30.842204+0000","last_change":"2026-02-24T07:51:22.471952+0000","last_active":"2026-02-24T07:51:30.842204+0000","last_peered":"2026-02-24T07:51:30.842204+0000","last_clean":"2026-02-24T07:51:30.842204+0000","last_became_active":"2026-02-24T07:48:57.902313+0000","last_became_peered":"2026-02-24T07:48:57.902313+0000","last_unstale":"2026-02-24T07:51:30.842204+0000","last_undegraded":"2026-02-24T07:51:30.842204+0000","last_fullsized":"2026-02-24T07:51:30.842204+0000","mapping_epoch":55,"log_start":"66'74359","ondisk_log_start":"66'74359","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"65'69619","last_scrub_stamp":"2026-02-24T07:51:22.471822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:51:22.471822+0000","objects_scrubbed":12055,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:52:37.258331+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":457310208,"num_objects":6978,"num_object_clones":0,"num_object_copies":20934,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6978,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":74362,"num_write_kb":2602880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"66'74138","reported_seq":74547,"reported_epoch":66,"state":"active+clean","last_fresh":"2026-02-24T07:51:30.738132+0000","last_change":"2026-02-24T07:50:21.065676+0000","last_active":"2026-02-24T07:51:30.738132+0000","last_peered":"2026-02-24T07:51:30.738132+0000","last_clean":"2026-02-24T07:51:30.738132+0000","last_became_active":"2026-02-24T07:48:57.903584+0000","last_became_peered":"2026-02-24T07:48:57.903584+0000","last_unstale":"2026-02-24T07:51:30.738132+0000","last_undegraded":"2026-02-24T07:51:30.738132+0000","last_fullsized":"2026-02-24T07:51:30.738132+0000","mapping_epoch":55,"log_start":"66'74135","ondisk_log_start":"66'74135","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"61'37487","last_scrub_stamp":"2026-02-24T07:50:21.065638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:50:21.065638+0000","objects_scrubbed":36748,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:51:31.991874+0000","scrub_duration":3643,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":461373440,"num_objects":7040,"num_object_clones":0,"num_object_copies":21120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7040,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":74138,"num_write_kb":2597696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.d","version":"66'73840","reported_seq":74313,"reported_epoch":66,"state":"active+clean","last_fresh":"2026-02-24T07:51:30.748473+0000","last_change":"2026-02-24T07:51:24.114191+0000","last_active":"2026-02-24T07:51:30.748473+0000","last_peered":"2026-02-24T07:51:30.748473+0000","last_clean":"2026-02-24T07:51:30.748473+0000","last_became_active":"2026-02-24T07:48:57.905031+0000","last_became_peered":"2026-02-24T07:48:57.905031+0000","last_unstale":"2026-02-24T07:51:30.748473+0000","last_undegraded":"2026-02-24T07:51:30.748473+0000","last_fullsized":"2026-02-24T07:51:30.748473+0000","mapping_epoch":55,"log_start":"66'73837","ondisk_log_start":"66'73837","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"65'69926","last_scrub_stamp":"2026-02-24T07:51:24.113625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:51:24.113625+0000","objects_scrubbed":11114,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:52:43.886535+0000","scrub_duration":1132,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":455540760,"num_objects":6952,"num_object_clones":0,"num_object_copies":20856,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6952,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":73840,"num_write_kb":2585281,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"66'73630","reported_seq":74141,"reported_epoch":66,"state":"active+clean","last_fresh":"2026-02-24T07:51:30.747170+0000","last_change":"2026-02-24T07:51:28.745731+0000","last_active":"2026-02-24T07:51:30.747170+0000","last_peered":"2026-02-24T07:51:30.747170+0000","last_clean":"2026-02-24T07:51:30.747170+0000","last_became_active":"2026-02-24T07:48:57.904683+0000","last_became_peered":"2026-02-24T07:48:57.904683+0000","last_unstale":"2026-02-24T07:51:30.747170+0000","last_undegraded":"2026-02-24T07:51:30.747170+0000","last_fullsized":"2026-02-24T07:51:30.747170+0000","mapping_epoch":55,"log_start":"66'73626","ondisk_log_start":"66'73626","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"65'72446","last_scrub_stamp":"2026-02-24T07:51:28.745680+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:51:28.745680+0000","objects_scrubbed":8236,"log_size":4,"log_dups_size":3000,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:52:34.536079+0000","scrub_duration":716,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":449708032,"num_objects":6862,"num_object_clones":0,"num_object_copies":20586,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6862,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":73630,"num_write_kb":2575744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.b","version":"66'73583","reported_seq":74015,"reported_epoch":66,"state":"active+clean","last_fresh":"2026-02-24T07:51:30.760323+0000","last_change":"2026-02-24T07:50:27.267008+0000","last_active":"2026-02-24T07:51:30.760323+0000","last_peered":"2026-02-24T07:51:30.760323+0000","last_clean":"2026-02-24T07:51:30.760323+0000","last_became_active":"2026-02-24T07:48:57.901700+0000","last_became_peered":"2026-02-24T07:48:57.901700+0000","last_unstale":"2026-02-24T07:51:30.760323+0000","last_undegraded":"2026-02-24T07:51:30.760323+0000","last_fullsized":"2026-02-24T07:51:30.760323+0000","mapping_epoch":55,"log_start":"66'73580","ondisk_log_start":"66'73580","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"61'39639","last_scrub_stamp":"2026-02-24T07:50:27.266943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:50:27.266943+0000","objects_scrubbed":38982,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:51:34.346751+0000","scrub_duration":3744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":463405056,"num_objects":7071,"num_object_clones":0,"num_object_copies":21213,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7071,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":73583,"num_write_kb":2580928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.a","version":"66'74052","reported_seq":74487,"reported_epoch":66,"state":"active+clean","last_fresh":"2026-02-24T07:51:30.737550+0000","last_change":"2026-02-24T07:50:33.043786+0000","last_active":"2026-02-24T07:51:30.737550+0000","last_peered":"2026-02-24T07:51:30.737550+0000","last_clean":"2026-02-24T07:51:30.737550+0000","last_became_active":"2026-02-24T07:48:57.903228+0000","last_became_peered":"2026-02-24T07:48:57.903228+0000","last_unstale":"2026-02-24T07:51:30.737550+0000","last_undegraded":"2026-02-24T07:51:30.737550+0000","last_fullsized":"2026-02-24T07:51:30.737550+0000","mapping_epoch":55,"log_start":"66'74049","ondisk_log_start":"66'74049","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"61'42539","last_scrub_stamp":"2026-02-24T07:50:33.043726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:50:33.043726+0000","objects_scrubbed":39246,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:51:51.239665+0000","scrub_duration":3547,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":447873024,"num_objects":6834,"num_object_clones":0,"num_object_copies":20502,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6834,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":74052,"num_write_kb":2588352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"66'73942","reported_seq":74317,"reported_epoch":66,"state":"active+clean","last_fresh":"2026-02-24T07:51:30.737897+0000","last_change":"2026-02-24T07:50:13.165492+0000","last_active":"2026-02-24T07:51:30.737897+0000","last_peered":"2026-02-24T07:51:30.737897+0000","last_clean":"2026-02-24T07:51:30.737897+0000","last_became_active":"2026-02-24T07:48:57.901730+0000","last_became_peered":"2026-02-24T07:48:57.901730+0000","last_unstale":"2026-02-24T07:51:30.737897+0000","last_undegraded":"2026-02-24T07:51:30.737897+0000","last_fullsized":"2026-02-24T07:51:30.737897+0000","mapping_epoch":55,"log_start":"66'73939","ondisk_log_start":"66'73939","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"61'33943","last_scrub_stamp":"2026-02-24T07:50:13.165459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:50:13.165459+0000","objects_scrubbed":33253,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:51:33.820708+0000","scrub_duration":3776,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":454033408,"num_objects":6928,"num_object_clones":0,"num_object_copies":20784,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6928,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":73942,"num_write_kb":2587840,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"66'73738","reported_seq":74258,"reported_epoch":66,"state":"active+clean","last_fresh":"2026-02-24T07:51:30.751280+0000","last_change":"2026-02-24T07:51:12.758378+0000","last_active":"2026-02-24T07:51:30.751280+0000","last_peered":"2026-02-24T07:51:30.751280+0000","last_clean":"2026-02-24T07:51:30.751280+0000","last_became_active":"2026-02-24T07:48:57.904376+0000","last_became_peered":"2026-02-24T07:48:57.904376+0000","last_unstale":"2026-02-24T07:51:30.751280+0000","last_undegraded":"2026-02-24T07:51:30.751280+0000","last_fullsized":"2026-02-24T07:51:30.751280+0000","mapping_epoch":55,"log_start":"66'73735","ondisk_log_start":"66'73735","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"65'64020","last_scrub_stamp":"2026-02-24T07:51:12.758336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:51:12.758336+0000","objects_scrubbed":17129,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:52:28.496810+0000","scrub_duration":1688,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":465960960,"num_objects":7110,"num_object_clones":0,"num_object_copies":21330,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7110,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":73738,"num_write_kb":2587136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.7","version":"66'73570","reported_seq":73936,"reported_epoch":66,"state":"active+clean","last_fresh":"2026-02-24T07:51:30.841210+0000","last_change":"2026-02-24T07:50:11.227608+0000","last_active":"2026-02-24T07:51:30.841210+0000","last_peered":"2026-02-24T07:51:30.841210+0000","last_clean":"2026-02-24T07:51:30.841210+0000","last_became_active":"2026-02-24T07:48:57.902484+0000","last_became_peered":"2026-02-24T07:48:57.902484+0000","last_unstale":"2026-02-24T07:51:30.841210+0000","last_undegraded":"2026-02-24T07:51:30.841210+0000","last_fullsized":"2026-02-24T07:51:30.841210+0000","mapping_epoch":55,"log_start":"66'73567","ondisk_log_start":"66'73567","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"61'32990","last_scrub_stamp":"2026-02-24T07:50:11.227573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:50:11.227573+0000","objects_scrubbed":32460,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"queued for scrub","scrub_duration":3701,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":463732736,"num_objects":7076,"num_object_clones":0,"num_object_copies":21228,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7076,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":73570,"num_write_kb":2580672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.6","version":"66'73591","reported_seq":74112,"reported_epoch":66,"state":"active+clean","last_fresh":"2026-02-24T07:51:30.759139+0000","last_change":"2026-02-24T07:51:19.852133+0000","last_active":"2026-02-24T07:51:30.759139+0000","last_peered":"2026-02-24T07:51:30.759139+0000","last_clean":"2026-02-24T07:51:30.759139+0000","last_became_active":"2026-02-24T07:48:57.901565+0000","last_became_peered":"2026-02-24T07:48:57.901565+0000","last_unstale":"2026-02-24T07:51:30.759139+0000","last_undegraded":"2026-02-24T07:51:30.759139+0000","last_fullsized":"2026-02-24T07:51:30.759139+0000","mapping_epoch":55,"log_start":"66'73588","ondisk_log_start":"66'73588","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"65'67636","last_scrub_stamp":"2026-02-24T07:51:19.852077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:51:19.852077+0000","objects_scrubbed":13301,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:52:23.813957+0000","scrub_duration":1519,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":456720384,"num_objects":6969,"num_object_clones":0,"num_object_copies":20907,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6969,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":73591,"num_write_kb":2577920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"66'74358","reported_seq":74749,"reported_epoch":66,"state":"active+clean","last_fresh":"2026-02-24T07:51:30.843109+0000","last_change":"2026-02-24T07:50:16.399238+0000","last_active":"2026-02-24T07:51:30.843109+0000","last_peered":"2026-02-24T07:51:30.843109+0000","last_clean":"2026-02-24T07:51:30.843109+0000","last_became_active":"2026-02-24T07:48:57.901824+0000","last_became_peered":"2026-02-24T07:48:57.901824+0000","last_unstale":"2026-02-24T07:51:30.843109+0000","last_undegraded":"2026-02-24T07:51:30.843109+0000","last_fullsized":"2026-02-24T07:51:30.843109+0000","mapping_epoch":55,"log_start":"66'74355","ondisk_log_start":"66'74355","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"61'35510","last_scrub_stamp":"2026-02-24T07:50:16.399194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:50:16.399194+0000","objects_scrubbed":34844,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:51:44.770701+0000","scrub_duration":2971,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":452067328,"num_objects":6898,"num_object_clones":0,"num_object_copies":20694,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6898,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":74358,"num_write_kb":2600192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[1,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"66'73468","reported_seq":74001,"reported_epoch":66,"state":"active+clean","last_fresh":"2026-02-24T07:51:30.759511+0000","last_change":"2026-02-24T07:51:15.067567+0000","last_active":"2026-02-24T07:51:30.759511+0000","last_peered":"2026-02-24T07:51:30.759511+0000","last_clean":"2026-02-24T07:51:30.759511+0000","last_became_active":"2026-02-24T07:48:57.902224+0000","last_became_peered":"2026-02-24T07:48:57.902224+0000","last_unstale":"2026-02-24T07:51:30.759511+0000","last_undegraded":"2026-02-24T07:51:30.759511+0000","last_fullsized":"2026-02-24T07:51:30.759511+0000","mapping_epoch":55,"log_start":"66'73465","ondisk_log_start":"66'73465","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"65'64892","last_scrub_stamp":"2026-02-24T07:51:15.067417+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:51:15.067417+0000","objects_scrubbed":15892,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:52:16.892433+0000","scrub_duration":1747,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":452853760,"num_objects":6910,"num_object_clones":0,"num_object_copies":20730,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6910,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":73468,"num_write_kb":2572096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"66'74253","reported_seq":74768,"reported_epoch":66,"state":"active+clean","last_fresh":"2026-02-24T07:51:30.841273+0000","last_change":"2026-02-24T07:51:13.251872+0000","last_active":"2026-02-24T07:51:30.841273+0000","last_peered":"2026-02-24T07:51:30.841273+0000","last_clean":"2026-02-24T07:51:30.841273+0000","last_became_active":"2026-02-24T07:48:57.902125+0000","last_became_peered":"2026-02-24T07:48:57.902125+0000","last_unstale":"2026-02-24T07:51:30.841273+0000","last_undegraded":"2026-02-24T07:51:30.841273+0000","last_fullsized":"2026-02-24T07:51:30.841273+0000","mapping_epoch":55,"log_start":"66'74250","ondisk_log_start":"66'74250","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"65'64550","last_scrub_stamp":"2026-02-24T07:51:13.251826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:51:13.251826+0000","objects_scrubbed":17042,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:52:21.793363+0000","scrub_duration":1966,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":454623232,"num_objects":6937,"num_object_clones":0,"num_object_copies":20811,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6937,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":74253,"num_write_kb":2598080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1","version":"66'74345","reported_seq":74787,"reported_epoch":66,"state":"active+clean","last_fresh":"2026-02-24T07:51:30.734567+0000","last_change":"2026-02-24T07:50:29.889880+0000","last_active":"2026-02-24T07:51:30.734567+0000","last_peered":"2026-02-24T07:51:30.734567+0000","last_clean":"2026-02-24T07:51:30.734567+0000","last_became_active":"2026-02-24T07:48:57.903439+0000","last_became_peered":"2026-02-24T07:48:57.903439+0000","last_unstale":"2026-02-24T07:51:30.734567+0000","last_undegraded":"2026-02-24T07:51:30.734567+0000","last_fullsized":"2026-02-24T07:51:30.734567+0000","mapping_epoch":55,"log_start":"66'74342","ondisk_log_start":"66'74342","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"61'41105","last_scrub_stamp":"2026-02-24T07:50:29.889816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:50:29.889816+0000","objects_scrubbed":40127,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:51:50.788414+0000","scrub_duration":4409,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":463011840,"num_objects":7065,"num_object_clones":0,"num_object_copies":21195,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7065,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":74345,"num_write_kb":2605120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.0","version":"66'74068","reported_seq":74510,"reported_epoch":66,"state":"active+clean","last_fresh":"2026-02-24T07:51:30.760404+0000","last_change":"2026-02-24T07:50:29.387323+0000","last_active":"2026-02-24T07:51:30.760404+0000","last_peered":"2026-02-24T07:51:30.760404+0000","last_clean":"2026-02-24T07:51:30.760404+0000","last_became_active":"2026-02-24T07:48:57.902565+0000","last_became_peered":"2026-02-24T07:48:57.902565+0000","last_unstale":"2026-02-24T07:51:30.760404+0000","last_undegraded":"2026-02-24T07:51:30.760404+0000","last_fullsized":"2026-02-24T07:51:30.760404+0000","mapping_epoch":55,"log_start":"66'74065","ondisk_log_start":"66'74065","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"61'40736","last_scrub_stamp":"2026-02-24T07:50:29.387206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:50:29.387206+0000","objects_scrubbed":39946,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:51:59.173732+0000","scrub_duration":3834,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":463601664,"num_objects":7074,"num_object_clones":0,"num_object_copies":21222,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7074,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":74068,"num_write_kb":2596544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,3],"acting":[0,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.5","version":"66'73923","reported_seq":74404,"reported_epoch":66,"state":"active+clean","last_fresh":"2026-02-24T07:51:30.747949+0000","last_change":"2026-02-24T07:51:21.557037+0000","last_active":"2026-02-24T07:51:30.747949+0000","last_peered":"2026-02-24T07:51:30.747949+0000","last_clean":"2026-02-24T07:51:30.747949+0000","last_became_active":"2026-02-24T07:48:57.905127+0000","last_became_peered":"2026-02-24T07:48:57.905127+0000","last_unstale":"2026-02-24T07:51:30.747949+0000","last_undegraded":"2026-02-24T07:51:30.747949+0000","last_fullsized":"2026-02-24T07:51:30.747949+0000","mapping_epoch":55,"log_start":"66'73920","ondisk_log_start":"66'73920","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"65'68757","last_scrub_stamp":"2026-02-24T07:51:21.556951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:51:21.556951+0000","objects_scrubbed":12622,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:52:49.364567+0000","scrub_duration":1535,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":469958656,"num_objects":7171,"num_object_clones":0,"num_object_copies":21513,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7171,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":73923,"num_write_kb":2595008,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"29'99","reported_seq":304,"reported_epoch":66,"state":"active+clean","last_fresh":"2026-02-24T07:51:29.888058+0000","last_change":"2026-02-24T07:50:30.401963+0000","last_active":"2026-02-24T07:51:29.888058+0000","last_peered":"2026-02-24T07:51:29.888058+0000","last_clean":"2026-02-24T07:51:29.888058+0000","last_became_active":"2026-02-24T07:42:54.186719+0000","last_became_peered":"2026-02-24T07:42:54.186719+0000","last_unstale":"2026-02-24T07:51:29.888058+0000","last_undegraded":"2026-02-24T07:51:29.888058+0000","last_fullsized":"2026-02-24T07:51:29.888058+0000","mapping_epoch":27,"log_start":"29'96","ondisk_log_start":"29'96","created":20,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'99","last_scrub_stamp":"2026-02-24T07:50:30.401929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:42:43.640776+0000","last_clean_scrub_stamp":"2026-02-24T07:50:30.401929+0000","objects_scrubbed":2,"log_size":3,"log_dups_size":96,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:51:52.990727+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":240,"num_write":135,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":16,"stat_sum":{"num_bytes":7331774488,"num_objects":111875,"num_object_clones":0,"num_object_copies":335625,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":111875,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1182861,"num_write_kb":41431489,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":22002085888,"data_stored":22002073672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":2750,"internal_metadata":0},"log_size":49,"ondisk_log_size":49,"up":48,"acting":48,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":240,"num_write":135,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1682,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":3,"up_from":25,"seq":107374182506,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":6534664,"kb_used_data":5667572,"kb_used_omap":12,"kb_used_meta":867059,"kb_avail":727468536,"statfs":{"total":751619276800,"available":744927780864,"internally_reserved":0,"allocated":5803593728,"data_stored":5803136777,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13130,"internal_metadata":887868598},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":19,"seq":81604378732,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":7476232,"kb_used_data":6346292,"kb_used_omap":14,"kb_used_meta":1129841,"kb_avail":726526968,"statfs":{"total":751619276800,"available":743963615232,"internally_reserved":0,"allocated":6498603008,"data_stored":6498146057,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":14441,"internal_metadata":1156958103},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[4,1],"upper_bound":4},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542253,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":6340480,"kb_used_data":4644524,"kb_used_omap":19,"kb_used_meta":1695852,"kb_avail":727662720,"statfs":{"total":751619276800,"available":745126625280,"internally_reserved":0,"allocated":4755992576,"data_stored":4755543249,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19509,"internal_metadata":1736553419},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705776,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":5992456,"kb_used_data":5492532,"kb_used_omap":11,"kb_used_meta":499828,"kb_avail":728010744,"statfs":{"total":751619276800,"available":745483001856,"internally_reserved":0,"allocated":5624352768,"data_stored":5623895817,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":11447,"internal_metadata":511824713},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1682,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":5521281024,"data_stored":5521276952,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1789,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4539875328,"data_stored":4539875328,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":124,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":6437605376,"data_stored":6437601304,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":5503324160,"data_stored":5503320088,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":837,"internal_metadata":0}]}} 2026-02-24T07:51:35.049 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph pg dump --format=json 2026-02-24T07:51:35.193 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:51:35.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:51:34 trial035 ceph-mon[39284]: 4.7 scrub ok 2026-02-24T07:51:35.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:51:34 trial035 ceph-mon[39284]: pgmap v299: 17 pgs: 17 active+clean; 6.8 GiB data, 25 GiB used, 2.7 TiB / 2.7 TiB avail; 0 B/s wr, 7.76k op/s 2026-02-24T07:51:35.320 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:51:35.317+0000 7f59c0de7640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:51:35.320 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:51:35.318+0000 7f59c0de7640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:51:35.471 INFO:teuthology.orchestra.run.trial035.stdout: 2026-02-24T07:51:35.472 INFO:teuthology.orchestra.run.trial035.stderr:dumped all 2026-02-24T07:51:35.514 INFO:teuthology.orchestra.run.trial035.stdout:{"pg_ready":true,"pg_map":{"version":300,"stamp":"2026-02-24T07:51:33.878933+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":7332364856,"num_objects":111877,"num_object_clones":0,"num_object_copies":335631,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":111877,"num_whiteouts":0,"num_read":137,"num_read_kb":241,"num_write":1182996,"num_write_kb":41433639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":52,"ondisk_log_size":52,"up":51,"acting":51,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":51,"num_osds":4,"num_per_pool_osds":4,"num_per_pool_omap_osds":4,"kb":2936012800,"kb_used":26343832,"kb_used_data":22150920,"kb_used_omap":57,"kb_used_meta":4192582,"kb_avail":2909668968,"statfs":{"total":3006477107200,"available":2979501023232,"internally_reserved":0,"allocated":22682542080,"data_stored":22680721900,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":58527,"internal_metadata":4293204833},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[4,1],"upper_bound":4},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-3052077056,"num_objects":-46571,"num_object_clones":0,"num_object_copies":-139713,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-46571,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":46571,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.152523"},"pg_stats":[{"pgid":"4.f","version":"66'74362","reported_seq":74834,"reported_epoch":66,"state":"active+clean","last_fresh":"2026-02-24T07:51:30.842204+0000","last_change":"2026-02-24T07:51:22.471952+0000","last_active":"2026-02-24T07:51:30.842204+0000","last_peered":"2026-02-24T07:51:30.842204+0000","last_clean":"2026-02-24T07:51:30.842204+0000","last_became_active":"2026-02-24T07:48:57.902313+0000","last_became_peered":"2026-02-24T07:48:57.902313+0000","last_unstale":"2026-02-24T07:51:30.842204+0000","last_undegraded":"2026-02-24T07:51:30.842204+0000","last_fullsized":"2026-02-24T07:51:30.842204+0000","mapping_epoch":55,"log_start":"66'74359","ondisk_log_start":"66'74359","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"65'69619","last_scrub_stamp":"2026-02-24T07:51:22.471822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:51:22.471822+0000","objects_scrubbed":12055,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:52:37.258331+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":457310208,"num_objects":6978,"num_object_clones":0,"num_object_copies":20934,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6978,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":74362,"num_write_kb":2602880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"66'74138","reported_seq":74547,"reported_epoch":66,"state":"active+clean","last_fresh":"2026-02-24T07:51:30.738132+0000","last_change":"2026-02-24T07:50:21.065676+0000","last_active":"2026-02-24T07:51:30.738132+0000","last_peered":"2026-02-24T07:51:30.738132+0000","last_clean":"2026-02-24T07:51:30.738132+0000","last_became_active":"2026-02-24T07:48:57.903584+0000","last_became_peered":"2026-02-24T07:48:57.903584+0000","last_unstale":"2026-02-24T07:51:30.738132+0000","last_undegraded":"2026-02-24T07:51:30.738132+0000","last_fullsized":"2026-02-24T07:51:30.738132+0000","mapping_epoch":55,"log_start":"66'74135","ondisk_log_start":"66'74135","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"61'37487","last_scrub_stamp":"2026-02-24T07:50:21.065638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:50:21.065638+0000","objects_scrubbed":36748,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:51:31.991874+0000","scrub_duration":3643,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":461373440,"num_objects":7040,"num_object_clones":0,"num_object_copies":21120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7040,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":74138,"num_write_kb":2597696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.d","version":"66'73840","reported_seq":74313,"reported_epoch":66,"state":"active+clean","last_fresh":"2026-02-24T07:51:30.748473+0000","last_change":"2026-02-24T07:51:24.114191+0000","last_active":"2026-02-24T07:51:30.748473+0000","last_peered":"2026-02-24T07:51:30.748473+0000","last_clean":"2026-02-24T07:51:30.748473+0000","last_became_active":"2026-02-24T07:48:57.905031+0000","last_became_peered":"2026-02-24T07:48:57.905031+0000","last_unstale":"2026-02-24T07:51:30.748473+0000","last_undegraded":"2026-02-24T07:51:30.748473+0000","last_fullsized":"2026-02-24T07:51:30.748473+0000","mapping_epoch":55,"log_start":"66'73837","ondisk_log_start":"66'73837","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"65'69926","last_scrub_stamp":"2026-02-24T07:51:24.113625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:51:24.113625+0000","objects_scrubbed":11114,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:52:43.886535+0000","scrub_duration":1132,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":455540760,"num_objects":6952,"num_object_clones":0,"num_object_copies":20856,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6952,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":73840,"num_write_kb":2585281,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"66'73630","reported_seq":74141,"reported_epoch":66,"state":"active+clean","last_fresh":"2026-02-24T07:51:30.747170+0000","last_change":"2026-02-24T07:51:28.745731+0000","last_active":"2026-02-24T07:51:30.747170+0000","last_peered":"2026-02-24T07:51:30.747170+0000","last_clean":"2026-02-24T07:51:30.747170+0000","last_became_active":"2026-02-24T07:48:57.904683+0000","last_became_peered":"2026-02-24T07:48:57.904683+0000","last_unstale":"2026-02-24T07:51:30.747170+0000","last_undegraded":"2026-02-24T07:51:30.747170+0000","last_fullsized":"2026-02-24T07:51:30.747170+0000","mapping_epoch":55,"log_start":"66'73626","ondisk_log_start":"66'73626","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"65'72446","last_scrub_stamp":"2026-02-24T07:51:28.745680+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:51:28.745680+0000","objects_scrubbed":8236,"log_size":4,"log_dups_size":3000,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:52:34.536079+0000","scrub_duration":716,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":449708032,"num_objects":6862,"num_object_clones":0,"num_object_copies":20586,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6862,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":73630,"num_write_kb":2575744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.b","version":"66'73583","reported_seq":74015,"reported_epoch":66,"state":"active+clean","last_fresh":"2026-02-24T07:51:30.760323+0000","last_change":"2026-02-24T07:50:27.267008+0000","last_active":"2026-02-24T07:51:30.760323+0000","last_peered":"2026-02-24T07:51:30.760323+0000","last_clean":"2026-02-24T07:51:30.760323+0000","last_became_active":"2026-02-24T07:48:57.901700+0000","last_became_peered":"2026-02-24T07:48:57.901700+0000","last_unstale":"2026-02-24T07:51:30.760323+0000","last_undegraded":"2026-02-24T07:51:30.760323+0000","last_fullsized":"2026-02-24T07:51:30.760323+0000","mapping_epoch":55,"log_start":"66'73580","ondisk_log_start":"66'73580","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"61'39639","last_scrub_stamp":"2026-02-24T07:50:27.266943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:50:27.266943+0000","objects_scrubbed":38982,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:51:34.346751+0000","scrub_duration":3744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":463405056,"num_objects":7071,"num_object_clones":0,"num_object_copies":21213,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7071,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":73583,"num_write_kb":2580928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.a","version":"66'74052","reported_seq":74487,"reported_epoch":66,"state":"active+clean","last_fresh":"2026-02-24T07:51:30.737550+0000","last_change":"2026-02-24T07:50:33.043786+0000","last_active":"2026-02-24T07:51:30.737550+0000","last_peered":"2026-02-24T07:51:30.737550+0000","last_clean":"2026-02-24T07:51:30.737550+0000","last_became_active":"2026-02-24T07:48:57.903228+0000","last_became_peered":"2026-02-24T07:48:57.903228+0000","last_unstale":"2026-02-24T07:51:30.737550+0000","last_undegraded":"2026-02-24T07:51:30.737550+0000","last_fullsized":"2026-02-24T07:51:30.737550+0000","mapping_epoch":55,"log_start":"66'74049","ondisk_log_start":"66'74049","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"61'42539","last_scrub_stamp":"2026-02-24T07:50:33.043726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:50:33.043726+0000","objects_scrubbed":39246,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:51:51.239665+0000","scrub_duration":3547,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":447873024,"num_objects":6834,"num_object_clones":0,"num_object_copies":20502,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6834,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":74052,"num_write_kb":2588352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"66'73942","reported_seq":74317,"reported_epoch":66,"state":"active+clean","last_fresh":"2026-02-24T07:51:30.737897+0000","last_change":"2026-02-24T07:50:13.165492+0000","last_active":"2026-02-24T07:51:30.737897+0000","last_peered":"2026-02-24T07:51:30.737897+0000","last_clean":"2026-02-24T07:51:30.737897+0000","last_became_active":"2026-02-24T07:48:57.901730+0000","last_became_peered":"2026-02-24T07:48:57.901730+0000","last_unstale":"2026-02-24T07:51:30.737897+0000","last_undegraded":"2026-02-24T07:51:30.737897+0000","last_fullsized":"2026-02-24T07:51:30.737897+0000","mapping_epoch":55,"log_start":"66'73939","ondisk_log_start":"66'73939","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"61'33943","last_scrub_stamp":"2026-02-24T07:50:13.165459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:50:13.165459+0000","objects_scrubbed":33253,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:51:33.820708+0000","scrub_duration":3776,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":454033408,"num_objects":6928,"num_object_clones":0,"num_object_copies":20784,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6928,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":73942,"num_write_kb":2587840,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"66'73738","reported_seq":74258,"reported_epoch":66,"state":"active+clean","last_fresh":"2026-02-24T07:51:30.751280+0000","last_change":"2026-02-24T07:51:12.758378+0000","last_active":"2026-02-24T07:51:30.751280+0000","last_peered":"2026-02-24T07:51:30.751280+0000","last_clean":"2026-02-24T07:51:30.751280+0000","last_became_active":"2026-02-24T07:48:57.904376+0000","last_became_peered":"2026-02-24T07:48:57.904376+0000","last_unstale":"2026-02-24T07:51:30.751280+0000","last_undegraded":"2026-02-24T07:51:30.751280+0000","last_fullsized":"2026-02-24T07:51:30.751280+0000","mapping_epoch":55,"log_start":"66'73735","ondisk_log_start":"66'73735","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"65'64020","last_scrub_stamp":"2026-02-24T07:51:12.758336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:51:12.758336+0000","objects_scrubbed":17129,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:52:28.496810+0000","scrub_duration":1688,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":465960960,"num_objects":7110,"num_object_clones":0,"num_object_copies":21330,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7110,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":73738,"num_write_kb":2587136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.7","version":"66'73570","reported_seq":73936,"reported_epoch":66,"state":"active+clean","last_fresh":"2026-02-24T07:51:30.841210+0000","last_change":"2026-02-24T07:50:11.227608+0000","last_active":"2026-02-24T07:51:30.841210+0000","last_peered":"2026-02-24T07:51:30.841210+0000","last_clean":"2026-02-24T07:51:30.841210+0000","last_became_active":"2026-02-24T07:48:57.902484+0000","last_became_peered":"2026-02-24T07:48:57.902484+0000","last_unstale":"2026-02-24T07:51:30.841210+0000","last_undegraded":"2026-02-24T07:51:30.841210+0000","last_fullsized":"2026-02-24T07:51:30.841210+0000","mapping_epoch":55,"log_start":"66'73567","ondisk_log_start":"66'73567","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"61'32990","last_scrub_stamp":"2026-02-24T07:50:11.227573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:50:11.227573+0000","objects_scrubbed":32460,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"queued for scrub","scrub_duration":3701,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":463732736,"num_objects":7076,"num_object_clones":0,"num_object_copies":21228,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7076,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":73570,"num_write_kb":2580672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.6","version":"66'73591","reported_seq":74112,"reported_epoch":66,"state":"active+clean","last_fresh":"2026-02-24T07:51:30.759139+0000","last_change":"2026-02-24T07:51:19.852133+0000","last_active":"2026-02-24T07:51:30.759139+0000","last_peered":"2026-02-24T07:51:30.759139+0000","last_clean":"2026-02-24T07:51:30.759139+0000","last_became_active":"2026-02-24T07:48:57.901565+0000","last_became_peered":"2026-02-24T07:48:57.901565+0000","last_unstale":"2026-02-24T07:51:30.759139+0000","last_undegraded":"2026-02-24T07:51:30.759139+0000","last_fullsized":"2026-02-24T07:51:30.759139+0000","mapping_epoch":55,"log_start":"66'73588","ondisk_log_start":"66'73588","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"65'67636","last_scrub_stamp":"2026-02-24T07:51:19.852077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:51:19.852077+0000","objects_scrubbed":13301,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:52:23.813957+0000","scrub_duration":1519,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":456720384,"num_objects":6969,"num_object_clones":0,"num_object_copies":20907,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6969,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":73591,"num_write_kb":2577920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"66'74358","reported_seq":74749,"reported_epoch":66,"state":"active+clean","last_fresh":"2026-02-24T07:51:30.843109+0000","last_change":"2026-02-24T07:50:16.399238+0000","last_active":"2026-02-24T07:51:30.843109+0000","last_peered":"2026-02-24T07:51:30.843109+0000","last_clean":"2026-02-24T07:51:30.843109+0000","last_became_active":"2026-02-24T07:48:57.901824+0000","last_became_peered":"2026-02-24T07:48:57.901824+0000","last_unstale":"2026-02-24T07:51:30.843109+0000","last_undegraded":"2026-02-24T07:51:30.843109+0000","last_fullsized":"2026-02-24T07:51:30.843109+0000","mapping_epoch":55,"log_start":"66'74355","ondisk_log_start":"66'74355","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"61'35510","last_scrub_stamp":"2026-02-24T07:50:16.399194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:50:16.399194+0000","objects_scrubbed":34844,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:51:44.770701+0000","scrub_duration":2971,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":452067328,"num_objects":6898,"num_object_clones":0,"num_object_copies":20694,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6898,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":74358,"num_write_kb":2600192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[1,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"66'73468","reported_seq":74001,"reported_epoch":66,"state":"active+clean","last_fresh":"2026-02-24T07:51:30.759511+0000","last_change":"2026-02-24T07:51:15.067567+0000","last_active":"2026-02-24T07:51:30.759511+0000","last_peered":"2026-02-24T07:51:30.759511+0000","last_clean":"2026-02-24T07:51:30.759511+0000","last_became_active":"2026-02-24T07:48:57.902224+0000","last_became_peered":"2026-02-24T07:48:57.902224+0000","last_unstale":"2026-02-24T07:51:30.759511+0000","last_undegraded":"2026-02-24T07:51:30.759511+0000","last_fullsized":"2026-02-24T07:51:30.759511+0000","mapping_epoch":55,"log_start":"66'73465","ondisk_log_start":"66'73465","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"65'64892","last_scrub_stamp":"2026-02-24T07:51:15.067417+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:51:15.067417+0000","objects_scrubbed":15892,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:52:16.892433+0000","scrub_duration":1747,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":452853760,"num_objects":6910,"num_object_clones":0,"num_object_copies":20730,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6910,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":73468,"num_write_kb":2572096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"66'74253","reported_seq":74768,"reported_epoch":66,"state":"active+clean","last_fresh":"2026-02-24T07:51:30.841273+0000","last_change":"2026-02-24T07:51:13.251872+0000","last_active":"2026-02-24T07:51:30.841273+0000","last_peered":"2026-02-24T07:51:30.841273+0000","last_clean":"2026-02-24T07:51:30.841273+0000","last_became_active":"2026-02-24T07:48:57.902125+0000","last_became_peered":"2026-02-24T07:48:57.902125+0000","last_unstale":"2026-02-24T07:51:30.841273+0000","last_undegraded":"2026-02-24T07:51:30.841273+0000","last_fullsized":"2026-02-24T07:51:30.841273+0000","mapping_epoch":55,"log_start":"66'74250","ondisk_log_start":"66'74250","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"65'64550","last_scrub_stamp":"2026-02-24T07:51:13.251826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:51:13.251826+0000","objects_scrubbed":17042,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:52:21.793363+0000","scrub_duration":1966,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":454623232,"num_objects":6937,"num_object_clones":0,"num_object_copies":20811,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6937,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":74253,"num_write_kb":2598080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1","version":"66'74345","reported_seq":74787,"reported_epoch":66,"state":"active+clean","last_fresh":"2026-02-24T07:51:30.734567+0000","last_change":"2026-02-24T07:50:29.889880+0000","last_active":"2026-02-24T07:51:30.734567+0000","last_peered":"2026-02-24T07:51:30.734567+0000","last_clean":"2026-02-24T07:51:30.734567+0000","last_became_active":"2026-02-24T07:48:57.903439+0000","last_became_peered":"2026-02-24T07:48:57.903439+0000","last_unstale":"2026-02-24T07:51:30.734567+0000","last_undegraded":"2026-02-24T07:51:30.734567+0000","last_fullsized":"2026-02-24T07:51:30.734567+0000","mapping_epoch":55,"log_start":"66'74342","ondisk_log_start":"66'74342","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"61'41105","last_scrub_stamp":"2026-02-24T07:50:29.889816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:50:29.889816+0000","objects_scrubbed":40127,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:51:50.788414+0000","scrub_duration":4409,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":463011840,"num_objects":7065,"num_object_clones":0,"num_object_copies":21195,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7065,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":74345,"num_write_kb":2605120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.0","version":"66'74068","reported_seq":74510,"reported_epoch":66,"state":"active+clean","last_fresh":"2026-02-24T07:51:30.760404+0000","last_change":"2026-02-24T07:50:29.387323+0000","last_active":"2026-02-24T07:51:30.760404+0000","last_peered":"2026-02-24T07:51:30.760404+0000","last_clean":"2026-02-24T07:51:30.760404+0000","last_became_active":"2026-02-24T07:48:57.902565+0000","last_became_peered":"2026-02-24T07:48:57.902565+0000","last_unstale":"2026-02-24T07:51:30.760404+0000","last_undegraded":"2026-02-24T07:51:30.760404+0000","last_fullsized":"2026-02-24T07:51:30.760404+0000","mapping_epoch":55,"log_start":"66'74065","ondisk_log_start":"66'74065","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"61'40736","last_scrub_stamp":"2026-02-24T07:50:29.387206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:50:29.387206+0000","objects_scrubbed":39946,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:51:59.173732+0000","scrub_duration":3834,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":463601664,"num_objects":7074,"num_object_clones":0,"num_object_copies":21222,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7074,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":74068,"num_write_kb":2596544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,3],"acting":[0,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.5","version":"66'73923","reported_seq":74404,"reported_epoch":66,"state":"active+clean","last_fresh":"2026-02-24T07:51:30.747949+0000","last_change":"2026-02-24T07:51:21.557037+0000","last_active":"2026-02-24T07:51:30.747949+0000","last_peered":"2026-02-24T07:51:30.747949+0000","last_clean":"2026-02-24T07:51:30.747949+0000","last_became_active":"2026-02-24T07:48:57.905127+0000","last_became_peered":"2026-02-24T07:48:57.905127+0000","last_unstale":"2026-02-24T07:51:30.747949+0000","last_undegraded":"2026-02-24T07:51:30.747949+0000","last_fullsized":"2026-02-24T07:51:30.747949+0000","mapping_epoch":55,"log_start":"66'73920","ondisk_log_start":"66'73920","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"65'68757","last_scrub_stamp":"2026-02-24T07:51:21.556951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:51:21.556951+0000","objects_scrubbed":12622,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:52:49.364567+0000","scrub_duration":1535,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":469958656,"num_objects":7171,"num_object_clones":0,"num_object_copies":21513,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7171,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":73923,"num_write_kb":2595008,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"29'99","reported_seq":304,"reported_epoch":66,"state":"active+clean","last_fresh":"2026-02-24T07:51:29.888058+0000","last_change":"2026-02-24T07:50:30.401963+0000","last_active":"2026-02-24T07:51:29.888058+0000","last_peered":"2026-02-24T07:51:29.888058+0000","last_clean":"2026-02-24T07:51:29.888058+0000","last_became_active":"2026-02-24T07:42:54.186719+0000","last_became_peered":"2026-02-24T07:42:54.186719+0000","last_unstale":"2026-02-24T07:51:29.888058+0000","last_undegraded":"2026-02-24T07:51:29.888058+0000","last_fullsized":"2026-02-24T07:51:29.888058+0000","mapping_epoch":27,"log_start":"29'96","ondisk_log_start":"29'96","created":20,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'99","last_scrub_stamp":"2026-02-24T07:50:30.401929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:42:43.640776+0000","last_clean_scrub_stamp":"2026-02-24T07:50:30.401929+0000","objects_scrubbed":2,"log_size":3,"log_dups_size":96,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:51:52.990727+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":240,"num_write":135,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":16,"stat_sum":{"num_bytes":7331774488,"num_objects":111875,"num_object_clones":0,"num_object_copies":335625,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":111875,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1182861,"num_write_kb":41431489,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":22002085888,"data_stored":22002073672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":2750,"internal_metadata":0},"log_size":49,"ondisk_log_size":49,"up":48,"acting":48,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":240,"num_write":135,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1682,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":3,"up_from":25,"seq":107374182506,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":6534664,"kb_used_data":5667572,"kb_used_omap":12,"kb_used_meta":867059,"kb_avail":727468536,"statfs":{"total":751619276800,"available":744927780864,"internally_reserved":0,"allocated":5803593728,"data_stored":5803136777,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13130,"internal_metadata":887868598},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":19,"seq":81604378732,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":7476232,"kb_used_data":6346292,"kb_used_omap":14,"kb_used_meta":1129841,"kb_avail":726526968,"statfs":{"total":751619276800,"available":743963615232,"internally_reserved":0,"allocated":6498603008,"data_stored":6498146057,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":14441,"internal_metadata":1156958103},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[4,1],"upper_bound":4},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542253,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":6340480,"kb_used_data":4644524,"kb_used_omap":19,"kb_used_meta":1695852,"kb_avail":727662720,"statfs":{"total":751619276800,"available":745126625280,"internally_reserved":0,"allocated":4755992576,"data_stored":4755543249,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19509,"internal_metadata":1736553419},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705776,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":5992456,"kb_used_data":5492532,"kb_used_omap":11,"kb_used_meta":499828,"kb_avail":728010744,"statfs":{"total":751619276800,"available":745483001856,"internally_reserved":0,"allocated":5624352768,"data_stored":5623895817,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":11447,"internal_metadata":511824713},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1682,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":5521281024,"data_stored":5521276952,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1789,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4539875328,"data_stored":4539875328,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":124,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":6437605376,"data_stored":6437601304,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":5503324160,"data_stored":5503320088,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":837,"internal_metadata":0}]}} 2026-02-24T07:51:35.518 INFO:tasks.cephadm.ceph_manager.ceph:recovered! 2026-02-24T07:51:35.518 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2026-02-24T07:51:35.518 INFO:tasks.thrashosds.thrasher:Growing pool unique_pool_2 2026-02-24T07:51:35.518 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph pg dump --format=json 2026-02-24T07:51:35.656 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:51:35.775 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:51:35.773+0000 7f344a72b640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:51:35.775 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:51:35.773+0000 7f344a72b640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:51:35.920 INFO:teuthology.orchestra.run.trial035.stdout: 2026-02-24T07:51:35.920 INFO:teuthology.orchestra.run.trial035.stderr:dumped all 2026-02-24T07:51:35.966 INFO:teuthology.orchestra.run.trial035.stdout:{"pg_ready":true,"pg_map":{"version":301,"stamp":"2026-02-24T07:51:35.727213+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":7332364856,"num_objects":111877,"num_object_clones":0,"num_object_copies":335631,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":111877,"num_whiteouts":0,"num_read":137,"num_read_kb":241,"num_write":1182996,"num_write_kb":41433639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":52,"ondisk_log_size":52,"up":51,"acting":51,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":51,"num_osds":4,"num_per_pool_osds":4,"num_per_pool_omap_osds":4,"kb":2936012800,"kb_used":26343832,"kb_used_data":22150920,"kb_used_omap":57,"kb_used_meta":4192582,"kb_avail":2909668968,"statfs":{"total":3006477107200,"available":2979501023232,"internally_reserved":0,"allocated":22682542080,"data_stored":22680721900,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":58527,"internal_metadata":4293204833},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[4,1],"upper_bound":4},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-3052077056,"num_objects":-46571,"num_object_clones":0,"num_object_copies":-139713,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-46571,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":46571,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.839865"},"pg_stats":[{"pgid":"4.f","version":"66'74362","reported_seq":74834,"reported_epoch":66,"state":"active+clean","last_fresh":"2026-02-24T07:51:30.842204+0000","last_change":"2026-02-24T07:51:22.471952+0000","last_active":"2026-02-24T07:51:30.842204+0000","last_peered":"2026-02-24T07:51:30.842204+0000","last_clean":"2026-02-24T07:51:30.842204+0000","last_became_active":"2026-02-24T07:48:57.902313+0000","last_became_peered":"2026-02-24T07:48:57.902313+0000","last_unstale":"2026-02-24T07:51:30.842204+0000","last_undegraded":"2026-02-24T07:51:30.842204+0000","last_fullsized":"2026-02-24T07:51:30.842204+0000","mapping_epoch":55,"log_start":"66'74359","ondisk_log_start":"66'74359","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"65'69619","last_scrub_stamp":"2026-02-24T07:51:22.471822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:51:22.471822+0000","objects_scrubbed":12055,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:52:37.258331+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":457310208,"num_objects":6978,"num_object_clones":0,"num_object_copies":20934,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6978,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":74362,"num_write_kb":2602880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"66'74138","reported_seq":74547,"reported_epoch":66,"state":"active+clean","last_fresh":"2026-02-24T07:51:30.738132+0000","last_change":"2026-02-24T07:50:21.065676+0000","last_active":"2026-02-24T07:51:30.738132+0000","last_peered":"2026-02-24T07:51:30.738132+0000","last_clean":"2026-02-24T07:51:30.738132+0000","last_became_active":"2026-02-24T07:48:57.903584+0000","last_became_peered":"2026-02-24T07:48:57.903584+0000","last_unstale":"2026-02-24T07:51:30.738132+0000","last_undegraded":"2026-02-24T07:51:30.738132+0000","last_fullsized":"2026-02-24T07:51:30.738132+0000","mapping_epoch":55,"log_start":"66'74135","ondisk_log_start":"66'74135","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"61'37487","last_scrub_stamp":"2026-02-24T07:50:21.065638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:50:21.065638+0000","objects_scrubbed":36748,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:51:31.991874+0000","scrub_duration":3643,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":461373440,"num_objects":7040,"num_object_clones":0,"num_object_copies":21120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7040,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":74138,"num_write_kb":2597696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.d","version":"66'73840","reported_seq":74313,"reported_epoch":66,"state":"active+clean","last_fresh":"2026-02-24T07:51:30.748473+0000","last_change":"2026-02-24T07:51:24.114191+0000","last_active":"2026-02-24T07:51:30.748473+0000","last_peered":"2026-02-24T07:51:30.748473+0000","last_clean":"2026-02-24T07:51:30.748473+0000","last_became_active":"2026-02-24T07:48:57.905031+0000","last_became_peered":"2026-02-24T07:48:57.905031+0000","last_unstale":"2026-02-24T07:51:30.748473+0000","last_undegraded":"2026-02-24T07:51:30.748473+0000","last_fullsized":"2026-02-24T07:51:30.748473+0000","mapping_epoch":55,"log_start":"66'73837","ondisk_log_start":"66'73837","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"65'69926","last_scrub_stamp":"2026-02-24T07:51:24.113625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:51:24.113625+0000","objects_scrubbed":11114,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:52:43.886535+0000","scrub_duration":1132,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":455540760,"num_objects":6952,"num_object_clones":0,"num_object_copies":20856,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6952,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":73840,"num_write_kb":2585281,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"66'73630","reported_seq":74141,"reported_epoch":66,"state":"active+clean","last_fresh":"2026-02-24T07:51:30.747170+0000","last_change":"2026-02-24T07:51:28.745731+0000","last_active":"2026-02-24T07:51:30.747170+0000","last_peered":"2026-02-24T07:51:30.747170+0000","last_clean":"2026-02-24T07:51:30.747170+0000","last_became_active":"2026-02-24T07:48:57.904683+0000","last_became_peered":"2026-02-24T07:48:57.904683+0000","last_unstale":"2026-02-24T07:51:30.747170+0000","last_undegraded":"2026-02-24T07:51:30.747170+0000","last_fullsized":"2026-02-24T07:51:30.747170+0000","mapping_epoch":55,"log_start":"66'73626","ondisk_log_start":"66'73626","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"65'72446","last_scrub_stamp":"2026-02-24T07:51:28.745680+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:51:28.745680+0000","objects_scrubbed":8236,"log_size":4,"log_dups_size":3000,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:52:34.536079+0000","scrub_duration":716,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":449708032,"num_objects":6862,"num_object_clones":0,"num_object_copies":20586,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6862,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":73630,"num_write_kb":2575744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.b","version":"66'73583","reported_seq":74015,"reported_epoch":66,"state":"active+clean","last_fresh":"2026-02-24T07:51:30.760323+0000","last_change":"2026-02-24T07:50:27.267008+0000","last_active":"2026-02-24T07:51:30.760323+0000","last_peered":"2026-02-24T07:51:30.760323+0000","last_clean":"2026-02-24T07:51:30.760323+0000","last_became_active":"2026-02-24T07:48:57.901700+0000","last_became_peered":"2026-02-24T07:48:57.901700+0000","last_unstale":"2026-02-24T07:51:30.760323+0000","last_undegraded":"2026-02-24T07:51:30.760323+0000","last_fullsized":"2026-02-24T07:51:30.760323+0000","mapping_epoch":55,"log_start":"66'73580","ondisk_log_start":"66'73580","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"61'39639","last_scrub_stamp":"2026-02-24T07:50:27.266943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:50:27.266943+0000","objects_scrubbed":38982,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:51:34.346751+0000","scrub_duration":3744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":463405056,"num_objects":7071,"num_object_clones":0,"num_object_copies":21213,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7071,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":73583,"num_write_kb":2580928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.a","version":"66'74052","reported_seq":74487,"reported_epoch":66,"state":"active+clean","last_fresh":"2026-02-24T07:51:30.737550+0000","last_change":"2026-02-24T07:50:33.043786+0000","last_active":"2026-02-24T07:51:30.737550+0000","last_peered":"2026-02-24T07:51:30.737550+0000","last_clean":"2026-02-24T07:51:30.737550+0000","last_became_active":"2026-02-24T07:48:57.903228+0000","last_became_peered":"2026-02-24T07:48:57.903228+0000","last_unstale":"2026-02-24T07:51:30.737550+0000","last_undegraded":"2026-02-24T07:51:30.737550+0000","last_fullsized":"2026-02-24T07:51:30.737550+0000","mapping_epoch":55,"log_start":"66'74049","ondisk_log_start":"66'74049","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"61'42539","last_scrub_stamp":"2026-02-24T07:50:33.043726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:50:33.043726+0000","objects_scrubbed":39246,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:51:51.239665+0000","scrub_duration":3547,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":447873024,"num_objects":6834,"num_object_clones":0,"num_object_copies":20502,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6834,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":74052,"num_write_kb":2588352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"66'73942","reported_seq":74317,"reported_epoch":66,"state":"active+clean","last_fresh":"2026-02-24T07:51:30.737897+0000","last_change":"2026-02-24T07:50:13.165492+0000","last_active":"2026-02-24T07:51:30.737897+0000","last_peered":"2026-02-24T07:51:30.737897+0000","last_clean":"2026-02-24T07:51:30.737897+0000","last_became_active":"2026-02-24T07:48:57.901730+0000","last_became_peered":"2026-02-24T07:48:57.901730+0000","last_unstale":"2026-02-24T07:51:30.737897+0000","last_undegraded":"2026-02-24T07:51:30.737897+0000","last_fullsized":"2026-02-24T07:51:30.737897+0000","mapping_epoch":55,"log_start":"66'73939","ondisk_log_start":"66'73939","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"61'33943","last_scrub_stamp":"2026-02-24T07:50:13.165459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:50:13.165459+0000","objects_scrubbed":33253,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:51:33.820708+0000","scrub_duration":3776,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":454033408,"num_objects":6928,"num_object_clones":0,"num_object_copies":20784,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6928,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":73942,"num_write_kb":2587840,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"66'73738","reported_seq":74258,"reported_epoch":66,"state":"active+clean","last_fresh":"2026-02-24T07:51:30.751280+0000","last_change":"2026-02-24T07:51:12.758378+0000","last_active":"2026-02-24T07:51:30.751280+0000","last_peered":"2026-02-24T07:51:30.751280+0000","last_clean":"2026-02-24T07:51:30.751280+0000","last_became_active":"2026-02-24T07:48:57.904376+0000","last_became_peered":"2026-02-24T07:48:57.904376+0000","last_unstale":"2026-02-24T07:51:30.751280+0000","last_undegraded":"2026-02-24T07:51:30.751280+0000","last_fullsized":"2026-02-24T07:51:30.751280+0000","mapping_epoch":55,"log_start":"66'73735","ondisk_log_start":"66'73735","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"65'64020","last_scrub_stamp":"2026-02-24T07:51:12.758336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:51:12.758336+0000","objects_scrubbed":17129,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:52:28.496810+0000","scrub_duration":1688,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":465960960,"num_objects":7110,"num_object_clones":0,"num_object_copies":21330,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7110,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":73738,"num_write_kb":2587136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.7","version":"66'73570","reported_seq":73936,"reported_epoch":66,"state":"active+clean","last_fresh":"2026-02-24T07:51:30.841210+0000","last_change":"2026-02-24T07:50:11.227608+0000","last_active":"2026-02-24T07:51:30.841210+0000","last_peered":"2026-02-24T07:51:30.841210+0000","last_clean":"2026-02-24T07:51:30.841210+0000","last_became_active":"2026-02-24T07:48:57.902484+0000","last_became_peered":"2026-02-24T07:48:57.902484+0000","last_unstale":"2026-02-24T07:51:30.841210+0000","last_undegraded":"2026-02-24T07:51:30.841210+0000","last_fullsized":"2026-02-24T07:51:30.841210+0000","mapping_epoch":55,"log_start":"66'73567","ondisk_log_start":"66'73567","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"61'32990","last_scrub_stamp":"2026-02-24T07:50:11.227573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:50:11.227573+0000","objects_scrubbed":32460,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"queued for scrub","scrub_duration":3701,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":463732736,"num_objects":7076,"num_object_clones":0,"num_object_copies":21228,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7076,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":73570,"num_write_kb":2580672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.6","version":"66'73591","reported_seq":74112,"reported_epoch":66,"state":"active+clean","last_fresh":"2026-02-24T07:51:30.759139+0000","last_change":"2026-02-24T07:51:19.852133+0000","last_active":"2026-02-24T07:51:30.759139+0000","last_peered":"2026-02-24T07:51:30.759139+0000","last_clean":"2026-02-24T07:51:30.759139+0000","last_became_active":"2026-02-24T07:48:57.901565+0000","last_became_peered":"2026-02-24T07:48:57.901565+0000","last_unstale":"2026-02-24T07:51:30.759139+0000","last_undegraded":"2026-02-24T07:51:30.759139+0000","last_fullsized":"2026-02-24T07:51:30.759139+0000","mapping_epoch":55,"log_start":"66'73588","ondisk_log_start":"66'73588","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"65'67636","last_scrub_stamp":"2026-02-24T07:51:19.852077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:51:19.852077+0000","objects_scrubbed":13301,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:52:23.813957+0000","scrub_duration":1519,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":456720384,"num_objects":6969,"num_object_clones":0,"num_object_copies":20907,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6969,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":73591,"num_write_kb":2577920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"66'74358","reported_seq":74749,"reported_epoch":66,"state":"active+clean","last_fresh":"2026-02-24T07:51:30.843109+0000","last_change":"2026-02-24T07:50:16.399238+0000","last_active":"2026-02-24T07:51:30.843109+0000","last_peered":"2026-02-24T07:51:30.843109+0000","last_clean":"2026-02-24T07:51:30.843109+0000","last_became_active":"2026-02-24T07:48:57.901824+0000","last_became_peered":"2026-02-24T07:48:57.901824+0000","last_unstale":"2026-02-24T07:51:30.843109+0000","last_undegraded":"2026-02-24T07:51:30.843109+0000","last_fullsized":"2026-02-24T07:51:30.843109+0000","mapping_epoch":55,"log_start":"66'74355","ondisk_log_start":"66'74355","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"61'35510","last_scrub_stamp":"2026-02-24T07:50:16.399194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:50:16.399194+0000","objects_scrubbed":34844,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:51:44.770701+0000","scrub_duration":2971,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":452067328,"num_objects":6898,"num_object_clones":0,"num_object_copies":20694,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6898,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":74358,"num_write_kb":2600192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[1,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"66'73468","reported_seq":74001,"reported_epoch":66,"state":"active+clean","last_fresh":"2026-02-24T07:51:30.759511+0000","last_change":"2026-02-24T07:51:15.067567+0000","last_active":"2026-02-24T07:51:30.759511+0000","last_peered":"2026-02-24T07:51:30.759511+0000","last_clean":"2026-02-24T07:51:30.759511+0000","last_became_active":"2026-02-24T07:48:57.902224+0000","last_became_peered":"2026-02-24T07:48:57.902224+0000","last_unstale":"2026-02-24T07:51:30.759511+0000","last_undegraded":"2026-02-24T07:51:30.759511+0000","last_fullsized":"2026-02-24T07:51:30.759511+0000","mapping_epoch":55,"log_start":"66'73465","ondisk_log_start":"66'73465","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"65'64892","last_scrub_stamp":"2026-02-24T07:51:15.067417+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:51:15.067417+0000","objects_scrubbed":15892,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:52:16.892433+0000","scrub_duration":1747,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":452853760,"num_objects":6910,"num_object_clones":0,"num_object_copies":20730,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6910,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":73468,"num_write_kb":2572096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"66'74253","reported_seq":74768,"reported_epoch":66,"state":"active+clean","last_fresh":"2026-02-24T07:51:30.841273+0000","last_change":"2026-02-24T07:51:13.251872+0000","last_active":"2026-02-24T07:51:30.841273+0000","last_peered":"2026-02-24T07:51:30.841273+0000","last_clean":"2026-02-24T07:51:30.841273+0000","last_became_active":"2026-02-24T07:48:57.902125+0000","last_became_peered":"2026-02-24T07:48:57.902125+0000","last_unstale":"2026-02-24T07:51:30.841273+0000","last_undegraded":"2026-02-24T07:51:30.841273+0000","last_fullsized":"2026-02-24T07:51:30.841273+0000","mapping_epoch":55,"log_start":"66'74250","ondisk_log_start":"66'74250","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"65'64550","last_scrub_stamp":"2026-02-24T07:51:13.251826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:51:13.251826+0000","objects_scrubbed":17042,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:52:21.793363+0000","scrub_duration":1966,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":454623232,"num_objects":6937,"num_object_clones":0,"num_object_copies":20811,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6937,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":74253,"num_write_kb":2598080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1","version":"66'74345","reported_seq":74787,"reported_epoch":66,"state":"active+clean","last_fresh":"2026-02-24T07:51:30.734567+0000","last_change":"2026-02-24T07:50:29.889880+0000","last_active":"2026-02-24T07:51:30.734567+0000","last_peered":"2026-02-24T07:51:30.734567+0000","last_clean":"2026-02-24T07:51:30.734567+0000","last_became_active":"2026-02-24T07:48:57.903439+0000","last_became_peered":"2026-02-24T07:48:57.903439+0000","last_unstale":"2026-02-24T07:51:30.734567+0000","last_undegraded":"2026-02-24T07:51:30.734567+0000","last_fullsized":"2026-02-24T07:51:30.734567+0000","mapping_epoch":55,"log_start":"66'74342","ondisk_log_start":"66'74342","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"61'41105","last_scrub_stamp":"2026-02-24T07:50:29.889816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:50:29.889816+0000","objects_scrubbed":40127,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:51:50.788414+0000","scrub_duration":4409,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":463011840,"num_objects":7065,"num_object_clones":0,"num_object_copies":21195,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7065,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":74345,"num_write_kb":2605120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.0","version":"66'74068","reported_seq":74510,"reported_epoch":66,"state":"active+clean","last_fresh":"2026-02-24T07:51:30.760404+0000","last_change":"2026-02-24T07:50:29.387323+0000","last_active":"2026-02-24T07:51:30.760404+0000","last_peered":"2026-02-24T07:51:30.760404+0000","last_clean":"2026-02-24T07:51:30.760404+0000","last_became_active":"2026-02-24T07:48:57.902565+0000","last_became_peered":"2026-02-24T07:48:57.902565+0000","last_unstale":"2026-02-24T07:51:30.760404+0000","last_undegraded":"2026-02-24T07:51:30.760404+0000","last_fullsized":"2026-02-24T07:51:30.760404+0000","mapping_epoch":55,"log_start":"66'74065","ondisk_log_start":"66'74065","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"61'40736","last_scrub_stamp":"2026-02-24T07:50:29.387206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:50:29.387206+0000","objects_scrubbed":39946,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:51:59.173732+0000","scrub_duration":3834,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":463601664,"num_objects":7074,"num_object_clones":0,"num_object_copies":21222,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7074,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":74068,"num_write_kb":2596544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,3],"acting":[0,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.5","version":"66'73923","reported_seq":74404,"reported_epoch":66,"state":"active+clean","last_fresh":"2026-02-24T07:51:30.747949+0000","last_change":"2026-02-24T07:51:21.557037+0000","last_active":"2026-02-24T07:51:30.747949+0000","last_peered":"2026-02-24T07:51:30.747949+0000","last_clean":"2026-02-24T07:51:30.747949+0000","last_became_active":"2026-02-24T07:48:57.905127+0000","last_became_peered":"2026-02-24T07:48:57.905127+0000","last_unstale":"2026-02-24T07:51:30.747949+0000","last_undegraded":"2026-02-24T07:51:30.747949+0000","last_fullsized":"2026-02-24T07:51:30.747949+0000","mapping_epoch":55,"log_start":"66'73920","ondisk_log_start":"66'73920","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"65'68757","last_scrub_stamp":"2026-02-24T07:51:21.556951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:51:21.556951+0000","objects_scrubbed":12622,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:52:49.364567+0000","scrub_duration":1535,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":469958656,"num_objects":7171,"num_object_clones":0,"num_object_copies":21513,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7171,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":73923,"num_write_kb":2595008,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"29'99","reported_seq":304,"reported_epoch":66,"state":"active+clean","last_fresh":"2026-02-24T07:51:29.888058+0000","last_change":"2026-02-24T07:50:30.401963+0000","last_active":"2026-02-24T07:51:29.888058+0000","last_peered":"2026-02-24T07:51:29.888058+0000","last_clean":"2026-02-24T07:51:29.888058+0000","last_became_active":"2026-02-24T07:42:54.186719+0000","last_became_peered":"2026-02-24T07:42:54.186719+0000","last_unstale":"2026-02-24T07:51:29.888058+0000","last_undegraded":"2026-02-24T07:51:29.888058+0000","last_fullsized":"2026-02-24T07:51:29.888058+0000","mapping_epoch":27,"log_start":"29'96","ondisk_log_start":"29'96","created":20,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'99","last_scrub_stamp":"2026-02-24T07:50:30.401929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:42:43.640776+0000","last_clean_scrub_stamp":"2026-02-24T07:50:30.401929+0000","objects_scrubbed":2,"log_size":3,"log_dups_size":96,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:51:52.990727+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":240,"num_write":135,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":16,"stat_sum":{"num_bytes":7331774488,"num_objects":111875,"num_object_clones":0,"num_object_copies":335625,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":111875,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1182861,"num_write_kb":41431489,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":22002085888,"data_stored":22002073672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":2750,"internal_metadata":0},"log_size":49,"ondisk_log_size":49,"up":48,"acting":48,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":240,"num_write":135,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1682,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":3,"up_from":25,"seq":107374182506,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":6534664,"kb_used_data":5667572,"kb_used_omap":12,"kb_used_meta":867059,"kb_avail":727468536,"statfs":{"total":751619276800,"available":744927780864,"internally_reserved":0,"allocated":5803593728,"data_stored":5803136777,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13130,"internal_metadata":887868598},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":19,"seq":81604378732,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":7476232,"kb_used_data":6346292,"kb_used_omap":14,"kb_used_meta":1129841,"kb_avail":726526968,"statfs":{"total":751619276800,"available":743963615232,"internally_reserved":0,"allocated":6498603008,"data_stored":6498146057,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":14441,"internal_metadata":1156958103},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[4,1],"upper_bound":4},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542253,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":6340480,"kb_used_data":4644524,"kb_used_omap":19,"kb_used_meta":1695852,"kb_avail":727662720,"statfs":{"total":751619276800,"available":745126625280,"internally_reserved":0,"allocated":4755992576,"data_stored":4755543249,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19509,"internal_metadata":1736553419},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705776,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":5992456,"kb_used_data":5492532,"kb_used_omap":11,"kb_used_meta":499828,"kb_avail":728010744,"statfs":{"total":751619276800,"available":745483001856,"internally_reserved":0,"allocated":5624352768,"data_stored":5623895817,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":11447,"internal_metadata":511824713},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1682,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":5521281024,"data_stored":5521276952,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1789,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4539875328,"data_stored":4539875328,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":124,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":6437605376,"data_stored":6437601304,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":5503324160,"data_stored":5503320088,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":837,"internal_metadata":0}]}} 2026-02-24T07:51:35.969 INFO:tasks.cephadm.ceph_manager.ceph:increase pool size by 10 2026-02-24T07:51:35.969 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph osd pool set unique_pool_2 pg_num 76 2026-02-24T07:51:35.994 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:51:35 trial035 ceph-mon[39284]: 4.b scrub starts 2026-02-24T07:51:35.994 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:51:35 trial035 ceph-mon[39284]: purged_snaps scrub starts 2026-02-24T07:51:35.994 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:51:35 trial035 ceph-mon[39284]: purged_snaps scrub ok 2026-02-24T07:51:35.994 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:51:35 trial035 ceph-mon[39284]: 4.b scrub ok 2026-02-24T07:51:36.107 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:51:36.233 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:51:36.230+0000 7f19764f8640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:51:36.233 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:51:36.230+0000 7f19764f8640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:51:36.248 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2026-02-24T07:51:36.390 INFO:teuthology.orchestra.run.trial035.stderr:Error EPERM: crimson_allow_pg_split is false; pg_num increase denied for crimson pool 2026-02-24T07:51:36.396 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:51:36.430 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-24T07:51:36.519 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:51:36.517+0000 7fda6e1d0640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:51:36.520 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:51:36.517+0000 7fda6e1d0640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:51:36.534 INFO:teuthology.orchestra.run.trial035.stdout:osd.0: {} 2026-02-24T07:51:36.535 INFO:teuthology.orchestra.run.trial035.stderr:osd.0: osd_enable_op_tracker = '' 2026-02-24T07:51:36.547 INFO:teuthology.orchestra.run.trial035.stdout:osd.1: {} 2026-02-24T07:51:36.548 INFO:teuthology.orchestra.run.trial035.stderr:osd.1: osd_enable_op_tracker = '' 2026-02-24T07:51:36.561 INFO:teuthology.orchestra.run.trial035.stdout:osd.2: {} 2026-02-24T07:51:36.562 INFO:teuthology.orchestra.run.trial035.stderr:osd.2: osd_enable_op_tracker = '' 2026-02-24T07:51:36.574 INFO:teuthology.orchestra.run.trial035.stdout:osd.3: {} 2026-02-24T07:51:36.574 INFO:teuthology.orchestra.run.trial035.stderr:osd.3: osd_enable_op_tracker = '' 2026-02-24T07:51:37.204 INFO:journalctl@ceph.mgr.x.trial035.stdout:Feb 24 07:51:36 trial035 ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-mgr-x[39621]: ::ffff:10.20.193.91 - - [24/Feb/2026:07:51:36] "GET /metrics HTTP/1.1" 200 4234 "" "Prometheus/3.6.0" 2026-02-24T07:51:37.205 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:51:36 trial035 ceph-mon[39284]: from='client.15107 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2026-02-24T07:51:37.205 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:51:36 trial035 ceph-mon[39284]: from='client.15109 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2026-02-24T07:51:37.205 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:51:36 trial035 ceph-mon[39284]: pgmap v301: 17 pgs: 17 active+clean; 6.8 GiB data, 25 GiB used, 2.7 TiB / 2.7 TiB avail; 0 B/s wr, 7.97k op/s 2026-02-24T07:51:37.205 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:51:36 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/1564895269' entity='client.admin' cmd={"prefix": "osd pool set", "pool": "unique_pool_2", "var": "pg_num", "val": "76"} : dispatch 2026-02-24T07:51:37.205 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:51:36 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/2062314973' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2026-02-24T07:51:37.356 INFO:journalctl@ceph.prometheus.a.trial091.stdout:Feb 24 07:51:36 trial091 ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-prometheus-a[66007]: time=2026-02-24T07:51:36.973Z level=ERROR source=scrape.go:1630 msg="Failed to determine correct type of scrape target." component="scrape manager" scrape_pool=ceph target=http://trial035:9283/metrics content_type="text/html;charset=utf-8" fallback_media_type="" err="received unsupported Content-Type \"text/html;charset=utf-8\" and no fallback_scrape_protocol specified for target" 2026-02-24T07:51:38.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:51:37 trial035 ceph-mon[39284]: from='client.15111 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2026-02-24T07:51:38.619 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2026-02-24T07:51:38.763 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:51:39.176 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:51:38 trial035 ceph-mon[39284]: pgmap v302: 17 pgs: 17 active+clean; 4.6 GiB data, 19 GiB used, 2.7 TiB / 2.7 TiB avail; 0 B/s wr, 5.43k op/s 2026-02-24T07:51:39.177 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:51:38 trial035 ceph-mon[39284]: osdmap e70: 4 total, 4 up, 4 in 2026-02-24T07:51:39.240 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:51:39.174+0000 7f3def806640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:51:39.240 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:51:39.175+0000 7f3def806640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:51:39.241 INFO:teuthology.orchestra.run.trial035.stderr:osd.0: osd_enable_op_tracker = '' 2026-02-24T07:51:39.241 INFO:teuthology.orchestra.run.trial035.stdout:osd.0: {} 2026-02-24T07:51:39.241 INFO:teuthology.orchestra.run.trial035.stdout:osd.1: {} 2026-02-24T07:51:39.241 INFO:teuthology.orchestra.run.trial035.stderr:osd.1: osd_enable_op_tracker = '' 2026-02-24T07:51:39.242 INFO:teuthology.orchestra.run.trial035.stdout:osd.2: {} 2026-02-24T07:51:39.242 INFO:teuthology.orchestra.run.trial035.stderr:osd.2: osd_enable_op_tracker = '' 2026-02-24T07:51:39.254 INFO:teuthology.orchestra.run.trial035.stdout:osd.3: {} 2026-02-24T07:51:39.254 INFO:teuthology.orchestra.run.trial035.stderr:osd.3: osd_enable_op_tracker = '' 2026-02-24T07:51:40.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:51:39 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/267660959' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2026-02-24T07:51:41.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:51:40 trial035 ceph-mon[39284]: pgmap v304: 17 pgs: 17 active+clean; 4.6 GiB data, 19 GiB used, 2.7 TiB / 2.7 TiB avail; 0 B/s wr, 4.74k op/s 2026-02-24T07:51:41.431 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2026-02-24T07:51:41.431 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2026-02-24T07:51:41.432 INFO:tasks.thrashosds.thrasher:fixing pg num pool .mgr 2026-02-24T07:51:41.432 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph pg dump --format=json 2026-02-24T07:51:41.575 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:51:41.696 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:51:41.693+0000 7f33a5762640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:51:41.696 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:51:41.694+0000 7f33a5762640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:51:41.833 INFO:teuthology.orchestra.run.trial035.stdout: 2026-02-24T07:51:41.833 INFO:teuthology.orchestra.run.trial035.stderr:dumped all 2026-02-24T07:51:41.880 INFO:teuthology.orchestra.run.trial035.stdout:{"pg_ready":true,"pg_map":{"version":305,"stamp":"2026-02-24T07:51:41.728020+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":2077688376,"num_objects":31697,"num_object_clones":0,"num_object_copies":95091,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31697,"num_whiteouts":0,"num_read":137,"num_read_kb":241,"num_write":1263176,"num_write_kb":41433639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":53,"ondisk_log_size":53,"up":51,"acting":51,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":51,"num_osds":4,"num_per_pool_osds":4,"num_per_pool_omap_osds":4,"kb":2936012800,"kb_used":11283736,"kb_used_data":6621640,"kb_used_omap":68,"kb_used_meta":4661819,"kb_avail":2924729064,"statfs":{"total":3006477107200,"available":2994922561536,"internally_reserved":0,"allocated":6780559360,"data_stored":6778613084,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":69889,"internal_metadata":4773703423},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[11,2,1],"upper_bound":8},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-5254676480,"num_objects":-80180,"num_object_clones":0,"num_object_copies":-240540,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-80180,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":80180,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001087"},"pg_stats":[{"pgid":"4.f","version":"70'79397","reported_seq":79878,"reported_epoch":70,"state":"active+clean","last_fresh":"2026-02-24T07:51:40.838983+0000","last_change":"2026-02-24T07:51:22.471952+0000","last_active":"2026-02-24T07:51:40.838983+0000","last_peered":"2026-02-24T07:51:40.838983+0000","last_clean":"2026-02-24T07:51:40.838983+0000","last_became_active":"2026-02-24T07:48:57.902313+0000","last_became_peered":"2026-02-24T07:48:57.902313+0000","last_unstale":"2026-02-24T07:51:40.838983+0000","last_undegraded":"2026-02-24T07:51:40.838983+0000","last_fullsized":"2026-02-24T07:51:40.838983+0000","mapping_epoch":55,"log_start":"70'79394","ondisk_log_start":"70'79394","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"65'69619","last_scrub_stamp":"2026-02-24T07:51:22.471822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:51:22.471822+0000","objects_scrubbed":12055,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:52:37.258331+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":127336448,"num_objects":1943,"num_object_clones":0,"num_object_copies":5829,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1943,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":79397,"num_write_kb":2602880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"70'79180","reported_seq":79667,"reported_epoch":70,"state":"active+clean","last_fresh":"2026-02-24T07:51:40.740264+0000","last_change":"2026-02-24T07:51:33.544109+0000","last_active":"2026-02-24T07:51:40.740264+0000","last_peered":"2026-02-24T07:51:40.740264+0000","last_clean":"2026-02-24T07:51:40.740264+0000","last_became_active":"2026-02-24T07:48:57.903584+0000","last_became_peered":"2026-02-24T07:48:57.903584+0000","last_unstale":"2026-02-24T07:51:40.740264+0000","last_undegraded":"2026-02-24T07:51:40.740264+0000","last_fullsized":"2026-02-24T07:51:40.740264+0000","mapping_epoch":55,"log_start":"70'79177","ondisk_log_start":"70'79177","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"68'75450","last_scrub_stamp":"2026-02-24T07:51:33.543750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:51:33.543750+0000","objects_scrubbed":5899,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:52:54.064124+0000","scrub_duration":882,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":130940928,"num_objects":1998,"num_object_clones":0,"num_object_copies":5994,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1998,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":79180,"num_write_kb":2597696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.d","version":"70'78743","reported_seq":79224,"reported_epoch":70,"state":"active+clean","last_fresh":"2026-02-24T07:51:40.751509+0000","last_change":"2026-02-24T07:51:24.114191+0000","last_active":"2026-02-24T07:51:40.751509+0000","last_peered":"2026-02-24T07:51:40.751509+0000","last_clean":"2026-02-24T07:51:40.751509+0000","last_became_active":"2026-02-24T07:48:57.905031+0000","last_became_peered":"2026-02-24T07:48:57.905031+0000","last_unstale":"2026-02-24T07:51:40.751509+0000","last_undegraded":"2026-02-24T07:51:40.751509+0000","last_fullsized":"2026-02-24T07:51:40.751509+0000","mapping_epoch":55,"log_start":"70'78740","ondisk_log_start":"70'78740","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"65'69926","last_scrub_stamp":"2026-02-24T07:51:24.113625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:51:24.113625+0000","objects_scrubbed":11114,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:52:43.886535+0000","scrub_duration":1132,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":134217752,"num_objects":2049,"num_object_clones":0,"num_object_copies":6147,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2049,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":78743,"num_write_kb":2585281,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"70'78491","reported_seq":79013,"reported_epoch":70,"state":"active+clean","last_fresh":"2026-02-24T07:51:40.751653+0000","last_change":"2026-02-24T07:51:28.745731+0000","last_active":"2026-02-24T07:51:40.751653+0000","last_peered":"2026-02-24T07:51:40.751653+0000","last_clean":"2026-02-24T07:51:40.751653+0000","last_became_active":"2026-02-24T07:48:57.904683+0000","last_became_peered":"2026-02-24T07:48:57.904683+0000","last_unstale":"2026-02-24T07:51:40.751653+0000","last_undegraded":"2026-02-24T07:51:40.751653+0000","last_fullsized":"2026-02-24T07:51:40.751653+0000","mapping_epoch":55,"log_start":"70'78488","ondisk_log_start":"70'78488","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"65'72446","last_scrub_stamp":"2026-02-24T07:51:28.745680+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:51:28.745680+0000","objects_scrubbed":8236,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:52:34.536079+0000","scrub_duration":716,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":131137536,"num_objects":2001,"num_object_clones":0,"num_object_copies":6003,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2001,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":78491,"num_write_kb":2575744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.b","version":"70'78632","reported_seq":79131,"reported_epoch":70,"state":"active+clean","last_fresh":"2026-02-24T07:51:40.758326+0000","last_change":"2026-02-24T07:51:35.203746+0000","last_active":"2026-02-24T07:51:40.758326+0000","last_peered":"2026-02-24T07:51:40.758326+0000","last_clean":"2026-02-24T07:51:40.758326+0000","last_became_active":"2026-02-24T07:48:57.901700+0000","last_became_peered":"2026-02-24T07:48:57.901700+0000","last_unstale":"2026-02-24T07:51:40.758326+0000","last_undegraded":"2026-02-24T07:51:40.758326+0000","last_fullsized":"2026-02-24T07:51:40.758326+0000","mapping_epoch":55,"log_start":"70'78628","ondisk_log_start":"70'78628","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"69'75555","last_scrub_stamp":"2026-02-24T07:51:35.203709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:51:35.203709+0000","objects_scrubbed":5254,"log_size":4,"log_dups_size":3000,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:52:52.009840+0000","scrub_duration":634,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":132513792,"num_objects":2022,"num_object_clones":0,"num_object_copies":6066,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2022,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":78632,"num_write_kb":2580928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.a","version":"70'78916","reported_seq":79359,"reported_epoch":70,"state":"active+clean","last_fresh":"2026-02-24T07:51:40.739505+0000","last_change":"2026-02-24T07:50:33.043786+0000","last_active":"2026-02-24T07:51:40.739505+0000","last_peered":"2026-02-24T07:51:40.739505+0000","last_clean":"2026-02-24T07:51:40.739505+0000","last_became_active":"2026-02-24T07:48:57.903228+0000","last_became_peered":"2026-02-24T07:48:57.903228+0000","last_unstale":"2026-02-24T07:51:40.739505+0000","last_undegraded":"2026-02-24T07:51:40.739505+0000","last_fullsized":"2026-02-24T07:51:40.739505+0000","mapping_epoch":55,"log_start":"70'78913","ondisk_log_start":"70'78913","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"61'42539","last_scrub_stamp":"2026-02-24T07:50:33.043726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:50:33.043726+0000","objects_scrubbed":39246,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:51:51.239665+0000","scrub_duration":3547,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":129105920,"num_objects":1970,"num_object_clones":0,"num_object_copies":5910,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1970,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":78916,"num_write_kb":2588352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"70'78953","reported_seq":79336,"reported_epoch":70,"state":"active+clean","last_fresh":"2026-02-24T07:51:40.736727+0000","last_change":"2026-02-24T07:50:13.165492+0000","last_active":"2026-02-24T07:51:40.736727+0000","last_peered":"2026-02-24T07:51:40.736727+0000","last_clean":"2026-02-24T07:51:40.736727+0000","last_became_active":"2026-02-24T07:48:57.901730+0000","last_became_peered":"2026-02-24T07:48:57.901730+0000","last_unstale":"2026-02-24T07:51:40.736727+0000","last_undegraded":"2026-02-24T07:51:40.736727+0000","last_fullsized":"2026-02-24T07:51:40.736727+0000","mapping_epoch":55,"log_start":"70'78950","ondisk_log_start":"70'78950","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"61'33943","last_scrub_stamp":"2026-02-24T07:50:13.165459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:50:13.165459+0000","objects_scrubbed":33253,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"queued for scrub","scrub_duration":3776,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":125632512,"num_objects":1917,"num_object_clones":0,"num_object_copies":5751,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1917,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":78953,"num_write_kb":2587840,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"70'78891","reported_seq":79419,"reported_epoch":70,"state":"active+clean","last_fresh":"2026-02-24T07:51:40.747986+0000","last_change":"2026-02-24T07:51:12.758378+0000","last_active":"2026-02-24T07:51:40.747986+0000","last_peered":"2026-02-24T07:51:40.747986+0000","last_clean":"2026-02-24T07:51:40.747986+0000","last_became_active":"2026-02-24T07:48:57.904376+0000","last_became_peered":"2026-02-24T07:48:57.904376+0000","last_unstale":"2026-02-24T07:51:40.747986+0000","last_undegraded":"2026-02-24T07:51:40.747986+0000","last_fullsized":"2026-02-24T07:51:40.747986+0000","mapping_epoch":55,"log_start":"70'78888","ondisk_log_start":"70'78888","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"65'64020","last_scrub_stamp":"2026-02-24T07:51:12.758336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:51:12.758336+0000","objects_scrubbed":17129,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:52:28.496810+0000","scrub_duration":1688,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":128253952,"num_objects":1957,"num_object_clones":0,"num_object_copies":5871,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1957,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":78891,"num_write_kb":2587136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.7","version":"70'78688","reported_seq":79133,"reported_epoch":70,"state":"active+clean","last_fresh":"2026-02-24T07:51:40.843214+0000","last_change":"2026-02-24T07:51:33.202772+0000","last_active":"2026-02-24T07:51:40.843214+0000","last_peered":"2026-02-24T07:51:40.843214+0000","last_clean":"2026-02-24T07:51:40.843214+0000","last_became_active":"2026-02-24T07:48:57.902484+0000","last_became_peered":"2026-02-24T07:48:57.902484+0000","last_unstale":"2026-02-24T07:51:40.843214+0000","last_undegraded":"2026-02-24T07:51:40.843214+0000","last_fullsized":"2026-02-24T07:51:40.843214+0000","mapping_epoch":55,"log_start":"70'78684","ondisk_log_start":"70'78684","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"68'74654","last_scrub_stamp":"2026-02-24T07:51:33.202731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:51:33.202731+0000","objects_scrubbed":6129,"log_size":4,"log_dups_size":3000,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:52:46.449758+0000","scrub_duration":904,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":128319488,"num_objects":1958,"num_object_clones":0,"num_object_copies":5874,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1958,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":78688,"num_write_kb":2580672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.6","version":"70'78595","reported_seq":79124,"reported_epoch":70,"state":"active+clean","last_fresh":"2026-02-24T07:51:40.758410+0000","last_change":"2026-02-24T07:51:19.852133+0000","last_active":"2026-02-24T07:51:40.758410+0000","last_peered":"2026-02-24T07:51:40.758410+0000","last_clean":"2026-02-24T07:51:40.758410+0000","last_became_active":"2026-02-24T07:48:57.901565+0000","last_became_peered":"2026-02-24T07:48:57.901565+0000","last_unstale":"2026-02-24T07:51:40.758410+0000","last_undegraded":"2026-02-24T07:51:40.758410+0000","last_fullsized":"2026-02-24T07:51:40.758410+0000","mapping_epoch":55,"log_start":"70'78592","ondisk_log_start":"70'78592","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"65'67636","last_scrub_stamp":"2026-02-24T07:51:19.852077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:51:19.852077+0000","objects_scrubbed":13301,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:52:23.813957+0000","scrub_duration":1519,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":128778240,"num_objects":1965,"num_object_clones":0,"num_object_copies":5895,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1965,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":78595,"num_write_kb":2577920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"70'79292","reported_seq":79690,"reported_epoch":70,"state":"active+clean","last_fresh":"2026-02-24T07:51:40.835205+0000","last_change":"2026-02-24T07:50:16.399238+0000","last_active":"2026-02-24T07:51:40.835205+0000","last_peered":"2026-02-24T07:51:40.835205+0000","last_clean":"2026-02-24T07:51:40.835205+0000","last_became_active":"2026-02-24T07:48:57.901824+0000","last_became_peered":"2026-02-24T07:48:57.901824+0000","last_unstale":"2026-02-24T07:51:40.835205+0000","last_undegraded":"2026-02-24T07:51:40.835205+0000","last_fullsized":"2026-02-24T07:51:40.835205+0000","mapping_epoch":55,"log_start":"70'79289","ondisk_log_start":"70'79289","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"61'35510","last_scrub_stamp":"2026-02-24T07:50:16.399194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:50:16.399194+0000","objects_scrubbed":34844,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:51:44.770701+0000","scrub_duration":2971,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":128712704,"num_objects":1964,"num_object_clones":0,"num_object_copies":5892,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1964,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":79292,"num_write_kb":2600192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[1,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"70'78414","reported_seq":78955,"reported_epoch":70,"state":"active+clean","last_fresh":"2026-02-24T07:51:40.755843+0000","last_change":"2026-02-24T07:51:15.067567+0000","last_active":"2026-02-24T07:51:40.755843+0000","last_peered":"2026-02-24T07:51:40.755843+0000","last_clean":"2026-02-24T07:51:40.755843+0000","last_became_active":"2026-02-24T07:48:57.902224+0000","last_became_peered":"2026-02-24T07:48:57.902224+0000","last_unstale":"2026-02-24T07:51:40.755843+0000","last_undegraded":"2026-02-24T07:51:40.755843+0000","last_fullsized":"2026-02-24T07:51:40.755843+0000","mapping_epoch":55,"log_start":"70'78411","ondisk_log_start":"70'78411","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"65'64892","last_scrub_stamp":"2026-02-24T07:51:15.067417+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:51:15.067417+0000","objects_scrubbed":15892,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:52:16.892433+0000","scrub_duration":1747,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":128712704,"num_objects":1964,"num_object_clones":0,"num_object_copies":5892,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1964,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":78414,"num_write_kb":2572096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"70'79295","reported_seq":79818,"reported_epoch":70,"state":"active+clean","last_fresh":"2026-02-24T07:51:40.841349+0000","last_change":"2026-02-24T07:51:13.251872+0000","last_active":"2026-02-24T07:51:40.841349+0000","last_peered":"2026-02-24T07:51:40.841349+0000","last_clean":"2026-02-24T07:51:40.841349+0000","last_became_active":"2026-02-24T07:48:57.902125+0000","last_became_peered":"2026-02-24T07:48:57.902125+0000","last_unstale":"2026-02-24T07:51:40.841349+0000","last_undegraded":"2026-02-24T07:51:40.841349+0000","last_fullsized":"2026-02-24T07:51:40.841349+0000","mapping_epoch":55,"log_start":"70'79292","ondisk_log_start":"70'79292","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"65'64550","last_scrub_stamp":"2026-02-24T07:51:13.251826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:51:13.251826+0000","objects_scrubbed":17042,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:52:21.793363+0000","scrub_duration":1966,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":124190720,"num_objects":1895,"num_object_clones":0,"num_object_copies":5685,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1895,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":79295,"num_write_kb":2598080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1","version":"70'79345","reported_seq":79795,"reported_epoch":70,"state":"active+clean","last_fresh":"2026-02-24T07:51:40.741359+0000","last_change":"2026-02-24T07:50:29.889880+0000","last_active":"2026-02-24T07:51:40.741359+0000","last_peered":"2026-02-24T07:51:40.741359+0000","last_clean":"2026-02-24T07:51:40.741359+0000","last_became_active":"2026-02-24T07:48:57.903439+0000","last_became_peered":"2026-02-24T07:48:57.903439+0000","last_unstale":"2026-02-24T07:51:40.741359+0000","last_undegraded":"2026-02-24T07:51:40.741359+0000","last_fullsized":"2026-02-24T07:51:40.741359+0000","mapping_epoch":55,"log_start":"70'79342","ondisk_log_start":"70'79342","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"61'41105","last_scrub_stamp":"2026-02-24T07:50:29.889816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:50:29.889816+0000","objects_scrubbed":40127,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:51:50.788414+0000","scrub_duration":4409,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":135331840,"num_objects":2065,"num_object_clones":0,"num_object_copies":6195,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2065,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":79345,"num_write_kb":2605120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.0","version":"70'79113","reported_seq":79563,"reported_epoch":70,"state":"active+clean","last_fresh":"2026-02-24T07:51:40.760652+0000","last_change":"2026-02-24T07:50:29.387323+0000","last_active":"2026-02-24T07:51:40.760652+0000","last_peered":"2026-02-24T07:51:40.760652+0000","last_clean":"2026-02-24T07:51:40.760652+0000","last_became_active":"2026-02-24T07:48:57.902565+0000","last_became_peered":"2026-02-24T07:48:57.902565+0000","last_unstale":"2026-02-24T07:51:40.760652+0000","last_undegraded":"2026-02-24T07:51:40.760652+0000","last_fullsized":"2026-02-24T07:51:40.760652+0000","mapping_epoch":55,"log_start":"70'79110","ondisk_log_start":"70'79110","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"61'40736","last_scrub_stamp":"2026-02-24T07:50:29.387206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:50:29.387206+0000","objects_scrubbed":39946,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:51:59.173732+0000","scrub_duration":3834,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":132972544,"num_objects":2029,"num_object_clones":0,"num_object_copies":6087,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2029,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":79113,"num_write_kb":2596544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,3],"acting":[0,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.5","version":"70'79096","reported_seq":79586,"reported_epoch":70,"state":"active+clean","last_fresh":"2026-02-24T07:51:40.749506+0000","last_change":"2026-02-24T07:51:21.557037+0000","last_active":"2026-02-24T07:51:40.749506+0000","last_peered":"2026-02-24T07:51:40.749506+0000","last_clean":"2026-02-24T07:51:40.749506+0000","last_became_active":"2026-02-24T07:48:57.905127+0000","last_became_peered":"2026-02-24T07:48:57.905127+0000","last_unstale":"2026-02-24T07:51:40.749506+0000","last_undegraded":"2026-02-24T07:51:40.749506+0000","last_fullsized":"2026-02-24T07:51:40.749506+0000","mapping_epoch":55,"log_start":"70'79093","ondisk_log_start":"70'79093","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"65'68757","last_scrub_stamp":"2026-02-24T07:51:21.556951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:51:21.556951+0000","objects_scrubbed":12622,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:52:49.364567+0000","scrub_duration":1535,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":130940928,"num_objects":1998,"num_object_clones":0,"num_object_copies":5994,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1998,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":79096,"num_write_kb":2595008,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"29'99","reported_seq":312,"reported_epoch":70,"state":"active+clean","last_fresh":"2026-02-24T07:51:38.888268+0000","last_change":"2026-02-24T07:50:30.401963+0000","last_active":"2026-02-24T07:51:38.888268+0000","last_peered":"2026-02-24T07:51:38.888268+0000","last_clean":"2026-02-24T07:51:38.888268+0000","last_became_active":"2026-02-24T07:42:54.186719+0000","last_became_peered":"2026-02-24T07:42:54.186719+0000","last_unstale":"2026-02-24T07:51:38.888268+0000","last_undegraded":"2026-02-24T07:51:38.888268+0000","last_fullsized":"2026-02-24T07:51:38.888268+0000","mapping_epoch":27,"log_start":"29'96","ondisk_log_start":"29'96","created":20,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'99","last_scrub_stamp":"2026-02-24T07:50:30.401929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:42:43.640776+0000","last_clean_scrub_stamp":"2026-02-24T07:50:30.401929+0000","objects_scrubbed":2,"log_size":3,"log_dups_size":96,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:51:52.990727+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":240,"num_write":135,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":16,"stat_sum":{"num_bytes":2077098008,"num_objects":31695,"num_object_clones":0,"num_object_copies":95085,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31695,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1263041,"num_write_kb":41431489,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":6236811264,"data_stored":6236799048,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4892,"internal_metadata":0},"log_size":50,"ondisk_log_size":50,"up":48,"acting":48,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":240,"num_write":135,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1682,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":3,"up_from":25,"seq":107374182508,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":2777656,"kb_used_data":1802212,"kb_used_omap":15,"kb_used_meta":975344,"kb_avail":731225544,"statfs":{"total":751619276800,"available":748774957056,"internally_reserved":0,"allocated":1845465088,"data_stored":1844976613,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":15738,"internal_metadata":998752902},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":19,"seq":81604378734,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":3104632,"kb_used_data":1794276,"kb_used_omap":16,"kb_used_meta":1310319,"kb_avail":730898568,"statfs":{"total":751619276800,"available":748440133632,"internally_reserved":0,"allocated":1837338624,"data_stored":1836850149,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":17142,"internal_metadata":1341766922},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[5,1],"upper_bound":4},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542255,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":3184688,"kb_used_data":1416604,"kb_used_omap":22,"kb_used_meta":1768041,"kb_avail":730818512,"statfs":{"total":751619276800,"available":748358156288,"internally_reserved":0,"allocated":1450602496,"data_stored":1450121645,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":22862,"internal_metadata":1810474674},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[6,1,1],"upper_bound":8},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705778,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":2216760,"kb_used_data":1608548,"kb_used_omap":13,"kb_used_meta":608114,"kb_avail":731786440,"statfs":{"total":751619276800,"available":749349314560,"internally_reserved":0,"allocated":1647153152,"data_stored":1646664677,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":14147,"internal_metadata":622708925},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1682,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":1564676096,"data_stored":1564672024,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":3046,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":1266352128,"data_stored":1266352128,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":217,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":1835601920,"data_stored":1835597848,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":93,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":1570181120,"data_stored":1570177048,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1536,"internal_metadata":0}]}} 2026-02-24T07:51:41.882 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph osd pool set .mgr pgp_num 281 2026-02-24T07:51:41.963 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2026-02-24T07:51:42.022 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:51:42.087 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:51:41 trial035 ceph-mon[39284]: pgmap v305: 17 pgs: 17 active+clean; 1.9 GiB data, 11 GiB used, 2.7 TiB / 2.7 TiB avail; 0 B/s wr, 10.02k op/s 2026-02-24T07:51:42.087 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:51:41 trial035 ceph-mon[39284]: from='client.15119 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2026-02-24T07:51:42.103 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:51:42.150 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:51:42.148+0000 7fd8baa6b640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:51:42.150 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:51:42.148+0000 7fd8baa6b640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:51:42.220 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:51:42.218+0000 7f7b90dcc640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:51:42.221 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:51:42.218+0000 7f7b90dcc640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:51:42.239 INFO:teuthology.orchestra.run.trial035.stdout:osd.0: {} 2026-02-24T07:51:42.239 INFO:teuthology.orchestra.run.trial035.stderr:osd.0: osd_enable_op_tracker = '' 2026-02-24T07:51:42.252 INFO:teuthology.orchestra.run.trial035.stdout:osd.1: {} 2026-02-24T07:51:42.252 INFO:teuthology.orchestra.run.trial035.stderr:osd.1: osd_enable_op_tracker = '' 2026-02-24T07:51:42.265 INFO:teuthology.orchestra.run.trial035.stdout:osd.2: {} 2026-02-24T07:51:42.266 INFO:teuthology.orchestra.run.trial035.stderr:osd.2: osd_enable_op_tracker = '' 2026-02-24T07:51:42.278 INFO:teuthology.orchestra.run.trial035.stdout:osd.3: {} 2026-02-24T07:51:42.279 INFO:teuthology.orchestra.run.trial035.stderr:osd.3: osd_enable_op_tracker = '' 2026-02-24T07:51:42.298 INFO:teuthology.orchestra.run.trial035.stderr:Error EPERM: crimson_allow_pg_split is false; pgp_num increase denied 2026-02-24T07:51:42.337 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-24T07:51:43.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:51:42 trial035 ceph-mon[39284]: 4.9 scrub starts 2026-02-24T07:51:43.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:51:42 trial035 ceph-mon[39284]: 4.9 scrub ok 2026-02-24T07:51:43.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:51:42 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/2530039855' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2026-02-24T07:51:43.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:51:42 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/4054125997' entity='client.admin' cmd={"prefix": "osd pool set", "pool": ".mgr", "var": "pgp_num", "val": "281"} : dispatch 2026-02-24T07:51:44.180 INFO:tasks.radosbench.radosbench.0.trial035.stdout:Removed 647367 objects 2026-02-24T07:51:44.180 INFO:tasks.radosbench.radosbench.0.trial035.stdout:Clean up completed and total clean up time :75.1768 2026-02-24T07:51:44.327 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2026-02-24T07:51:44.446 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:51:44.539 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:51:44.538+0000 7fbf62623640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:51:44.540 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:51:44.538+0000 7fbf62623640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:51:44.551 INFO:teuthology.orchestra.run.trial035.stdout:osd.0: {} 2026-02-24T07:51:44.552 INFO:teuthology.orchestra.run.trial035.stderr:osd.0: osd_enable_op_tracker = '' 2026-02-24T07:51:44.562 INFO:teuthology.orchestra.run.trial035.stdout:osd.1: {} 2026-02-24T07:51:44.562 INFO:teuthology.orchestra.run.trial035.stderr:osd.1: osd_enable_op_tracker = '' 2026-02-24T07:51:44.572 INFO:teuthology.orchestra.run.trial035.stdout:osd.2: {} 2026-02-24T07:51:44.572 INFO:teuthology.orchestra.run.trial035.stderr:osd.2: osd_enable_op_tracker = '' 2026-02-24T07:51:44.583 INFO:teuthology.orchestra.run.trial035.stdout:osd.3: {} 2026-02-24T07:51:44.583 INFO:teuthology.orchestra.run.trial035.stderr:osd.3: osd_enable_op_tracker = '' 2026-02-24T07:51:45.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:51:44 trial035 ceph-mon[39284]: pgmap v306: 17 pgs: 17 active+clean; 1.9 GiB data, 11 GiB used, 2.7 TiB / 2.7 TiB avail; 0 B/s wr, 8.14k op/s 2026-02-24T07:51:45.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:51:44 trial035 ceph-mon[39284]: osdmap e71: 4 total, 4 up, 4 in 2026-02-24T07:51:45.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:51:44 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/2557532759' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2026-02-24T07:51:45.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:51:44 trial035 ceph-mon[39284]: from='mgr.14284 10.20.193.35:0/3681373222' entity='mgr.x' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2026-02-24T07:51:46.619 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2026-02-24T07:51:46.738 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:51:46.845 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:51:46.844+0000 7f503335e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:51:46.845 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:51:46.845+0000 7f503335e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:51:46.857 INFO:teuthology.orchestra.run.trial035.stdout:osd.0: {} 2026-02-24T07:51:46.857 INFO:teuthology.orchestra.run.trial035.stderr:osd.0: osd_enable_op_tracker = '' 2026-02-24T07:51:46.867 INFO:teuthology.orchestra.run.trial035.stdout:osd.1: {} 2026-02-24T07:51:46.867 INFO:teuthology.orchestra.run.trial035.stderr:osd.1: osd_enable_op_tracker = '' 2026-02-24T07:51:46.878 INFO:teuthology.orchestra.run.trial035.stdout:osd.2: {} 2026-02-24T07:51:46.878 INFO:teuthology.orchestra.run.trial035.stderr:osd.2: osd_enable_op_tracker = '' 2026-02-24T07:51:46.888 INFO:teuthology.orchestra.run.trial035.stdout:osd.3: {} 2026-02-24T07:51:46.888 INFO:teuthology.orchestra.run.trial035.stderr:osd.3: osd_enable_op_tracker = '' 2026-02-24T07:51:46.894 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:51:46 trial035 ceph-mon[39284]: pgmap v308: 17 pgs: 17 active+clean; 1.9 GiB data, 11 GiB used, 2.7 TiB / 2.7 TiB avail; 0 B/s wr, 5.38k op/s 2026-02-24T07:51:46.894 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:51:46 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/1940676290' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2026-02-24T07:51:47.204 INFO:journalctl@ceph.mgr.x.trial035.stdout:Feb 24 07:51:46 trial035 ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-mgr-x[39621]: ::ffff:10.20.193.91 - - [24/Feb/2026:07:51:46] "GET /metrics HTTP/1.1" 200 4230 "" "Prometheus/3.6.0" 2026-02-24T07:51:47.264 INFO:journalctl@ceph.prometheus.a.trial091.stdout:Feb 24 07:51:46 trial091 ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-prometheus-a[66007]: time=2026-02-24T07:51:46.972Z level=ERROR source=scrape.go:1630 msg="Failed to determine correct type of scrape target." component="scrape manager" scrape_pool=ceph target=http://trial035:9283/metrics content_type="text/html;charset=utf-8" fallback_media_type="" err="received unsupported Content-Type \"text/html;charset=utf-8\" and no fallback_scrape_protocol specified for target" 2026-02-24T07:51:47.338 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2026-02-24T07:51:47.338 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2026-02-24T07:51:47.338 INFO:tasks.thrashosds.thrasher:Growing pool .mgr 2026-02-24T07:51:47.339 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph pg dump --format=json 2026-02-24T07:51:47.454 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:51:47.557 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:51:47.556+0000 7fdf76a8d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:51:47.557 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:51:47.556+0000 7fdf76a8d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:51:47.665 INFO:teuthology.orchestra.run.trial035.stdout: 2026-02-24T07:51:47.666 INFO:teuthology.orchestra.run.trial035.stderr:dumped all 2026-02-24T07:51:47.701 INFO:teuthology.orchestra.run.trial035.stdout:{"pg_ready":true,"pg_map":{"version":308,"stamp":"2026-02-24T07:51:45.728574+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":2077688376,"num_objects":31697,"num_object_clones":0,"num_object_copies":95091,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31697,"num_whiteouts":0,"num_read":137,"num_read_kb":241,"num_write":1263176,"num_write_kb":41433639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":53,"ondisk_log_size":53,"up":51,"acting":51,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":51,"num_osds":4,"num_per_pool_osds":4,"num_per_pool_omap_osds":4,"kb":2936012800,"kb_used":11283736,"kb_used_data":6621640,"kb_used_omap":68,"kb_used_meta":4661819,"kb_avail":2924729064,"statfs":{"total":3006477107200,"available":2994922561536,"internally_reserved":0,"allocated":6780559360,"data_stored":6778613084,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":69889,"internal_metadata":4773703423},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[11,2,1],"upper_bound":8},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-2818506752,"num_objects":-43007,"num_object_clones":0,"num_object_copies":-129021,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-43007,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":43007,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001085"},"pg_stats":[{"pgid":"4.f","version":"70'79397","reported_seq":79878,"reported_epoch":70,"state":"active+clean","last_fresh":"2026-02-24T07:51:40.838983+0000","last_change":"2026-02-24T07:51:22.471952+0000","last_active":"2026-02-24T07:51:40.838983+0000","last_peered":"2026-02-24T07:51:40.838983+0000","last_clean":"2026-02-24T07:51:40.838983+0000","last_became_active":"2026-02-24T07:48:57.902313+0000","last_became_peered":"2026-02-24T07:48:57.902313+0000","last_unstale":"2026-02-24T07:51:40.838983+0000","last_undegraded":"2026-02-24T07:51:40.838983+0000","last_fullsized":"2026-02-24T07:51:40.838983+0000","mapping_epoch":55,"log_start":"70'79394","ondisk_log_start":"70'79394","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"65'69619","last_scrub_stamp":"2026-02-24T07:51:22.471822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:51:22.471822+0000","objects_scrubbed":12055,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:52:37.258331+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":127336448,"num_objects":1943,"num_object_clones":0,"num_object_copies":5829,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1943,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":79397,"num_write_kb":2602880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"70'79180","reported_seq":79667,"reported_epoch":70,"state":"active+clean","last_fresh":"2026-02-24T07:51:40.740264+0000","last_change":"2026-02-24T07:51:33.544109+0000","last_active":"2026-02-24T07:51:40.740264+0000","last_peered":"2026-02-24T07:51:40.740264+0000","last_clean":"2026-02-24T07:51:40.740264+0000","last_became_active":"2026-02-24T07:48:57.903584+0000","last_became_peered":"2026-02-24T07:48:57.903584+0000","last_unstale":"2026-02-24T07:51:40.740264+0000","last_undegraded":"2026-02-24T07:51:40.740264+0000","last_fullsized":"2026-02-24T07:51:40.740264+0000","mapping_epoch":55,"log_start":"70'79177","ondisk_log_start":"70'79177","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"68'75450","last_scrub_stamp":"2026-02-24T07:51:33.543750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:51:33.543750+0000","objects_scrubbed":5899,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:52:54.064124+0000","scrub_duration":882,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":130940928,"num_objects":1998,"num_object_clones":0,"num_object_copies":5994,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1998,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":79180,"num_write_kb":2597696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.d","version":"70'78743","reported_seq":79224,"reported_epoch":70,"state":"active+clean","last_fresh":"2026-02-24T07:51:40.751509+0000","last_change":"2026-02-24T07:51:24.114191+0000","last_active":"2026-02-24T07:51:40.751509+0000","last_peered":"2026-02-24T07:51:40.751509+0000","last_clean":"2026-02-24T07:51:40.751509+0000","last_became_active":"2026-02-24T07:48:57.905031+0000","last_became_peered":"2026-02-24T07:48:57.905031+0000","last_unstale":"2026-02-24T07:51:40.751509+0000","last_undegraded":"2026-02-24T07:51:40.751509+0000","last_fullsized":"2026-02-24T07:51:40.751509+0000","mapping_epoch":55,"log_start":"70'78740","ondisk_log_start":"70'78740","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"65'69926","last_scrub_stamp":"2026-02-24T07:51:24.113625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:51:24.113625+0000","objects_scrubbed":11114,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:52:43.886535+0000","scrub_duration":1132,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":134217752,"num_objects":2049,"num_object_clones":0,"num_object_copies":6147,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2049,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":78743,"num_write_kb":2585281,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"70'78491","reported_seq":79013,"reported_epoch":70,"state":"active+clean","last_fresh":"2026-02-24T07:51:40.751653+0000","last_change":"2026-02-24T07:51:28.745731+0000","last_active":"2026-02-24T07:51:40.751653+0000","last_peered":"2026-02-24T07:51:40.751653+0000","last_clean":"2026-02-24T07:51:40.751653+0000","last_became_active":"2026-02-24T07:48:57.904683+0000","last_became_peered":"2026-02-24T07:48:57.904683+0000","last_unstale":"2026-02-24T07:51:40.751653+0000","last_undegraded":"2026-02-24T07:51:40.751653+0000","last_fullsized":"2026-02-24T07:51:40.751653+0000","mapping_epoch":55,"log_start":"70'78488","ondisk_log_start":"70'78488","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"65'72446","last_scrub_stamp":"2026-02-24T07:51:28.745680+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:51:28.745680+0000","objects_scrubbed":8236,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:52:34.536079+0000","scrub_duration":716,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":131137536,"num_objects":2001,"num_object_clones":0,"num_object_copies":6003,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2001,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":78491,"num_write_kb":2575744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.b","version":"70'78632","reported_seq":79131,"reported_epoch":70,"state":"active+clean","last_fresh":"2026-02-24T07:51:40.758326+0000","last_change":"2026-02-24T07:51:35.203746+0000","last_active":"2026-02-24T07:51:40.758326+0000","last_peered":"2026-02-24T07:51:40.758326+0000","last_clean":"2026-02-24T07:51:40.758326+0000","last_became_active":"2026-02-24T07:48:57.901700+0000","last_became_peered":"2026-02-24T07:48:57.901700+0000","last_unstale":"2026-02-24T07:51:40.758326+0000","last_undegraded":"2026-02-24T07:51:40.758326+0000","last_fullsized":"2026-02-24T07:51:40.758326+0000","mapping_epoch":55,"log_start":"70'78628","ondisk_log_start":"70'78628","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"69'75555","last_scrub_stamp":"2026-02-24T07:51:35.203709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:51:35.203709+0000","objects_scrubbed":5254,"log_size":4,"log_dups_size":3000,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:52:52.009840+0000","scrub_duration":634,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":132513792,"num_objects":2022,"num_object_clones":0,"num_object_copies":6066,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2022,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":78632,"num_write_kb":2580928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.a","version":"70'78916","reported_seq":79359,"reported_epoch":70,"state":"active+clean","last_fresh":"2026-02-24T07:51:40.739505+0000","last_change":"2026-02-24T07:50:33.043786+0000","last_active":"2026-02-24T07:51:40.739505+0000","last_peered":"2026-02-24T07:51:40.739505+0000","last_clean":"2026-02-24T07:51:40.739505+0000","last_became_active":"2026-02-24T07:48:57.903228+0000","last_became_peered":"2026-02-24T07:48:57.903228+0000","last_unstale":"2026-02-24T07:51:40.739505+0000","last_undegraded":"2026-02-24T07:51:40.739505+0000","last_fullsized":"2026-02-24T07:51:40.739505+0000","mapping_epoch":55,"log_start":"70'78913","ondisk_log_start":"70'78913","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"61'42539","last_scrub_stamp":"2026-02-24T07:50:33.043726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:50:33.043726+0000","objects_scrubbed":39246,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:51:51.239665+0000","scrub_duration":3547,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":129105920,"num_objects":1970,"num_object_clones":0,"num_object_copies":5910,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1970,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":78916,"num_write_kb":2588352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"70'78953","reported_seq":79336,"reported_epoch":70,"state":"active+clean","last_fresh":"2026-02-24T07:51:40.736727+0000","last_change":"2026-02-24T07:50:13.165492+0000","last_active":"2026-02-24T07:51:40.736727+0000","last_peered":"2026-02-24T07:51:40.736727+0000","last_clean":"2026-02-24T07:51:40.736727+0000","last_became_active":"2026-02-24T07:48:57.901730+0000","last_became_peered":"2026-02-24T07:48:57.901730+0000","last_unstale":"2026-02-24T07:51:40.736727+0000","last_undegraded":"2026-02-24T07:51:40.736727+0000","last_fullsized":"2026-02-24T07:51:40.736727+0000","mapping_epoch":55,"log_start":"70'78950","ondisk_log_start":"70'78950","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"61'33943","last_scrub_stamp":"2026-02-24T07:50:13.165459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:50:13.165459+0000","objects_scrubbed":33253,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"queued for scrub","scrub_duration":3776,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":125632512,"num_objects":1917,"num_object_clones":0,"num_object_copies":5751,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1917,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":78953,"num_write_kb":2587840,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"70'78891","reported_seq":79419,"reported_epoch":70,"state":"active+clean","last_fresh":"2026-02-24T07:51:40.747986+0000","last_change":"2026-02-24T07:51:12.758378+0000","last_active":"2026-02-24T07:51:40.747986+0000","last_peered":"2026-02-24T07:51:40.747986+0000","last_clean":"2026-02-24T07:51:40.747986+0000","last_became_active":"2026-02-24T07:48:57.904376+0000","last_became_peered":"2026-02-24T07:48:57.904376+0000","last_unstale":"2026-02-24T07:51:40.747986+0000","last_undegraded":"2026-02-24T07:51:40.747986+0000","last_fullsized":"2026-02-24T07:51:40.747986+0000","mapping_epoch":55,"log_start":"70'78888","ondisk_log_start":"70'78888","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"65'64020","last_scrub_stamp":"2026-02-24T07:51:12.758336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:51:12.758336+0000","objects_scrubbed":17129,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:52:28.496810+0000","scrub_duration":1688,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":128253952,"num_objects":1957,"num_object_clones":0,"num_object_copies":5871,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1957,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":78891,"num_write_kb":2587136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.7","version":"70'78688","reported_seq":79133,"reported_epoch":70,"state":"active+clean","last_fresh":"2026-02-24T07:51:40.843214+0000","last_change":"2026-02-24T07:51:33.202772+0000","last_active":"2026-02-24T07:51:40.843214+0000","last_peered":"2026-02-24T07:51:40.843214+0000","last_clean":"2026-02-24T07:51:40.843214+0000","last_became_active":"2026-02-24T07:48:57.902484+0000","last_became_peered":"2026-02-24T07:48:57.902484+0000","last_unstale":"2026-02-24T07:51:40.843214+0000","last_undegraded":"2026-02-24T07:51:40.843214+0000","last_fullsized":"2026-02-24T07:51:40.843214+0000","mapping_epoch":55,"log_start":"70'78684","ondisk_log_start":"70'78684","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"68'74654","last_scrub_stamp":"2026-02-24T07:51:33.202731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:51:33.202731+0000","objects_scrubbed":6129,"log_size":4,"log_dups_size":3000,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:52:46.449758+0000","scrub_duration":904,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":128319488,"num_objects":1958,"num_object_clones":0,"num_object_copies":5874,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1958,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":78688,"num_write_kb":2580672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.6","version":"70'78595","reported_seq":79124,"reported_epoch":70,"state":"active+clean","last_fresh":"2026-02-24T07:51:40.758410+0000","last_change":"2026-02-24T07:51:19.852133+0000","last_active":"2026-02-24T07:51:40.758410+0000","last_peered":"2026-02-24T07:51:40.758410+0000","last_clean":"2026-02-24T07:51:40.758410+0000","last_became_active":"2026-02-24T07:48:57.901565+0000","last_became_peered":"2026-02-24T07:48:57.901565+0000","last_unstale":"2026-02-24T07:51:40.758410+0000","last_undegraded":"2026-02-24T07:51:40.758410+0000","last_fullsized":"2026-02-24T07:51:40.758410+0000","mapping_epoch":55,"log_start":"70'78592","ondisk_log_start":"70'78592","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"65'67636","last_scrub_stamp":"2026-02-24T07:51:19.852077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:51:19.852077+0000","objects_scrubbed":13301,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:52:23.813957+0000","scrub_duration":1519,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":128778240,"num_objects":1965,"num_object_clones":0,"num_object_copies":5895,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1965,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":78595,"num_write_kb":2577920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"70'79292","reported_seq":79690,"reported_epoch":70,"state":"active+clean","last_fresh":"2026-02-24T07:51:40.835205+0000","last_change":"2026-02-24T07:50:16.399238+0000","last_active":"2026-02-24T07:51:40.835205+0000","last_peered":"2026-02-24T07:51:40.835205+0000","last_clean":"2026-02-24T07:51:40.835205+0000","last_became_active":"2026-02-24T07:48:57.901824+0000","last_became_peered":"2026-02-24T07:48:57.901824+0000","last_unstale":"2026-02-24T07:51:40.835205+0000","last_undegraded":"2026-02-24T07:51:40.835205+0000","last_fullsized":"2026-02-24T07:51:40.835205+0000","mapping_epoch":55,"log_start":"70'79289","ondisk_log_start":"70'79289","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"61'35510","last_scrub_stamp":"2026-02-24T07:50:16.399194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:50:16.399194+0000","objects_scrubbed":34844,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:51:44.770701+0000","scrub_duration":2971,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":128712704,"num_objects":1964,"num_object_clones":0,"num_object_copies":5892,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1964,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":79292,"num_write_kb":2600192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[1,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"70'78414","reported_seq":78955,"reported_epoch":70,"state":"active+clean","last_fresh":"2026-02-24T07:51:40.755843+0000","last_change":"2026-02-24T07:51:15.067567+0000","last_active":"2026-02-24T07:51:40.755843+0000","last_peered":"2026-02-24T07:51:40.755843+0000","last_clean":"2026-02-24T07:51:40.755843+0000","last_became_active":"2026-02-24T07:48:57.902224+0000","last_became_peered":"2026-02-24T07:48:57.902224+0000","last_unstale":"2026-02-24T07:51:40.755843+0000","last_undegraded":"2026-02-24T07:51:40.755843+0000","last_fullsized":"2026-02-24T07:51:40.755843+0000","mapping_epoch":55,"log_start":"70'78411","ondisk_log_start":"70'78411","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"65'64892","last_scrub_stamp":"2026-02-24T07:51:15.067417+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:51:15.067417+0000","objects_scrubbed":15892,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:52:16.892433+0000","scrub_duration":1747,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":128712704,"num_objects":1964,"num_object_clones":0,"num_object_copies":5892,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1964,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":78414,"num_write_kb":2572096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"70'79295","reported_seq":79818,"reported_epoch":70,"state":"active+clean","last_fresh":"2026-02-24T07:51:40.841349+0000","last_change":"2026-02-24T07:51:13.251872+0000","last_active":"2026-02-24T07:51:40.841349+0000","last_peered":"2026-02-24T07:51:40.841349+0000","last_clean":"2026-02-24T07:51:40.841349+0000","last_became_active":"2026-02-24T07:48:57.902125+0000","last_became_peered":"2026-02-24T07:48:57.902125+0000","last_unstale":"2026-02-24T07:51:40.841349+0000","last_undegraded":"2026-02-24T07:51:40.841349+0000","last_fullsized":"2026-02-24T07:51:40.841349+0000","mapping_epoch":55,"log_start":"70'79292","ondisk_log_start":"70'79292","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"65'64550","last_scrub_stamp":"2026-02-24T07:51:13.251826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:51:13.251826+0000","objects_scrubbed":17042,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:52:21.793363+0000","scrub_duration":1966,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":124190720,"num_objects":1895,"num_object_clones":0,"num_object_copies":5685,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1895,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":79295,"num_write_kb":2598080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1","version":"70'79345","reported_seq":79795,"reported_epoch":70,"state":"active+clean","last_fresh":"2026-02-24T07:51:40.741359+0000","last_change":"2026-02-24T07:50:29.889880+0000","last_active":"2026-02-24T07:51:40.741359+0000","last_peered":"2026-02-24T07:51:40.741359+0000","last_clean":"2026-02-24T07:51:40.741359+0000","last_became_active":"2026-02-24T07:48:57.903439+0000","last_became_peered":"2026-02-24T07:48:57.903439+0000","last_unstale":"2026-02-24T07:51:40.741359+0000","last_undegraded":"2026-02-24T07:51:40.741359+0000","last_fullsized":"2026-02-24T07:51:40.741359+0000","mapping_epoch":55,"log_start":"70'79342","ondisk_log_start":"70'79342","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"61'41105","last_scrub_stamp":"2026-02-24T07:50:29.889816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:50:29.889816+0000","objects_scrubbed":40127,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:51:50.788414+0000","scrub_duration":4409,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":135331840,"num_objects":2065,"num_object_clones":0,"num_object_copies":6195,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2065,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":79345,"num_write_kb":2605120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.0","version":"70'79113","reported_seq":79563,"reported_epoch":70,"state":"active+clean","last_fresh":"2026-02-24T07:51:40.760652+0000","last_change":"2026-02-24T07:50:29.387323+0000","last_active":"2026-02-24T07:51:40.760652+0000","last_peered":"2026-02-24T07:51:40.760652+0000","last_clean":"2026-02-24T07:51:40.760652+0000","last_became_active":"2026-02-24T07:48:57.902565+0000","last_became_peered":"2026-02-24T07:48:57.902565+0000","last_unstale":"2026-02-24T07:51:40.760652+0000","last_undegraded":"2026-02-24T07:51:40.760652+0000","last_fullsized":"2026-02-24T07:51:40.760652+0000","mapping_epoch":55,"log_start":"70'79110","ondisk_log_start":"70'79110","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"61'40736","last_scrub_stamp":"2026-02-24T07:50:29.387206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:50:29.387206+0000","objects_scrubbed":39946,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:51:59.173732+0000","scrub_duration":3834,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":132972544,"num_objects":2029,"num_object_clones":0,"num_object_copies":6087,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2029,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":79113,"num_write_kb":2596544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,3],"acting":[0,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.5","version":"70'79096","reported_seq":79586,"reported_epoch":70,"state":"active+clean","last_fresh":"2026-02-24T07:51:40.749506+0000","last_change":"2026-02-24T07:51:21.557037+0000","last_active":"2026-02-24T07:51:40.749506+0000","last_peered":"2026-02-24T07:51:40.749506+0000","last_clean":"2026-02-24T07:51:40.749506+0000","last_became_active":"2026-02-24T07:48:57.905127+0000","last_became_peered":"2026-02-24T07:48:57.905127+0000","last_unstale":"2026-02-24T07:51:40.749506+0000","last_undegraded":"2026-02-24T07:51:40.749506+0000","last_fullsized":"2026-02-24T07:51:40.749506+0000","mapping_epoch":55,"log_start":"70'79093","ondisk_log_start":"70'79093","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"65'68757","last_scrub_stamp":"2026-02-24T07:51:21.556951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:48:56.890768+0000","last_clean_scrub_stamp":"2026-02-24T07:51:21.556951+0000","objects_scrubbed":12622,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:52:49.364567+0000","scrub_duration":1535,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":130940928,"num_objects":1998,"num_object_clones":0,"num_object_copies":5994,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1998,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":79096,"num_write_kb":2595008,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"29'99","reported_seq":312,"reported_epoch":70,"state":"active+clean","last_fresh":"2026-02-24T07:51:38.888268+0000","last_change":"2026-02-24T07:50:30.401963+0000","last_active":"2026-02-24T07:51:38.888268+0000","last_peered":"2026-02-24T07:51:38.888268+0000","last_clean":"2026-02-24T07:51:38.888268+0000","last_became_active":"2026-02-24T07:42:54.186719+0000","last_became_peered":"2026-02-24T07:42:54.186719+0000","last_unstale":"2026-02-24T07:51:38.888268+0000","last_undegraded":"2026-02-24T07:51:38.888268+0000","last_fullsized":"2026-02-24T07:51:38.888268+0000","mapping_epoch":27,"log_start":"29'96","ondisk_log_start":"29'96","created":20,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'99","last_scrub_stamp":"2026-02-24T07:50:30.401929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:42:43.640776+0000","last_clean_scrub_stamp":"2026-02-24T07:50:30.401929+0000","objects_scrubbed":2,"log_size":3,"log_dups_size":96,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:51:52.990727+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":240,"num_write":135,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":16,"stat_sum":{"num_bytes":2077098008,"num_objects":31695,"num_object_clones":0,"num_object_copies":95085,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31695,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1263041,"num_write_kb":41431489,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":6236811264,"data_stored":6236799048,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4892,"internal_metadata":0},"log_size":50,"ondisk_log_size":50,"up":48,"acting":48,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":240,"num_write":135,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1682,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":3,"up_from":25,"seq":107374182508,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":2777656,"kb_used_data":1802212,"kb_used_omap":15,"kb_used_meta":975344,"kb_avail":731225544,"statfs":{"total":751619276800,"available":748774957056,"internally_reserved":0,"allocated":1845465088,"data_stored":1844976613,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":15738,"internal_metadata":998752902},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":19,"seq":81604378734,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":3104632,"kb_used_data":1794276,"kb_used_omap":16,"kb_used_meta":1310319,"kb_avail":730898568,"statfs":{"total":751619276800,"available":748440133632,"internally_reserved":0,"allocated":1837338624,"data_stored":1836850149,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":17142,"internal_metadata":1341766922},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[5,1],"upper_bound":4},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542255,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":3184688,"kb_used_data":1416604,"kb_used_omap":22,"kb_used_meta":1768041,"kb_avail":730818512,"statfs":{"total":751619276800,"available":748358156288,"internally_reserved":0,"allocated":1450602496,"data_stored":1450121645,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":22862,"internal_metadata":1810474674},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[6,1,1],"upper_bound":8},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705778,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":2216760,"kb_used_data":1608548,"kb_used_omap":13,"kb_used_meta":608114,"kb_avail":731786440,"statfs":{"total":751619276800,"available":749349314560,"internally_reserved":0,"allocated":1647153152,"data_stored":1646664677,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":14147,"internal_metadata":622708925},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1682,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":1564676096,"data_stored":1564672024,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":3046,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":1266352128,"data_stored":1266352128,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":217,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":1835601920,"data_stored":1835597848,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":93,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":1570181120,"data_stored":1570177048,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1536,"internal_metadata":0}]}} 2026-02-24T07:51:47.703 INFO:tasks.cephadm.ceph_manager.ceph:increase pool size by 10 2026-02-24T07:51:47.703 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph osd pool set .mgr pg_num 291 2026-02-24T07:51:47.818 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:51:47.917 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:51:47.916+0000 7fe68e73f640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:51:47.918 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:51:47.916+0000 7fe68e73f640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:51:48.027 INFO:teuthology.orchestra.run.trial035.stderr:Error EPERM: crimson_allow_pg_split is false; pg_num increase denied for crimson pool 2026-02-24T07:51:48.059 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-24T07:51:48.924 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2026-02-24T07:51:49.040 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:51:49.134 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:51:49.133+0000 7fdf6f15e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:51:49.135 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:51:48 trial035 ceph-mon[39284]: 4.4 scrub starts 2026-02-24T07:51:49.135 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:51:48 trial035 ceph-mon[39284]: 4.4 scrub ok 2026-02-24T07:51:49.135 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:51:48 trial035 ceph-mon[39284]: from='client.15129 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2026-02-24T07:51:49.135 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:51:48 trial035 ceph-mon[39284]: pgmap v309: 17 pgs: 17 active+clean; 577 KiB data, 4.4 GiB used, 2.7 TiB / 2.7 TiB avail; 0 B/s wr, 8.44k op/s 2026-02-24T07:51:49.135 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:51:48 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/2470517396' entity='client.admin' cmd={"prefix": "osd pool set", "pool": ".mgr", "var": "pg_num", "val": "291"} : dispatch 2026-02-24T07:51:49.136 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:51:49.133+0000 7fdf6f15e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:51:49.146 INFO:teuthology.orchestra.run.trial035.stdout:osd.0: {} 2026-02-24T07:51:49.147 INFO:teuthology.orchestra.run.trial035.stderr:osd.0: osd_enable_op_tracker = '' 2026-02-24T07:51:49.157 INFO:teuthology.orchestra.run.trial035.stdout:osd.1: {} 2026-02-24T07:51:49.157 INFO:teuthology.orchestra.run.trial035.stderr:osd.1: osd_enable_op_tracker = '' 2026-02-24T07:51:49.166 INFO:teuthology.orchestra.run.trial035.stdout:osd.2: {} 2026-02-24T07:51:49.167 INFO:teuthology.orchestra.run.trial035.stderr:osd.2: osd_enable_op_tracker = '' 2026-02-24T07:51:49.177 INFO:teuthology.orchestra.run.trial035.stdout:osd.3: {} 2026-02-24T07:51:49.177 INFO:teuthology.orchestra.run.trial035.stderr:osd.3: osd_enable_op_tracker = '' 2026-02-24T07:51:49.781 INFO:tasks.cephadm.ceph_manager.ceph:removing pool_name unique_pool_2 2026-02-24T07:51:49.781 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph osd pool rm unique_pool_2 unique_pool_2 --yes-i-really-really-mean-it 2026-02-24T07:51:49.902 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:51:50.006 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:51:50.005+0000 7f233665e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:51:50.006 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:51:50.005+0000 7f233665e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:51:50.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:51:49 trial035 ceph-mon[39284]: osdmap e72: 4 total, 4 up, 4 in 2026-02-24T07:51:50.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:51:49 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/1146796755' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2026-02-24T07:51:50.886 INFO:teuthology.orchestra.run.trial035.stderr:pool 'unique_pool_2' removed 2026-02-24T07:51:51.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:51:50 trial035 ceph-mon[39284]: pgmap v311: 17 pgs: 17 active+clean; 577 KiB data, 4.4 GiB used, 2.7 TiB / 2.7 TiB avail; 0 B/s wr, 3.96k op/s 2026-02-24T07:51:51.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:51:50 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/1343998463' entity='client.admin' cmd={"prefix": "osd pool rm", "pool": "unique_pool_2", "pool2": "unique_pool_2", "yes_i_really_really_mean_it": true} : dispatch 2026-02-24T07:51:51.212 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2026-02-24T07:51:51.451 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:51:51.523 INFO:teuthology.task.full_sequential:In full_sequential, running task radosbench... 2026-02-24T07:51:51.524 INFO:tasks.radosbench:Beginning radosbench... 2026-02-24T07:51:51.525 INFO:tasks.cephadm.ceph_manager.ceph:creating pool_name unique_pool_3 2026-02-24T07:51:51.525 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph osd pool create unique_pool_3 16 2026-02-24T07:51:51.747 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:51:51.937 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:51:51.875+0000 7f17a4439640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:51:51.937 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:51:51.875+0000 7f17a4439640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:51:51.937 INFO:teuthology.orchestra.run.trial035.stderr:osd.0: osd_enable_op_tracker = '' 2026-02-24T07:51:51.937 INFO:teuthology.orchestra.run.trial035.stderr:osd.1: osd_enable_op_tracker = '' 2026-02-24T07:51:51.937 INFO:teuthology.orchestra.run.trial035.stdout:osd.0: {} 2026-02-24T07:51:51.937 INFO:teuthology.orchestra.run.trial035.stdout:osd.1: {} 2026-02-24T07:51:51.937 INFO:teuthology.orchestra.run.trial035.stdout:osd.2: {} 2026-02-24T07:51:51.937 INFO:teuthology.orchestra.run.trial035.stdout:osd.3: {} 2026-02-24T07:51:51.938 INFO:teuthology.orchestra.run.trial035.stderr:osd.2: osd_enable_op_tracker = '' 2026-02-24T07:51:51.938 INFO:teuthology.orchestra.run.trial035.stderr:osd.3: osd_enable_op_tracker = '' 2026-02-24T07:51:52.082 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:51:51 trial035 ceph-mon[39284]: 4.1 scrub starts 2026-02-24T07:51:52.082 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:51:51 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/1343998463' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "unique_pool_2", "pool2": "unique_pool_2", "yes_i_really_really_mean_it": true}]': finished 2026-02-24T07:51:52.082 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:51:51 trial035 ceph-mon[39284]: osdmap e73: 4 total, 4 up, 4 in 2026-02-24T07:51:52.082 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:51:51 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/1525069249' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2026-02-24T07:51:52.529 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:51:52.456+0000 7f96d9abc640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:51:52.529 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:51:52.456+0000 7f96d9abc640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:51:52.893 INFO:teuthology.orchestra.run.trial035.stderr:pool 'unique_pool_3' created 2026-02-24T07:51:52.934 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph osd pool application enable unique_pool_3 rados --yes-i-really-mean-it || true 2026-02-24T07:51:53.049 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:51:53.061 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2026-02-24T07:51:53.061 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2026-02-24T07:51:53.149 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:51:53.148+0000 7f2404386640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:51:53.149 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:51:53.148+0000 7f2404386640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:51:53.149 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:51:52 trial035 ceph-mon[39284]: pgmap v313: 1 pgs: 1 active+clean; 577 KiB data, 4.4 GiB used, 2.7 TiB / 2.7 TiB avail 2026-02-24T07:51:53.149 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:51:52 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/2425862922' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "unique_pool_3", "pg_num": 16} : dispatch 2026-02-24T07:51:53.895 INFO:teuthology.orchestra.run.trial035.stderr:enabled application 'rados' on pool 'unique_pool_3' 2026-02-24T07:51:54.084 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2026-02-24T07:51:54.202 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:51:54.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:51:53 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/2425862922' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "unique_pool_3", "pg_num": 16}]': finished 2026-02-24T07:51:54.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:51:53 trial035 ceph-mon[39284]: osdmap e74: 4 total, 4 up, 4 in 2026-02-24T07:51:54.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:51:53 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/3056259024' entity='client.admin' cmd={"prefix": "osd pool application enable", "pool": "unique_pool_3", "app": "rados", "yes_i_really_mean_it": true} : dispatch 2026-02-24T07:51:54.304 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:51:54.303+0000 7fa2e5592640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:51:54.304 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:51:54.303+0000 7fa2e5592640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:51:54.316 INFO:teuthology.orchestra.run.trial035.stdout:osd.0: {} 2026-02-24T07:51:54.316 INFO:teuthology.orchestra.run.trial035.stderr:osd.0: osd_enable_op_tracker = '' 2026-02-24T07:51:54.326 INFO:teuthology.orchestra.run.trial035.stdout:osd.1: {} 2026-02-24T07:51:54.326 INFO:teuthology.orchestra.run.trial035.stderr:osd.1: osd_enable_op_tracker = '' 2026-02-24T07:51:54.337 INFO:teuthology.orchestra.run.trial035.stdout:osd.2: {} 2026-02-24T07:51:54.337 INFO:teuthology.orchestra.run.trial035.stderr:osd.2: osd_enable_op_tracker = '' 2026-02-24T07:51:54.347 INFO:teuthology.orchestra.run.trial035.stdout:osd.3: {} 2026-02-24T07:51:54.347 INFO:teuthology.orchestra.run.trial035.stderr:osd.3: osd_enable_op_tracker = '' 2026-02-24T07:51:54.939 DEBUG:teuthology.orchestra.run.trial035:> /bin/sh -c 'adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --no-log-to-stderr --name client.0 -t 16 -b 65536 --object-size 65536 -p unique_pool_3 bench 90 write' 2026-02-24T07:51:54.940 INFO:tasks.thrashosds.thrasher:fixing pg num pool .mgr 2026-02-24T07:51:54.940 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph pg dump --format=json 2026-02-24T07:51:54.943 INFO:tasks.radosbench:joining radosbench (timing out after 3000s) 2026-02-24T07:51:54.943 INFO:teuthology.orchestra.run:waiting for 3000 2026-02-24T07:51:54.990 INFO:tasks.radosbench.radosbench.0.trial035.stdout:hints = 1 2026-02-24T07:51:54.991 INFO:tasks.radosbench.radosbench.0.trial035.stdout:Maintaining 16 concurrent writes of 65536 bytes to objects of size 65536 for up to 90 seconds or 0 objects 2026-02-24T07:51:54.991 INFO:tasks.radosbench.radosbench.0.trial035.stdout:Object prefix: benchmark_data_trial035_120723 2026-02-24T07:51:54.991 INFO:tasks.radosbench.radosbench.0.trial035.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2026-02-24T07:51:54.991 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 0 0 0 0 0 0 - 0 2026-02-24T07:51:55.073 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:51:55.147 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:51:54 trial035 ceph-mon[39284]: pgmap v315: 17 pgs: 16 unknown, 1 active+clean; 577 KiB data, 4.4 GiB used, 2.7 TiB / 2.7 TiB avail 2026-02-24T07:51:55.147 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:51:54 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/3056259024' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "unique_pool_3", "app": "rados", "yes_i_really_mean_it": true}]': finished 2026-02-24T07:51:55.148 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:51:54 trial035 ceph-mon[39284]: osdmap e75: 4 total, 4 up, 4 in 2026-02-24T07:51:55.148 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:51:54 trial035 ceph-mon[39284]: 1.0 scrub starts 2026-02-24T07:51:55.148 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:51:54 trial035 ceph-mon[39284]: 1.0 scrub ok 2026-02-24T07:51:55.148 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:51:54 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/4135512012' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2026-02-24T07:51:55.213 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:51:55.211+0000 7f10b6234640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:51:55.213 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:51:55.211+0000 7f10b6234640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:51:55.357 INFO:teuthology.orchestra.run.trial035.stdout: 2026-02-24T07:51:55.357 INFO:teuthology.orchestra.run.trial035.stderr:dumped all 2026-02-24T07:51:55.411 INFO:teuthology.orchestra.run.trial035.stdout:{"pg_ready":true,"pg_map":{"version":317,"stamp":"2026-02-24T07:51:54.898763+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":240,"num_write":135,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":3,"acting":3,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":51,"num_osds":4,"num_per_pool_osds":4,"num_per_pool_omap_osds":4,"kb":2936012800,"kb_used":4594580,"kb_used_data":4740,"kb_used_omap":64,"kb_used_meta":4589695,"kb_avail":2931418220,"statfs":{"total":3006477107200,"available":3001772257280,"internally_reserved":0,"allocated":4853760,"data_stored":2872378,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":65860,"internal_metadata":4699848380},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":-1,"num_read_kb":-1,"num_write":-1294736,"num_write_kb":-41431489,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.169612"},"pg_stats":[{"pgid":"5.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2026-02-24T07:51:52.890170+0000","last_change":"2026-02-24T07:51:52.890170+0000","last_active":"2026-02-24T07:51:52.890170+0000","last_peered":"2026-02-24T07:51:52.890170+0000","last_clean":"2026-02-24T07:51:52.890170+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2026-02-24T07:51:52.890170+0000","last_undegraded":"2026-02-24T07:51:52.890170+0000","last_fullsized":"2026-02-24T07:51:52.890170+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_clean_scrub_stamp":"2026-02-24T07:51:52.890170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2026-02-24T07:51:52.890170+0000","last_change":"2026-02-24T07:51:52.890170+0000","last_active":"2026-02-24T07:51:52.890170+0000","last_peered":"2026-02-24T07:51:52.890170+0000","last_clean":"2026-02-24T07:51:52.890170+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2026-02-24T07:51:52.890170+0000","last_undegraded":"2026-02-24T07:51:52.890170+0000","last_fullsized":"2026-02-24T07:51:52.890170+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_clean_scrub_stamp":"2026-02-24T07:51:52.890170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2026-02-24T07:51:52.890170+0000","last_change":"2026-02-24T07:51:52.890170+0000","last_active":"2026-02-24T07:51:52.890170+0000","last_peered":"2026-02-24T07:51:52.890170+0000","last_clean":"2026-02-24T07:51:52.890170+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2026-02-24T07:51:52.890170+0000","last_undegraded":"2026-02-24T07:51:52.890170+0000","last_fullsized":"2026-02-24T07:51:52.890170+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_clean_scrub_stamp":"2026-02-24T07:51:52.890170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2026-02-24T07:51:52.890170+0000","last_change":"2026-02-24T07:51:52.890170+0000","last_active":"2026-02-24T07:51:52.890170+0000","last_peered":"2026-02-24T07:51:52.890170+0000","last_clean":"2026-02-24T07:51:52.890170+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2026-02-24T07:51:52.890170+0000","last_undegraded":"2026-02-24T07:51:52.890170+0000","last_fullsized":"2026-02-24T07:51:52.890170+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_clean_scrub_stamp":"2026-02-24T07:51:52.890170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2026-02-24T07:51:52.890170+0000","last_change":"2026-02-24T07:51:52.890170+0000","last_active":"2026-02-24T07:51:52.890170+0000","last_peered":"2026-02-24T07:51:52.890170+0000","last_clean":"2026-02-24T07:51:52.890170+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2026-02-24T07:51:52.890170+0000","last_undegraded":"2026-02-24T07:51:52.890170+0000","last_fullsized":"2026-02-24T07:51:52.890170+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_clean_scrub_stamp":"2026-02-24T07:51:52.890170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2026-02-24T07:51:52.890170+0000","last_change":"2026-02-24T07:51:52.890170+0000","last_active":"2026-02-24T07:51:52.890170+0000","last_peered":"2026-02-24T07:51:52.890170+0000","last_clean":"2026-02-24T07:51:52.890170+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2026-02-24T07:51:52.890170+0000","last_undegraded":"2026-02-24T07:51:52.890170+0000","last_fullsized":"2026-02-24T07:51:52.890170+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_clean_scrub_stamp":"2026-02-24T07:51:52.890170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2026-02-24T07:51:52.890170+0000","last_change":"2026-02-24T07:51:52.890170+0000","last_active":"2026-02-24T07:51:52.890170+0000","last_peered":"2026-02-24T07:51:52.890170+0000","last_clean":"2026-02-24T07:51:52.890170+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2026-02-24T07:51:52.890170+0000","last_undegraded":"2026-02-24T07:51:52.890170+0000","last_fullsized":"2026-02-24T07:51:52.890170+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_clean_scrub_stamp":"2026-02-24T07:51:52.890170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2026-02-24T07:51:52.890170+0000","last_change":"2026-02-24T07:51:52.890170+0000","last_active":"2026-02-24T07:51:52.890170+0000","last_peered":"2026-02-24T07:51:52.890170+0000","last_clean":"2026-02-24T07:51:52.890170+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2026-02-24T07:51:52.890170+0000","last_undegraded":"2026-02-24T07:51:52.890170+0000","last_fullsized":"2026-02-24T07:51:52.890170+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_clean_scrub_stamp":"2026-02-24T07:51:52.890170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2026-02-24T07:51:52.890170+0000","last_change":"2026-02-24T07:51:52.890170+0000","last_active":"2026-02-24T07:51:52.890170+0000","last_peered":"2026-02-24T07:51:52.890170+0000","last_clean":"2026-02-24T07:51:52.890170+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2026-02-24T07:51:52.890170+0000","last_undegraded":"2026-02-24T07:51:52.890170+0000","last_fullsized":"2026-02-24T07:51:52.890170+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_clean_scrub_stamp":"2026-02-24T07:51:52.890170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2026-02-24T07:51:52.890170+0000","last_change":"2026-02-24T07:51:52.890170+0000","last_active":"2026-02-24T07:51:52.890170+0000","last_peered":"2026-02-24T07:51:52.890170+0000","last_clean":"2026-02-24T07:51:52.890170+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2026-02-24T07:51:52.890170+0000","last_undegraded":"2026-02-24T07:51:52.890170+0000","last_fullsized":"2026-02-24T07:51:52.890170+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_clean_scrub_stamp":"2026-02-24T07:51:52.890170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2026-02-24T07:51:52.890170+0000","last_change":"2026-02-24T07:51:52.890170+0000","last_active":"2026-02-24T07:51:52.890170+0000","last_peered":"2026-02-24T07:51:52.890170+0000","last_clean":"2026-02-24T07:51:52.890170+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2026-02-24T07:51:52.890170+0000","last_undegraded":"2026-02-24T07:51:52.890170+0000","last_fullsized":"2026-02-24T07:51:52.890170+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_clean_scrub_stamp":"2026-02-24T07:51:52.890170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2026-02-24T07:51:52.890170+0000","last_change":"2026-02-24T07:51:52.890170+0000","last_active":"2026-02-24T07:51:52.890170+0000","last_peered":"2026-02-24T07:51:52.890170+0000","last_clean":"2026-02-24T07:51:52.890170+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2026-02-24T07:51:52.890170+0000","last_undegraded":"2026-02-24T07:51:52.890170+0000","last_fullsized":"2026-02-24T07:51:52.890170+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_clean_scrub_stamp":"2026-02-24T07:51:52.890170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2026-02-24T07:51:52.890170+0000","last_change":"2026-02-24T07:51:52.890170+0000","last_active":"2026-02-24T07:51:52.890170+0000","last_peered":"2026-02-24T07:51:52.890170+0000","last_clean":"2026-02-24T07:51:52.890170+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2026-02-24T07:51:52.890170+0000","last_undegraded":"2026-02-24T07:51:52.890170+0000","last_fullsized":"2026-02-24T07:51:52.890170+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_clean_scrub_stamp":"2026-02-24T07:51:52.890170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2026-02-24T07:51:52.890170+0000","last_change":"2026-02-24T07:51:52.890170+0000","last_active":"2026-02-24T07:51:52.890170+0000","last_peered":"2026-02-24T07:51:52.890170+0000","last_clean":"2026-02-24T07:51:52.890170+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2026-02-24T07:51:52.890170+0000","last_undegraded":"2026-02-24T07:51:52.890170+0000","last_fullsized":"2026-02-24T07:51:52.890170+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_clean_scrub_stamp":"2026-02-24T07:51:52.890170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2026-02-24T07:51:52.890170+0000","last_change":"2026-02-24T07:51:52.890170+0000","last_active":"2026-02-24T07:51:52.890170+0000","last_peered":"2026-02-24T07:51:52.890170+0000","last_clean":"2026-02-24T07:51:52.890170+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2026-02-24T07:51:52.890170+0000","last_undegraded":"2026-02-24T07:51:52.890170+0000","last_fullsized":"2026-02-24T07:51:52.890170+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_clean_scrub_stamp":"2026-02-24T07:51:52.890170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2026-02-24T07:51:52.890170+0000","last_change":"2026-02-24T07:51:52.890170+0000","last_active":"2026-02-24T07:51:52.890170+0000","last_peered":"2026-02-24T07:51:52.890170+0000","last_clean":"2026-02-24T07:51:52.890170+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2026-02-24T07:51:52.890170+0000","last_undegraded":"2026-02-24T07:51:52.890170+0000","last_fullsized":"2026-02-24T07:51:52.890170+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_clean_scrub_stamp":"2026-02-24T07:51:52.890170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"29'99","reported_seq":316,"reported_epoch":72,"state":"active+clean","last_fresh":"2026-02-24T07:51:48.882290+0000","last_change":"2026-02-24T07:50:30.401963+0000","last_active":"2026-02-24T07:51:48.882290+0000","last_peered":"2026-02-24T07:51:48.882290+0000","last_clean":"2026-02-24T07:51:48.882290+0000","last_became_active":"2026-02-24T07:42:54.186719+0000","last_became_peered":"2026-02-24T07:42:54.186719+0000","last_unstale":"2026-02-24T07:51:48.882290+0000","last_undegraded":"2026-02-24T07:51:48.882290+0000","last_fullsized":"2026-02-24T07:51:48.882290+0000","mapping_epoch":27,"log_start":"29'96","ondisk_log_start":"29'96","created":20,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'99","last_scrub_stamp":"2026-02-24T07:50:30.401929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:42:43.640776+0000","last_clean_scrub_stamp":"2026-02-24T07:50:30.401929+0000","objects_scrubbed":2,"log_size":3,"log_dups_size":96,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:51:52.990727+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":240,"num_write":135,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":16,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":240,"num_write":135,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1682,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":3,"up_from":25,"seq":107374182510,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":1012812,"kb_used_data":1336,"kb_used_omap":16,"kb_used_meta":1011439,"kb_avail":732990388,"statfs":{"total":751619276800,"available":750582157312,"internally_reserved":0,"allocated":1368064,"data_stored":867887,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":17036,"internal_metadata":1035713908},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":19,"seq":81604378736,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":1347776,"kb_used_data":1324,"kb_used_omap":17,"kb_used_meta":1346414,"kb_avail":732655424,"statfs":{"total":751619276800,"available":750239154176,"internally_reserved":0,"allocated":1355776,"data_stored":863486,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":17880,"internal_metadata":1378728488},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542257,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":1570364,"kb_used_data":744,"kb_used_omap":15,"kb_used_meta":1569584,"kb_avail":732432836,"statfs":{"total":751619276800,"available":750011224064,"internally_reserved":0,"allocated":761856,"data_stored":273118,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":15499,"internal_metadata":1607254901},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705780,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":663628,"kb_used_data":1336,"kb_used_omap":15,"kb_used_meta":662256,"kb_avail":733339572,"statfs":{"total":751619276800,"available":750939721728,"internally_reserved":0,"allocated":1368064,"data_stored":867887,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":15445,"internal_metadata":678151083},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1682,"internal_metadata":0}]}} 2026-02-24T07:51:55.413 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph osd pool set .mgr pgp_num 291 2026-02-24T07:51:55.556 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:51:55.687 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:51:55.685+0000 7fc40e30a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:51:55.687 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:51:55.685+0000 7fc40e30a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:51:55.824 INFO:teuthology.orchestra.run.trial035.stderr:Error EPERM: crimson_allow_pg_split is false; pgp_num increase denied 2026-02-24T07:51:55.867 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-24T07:51:55.954 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:51:55 trial035 ceph-mon[39284]: osdmap e76: 4 total, 4 up, 4 in 2026-02-24T07:51:55.954 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:51:55 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/3887084305' entity='client.admin' cmd={"prefix": "osd pool set", "pool": ".mgr", "var": "pgp_num", "val": "291"} : dispatch 2026-02-24T07:51:55.991 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 1 16 6583 6567 410.403 410.438 0.00129074 0.00243181 2026-02-24T07:51:56.387 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2026-02-24T07:51:56.524 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:51:56.636 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:51:56.634+0000 7ff7352a1640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:51:56.636 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:51:56.634+0000 7ff7352a1640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:51:56.652 INFO:teuthology.orchestra.run.trial035.stdout:osd.0: {} 2026-02-24T07:51:56.652 INFO:teuthology.orchestra.run.trial035.stderr:osd.0: osd_enable_op_tracker = '' 2026-02-24T07:51:56.665 INFO:teuthology.orchestra.run.trial035.stdout:osd.1: {} 2026-02-24T07:51:56.665 INFO:teuthology.orchestra.run.trial035.stderr:osd.1: osd_enable_op_tracker = '' 2026-02-24T07:51:56.678 INFO:teuthology.orchestra.run.trial035.stdout:osd.2: {} 2026-02-24T07:51:56.679 INFO:teuthology.orchestra.run.trial035.stderr:osd.2: osd_enable_op_tracker = '' 2026-02-24T07:51:56.691 INFO:teuthology.orchestra.run.trial035.stdout:osd.3: {} 2026-02-24T07:51:56.692 INFO:teuthology.orchestra.run.trial035.stderr:osd.3: osd_enable_op_tracker = '' 2026-02-24T07:51:56.954 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:51:56 trial035 ceph-mon[39284]: from='client.15148 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2026-02-24T07:51:56.954 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:51:56 trial035 ceph-mon[39284]: pgmap v318: 17 pgs: 16 unknown, 1 active+clean; 577 KiB data, 4.4 GiB used, 2.7 TiB / 2.7 TiB avail 2026-02-24T07:51:56.954 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:51:56 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/4079264152' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2026-02-24T07:51:56.991 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 2 15 13283 13268 414.591 418.812 0.00304322 0.00240924 2026-02-24T07:51:57.356 INFO:journalctl@ceph.prometheus.a.trial091.stdout:Feb 24 07:51:56 trial091 ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-prometheus-a[66007]: time=2026-02-24T07:51:56.973Z level=ERROR source=scrape.go:1630 msg="Failed to determine correct type of scrape target." component="scrape manager" scrape_pool=ceph target=http://trial035:9283/metrics content_type="text/html;charset=utf-8" fallback_media_type="" err="received unsupported Content-Type \"text/html;charset=utf-8\" and no fallback_scrape_protocol specified for target" 2026-02-24T07:51:57.454 INFO:journalctl@ceph.mgr.x.trial035.stdout:Feb 24 07:51:56 trial035 ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-mgr-x[39621]: ::ffff:10.20.193.91 - - [24/Feb/2026:07:51:56] "GET /metrics HTTP/1.1" 200 4230 "" "Prometheus/3.6.0" 2026-02-24T07:51:57.991 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 3 16 20160 20144 419.633 429.75 0.00251591 0.00238086 2026-02-24T07:51:58.731 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2026-02-24T07:51:58.868 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:51:58.988 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:51:58.986+0000 7f379201c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:51:58.988 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:51:58.986+0000 7f379201c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:51:58.991 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 4 16 27204 27188 424.779 440.25 0.00254938 0.00235223 2026-02-24T07:51:59.003 INFO:teuthology.orchestra.run.trial035.stdout:osd.0: {} 2026-02-24T07:51:59.003 INFO:teuthology.orchestra.run.trial035.stderr:osd.0: osd_enable_op_tracker = '' 2026-02-24T07:51:59.015 INFO:teuthology.orchestra.run.trial035.stdout:osd.1: {} 2026-02-24T07:51:59.016 INFO:teuthology.orchestra.run.trial035.stderr:osd.1: osd_enable_op_tracker = '' 2026-02-24T07:51:59.016 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:51:58 trial035 ceph-mon[39284]: pgmap v319: 17 pgs: 17 active+clean; 316 MiB data, 4.7 GiB used, 2.7 TiB / 2.7 TiB avail; 53 MiB/s wr, 840 op/s 2026-02-24T07:51:59.028 INFO:teuthology.orchestra.run.trial035.stdout:osd.2: {} 2026-02-24T07:51:59.029 INFO:teuthology.orchestra.run.trial035.stderr:osd.2: osd_enable_op_tracker = '' 2026-02-24T07:51:59.043 INFO:teuthology.orchestra.run.trial035.stdout:osd.3: {} 2026-02-24T07:51:59.043 INFO:teuthology.orchestra.run.trial035.stderr:osd.3: osd_enable_op_tracker = '' 2026-02-24T07:51:59.991 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 5 16 35129 35113 438.879 495.312 0.0014334 0.00227685 2026-02-24T07:52:00.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:51:59 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/2339449168' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2026-02-24T07:52:00.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:51:59 trial035 ceph-mon[39284]: from='mgr.14284 10.20.193.35:0/3681373222' entity='mgr.x' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2026-02-24T07:52:00.868 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2026-02-24T07:52:00.869 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2026-02-24T07:52:00.869 INFO:tasks.thrashosds.thrasher:Growing pool .mgr 2026-02-24T07:52:00.869 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph pg dump --format=json 2026-02-24T07:52:00.991 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 6 16 42714 42698 444.737 474.062 0.0017688 0.00224692 2026-02-24T07:52:01.009 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:52:01.107 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2026-02-24T07:52:01.143 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:52:01.141+0000 7fe7d3d5b640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:52:01.144 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:52:01.141+0000 7fe7d3d5b640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:52:01.144 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:52:00 trial035 ceph-mon[39284]: pgmap v320: 17 pgs: 17 active+clean; 316 MiB data, 4.7 GiB used, 2.7 TiB / 2.7 TiB avail; 46 MiB/s wr, 737 op/s 2026-02-24T07:52:01.245 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:52:01.282 INFO:teuthology.orchestra.run.trial035.stdout: 2026-02-24T07:52:01.283 INFO:teuthology.orchestra.run.trial035.stderr:dumped all 2026-02-24T07:52:01.337 INFO:teuthology.orchestra.run.trial035.stdout:{"pg_ready":true,"pg_map":{"version":320,"stamp":"2026-02-24T07:51:59.730507+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":331153952,"num_objects":5046,"num_object_clones":0,"num_object_copies":15138,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5046,"num_whiteouts":0,"num_read":136,"num_read_kb":240,"num_write":5179,"num_write_kb":324966,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":51,"ondisk_log_size":51,"up":51,"acting":51,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":34,"num_osds":4,"num_per_pool_osds":4,"num_per_pool_omap_osds":4,"kb":2936012800,"kb_used":4956256,"kb_used_data":330320,"kb_used_omap":73,"kb_used_meta":4625782,"kb_avail":2931056544,"statfs":{"total":3006477107200,"available":3001401901056,"internally_reserved":0,"allocated":338247680,"data_stored":336136228,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":75266,"internal_metadata":4736801278},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":330563584,"num_objects":5044,"num_object_clones":0,"num_object_copies":15132,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5044,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5044,"num_write_kb":322816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.837147"},"pg_stats":[{"pgid":"5.f","version":"76'284","reported_seq":298,"reported_epoch":76,"state":"active+clean","last_fresh":"2026-02-24T07:51:55.738890+0000","last_change":"2026-02-24T07:51:53.900797+0000","last_active":"2026-02-24T07:51:55.738890+0000","last_peered":"2026-02-24T07:51:55.738890+0000","last_clean":"2026-02-24T07:51:55.738890+0000","last_became_active":"2026-02-24T07:51:53.900431+0000","last_became_peered":"2026-02-24T07:51:53.900431+0000","last_unstale":"2026-02-24T07:51:55.738890+0000","last_undegraded":"2026-02-24T07:51:55.738890+0000","last_fullsized":"2026-02-24T07:51:55.738890+0000","mapping_epoch":74,"log_start":"76'281","ondisk_log_start":"76'281","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_clean_scrub_stamp":"2026-02-24T07:51:52.890170+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":281,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:52:57.614871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18612224,"num_objects":284,"num_object_clones":0,"num_object_copies":852,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":284,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":284,"num_write_kb":18176,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.e","version":"76'325","reported_seq":339,"reported_epoch":76,"state":"active+clean","last_fresh":"2026-02-24T07:51:55.751929+0000","last_change":"2026-02-24T07:51:53.899686+0000","last_active":"2026-02-24T07:51:55.751929+0000","last_peered":"2026-02-24T07:51:55.751929+0000","last_clean":"2026-02-24T07:51:55.751929+0000","last_became_active":"2026-02-24T07:51:53.899294+0000","last_became_peered":"2026-02-24T07:51:53.899294+0000","last_unstale":"2026-02-24T07:51:55.751929+0000","last_undegraded":"2026-02-24T07:51:55.751929+0000","last_fullsized":"2026-02-24T07:51:55.751929+0000","mapping_epoch":74,"log_start":"76'322","ondisk_log_start":"76'322","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_clean_scrub_stamp":"2026-02-24T07:51:52.890170+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":322,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:52:53.084042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21299200,"num_objects":325,"num_object_clones":0,"num_object_copies":975,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":325,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":325,"num_write_kb":20800,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"76'316","reported_seq":328,"reported_epoch":76,"state":"active+clean","last_fresh":"2026-02-24T07:51:55.759877+0000","last_change":"2026-02-24T07:51:53.900129+0000","last_active":"2026-02-24T07:51:55.759877+0000","last_peered":"2026-02-24T07:51:55.759877+0000","last_clean":"2026-02-24T07:51:55.759877+0000","last_became_active":"2026-02-24T07:51:53.899772+0000","last_became_peered":"2026-02-24T07:51:53.899772+0000","last_unstale":"2026-02-24T07:51:55.759877+0000","last_undegraded":"2026-02-24T07:51:55.759877+0000","last_fullsized":"2026-02-24T07:51:55.759877+0000","mapping_epoch":74,"log_start":"76'313","ondisk_log_start":"76'313","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_clean_scrub_stamp":"2026-02-24T07:51:52.890170+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":313,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:53:19.516698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20709376,"num_objects":316,"num_object_clones":0,"num_object_copies":948,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":316,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":316,"num_write_kb":20224,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,3],"acting":[0,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.c","version":"76'353","reported_seq":366,"reported_epoch":76,"state":"active+clean","last_fresh":"2026-02-24T07:51:55.842968+0000","last_change":"2026-02-24T07:51:53.898484+0000","last_active":"2026-02-24T07:51:55.842968+0000","last_peered":"2026-02-24T07:51:55.842968+0000","last_clean":"2026-02-24T07:51:55.842968+0000","last_became_active":"2026-02-24T07:51:53.898285+0000","last_became_peered":"2026-02-24T07:51:53.898285+0000","last_unstale":"2026-02-24T07:51:55.842968+0000","last_undegraded":"2026-02-24T07:51:55.842968+0000","last_fullsized":"2026-02-24T07:51:55.842968+0000","mapping_epoch":74,"log_start":"76'350","ondisk_log_start":"76'350","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_clean_scrub_stamp":"2026-02-24T07:51:52.890170+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":350,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:53:11.947668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23134208,"num_objects":353,"num_object_clones":0,"num_object_copies":1059,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":353,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":353,"num_write_kb":22592,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.b","version":"76'333","reported_seq":347,"reported_epoch":76,"state":"active+clean","last_fresh":"2026-02-24T07:51:55.762233+0000","last_change":"2026-02-24T07:51:53.900298+0000","last_active":"2026-02-24T07:51:55.762233+0000","last_peered":"2026-02-24T07:51:55.762233+0000","last_clean":"2026-02-24T07:51:55.762233+0000","last_became_active":"2026-02-24T07:51:53.899901+0000","last_became_peered":"2026-02-24T07:51:53.899901+0000","last_unstale":"2026-02-24T07:51:55.762233+0000","last_undegraded":"2026-02-24T07:51:55.762233+0000","last_fullsized":"2026-02-24T07:51:55.762233+0000","mapping_epoch":74,"log_start":"76'330","ondisk_log_start":"76'330","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_clean_scrub_stamp":"2026-02-24T07:51:52.890170+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":330,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:53:12.321784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21823488,"num_objects":333,"num_object_clones":0,"num_object_copies":999,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":333,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":333,"num_write_kb":21312,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"76'302","reported_seq":316,"reported_epoch":76,"state":"active+clean","last_fresh":"2026-02-24T07:51:55.763271+0000","last_change":"2026-02-24T07:51:53.899828+0000","last_active":"2026-02-24T07:51:55.763271+0000","last_peered":"2026-02-24T07:51:55.763271+0000","last_clean":"2026-02-24T07:51:55.763271+0000","last_became_active":"2026-02-24T07:51:53.899549+0000","last_became_peered":"2026-02-24T07:51:53.899549+0000","last_unstale":"2026-02-24T07:51:55.763271+0000","last_undegraded":"2026-02-24T07:51:55.763271+0000","last_fullsized":"2026-02-24T07:51:55.763271+0000","mapping_epoch":74,"log_start":"76'299","ondisk_log_start":"76'299","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_clean_scrub_stamp":"2026-02-24T07:51:52.890170+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":299,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:53:07.446496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19791872,"num_objects":302,"num_object_clones":0,"num_object_copies":906,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":302,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":302,"num_write_kb":19328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,1],"acting":[0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.9","version":"76'327","reported_seq":340,"reported_epoch":76,"state":"active+clean","last_fresh":"2026-02-24T07:51:55.752928+0000","last_change":"2026-02-24T07:51:53.901985+0000","last_active":"2026-02-24T07:51:55.752928+0000","last_peered":"2026-02-24T07:51:55.752928+0000","last_clean":"2026-02-24T07:51:55.752928+0000","last_became_active":"2026-02-24T07:51:53.901610+0000","last_became_peered":"2026-02-24T07:51:53.901610+0000","last_unstale":"2026-02-24T07:51:55.752928+0000","last_undegraded":"2026-02-24T07:51:55.752928+0000","last_fullsized":"2026-02-24T07:51:55.752928+0000","mapping_epoch":74,"log_start":"76'324","ondisk_log_start":"76'324","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_clean_scrub_stamp":"2026-02-24T07:51:52.890170+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":324,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:53:13.073570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21430272,"num_objects":327,"num_object_clones":0,"num_object_copies":981,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":327,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":327,"num_write_kb":20928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.8","version":"76'303","reported_seq":317,"reported_epoch":76,"state":"active+clean","last_fresh":"2026-02-24T07:51:55.764581+0000","last_change":"2026-02-24T07:51:53.900477+0000","last_active":"2026-02-24T07:51:55.764581+0000","last_peered":"2026-02-24T07:51:55.764581+0000","last_clean":"2026-02-24T07:51:55.764581+0000","last_became_active":"2026-02-24T07:51:53.900207+0000","last_became_peered":"2026-02-24T07:51:53.900207+0000","last_unstale":"2026-02-24T07:51:55.764581+0000","last_undegraded":"2026-02-24T07:51:55.764581+0000","last_fullsized":"2026-02-24T07:51:55.764581+0000","mapping_epoch":74,"log_start":"76'300","ondisk_log_start":"76'300","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_clean_scrub_stamp":"2026-02-24T07:51:52.890170+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":300,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:52:57.062307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19857408,"num_objects":303,"num_object_clones":0,"num_object_copies":909,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":303,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":303,"num_write_kb":19392,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.7","version":"76'325","reported_seq":339,"reported_epoch":76,"state":"active+clean","last_fresh":"2026-02-24T07:51:55.755272+0000","last_change":"2026-02-24T07:51:53.899854+0000","last_active":"2026-02-24T07:51:55.755272+0000","last_peered":"2026-02-24T07:51:55.755272+0000","last_clean":"2026-02-24T07:51:55.755272+0000","last_became_active":"2026-02-24T07:51:53.899472+0000","last_became_peered":"2026-02-24T07:51:53.899472+0000","last_unstale":"2026-02-24T07:51:55.755272+0000","last_undegraded":"2026-02-24T07:51:55.755272+0000","last_fullsized":"2026-02-24T07:51:55.755272+0000","mapping_epoch":74,"log_start":"76'322","ondisk_log_start":"76'322","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_clean_scrub_stamp":"2026-02-24T07:51:52.890170+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":322,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:52:55.115725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21299200,"num_objects":325,"num_object_clones":0,"num_object_copies":975,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":325,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":325,"num_write_kb":20800,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.6","version":"76'348","reported_seq":362,"reported_epoch":76,"state":"active+clean","last_fresh":"2026-02-24T07:51:55.840191+0000","last_change":"2026-02-24T07:51:53.898841+0000","last_active":"2026-02-24T07:51:55.840191+0000","last_peered":"2026-02-24T07:51:55.840191+0000","last_clean":"2026-02-24T07:51:55.840191+0000","last_became_active":"2026-02-24T07:51:53.898308+0000","last_became_peered":"2026-02-24T07:51:53.898308+0000","last_unstale":"2026-02-24T07:51:55.840191+0000","last_undegraded":"2026-02-24T07:51:55.840191+0000","last_fullsized":"2026-02-24T07:51:55.840191+0000","mapping_epoch":74,"log_start":"76'345","ondisk_log_start":"76'345","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_clean_scrub_stamp":"2026-02-24T07:51:52.890170+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":345,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:53:04.955186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22806528,"num_objects":348,"num_object_clones":0,"num_object_copies":1044,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":348,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":348,"num_write_kb":22272,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.5","version":"76'303","reported_seq":317,"reported_epoch":76,"state":"active+clean","last_fresh":"2026-02-24T07:51:55.763384+0000","last_change":"2026-02-24T07:51:53.900139+0000","last_active":"2026-02-24T07:51:55.763384+0000","last_peered":"2026-02-24T07:51:55.763384+0000","last_clean":"2026-02-24T07:51:55.763384+0000","last_became_active":"2026-02-24T07:51:53.899749+0000","last_became_peered":"2026-02-24T07:51:53.899749+0000","last_unstale":"2026-02-24T07:51:55.763384+0000","last_undegraded":"2026-02-24T07:51:55.763384+0000","last_fullsized":"2026-02-24T07:51:55.763384+0000","mapping_epoch":74,"log_start":"76'300","ondisk_log_start":"76'300","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_clean_scrub_stamp":"2026-02-24T07:51:52.890170+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":300,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:53:09.489880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19857408,"num_objects":303,"num_object_clones":0,"num_object_copies":909,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":303,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":303,"num_write_kb":19392,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"76'314","reported_seq":326,"reported_epoch":76,"state":"active+clean","last_fresh":"2026-02-24T07:51:55.759976+0000","last_change":"2026-02-24T07:51:53.900361+0000","last_active":"2026-02-24T07:51:55.759976+0000","last_peered":"2026-02-24T07:51:55.759976+0000","last_clean":"2026-02-24T07:51:55.759976+0000","last_became_active":"2026-02-24T07:51:53.900042+0000","last_became_peered":"2026-02-24T07:51:53.900042+0000","last_unstale":"2026-02-24T07:51:55.759976+0000","last_undegraded":"2026-02-24T07:51:55.759976+0000","last_fullsized":"2026-02-24T07:51:55.759976+0000","mapping_epoch":74,"log_start":"76'311","ondisk_log_start":"76'311","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_clean_scrub_stamp":"2026-02-24T07:51:52.890170+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":311,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:53:02.182725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20578304,"num_objects":314,"num_object_clones":0,"num_object_copies":942,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":314,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":314,"num_write_kb":20096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.2","version":"76'295","reported_seq":309,"reported_epoch":76,"state":"active+clean","last_fresh":"2026-02-24T07:51:55.755503+0000","last_change":"2026-02-24T07:51:53.900424+0000","last_active":"2026-02-24T07:51:55.755503+0000","last_peered":"2026-02-24T07:51:55.755503+0000","last_clean":"2026-02-24T07:51:55.755503+0000","last_became_active":"2026-02-24T07:51:53.900096+0000","last_became_peered":"2026-02-24T07:51:53.900096+0000","last_unstale":"2026-02-24T07:51:55.755503+0000","last_undegraded":"2026-02-24T07:51:55.755503+0000","last_fullsized":"2026-02-24T07:51:55.755503+0000","mapping_epoch":74,"log_start":"76'292","ondisk_log_start":"76'292","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_clean_scrub_stamp":"2026-02-24T07:51:52.890170+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":292,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:53:10.802005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19333120,"num_objects":295,"num_object_clones":0,"num_object_copies":885,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":295,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":295,"num_write_kb":18880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"76'306","reported_seq":319,"reported_epoch":76,"state":"active+clean","last_fresh":"2026-02-24T07:51:55.751848+0000","last_change":"2026-02-24T07:51:53.901421+0000","last_active":"2026-02-24T07:51:55.751848+0000","last_peered":"2026-02-24T07:51:55.751848+0000","last_clean":"2026-02-24T07:51:55.751848+0000","last_became_active":"2026-02-24T07:51:53.901027+0000","last_became_peered":"2026-02-24T07:51:53.901027+0000","last_unstale":"2026-02-24T07:51:55.751848+0000","last_undegraded":"2026-02-24T07:51:55.751848+0000","last_fullsized":"2026-02-24T07:51:55.751848+0000","mapping_epoch":74,"log_start":"76'303","ondisk_log_start":"76'303","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_clean_scrub_stamp":"2026-02-24T07:51:52.890170+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":303,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:53:04.436560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20054016,"num_objects":306,"num_object_clones":0,"num_object_copies":918,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":306,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":306,"num_write_kb":19584,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.0","version":"76'286","reported_seq":299,"reported_epoch":76,"state":"active+clean","last_fresh":"2026-02-24T07:51:55.762951+0000","last_change":"2026-02-24T07:51:53.898740+0000","last_active":"2026-02-24T07:51:55.762951+0000","last_peered":"2026-02-24T07:51:55.762951+0000","last_clean":"2026-02-24T07:51:55.762951+0000","last_became_active":"2026-02-24T07:51:53.898551+0000","last_became_peered":"2026-02-24T07:51:53.898551+0000","last_unstale":"2026-02-24T07:51:55.762951+0000","last_undegraded":"2026-02-24T07:51:55.762951+0000","last_fullsized":"2026-02-24T07:51:55.762951+0000","mapping_epoch":74,"log_start":"76'283","ondisk_log_start":"76'283","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_clean_scrub_stamp":"2026-02-24T07:51:52.890170+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":283,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:53:07.664864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18743296,"num_objects":286,"num_object_clones":0,"num_object_copies":858,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":286,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":286,"num_write_kb":18304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.4","version":"76'324","reported_seq":338,"reported_epoch":76,"state":"active+clean","last_fresh":"2026-02-24T07:51:55.738930+0000","last_change":"2026-02-24T07:51:53.900179+0000","last_active":"2026-02-24T07:51:55.738930+0000","last_peered":"2026-02-24T07:51:55.738930+0000","last_clean":"2026-02-24T07:51:55.738930+0000","last_became_active":"2026-02-24T07:51:53.899675+0000","last_became_peered":"2026-02-24T07:51:53.899675+0000","last_unstale":"2026-02-24T07:51:55.738930+0000","last_undegraded":"2026-02-24T07:51:55.738930+0000","last_fullsized":"2026-02-24T07:51:55.738930+0000","mapping_epoch":74,"log_start":"76'321","ondisk_log_start":"76'321","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_clean_scrub_stamp":"2026-02-24T07:51:52.890170+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":321,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:53:06.339989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21233664,"num_objects":324,"num_object_clones":0,"num_object_copies":972,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":324,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":324,"num_write_kb":20736,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"29'99","reported_seq":332,"reported_epoch":76,"state":"active+clean","last_fresh":"2026-02-24T07:51:54.993390+0000","last_change":"2026-02-24T07:51:54.101629+0000","last_active":"2026-02-24T07:51:54.993390+0000","last_peered":"2026-02-24T07:51:54.993390+0000","last_clean":"2026-02-24T07:51:54.993390+0000","last_became_active":"2026-02-24T07:42:54.186719+0000","last_became_peered":"2026-02-24T07:42:54.186719+0000","last_unstale":"2026-02-24T07:51:54.993390+0000","last_undegraded":"2026-02-24T07:51:54.993390+0000","last_fullsized":"2026-02-24T07:51:54.993390+0000","mapping_epoch":27,"log_start":"29'96","ondisk_log_start":"29'96","created":20,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'99","last_scrub_stamp":"2026-02-24T07:51:54.101595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:42:43.640776+0000","last_clean_scrub_stamp":"2026-02-24T07:51:54.101595+0000","objects_scrubbed":2,"log_size":3,"log_dups_size":96,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:52:54.995630+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":240,"num_write":135,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":16,"stat_sum":{"num_bytes":330563584,"num_objects":5044,"num_object_clones":0,"num_object_copies":15132,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5044,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5044,"num_write_kb":322816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":994312192,"data_stored":994312192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":48,"ondisk_log_size":48,"up":48,"acting":48,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":240,"num_write":135,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1682,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":3,"up_from":25,"seq":107374182511,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":1057404,"kb_used_data":27880,"kb_used_omap":17,"kb_used_meta":1029486,"kb_avail":732945796,"statfs":{"total":751619276800,"available":750536495104,"internally_reserved":0,"allocated":28549120,"data_stored":28018483,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":17663,"internal_metadata":1054194433},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":19,"seq":81604378737,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":1365872,"kb_used_data":1372,"kb_used_omap":20,"kb_used_meta":1364459,"kb_avail":732637328,"statfs":{"total":751619276800,"available":750220623872,"internally_reserved":0,"allocated":1404928,"data_stored":881835,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":20644,"internal_metadata":1397206876},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542258,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":1673080,"kb_used_data":103460,"kb_used_omap":18,"kb_used_meta":1569581,"kb_avail":732330120,"statfs":{"total":751619276800,"available":749906042880,"internally_reserved":0,"allocated":105943040,"data_stored":105415955,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":18821,"internal_metadata":1607251579},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705781,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":859900,"kb_used_data":197608,"kb_used_omap":17,"kb_used_meta":662254,"kb_avail":733143300,"statfs":{"total":751619276800,"available":750738739200,"internally_reserved":0,"allocated":202350592,"data_stored":201819955,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":18138,"internal_metadata":678148390},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1682,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":264372224,"data_stored":264372224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":226951168,"data_stored":226951168,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":218234880,"data_stored":218234880,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":284753920,"data_stored":284753920,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-24T07:52:01.338 INFO:tasks.cephadm.ceph_manager.ceph:increase pool size by 10 2026-02-24T07:52:01.338 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph osd pool set .mgr pg_num 301 2026-02-24T07:52:01.374 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:52:01.372+0000 7fd0b0a29640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:52:01.375 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:52:01.372+0000 7fd0b0a29640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:52:01.389 INFO:teuthology.orchestra.run.trial035.stdout:osd.0: {} 2026-02-24T07:52:01.389 INFO:teuthology.orchestra.run.trial035.stderr:osd.0: osd_enable_op_tracker = '' 2026-02-24T07:52:01.402 INFO:teuthology.orchestra.run.trial035.stdout:osd.1: {} 2026-02-24T07:52:01.403 INFO:teuthology.orchestra.run.trial035.stderr:osd.1: osd_enable_op_tracker = '' 2026-02-24T07:52:01.420 INFO:teuthology.orchestra.run.trial035.stdout:osd.2: {} 2026-02-24T07:52:01.420 INFO:teuthology.orchestra.run.trial035.stderr:osd.2: osd_enable_op_tracker = '' 2026-02-24T07:52:01.432 INFO:teuthology.orchestra.run.trial035.stdout:osd.3: {} 2026-02-24T07:52:01.432 INFO:teuthology.orchestra.run.trial035.stderr:osd.3: osd_enable_op_tracker = '' 2026-02-24T07:52:01.524 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:52:01.643 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:52:01.641+0000 7f03d5118640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:52:01.643 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:52:01.641+0000 7f03d5118640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:52:01.774 INFO:teuthology.orchestra.run.trial035.stderr:Error EPERM: crimson_allow_pg_split is false; pg_num increase denied for crimson pool 2026-02-24T07:52:01.814 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-24T07:52:01.992 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 7 16 49607 49591 442.743 430.812 0.00453883 0.00225714 2026-02-24T07:52:02.037 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:52:01 trial035 ceph-mon[39284]: from='client.15156 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2026-02-24T07:52:02.038 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:52:01 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/3021287796' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2026-02-24T07:52:02.038 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:52:01 trial035 ceph-mon[39284]: pgmap v321: 17 pgs: 17 active+clean; 2.5 GiB data, 12 GiB used, 2.7 TiB / 2.7 TiB avail; 321 MiB/s wr, 5.13k op/s 2026-02-24T07:52:02.038 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:52:01 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/2049691346' entity='client.admin' cmd={"prefix": "osd pool set", "pool": ".mgr", "var": "pg_num", "val": "301"} : dispatch 2026-02-24T07:52:02.991 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 8 16 56563 56547 441.739 434.75 0.00142058 0.00226228 2026-02-24T07:52:03.495 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2026-02-24T07:52:03.639 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:52:03.767 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:52:03.764+0000 7fa2b25c2640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:52:03.767 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:52:03.765+0000 7fa2b25c2640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:52:03.783 INFO:teuthology.orchestra.run.trial035.stdout:osd.0: {} 2026-02-24T07:52:03.783 INFO:teuthology.orchestra.run.trial035.stderr:osd.0: osd_enable_op_tracker = '' 2026-02-24T07:52:03.797 INFO:teuthology.orchestra.run.trial035.stdout:osd.1: {} 2026-02-24T07:52:03.797 INFO:teuthology.orchestra.run.trial035.stderr:osd.1: osd_enable_op_tracker = '' 2026-02-24T07:52:03.810 INFO:teuthology.orchestra.run.trial035.stdout:osd.2: {} 2026-02-24T07:52:03.810 INFO:teuthology.orchestra.run.trial035.stderr:osd.2: osd_enable_op_tracker = '' 2026-02-24T07:52:03.822 INFO:teuthology.orchestra.run.trial035.stdout:osd.3: {} 2026-02-24T07:52:03.822 INFO:teuthology.orchestra.run.trial035.stderr:osd.3: osd_enable_op_tracker = '' 2026-02-24T07:52:03.992 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 9 16 63337 63321 439.694 423.375 0.00115622 0.00227281 2026-02-24T07:52:04.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:52:03 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/2596775285' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2026-02-24T07:52:04.992 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 10 16 70007 69991 437.409 416.875 0.000864367 0.00228335 2026-02-24T07:52:05.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:52:04 trial035 ceph-mon[39284]: pgmap v322: 17 pgs: 17 active+clean; 2.5 GiB data, 12 GiB used, 2.7 TiB / 2.7 TiB avail; 261 MiB/s wr, 4.17k op/s 2026-02-24T07:52:05.867 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2026-02-24T07:52:05.992 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 11 16 76656 76640 435.421 415.562 0.0032335 0.00229529 2026-02-24T07:52:06.007 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:52:06.127 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:52:06.124+0000 7f6274e05640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:52:06.127 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:52:06.125+0000 7f6274e05640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:52:06.141 INFO:teuthology.orchestra.run.trial035.stdout:osd.0: {} 2026-02-24T07:52:06.141 INFO:teuthology.orchestra.run.trial035.stderr:osd.0: osd_enable_op_tracker = '' 2026-02-24T07:52:06.153 INFO:teuthology.orchestra.run.trial035.stdout:osd.1: {} 2026-02-24T07:52:06.153 INFO:teuthology.orchestra.run.trial035.stderr:osd.1: osd_enable_op_tracker = '' 2026-02-24T07:52:06.165 INFO:teuthology.orchestra.run.trial035.stdout:osd.2: {} 2026-02-24T07:52:06.166 INFO:teuthology.orchestra.run.trial035.stderr:osd.2: osd_enable_op_tracker = '' 2026-02-24T07:52:06.178 INFO:teuthology.orchestra.run.trial035.stdout:osd.3: {} 2026-02-24T07:52:06.178 INFO:teuthology.orchestra.run.trial035.stderr:osd.3: osd_enable_op_tracker = '' 2026-02-24T07:52:06.816 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2026-02-24T07:52:06.816 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2026-02-24T07:52:06.816 INFO:tasks.thrashosds.thrasher:Growing pool .mgr 2026-02-24T07:52:06.816 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph pg dump --format=json 2026-02-24T07:52:06.956 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:52:06.992 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 12 15 83010 82995 432.232 397.188 0.00238095 0.00231228 2026-02-24T07:52:07.072 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:52:06 trial035 ceph-mon[39284]: pgmap v323: 17 pgs: 17 active+clean; 2.5 GiB data, 12 GiB used, 2.7 TiB / 2.7 TiB avail; 237 MiB/s wr, 3.79k op/s 2026-02-24T07:52:07.072 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:52:06 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/730458972' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2026-02-24T07:52:07.073 INFO:journalctl@ceph.mgr.x.trial035.stdout:Feb 24 07:52:06 trial035 ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-mgr-x[39621]: ::ffff:10.20.193.91 - - [24/Feb/2026:07:52:06] "GET /metrics HTTP/1.1" 200 4214 "" "Prometheus/3.6.0" 2026-02-24T07:52:07.076 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:52:07.074+0000 7fcf2cdc1640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:52:07.076 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:52:07.074+0000 7fcf2cdc1640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:52:07.211 INFO:teuthology.orchestra.run.trial035.stdout: 2026-02-24T07:52:07.211 INFO:teuthology.orchestra.run.trial035.stderr:dumped all 2026-02-24T07:52:07.255 INFO:teuthology.orchestra.run.trial035.stdout:{"pg_ready":true,"pg_map":{"version":323,"stamp":"2026-02-24T07:52:05.731357+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":2690843168,"num_objects":41052,"num_object_clones":0,"num_object_copies":123156,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41052,"num_whiteouts":0,"num_read":136,"num_read_kb":240,"num_write":41185,"num_write_kb":2629350,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":51,"ondisk_log_size":51,"up":51,"acting":51,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":51,"num_osds":4,"num_per_pool_osds":4,"num_per_pool_omap_osds":4,"kb":2936012800,"kb_used":12222380,"kb_used_data":7199324,"kb_used_omap":74,"kb_used_meta":5022837,"kb_avail":2923790420,"statfs":{"total":3006477107200,"available":2993961390080,"internally_reserved":0,"allocated":7372107776,"data_stored":7369988780,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":75912,"internal_metadata":5143385976},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[8,3],"upper_bound":4},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2690252800,"num_objects":41050,"num_object_clones":0,"num_object_copies":123150,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41050,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":41050,"num_write_kb":2627200,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.832594"},"pg_stats":[{"pgid":"5.f","version":"76'2553","reported_seq":2567,"reported_epoch":76,"state":"active+clean","last_fresh":"2026-02-24T07:52:00.741584+0000","last_change":"2026-02-24T07:51:53.900797+0000","last_active":"2026-02-24T07:52:00.741584+0000","last_peered":"2026-02-24T07:52:00.741584+0000","last_clean":"2026-02-24T07:52:00.741584+0000","last_became_active":"2026-02-24T07:51:53.900431+0000","last_became_peered":"2026-02-24T07:51:53.900431+0000","last_unstale":"2026-02-24T07:52:00.741584+0000","last_undegraded":"2026-02-24T07:52:00.741584+0000","last_fullsized":"2026-02-24T07:52:00.741584+0000","mapping_epoch":74,"log_start":"76'2550","ondisk_log_start":"76'2550","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_clean_scrub_stamp":"2026-02-24T07:51:52.890170+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":2550,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:52:57.614871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":167313408,"num_objects":2553,"num_object_clones":0,"num_object_copies":7659,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2553,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2553,"num_write_kb":163392,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.e","version":"76'2550","reported_seq":2564,"reported_epoch":76,"state":"active+clean","last_fresh":"2026-02-24T07:52:00.751910+0000","last_change":"2026-02-24T07:51:53.899686+0000","last_active":"2026-02-24T07:52:00.751910+0000","last_peered":"2026-02-24T07:52:00.751910+0000","last_clean":"2026-02-24T07:52:00.751910+0000","last_became_active":"2026-02-24T07:51:53.899294+0000","last_became_peered":"2026-02-24T07:51:53.899294+0000","last_unstale":"2026-02-24T07:52:00.751910+0000","last_undegraded":"2026-02-24T07:52:00.751910+0000","last_fullsized":"2026-02-24T07:52:00.751910+0000","mapping_epoch":74,"log_start":"76'2547","ondisk_log_start":"76'2547","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_clean_scrub_stamp":"2026-02-24T07:51:52.890170+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":2547,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:52:53.084042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":167116800,"num_objects":2550,"num_object_clones":0,"num_object_copies":7650,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2550,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2550,"num_write_kb":163200,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"76'2541","reported_seq":2555,"reported_epoch":76,"state":"active+clean","last_fresh":"2026-02-24T07:52:00.764120+0000","last_change":"2026-02-24T07:51:53.900129+0000","last_active":"2026-02-24T07:52:00.764120+0000","last_peered":"2026-02-24T07:52:00.764120+0000","last_clean":"2026-02-24T07:52:00.764120+0000","last_became_active":"2026-02-24T07:51:53.899772+0000","last_became_peered":"2026-02-24T07:51:53.899772+0000","last_unstale":"2026-02-24T07:52:00.764120+0000","last_undegraded":"2026-02-24T07:52:00.764120+0000","last_fullsized":"2026-02-24T07:52:00.764120+0000","mapping_epoch":74,"log_start":"76'2538","ondisk_log_start":"76'2538","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_clean_scrub_stamp":"2026-02-24T07:51:52.890170+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":2538,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:53:19.516698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":166526976,"num_objects":2541,"num_object_clones":0,"num_object_copies":7623,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2541,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2541,"num_write_kb":162624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,3],"acting":[0,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.c","version":"76'2645","reported_seq":2659,"reported_epoch":76,"state":"active+clean","last_fresh":"2026-02-24T07:52:00.843079+0000","last_change":"2026-02-24T07:51:53.898484+0000","last_active":"2026-02-24T07:52:00.843079+0000","last_peered":"2026-02-24T07:52:00.843079+0000","last_clean":"2026-02-24T07:52:00.843079+0000","last_became_active":"2026-02-24T07:51:53.898285+0000","last_became_peered":"2026-02-24T07:51:53.898285+0000","last_unstale":"2026-02-24T07:52:00.843079+0000","last_undegraded":"2026-02-24T07:52:00.843079+0000","last_fullsized":"2026-02-24T07:52:00.843079+0000","mapping_epoch":74,"log_start":"76'2642","ondisk_log_start":"76'2642","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_clean_scrub_stamp":"2026-02-24T07:51:52.890170+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":2642,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:53:11.947668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":173342720,"num_objects":2645,"num_object_clones":0,"num_object_copies":7935,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2645,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2645,"num_write_kb":169280,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.b","version":"76'2580","reported_seq":2594,"reported_epoch":76,"state":"active+clean","last_fresh":"2026-02-24T07:52:00.760431+0000","last_change":"2026-02-24T07:51:53.900298+0000","last_active":"2026-02-24T07:52:00.760431+0000","last_peered":"2026-02-24T07:52:00.760431+0000","last_clean":"2026-02-24T07:52:00.760431+0000","last_became_active":"2026-02-24T07:51:53.899901+0000","last_became_peered":"2026-02-24T07:51:53.899901+0000","last_unstale":"2026-02-24T07:52:00.760431+0000","last_undegraded":"2026-02-24T07:52:00.760431+0000","last_fullsized":"2026-02-24T07:52:00.760431+0000","mapping_epoch":74,"log_start":"76'2577","ondisk_log_start":"76'2577","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_clean_scrub_stamp":"2026-02-24T07:51:52.890170+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":2577,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:53:12.321784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":169082880,"num_objects":2580,"num_object_clones":0,"num_object_copies":7740,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2580,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2580,"num_write_kb":165120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"76'2522","reported_seq":2536,"reported_epoch":76,"state":"active+clean","last_fresh":"2026-02-24T07:52:00.760373+0000","last_change":"2026-02-24T07:51:53.899828+0000","last_active":"2026-02-24T07:52:00.760373+0000","last_peered":"2026-02-24T07:52:00.760373+0000","last_clean":"2026-02-24T07:52:00.760373+0000","last_became_active":"2026-02-24T07:51:53.899549+0000","last_became_peered":"2026-02-24T07:51:53.899549+0000","last_unstale":"2026-02-24T07:52:00.760373+0000","last_undegraded":"2026-02-24T07:52:00.760373+0000","last_fullsized":"2026-02-24T07:52:00.760373+0000","mapping_epoch":74,"log_start":"76'2519","ondisk_log_start":"76'2519","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_clean_scrub_stamp":"2026-02-24T07:51:52.890170+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":2519,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:53:07.446496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":165281792,"num_objects":2522,"num_object_clones":0,"num_object_copies":7566,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2522,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2522,"num_write_kb":161408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,1],"acting":[0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.9","version":"76'2573","reported_seq":2587,"reported_epoch":76,"state":"active+clean","last_fresh":"2026-02-24T07:52:00.755275+0000","last_change":"2026-02-24T07:51:53.901985+0000","last_active":"2026-02-24T07:52:00.755275+0000","last_peered":"2026-02-24T07:52:00.755275+0000","last_clean":"2026-02-24T07:52:00.755275+0000","last_became_active":"2026-02-24T07:51:53.901610+0000","last_became_peered":"2026-02-24T07:51:53.901610+0000","last_unstale":"2026-02-24T07:52:00.755275+0000","last_undegraded":"2026-02-24T07:52:00.755275+0000","last_fullsized":"2026-02-24T07:52:00.755275+0000","mapping_epoch":74,"log_start":"76'2570","ondisk_log_start":"76'2570","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_clean_scrub_stamp":"2026-02-24T07:51:52.890170+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":2570,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:53:13.073570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":168624128,"num_objects":2573,"num_object_clones":0,"num_object_copies":7719,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2573,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2573,"num_write_kb":164672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.8","version":"76'2584","reported_seq":2598,"reported_epoch":76,"state":"active+clean","last_fresh":"2026-02-24T07:52:00.760329+0000","last_change":"2026-02-24T07:51:53.900477+0000","last_active":"2026-02-24T07:52:00.760329+0000","last_peered":"2026-02-24T07:52:00.760329+0000","last_clean":"2026-02-24T07:52:00.760329+0000","last_became_active":"2026-02-24T07:51:53.900207+0000","last_became_peered":"2026-02-24T07:51:53.900207+0000","last_unstale":"2026-02-24T07:52:00.760329+0000","last_undegraded":"2026-02-24T07:52:00.760329+0000","last_fullsized":"2026-02-24T07:52:00.760329+0000","mapping_epoch":74,"log_start":"76'2581","ondisk_log_start":"76'2581","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_clean_scrub_stamp":"2026-02-24T07:51:52.890170+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":2581,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:52:57.062307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":169345024,"num_objects":2584,"num_object_clones":0,"num_object_copies":7752,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2584,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2584,"num_write_kb":165376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.7","version":"76'2517","reported_seq":2531,"reported_epoch":76,"state":"active+clean","last_fresh":"2026-02-24T07:52:00.753272+0000","last_change":"2026-02-24T07:51:53.899854+0000","last_active":"2026-02-24T07:52:00.753272+0000","last_peered":"2026-02-24T07:52:00.753272+0000","last_clean":"2026-02-24T07:52:00.753272+0000","last_became_active":"2026-02-24T07:51:53.899472+0000","last_became_peered":"2026-02-24T07:51:53.899472+0000","last_unstale":"2026-02-24T07:52:00.753272+0000","last_undegraded":"2026-02-24T07:52:00.753272+0000","last_fullsized":"2026-02-24T07:52:00.753272+0000","mapping_epoch":74,"log_start":"76'2514","ondisk_log_start":"76'2514","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_clean_scrub_stamp":"2026-02-24T07:51:52.890170+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":2514,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:52:55.115725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":164954112,"num_objects":2517,"num_object_clones":0,"num_object_copies":7551,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2517,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2517,"num_write_kb":161088,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.6","version":"76'2555","reported_seq":2569,"reported_epoch":76,"state":"active+clean","last_fresh":"2026-02-24T07:52:00.842142+0000","last_change":"2026-02-24T07:51:53.898841+0000","last_active":"2026-02-24T07:52:00.842142+0000","last_peered":"2026-02-24T07:52:00.842142+0000","last_clean":"2026-02-24T07:52:00.842142+0000","last_became_active":"2026-02-24T07:51:53.898308+0000","last_became_peered":"2026-02-24T07:51:53.898308+0000","last_unstale":"2026-02-24T07:52:00.842142+0000","last_undegraded":"2026-02-24T07:52:00.842142+0000","last_fullsized":"2026-02-24T07:52:00.842142+0000","mapping_epoch":74,"log_start":"76'2552","ondisk_log_start":"76'2552","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_clean_scrub_stamp":"2026-02-24T07:51:52.890170+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":2552,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:53:04.955186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":167444480,"num_objects":2555,"num_object_clones":0,"num_object_copies":7665,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2555,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2555,"num_write_kb":163520,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.5","version":"76'2577","reported_seq":2591,"reported_epoch":76,"state":"active+clean","last_fresh":"2026-02-24T07:52:00.758169+0000","last_change":"2026-02-24T07:51:53.900139+0000","last_active":"2026-02-24T07:52:00.758169+0000","last_peered":"2026-02-24T07:52:00.758169+0000","last_clean":"2026-02-24T07:52:00.758169+0000","last_became_active":"2026-02-24T07:51:53.899749+0000","last_became_peered":"2026-02-24T07:51:53.899749+0000","last_unstale":"2026-02-24T07:52:00.758169+0000","last_undegraded":"2026-02-24T07:52:00.758169+0000","last_fullsized":"2026-02-24T07:52:00.758169+0000","mapping_epoch":74,"log_start":"76'2574","ondisk_log_start":"76'2574","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_clean_scrub_stamp":"2026-02-24T07:51:52.890170+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":2574,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:53:09.489880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":168886272,"num_objects":2577,"num_object_clones":0,"num_object_copies":7731,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2577,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2577,"num_write_kb":164928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"76'2612","reported_seq":2626,"reported_epoch":76,"state":"active+clean","last_fresh":"2026-02-24T07:52:00.762543+0000","last_change":"2026-02-24T07:51:53.900361+0000","last_active":"2026-02-24T07:52:00.762543+0000","last_peered":"2026-02-24T07:52:00.762543+0000","last_clean":"2026-02-24T07:52:00.762543+0000","last_became_active":"2026-02-24T07:51:53.900042+0000","last_became_peered":"2026-02-24T07:51:53.900042+0000","last_unstale":"2026-02-24T07:52:00.762543+0000","last_undegraded":"2026-02-24T07:52:00.762543+0000","last_fullsized":"2026-02-24T07:52:00.762543+0000","mapping_epoch":74,"log_start":"76'2609","ondisk_log_start":"76'2609","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_clean_scrub_stamp":"2026-02-24T07:51:52.890170+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":2609,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:53:02.182725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":171180032,"num_objects":2612,"num_object_clones":0,"num_object_copies":7836,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2612,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2612,"num_write_kb":167168,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.2","version":"76'2580","reported_seq":2594,"reported_epoch":76,"state":"active+clean","last_fresh":"2026-02-24T07:52:00.751847+0000","last_change":"2026-02-24T07:51:53.900424+0000","last_active":"2026-02-24T07:52:00.751847+0000","last_peered":"2026-02-24T07:52:00.751847+0000","last_clean":"2026-02-24T07:52:00.751847+0000","last_became_active":"2026-02-24T07:51:53.900096+0000","last_became_peered":"2026-02-24T07:51:53.900096+0000","last_unstale":"2026-02-24T07:52:00.751847+0000","last_undegraded":"2026-02-24T07:52:00.751847+0000","last_fullsized":"2026-02-24T07:52:00.751847+0000","mapping_epoch":74,"log_start":"76'2577","ondisk_log_start":"76'2577","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_clean_scrub_stamp":"2026-02-24T07:51:52.890170+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":2577,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:53:10.802005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":169082880,"num_objects":2580,"num_object_clones":0,"num_object_copies":7740,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2580,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2580,"num_write_kb":165120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"76'2523","reported_seq":2537,"reported_epoch":76,"state":"active+clean","last_fresh":"2026-02-24T07:52:00.748593+0000","last_change":"2026-02-24T07:51:53.901421+0000","last_active":"2026-02-24T07:52:00.748593+0000","last_peered":"2026-02-24T07:52:00.748593+0000","last_clean":"2026-02-24T07:52:00.748593+0000","last_became_active":"2026-02-24T07:51:53.901027+0000","last_became_peered":"2026-02-24T07:51:53.901027+0000","last_unstale":"2026-02-24T07:52:00.748593+0000","last_undegraded":"2026-02-24T07:52:00.748593+0000","last_fullsized":"2026-02-24T07:52:00.748593+0000","mapping_epoch":74,"log_start":"76'2520","ondisk_log_start":"76'2520","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_clean_scrub_stamp":"2026-02-24T07:51:52.890170+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":2520,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:53:04.436560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":165347328,"num_objects":2523,"num_object_clones":0,"num_object_copies":7569,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2523,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2523,"num_write_kb":161472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.0","version":"76'2566","reported_seq":2580,"reported_epoch":76,"state":"active+clean","last_fresh":"2026-02-24T07:52:00.762408+0000","last_change":"2026-02-24T07:51:53.898740+0000","last_active":"2026-02-24T07:52:00.762408+0000","last_peered":"2026-02-24T07:52:00.762408+0000","last_clean":"2026-02-24T07:52:00.762408+0000","last_became_active":"2026-02-24T07:51:53.898551+0000","last_became_peered":"2026-02-24T07:51:53.898551+0000","last_unstale":"2026-02-24T07:52:00.762408+0000","last_undegraded":"2026-02-24T07:52:00.762408+0000","last_fullsized":"2026-02-24T07:52:00.762408+0000","mapping_epoch":74,"log_start":"76'2563","ondisk_log_start":"76'2563","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_clean_scrub_stamp":"2026-02-24T07:51:52.890170+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":2563,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:53:07.664864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":168165376,"num_objects":2566,"num_object_clones":0,"num_object_copies":7698,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2566,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2566,"num_write_kb":164224,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.4","version":"76'2572","reported_seq":2586,"reported_epoch":76,"state":"active+clean","last_fresh":"2026-02-24T07:52:00.739660+0000","last_change":"2026-02-24T07:51:53.900179+0000","last_active":"2026-02-24T07:52:00.739660+0000","last_peered":"2026-02-24T07:52:00.739660+0000","last_clean":"2026-02-24T07:52:00.739660+0000","last_became_active":"2026-02-24T07:51:53.899675+0000","last_became_peered":"2026-02-24T07:51:53.899675+0000","last_unstale":"2026-02-24T07:52:00.739660+0000","last_undegraded":"2026-02-24T07:52:00.739660+0000","last_fullsized":"2026-02-24T07:52:00.739660+0000","mapping_epoch":74,"log_start":"76'2569","ondisk_log_start":"76'2569","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_clean_scrub_stamp":"2026-02-24T07:51:52.890170+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":2569,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:53:06.339989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":168558592,"num_objects":2572,"num_object_clones":0,"num_object_copies":7716,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2572,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2572,"num_write_kb":164608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"29'99","reported_seq":332,"reported_epoch":76,"state":"active+clean","last_fresh":"2026-02-24T07:51:54.993390+0000","last_change":"2026-02-24T07:51:54.101629+0000","last_active":"2026-02-24T07:51:54.993390+0000","last_peered":"2026-02-24T07:51:54.993390+0000","last_clean":"2026-02-24T07:51:54.993390+0000","last_became_active":"2026-02-24T07:42:54.186719+0000","last_became_peered":"2026-02-24T07:42:54.186719+0000","last_unstale":"2026-02-24T07:51:54.993390+0000","last_undegraded":"2026-02-24T07:51:54.993390+0000","last_fullsized":"2026-02-24T07:51:54.993390+0000","mapping_epoch":27,"log_start":"29'96","ondisk_log_start":"29'96","created":20,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'99","last_scrub_stamp":"2026-02-24T07:51:54.101595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:42:43.640776+0000","last_clean_scrub_stamp":"2026-02-24T07:51:54.101595+0000","objects_scrubbed":2,"log_size":3,"log_dups_size":96,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:52:54.995630+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":240,"num_write":135,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":16,"stat_sum":{"num_bytes":2690252800,"num_objects":41050,"num_object_clones":0,"num_object_copies":123150,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41050,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":41050,"num_write_kb":2627200,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8079474688,"data_stored":8079474688,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":48,"ondisk_log_size":48,"up":48,"acting":48,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":240,"num_write":135,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1682,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":3,"up_from":25,"seq":107374182512,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":3198844,"kb_used_data":2024936,"kb_used_omap":17,"kb_used_meta":1173870,"kb_avail":730804356,"statfs":{"total":751619276800,"available":748343660544,"internally_reserved":0,"allocated":2073534464,"data_stored":2073003827,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":17663,"internal_metadata":1202043649},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[4],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":19,"seq":81604378738,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":3014332,"kb_used_data":1577640,"kb_used_omap":20,"kb_used_meta":1436651,"kb_avail":730988868,"statfs":{"total":751619276800,"available":748532600832,"internally_reserved":0,"allocated":1615503360,"data_stored":1614972723,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":21290,"internal_metadata":1471130838},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542259,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":3162040,"kb_used_data":1520228,"kb_used_omap":18,"kb_used_meta":1641773,"kb_avail":730841160,"statfs":{"total":751619276800,"available":748381347840,"internally_reserved":0,"allocated":1556713472,"data_stored":1556186387,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":18821,"internal_metadata":1681176187},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705782,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":2847164,"kb_used_data":2076520,"kb_used_omap":17,"kb_used_meta":770542,"kb_avail":731156036,"statfs":{"total":751619276800,"available":748703780864,"internally_reserved":0,"allocated":2126356480,"data_stored":2125825843,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":18138,"internal_metadata":789035302},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,3],"upper_bound":4},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1682,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":2184839168,"data_stored":2184839168,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":1703346176,"data_stored":1703346176,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":1845035008,"data_stored":1845035008,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":2346254336,"data_stored":2346254336,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-24T07:52:07.257 INFO:tasks.cephadm.ceph_manager.ceph:increase pool size by 10 2026-02-24T07:52:07.257 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph osd pool set .mgr pg_num 311 2026-02-24T07:52:07.356 INFO:journalctl@ceph.prometheus.a.trial091.stdout:Feb 24 07:52:06 trial091 ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-prometheus-a[66007]: time=2026-02-24T07:52:06.973Z level=ERROR source=scrape.go:1630 msg="Failed to determine correct type of scrape target." component="scrape manager" scrape_pool=ceph target=http://trial035:9283/metrics content_type="text/html;charset=utf-8" fallback_media_type="" err="received unsupported Content-Type \"text/html;charset=utf-8\" and no fallback_scrape_protocol specified for target" 2026-02-24T07:52:07.404 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:52:07.539 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:52:07.537+0000 7fc9563a6640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:52:07.539 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:52:07.537+0000 7fc9563a6640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:52:07.679 INFO:teuthology.orchestra.run.trial035.stderr:Error EPERM: crimson_allow_pg_split is false; pg_num increase denied for crimson pool 2026-02-24T07:52:07.721 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-24T07:52:07.954 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:52:07 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/1243549606' entity='client.admin' cmd={"prefix": "osd pool set", "pool": ".mgr", "var": "pg_num", "val": "311"} : dispatch 2026-02-24T07:52:07.992 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 13 16 89381 89365 429.606 398.125 0.00186234 0.00232637 2026-02-24T07:52:08.239 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2026-02-24T07:52:08.381 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:52:08.506 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:52:08.504+0000 7f29bd76a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:52:08.506 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:52:08.504+0000 7f29bd76a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:52:08.522 INFO:teuthology.orchestra.run.trial035.stdout:osd.0: {} 2026-02-24T07:52:08.522 INFO:teuthology.orchestra.run.trial035.stderr:osd.0: osd_enable_op_tracker = '' 2026-02-24T07:52:08.535 INFO:teuthology.orchestra.run.trial035.stdout:osd.1: {} 2026-02-24T07:52:08.535 INFO:teuthology.orchestra.run.trial035.stderr:osd.1: osd_enable_op_tracker = '' 2026-02-24T07:52:08.547 INFO:teuthology.orchestra.run.trial035.stdout:osd.2: {} 2026-02-24T07:52:08.548 INFO:teuthology.orchestra.run.trial035.stderr:osd.2: osd_enable_op_tracker = '' 2026-02-24T07:52:08.559 INFO:teuthology.orchestra.run.trial035.stdout:osd.3: {} 2026-02-24T07:52:08.560 INFO:teuthology.orchestra.run.trial035.stderr:osd.3: osd_enable_op_tracker = '' 2026-02-24T07:52:08.992 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 14 16 95992 95976 428.431 413.188 0.00133817 0.00233275 2026-02-24T07:52:09.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:52:08 trial035 ceph-mon[39284]: from='client.15166 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2026-02-24T07:52:09.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:52:08 trial035 ceph-mon[39284]: pgmap v324: 17 pgs: 17 active+clean; 4.6 GiB data, 18 GiB used, 2.7 TiB / 2.7 TiB avail; 392 MiB/s wr, 6.27k op/s 2026-02-24T07:52:09.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:52:08 trial035 ceph-mon[39284]: from='mgr.14284 10.20.193.35:0/3681373222' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2026-02-24T07:52:09.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:52:08 trial035 ceph-mon[39284]: from='mgr.14284 10.20.193.35:0/3681373222' entity='mgr.x' 2026-02-24T07:52:09.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:52:08 trial035 ceph-mon[39284]: from='mgr.14284 10.20.193.35:0/3681373222' entity='mgr.x' 2026-02-24T07:52:09.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:52:08 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/1455746087' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2026-02-24T07:52:09.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:52:08 trial035 ceph-mon[39284]: from='mgr.14284 10.20.193.35:0/3681373222' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-24T07:52:09.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:52:08 trial035 ceph-mon[39284]: from='mgr.14284 10.20.193.35:0/3681373222' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2026-02-24T07:52:09.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:52:08 trial035 ceph-mon[39284]: from='mgr.14284 10.20.193.35:0/3681373222' entity='mgr.x' 2026-02-24T07:52:09.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:52:08 trial035 ceph-mon[39284]: from='mgr.14284 10.20.193.35:0/3681373222' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2026-02-24T07:52:09.992 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 15 16 102615 102599 427.463 413.938 0.00221315 0.00233811 2026-02-24T07:52:10.605 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2026-02-24T07:52:10.743 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:52:10.869 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:52:10.867+0000 7fc7fc4ae640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:52:10.869 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:52:10.867+0000 7fc7fc4ae640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:52:10.884 INFO:teuthology.orchestra.run.trial035.stdout:osd.0: {} 2026-02-24T07:52:10.884 INFO:teuthology.orchestra.run.trial035.stderr:osd.0: osd_enable_op_tracker = '' 2026-02-24T07:52:10.895 INFO:teuthology.orchestra.run.trial035.stdout:osd.1: {} 2026-02-24T07:52:10.896 INFO:teuthology.orchestra.run.trial035.stderr:osd.1: osd_enable_op_tracker = '' 2026-02-24T07:52:10.909 INFO:teuthology.orchestra.run.trial035.stdout:osd.2: {} 2026-02-24T07:52:10.909 INFO:teuthology.orchestra.run.trial035.stderr:osd.2: osd_enable_op_tracker = '' 2026-02-24T07:52:10.922 INFO:teuthology.orchestra.run.trial035.stdout:osd.3: {} 2026-02-24T07:52:10.922 INFO:teuthology.orchestra.run.trial035.stderr:osd.3: osd_enable_op_tracker = '' 2026-02-24T07:52:10.992 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 16 16 109188 109172 426.42 410.812 0.00221743 0.00234384 2026-02-24T07:52:11.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:52:10 trial035 ceph-mon[39284]: pgmap v325: 17 pgs: 17 active+clean; 4.6 GiB data, 18 GiB used, 2.7 TiB / 2.7 TiB avail; 366 MiB/s wr, 5.85k op/s 2026-02-24T07:52:11.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:52:10 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/1070657107' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2026-02-24T07:52:11.992 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 17 16 115731 115715 425.39 408.938 0.00197661 0.00234953 2026-02-24T07:52:12.723 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2026-02-24T07:52:12.723 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2026-02-24T07:52:12.723 INFO:tasks.thrashosds.thrasher:Growing pool .mgr 2026-02-24T07:52:12.724 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph pg dump --format=json 2026-02-24T07:52:12.867 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:52:12.965 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2026-02-24T07:52:12.992 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 18 16 122226 122210 424.307 405.938 0.0027783 0.00235185 2026-02-24T07:52:13.030 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:52:12 trial035 ceph-mon[39284]: pgmap v326: 17 pgs: 17 active+clean; 6.6 GiB data, 23 GiB used, 2.7 TiB / 2.7 TiB avail; 535 MiB/s wr, 8.55k op/s 2026-02-24T07:52:13.178 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:52:13.406 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:52:13.357+0000 7fed4800e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:52:13.406 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:52:13.357+0000 7fed4800e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:52:13.485 INFO:teuthology.orchestra.run.trial035.stdout: 2026-02-24T07:52:13.485 INFO:teuthology.orchestra.run.trial035.stderr:dumped all 2026-02-24T07:52:13.993 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 19 16 127140 127124 418.139 307.125 0.00177688 0.0023903 2026-02-24T07:52:14.430 INFO:teuthology.orchestra.run.trial035.stdout:{"pg_ready":true,"pg_map":{"version":326,"stamp":"2026-02-24T07:52:11.732176+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":7060718112,"num_objects":107731,"num_object_clones":0,"num_object_copies":323193,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":107731,"num_whiteouts":0,"num_read":136,"num_read_kb":240,"num_write":107864,"num_write_kb":6896806,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":51,"ondisk_log_size":51,"up":51,"acting":51,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":51,"num_osds":4,"num_per_pool_osds":4,"num_per_pool_omap_osds":4,"kb":2936012800,"kb_used":24627244,"kb_used_data":20101148,"kb_used_omap":74,"kb_used_meta":4525877,"kb_avail":2911385556,"statfs":{"total":3006477107200,"available":2981258809344,"internally_reserved":0,"allocated":20583575552,"data_stored":20581456556,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":75912,"internal_metadata":4634498936},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":6729564160,"num_objects":102685,"num_object_clones":0,"num_object_copies":308055,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102685,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":102685,"num_write_kb":6571840,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001668"},"pg_stats":[{"pgid":"5.f","version":"76'6696","reported_seq":6708,"reported_epoch":76,"state":"active+clean","last_fresh":"2026-02-24T07:52:10.741645+0000","last_change":"2026-02-24T07:51:53.900797+0000","last_active":"2026-02-24T07:52:10.741645+0000","last_peered":"2026-02-24T07:52:10.741645+0000","last_clean":"2026-02-24T07:52:10.741645+0000","last_became_active":"2026-02-24T07:51:53.900431+0000","last_became_peered":"2026-02-24T07:51:53.900431+0000","last_unstale":"2026-02-24T07:52:10.741645+0000","last_undegraded":"2026-02-24T07:52:10.741645+0000","last_fullsized":"2026-02-24T07:52:10.741645+0000","mapping_epoch":74,"log_start":"76'6693","ondisk_log_start":"76'6693","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_clean_scrub_stamp":"2026-02-24T07:51:52.890170+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:52:57.614871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":438829056,"num_objects":6696,"num_object_clones":0,"num_object_copies":20088,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6696,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6696,"num_write_kb":428544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.e","version":"76'6693","reported_seq":6707,"reported_epoch":76,"state":"active+clean","last_fresh":"2026-02-24T07:52:10.753023+0000","last_change":"2026-02-24T07:51:53.899686+0000","last_active":"2026-02-24T07:52:10.753023+0000","last_peered":"2026-02-24T07:52:10.753023+0000","last_clean":"2026-02-24T07:52:10.753023+0000","last_became_active":"2026-02-24T07:51:53.899294+0000","last_became_peered":"2026-02-24T07:51:53.899294+0000","last_unstale":"2026-02-24T07:52:10.753023+0000","last_undegraded":"2026-02-24T07:52:10.753023+0000","last_fullsized":"2026-02-24T07:52:10.753023+0000","mapping_epoch":74,"log_start":"76'6690","ondisk_log_start":"76'6690","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_clean_scrub_stamp":"2026-02-24T07:51:52.890170+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:52:53.084042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":438632448,"num_objects":6693,"num_object_clones":0,"num_object_copies":20079,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6693,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6693,"num_write_kb":428352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"76'6652","reported_seq":6666,"reported_epoch":76,"state":"active+clean","last_fresh":"2026-02-24T07:52:10.751909+0000","last_change":"2026-02-24T07:51:53.900129+0000","last_active":"2026-02-24T07:52:10.751909+0000","last_peered":"2026-02-24T07:52:10.751909+0000","last_clean":"2026-02-24T07:52:10.751909+0000","last_became_active":"2026-02-24T07:51:53.899772+0000","last_became_peered":"2026-02-24T07:51:53.899772+0000","last_unstale":"2026-02-24T07:52:10.751909+0000","last_undegraded":"2026-02-24T07:52:10.751909+0000","last_fullsized":"2026-02-24T07:52:10.751909+0000","mapping_epoch":74,"log_start":"76'6649","ondisk_log_start":"76'6649","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_clean_scrub_stamp":"2026-02-24T07:51:52.890170+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:53:19.516698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":435945472,"num_objects":6652,"num_object_clones":0,"num_object_copies":19956,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6652,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6652,"num_write_kb":425728,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,3],"acting":[0,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.c","version":"76'6825","reported_seq":6839,"reported_epoch":76,"state":"active+clean","last_fresh":"2026-02-24T07:52:10.841076+0000","last_change":"2026-02-24T07:51:53.898484+0000","last_active":"2026-02-24T07:52:10.841076+0000","last_peered":"2026-02-24T07:52:10.841076+0000","last_clean":"2026-02-24T07:52:10.841076+0000","last_became_active":"2026-02-24T07:51:53.898285+0000","last_became_peered":"2026-02-24T07:51:53.898285+0000","last_unstale":"2026-02-24T07:52:10.841076+0000","last_undegraded":"2026-02-24T07:52:10.841076+0000","last_fullsized":"2026-02-24T07:52:10.841076+0000","mapping_epoch":74,"log_start":"76'6822","ondisk_log_start":"76'6822","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_clean_scrub_stamp":"2026-02-24T07:51:52.890170+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:53:11.947668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":447283200,"num_objects":6825,"num_object_clones":0,"num_object_copies":20475,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6825,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6825,"num_write_kb":436800,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.b","version":"76'6798","reported_seq":6811,"reported_epoch":76,"state":"active+clean","last_fresh":"2026-02-24T07:52:10.762976+0000","last_change":"2026-02-24T07:51:53.900298+0000","last_active":"2026-02-24T07:52:10.762976+0000","last_peered":"2026-02-24T07:52:10.762976+0000","last_clean":"2026-02-24T07:52:10.762976+0000","last_became_active":"2026-02-24T07:51:53.899901+0000","last_became_peered":"2026-02-24T07:51:53.899901+0000","last_unstale":"2026-02-24T07:52:10.762976+0000","last_undegraded":"2026-02-24T07:52:10.762976+0000","last_fullsized":"2026-02-24T07:52:10.762976+0000","mapping_epoch":74,"log_start":"76'6795","ondisk_log_start":"76'6795","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_clean_scrub_stamp":"2026-02-24T07:51:52.890170+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:53:12.321784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":445513728,"num_objects":6798,"num_object_clones":0,"num_object_copies":20394,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6798,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6798,"num_write_kb":435072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"76'6709","reported_seq":6723,"reported_epoch":76,"state":"active+clean","last_fresh":"2026-02-24T07:52:10.763280+0000","last_change":"2026-02-24T07:51:53.899828+0000","last_active":"2026-02-24T07:52:10.763280+0000","last_peered":"2026-02-24T07:52:10.763280+0000","last_clean":"2026-02-24T07:52:10.763280+0000","last_became_active":"2026-02-24T07:51:53.899549+0000","last_became_peered":"2026-02-24T07:51:53.899549+0000","last_unstale":"2026-02-24T07:52:10.763280+0000","last_undegraded":"2026-02-24T07:52:10.763280+0000","last_fullsized":"2026-02-24T07:52:10.763280+0000","mapping_epoch":74,"log_start":"76'6706","ondisk_log_start":"76'6706","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_clean_scrub_stamp":"2026-02-24T07:51:52.890170+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:53:07.446496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":439681024,"num_objects":6709,"num_object_clones":0,"num_object_copies":20127,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6709,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6709,"num_write_kb":429376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,1],"acting":[0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.9","version":"76'6785","reported_seq":6799,"reported_epoch":76,"state":"active+clean","last_fresh":"2026-02-24T07:52:10.755047+0000","last_change":"2026-02-24T07:51:53.901985+0000","last_active":"2026-02-24T07:52:10.755047+0000","last_peered":"2026-02-24T07:52:10.755047+0000","last_clean":"2026-02-24T07:52:10.755047+0000","last_became_active":"2026-02-24T07:51:53.901610+0000","last_became_peered":"2026-02-24T07:51:53.901610+0000","last_unstale":"2026-02-24T07:52:10.755047+0000","last_undegraded":"2026-02-24T07:52:10.755047+0000","last_fullsized":"2026-02-24T07:52:10.755047+0000","mapping_epoch":74,"log_start":"76'6782","ondisk_log_start":"76'6782","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_clean_scrub_stamp":"2026-02-24T07:51:52.890170+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:53:13.073570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":444661760,"num_objects":6785,"num_object_clones":0,"num_object_copies":20355,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6785,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6785,"num_write_kb":434240,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.8","version":"76'6707","reported_seq":6721,"reported_epoch":76,"state":"active+clean","last_fresh":"2026-02-24T07:52:10.764459+0000","last_change":"2026-02-24T07:51:53.900477+0000","last_active":"2026-02-24T07:52:10.764459+0000","last_peered":"2026-02-24T07:52:10.764459+0000","last_clean":"2026-02-24T07:52:10.764459+0000","last_became_active":"2026-02-24T07:51:53.900207+0000","last_became_peered":"2026-02-24T07:51:53.900207+0000","last_unstale":"2026-02-24T07:52:10.764459+0000","last_undegraded":"2026-02-24T07:52:10.764459+0000","last_fullsized":"2026-02-24T07:52:10.764459+0000","mapping_epoch":74,"log_start":"76'6704","ondisk_log_start":"76'6704","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_clean_scrub_stamp":"2026-02-24T07:51:52.890170+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:52:57.062307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":439549952,"num_objects":6707,"num_object_clones":0,"num_object_copies":20121,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6707,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6707,"num_write_kb":429248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.7","version":"76'6684","reported_seq":6698,"reported_epoch":76,"state":"active+clean","last_fresh":"2026-02-24T07:52:10.747642+0000","last_change":"2026-02-24T07:51:53.899854+0000","last_active":"2026-02-24T07:52:10.747642+0000","last_peered":"2026-02-24T07:52:10.747642+0000","last_clean":"2026-02-24T07:52:10.747642+0000","last_became_active":"2026-02-24T07:51:53.899472+0000","last_became_peered":"2026-02-24T07:51:53.899472+0000","last_unstale":"2026-02-24T07:52:10.747642+0000","last_undegraded":"2026-02-24T07:52:10.747642+0000","last_fullsized":"2026-02-24T07:52:10.747642+0000","mapping_epoch":74,"log_start":"76'6681","ondisk_log_start":"76'6681","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_clean_scrub_stamp":"2026-02-24T07:51:52.890170+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:52:55.115725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":438042624,"num_objects":6684,"num_object_clones":0,"num_object_copies":20052,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6684,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6684,"num_write_kb":427776,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.6","version":"76'6607","reported_seq":6621,"reported_epoch":76,"state":"active+clean","last_fresh":"2026-02-24T07:52:10.844559+0000","last_change":"2026-02-24T07:51:53.898841+0000","last_active":"2026-02-24T07:52:10.844559+0000","last_peered":"2026-02-24T07:52:10.844559+0000","last_clean":"2026-02-24T07:52:10.844559+0000","last_became_active":"2026-02-24T07:51:53.898308+0000","last_became_peered":"2026-02-24T07:51:53.898308+0000","last_unstale":"2026-02-24T07:52:10.844559+0000","last_undegraded":"2026-02-24T07:52:10.844559+0000","last_fullsized":"2026-02-24T07:52:10.844559+0000","mapping_epoch":74,"log_start":"76'6604","ondisk_log_start":"76'6604","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_clean_scrub_stamp":"2026-02-24T07:51:52.890170+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:53:04.955186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":432996352,"num_objects":6607,"num_object_clones":0,"num_object_copies":19821,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6607,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6607,"num_write_kb":422848,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.5","version":"76'6754","reported_seq":6766,"reported_epoch":76,"state":"active+clean","last_fresh":"2026-02-24T07:52:10.756370+0000","last_change":"2026-02-24T07:51:53.900139+0000","last_active":"2026-02-24T07:52:10.756370+0000","last_peered":"2026-02-24T07:52:10.756370+0000","last_clean":"2026-02-24T07:52:10.756370+0000","last_became_active":"2026-02-24T07:51:53.899749+0000","last_became_peered":"2026-02-24T07:51:53.899749+0000","last_unstale":"2026-02-24T07:52:10.756370+0000","last_undegraded":"2026-02-24T07:52:10.756370+0000","last_fullsized":"2026-02-24T07:52:10.756370+0000","mapping_epoch":74,"log_start":"76'6751","ondisk_log_start":"76'6751","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_clean_scrub_stamp":"2026-02-24T07:51:52.890170+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:53:09.489880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":442630144,"num_objects":6754,"num_object_clones":0,"num_object_copies":20262,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6754,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6754,"num_write_kb":432256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"76'6747","reported_seq":6761,"reported_epoch":76,"state":"active+clean","last_fresh":"2026-02-24T07:52:10.762661+0000","last_change":"2026-02-24T07:51:53.900361+0000","last_active":"2026-02-24T07:52:10.762661+0000","last_peered":"2026-02-24T07:52:10.762661+0000","last_clean":"2026-02-24T07:52:10.762661+0000","last_became_active":"2026-02-24T07:51:53.900042+0000","last_became_peered":"2026-02-24T07:51:53.900042+0000","last_unstale":"2026-02-24T07:52:10.762661+0000","last_undegraded":"2026-02-24T07:52:10.762661+0000","last_fullsized":"2026-02-24T07:52:10.762661+0000","mapping_epoch":74,"log_start":"76'6744","ondisk_log_start":"76'6744","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_clean_scrub_stamp":"2026-02-24T07:51:52.890170+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:53:02.182725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":442171392,"num_objects":6747,"num_object_clones":0,"num_object_copies":20241,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6747,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6747,"num_write_kb":431808,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.2","version":"76'6815","reported_seq":6829,"reported_epoch":76,"state":"active+clean","last_fresh":"2026-02-24T07:52:10.751315+0000","last_change":"2026-02-24T07:51:53.900424+0000","last_active":"2026-02-24T07:52:10.751315+0000","last_peered":"2026-02-24T07:52:10.751315+0000","last_clean":"2026-02-24T07:52:10.751315+0000","last_became_active":"2026-02-24T07:51:53.900096+0000","last_became_peered":"2026-02-24T07:51:53.900096+0000","last_unstale":"2026-02-24T07:52:10.751315+0000","last_undegraded":"2026-02-24T07:52:10.751315+0000","last_fullsized":"2026-02-24T07:52:10.751315+0000","mapping_epoch":74,"log_start":"76'6812","ondisk_log_start":"76'6812","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_clean_scrub_stamp":"2026-02-24T07:51:52.890170+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:53:10.802005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":446627840,"num_objects":6815,"num_object_clones":0,"num_object_copies":20445,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6815,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6815,"num_write_kb":436160,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"76'6666","reported_seq":6680,"reported_epoch":76,"state":"active+clean","last_fresh":"2026-02-24T07:52:10.753768+0000","last_change":"2026-02-24T07:51:53.901421+0000","last_active":"2026-02-24T07:52:10.753768+0000","last_peered":"2026-02-24T07:52:10.753768+0000","last_clean":"2026-02-24T07:52:10.753768+0000","last_became_active":"2026-02-24T07:51:53.901027+0000","last_became_peered":"2026-02-24T07:51:53.901027+0000","last_unstale":"2026-02-24T07:52:10.753768+0000","last_undegraded":"2026-02-24T07:52:10.753768+0000","last_fullsized":"2026-02-24T07:52:10.753768+0000","mapping_epoch":74,"log_start":"76'6663","ondisk_log_start":"76'6663","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_clean_scrub_stamp":"2026-02-24T07:51:52.890170+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:53:04.436560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":436862976,"num_objects":6666,"num_object_clones":0,"num_object_copies":19998,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6666,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6666,"num_write_kb":426624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.0","version":"76'6910","reported_seq":6924,"reported_epoch":76,"state":"active+clean","last_fresh":"2026-02-24T07:52:10.763471+0000","last_change":"2026-02-24T07:51:53.898740+0000","last_active":"2026-02-24T07:52:10.763471+0000","last_peered":"2026-02-24T07:52:10.763471+0000","last_clean":"2026-02-24T07:52:10.763471+0000","last_became_active":"2026-02-24T07:51:53.898551+0000","last_became_peered":"2026-02-24T07:51:53.898551+0000","last_unstale":"2026-02-24T07:52:10.763471+0000","last_undegraded":"2026-02-24T07:52:10.763471+0000","last_fullsized":"2026-02-24T07:52:10.763471+0000","mapping_epoch":74,"log_start":"76'6907","ondisk_log_start":"76'6907","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_clean_scrub_stamp":"2026-02-24T07:51:52.890170+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:53:07.664864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":452853760,"num_objects":6910,"num_object_clones":0,"num_object_copies":20730,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6910,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6910,"num_write_kb":442240,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.4","version":"76'6681","reported_seq":6695,"reported_epoch":76,"state":"active+clean","last_fresh":"2026-02-24T07:52:10.741880+0000","last_change":"2026-02-24T07:51:53.900179+0000","last_active":"2026-02-24T07:52:10.741880+0000","last_peered":"2026-02-24T07:52:10.741880+0000","last_clean":"2026-02-24T07:52:10.741880+0000","last_became_active":"2026-02-24T07:51:53.899675+0000","last_became_peered":"2026-02-24T07:51:53.899675+0000","last_unstale":"2026-02-24T07:52:10.741880+0000","last_undegraded":"2026-02-24T07:52:10.741880+0000","last_fullsized":"2026-02-24T07:52:10.741880+0000","mapping_epoch":74,"log_start":"76'6678","ondisk_log_start":"76'6678","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_clean_scrub_stamp":"2026-02-24T07:51:52.890170+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:53:06.339989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":437846016,"num_objects":6681,"num_object_clones":0,"num_object_copies":20043,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6681,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6681,"num_write_kb":427584,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"29'99","reported_seq":332,"reported_epoch":76,"state":"active+clean","last_fresh":"2026-02-24T07:51:54.993390+0000","last_change":"2026-02-24T07:51:54.101629+0000","last_active":"2026-02-24T07:51:54.993390+0000","last_peered":"2026-02-24T07:51:54.993390+0000","last_clean":"2026-02-24T07:51:54.993390+0000","last_became_active":"2026-02-24T07:42:54.186719+0000","last_became_peered":"2026-02-24T07:42:54.186719+0000","last_unstale":"2026-02-24T07:51:54.993390+0000","last_undegraded":"2026-02-24T07:51:54.993390+0000","last_fullsized":"2026-02-24T07:51:54.993390+0000","mapping_epoch":27,"log_start":"29'96","ondisk_log_start":"29'96","created":20,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'99","last_scrub_stamp":"2026-02-24T07:51:54.101595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:42:43.640776+0000","last_clean_scrub_stamp":"2026-02-24T07:51:54.101595+0000","objects_scrubbed":2,"log_size":3,"log_dups_size":96,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:52:54.995630+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":240,"num_write":135,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":16,"stat_sum":{"num_bytes":7060127744,"num_objects":107729,"num_object_clones":0,"num_object_copies":323187,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":107729,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":107729,"num_write_kb":6894656,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":21186281472,"data_stored":21186281472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":48,"ondisk_log_size":48,"up":48,"acting":48,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":240,"num_write":135,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1682,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":3,"up_from":25,"seq":107374182514,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":7112956,"kb_used_data":5758504,"kb_used_omap":17,"kb_used_meta":1354350,"kb_avail":726890244,"statfs":{"total":751619276800,"available":744335609856,"internally_reserved":0,"allocated":5896708096,"data_stored":5896177459,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":17663,"internal_metadata":1386855169},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":19,"seq":81604378740,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":6008508,"kb_used_data":4535656,"kb_used_omap":20,"kb_used_meta":1472811,"kb_avail":727994692,"statfs":{"total":751619276800,"available":745466564608,"internally_reserved":0,"allocated":4644511744,"data_stored":4643981107,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":21290,"internal_metadata":1508158678},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542261,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":4896952,"kb_used_data":4203364,"kb_used_omap":18,"kb_used_meta":693549,"kb_avail":729106248,"statfs":{"total":751619276800,"available":746604797952,"internally_reserved":0,"allocated":4304244736,"data_stored":4303717651,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":18821,"internal_metadata":710194811},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705784,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":6608828,"kb_used_data":5603624,"kb_used_omap":17,"kb_used_meta":1005166,"kb_avail":727394372,"statfs":{"total":751619276800,"available":744851836928,"internally_reserved":0,"allocated":5738110976,"data_stored":5737580339,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":18138,"internal_metadata":1029290278},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1682,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":5744361472,"data_stored":5744361472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4432265216,"data_stored":4432265216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4852809728,"data_stored":4852809728,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":6156845056,"data_stored":6156845056,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-24T07:52:14.431 INFO:tasks.cephadm.ceph_manager.ceph:increase pool size by 10 2026-02-24T07:52:14.431 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph osd pool set .mgr pg_num 321 2026-02-24T07:52:14.669 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:52:14.576+0000 7fc862a20640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:52:14.669 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:52:14.576+0000 7fc862a20640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:52:14.669 INFO:teuthology.orchestra.run.trial035.stderr:osd.0: osd_enable_op_tracker = '' 2026-02-24T07:52:14.669 INFO:teuthology.orchestra.run.trial035.stderr:osd.1: osd_enable_op_tracker = '' 2026-02-24T07:52:14.669 INFO:teuthology.orchestra.run.trial035.stderr:osd.2: osd_enable_op_tracker = '' 2026-02-24T07:52:14.670 INFO:teuthology.orchestra.run.trial035.stdout:osd.0: {} 2026-02-24T07:52:14.670 INFO:teuthology.orchestra.run.trial035.stdout:osd.1: {} 2026-02-24T07:52:14.670 INFO:teuthology.orchestra.run.trial035.stdout:osd.2: {} 2026-02-24T07:52:14.670 INFO:teuthology.orchestra.run.trial035.stdout:osd.3: {} 2026-02-24T07:52:14.670 INFO:teuthology.orchestra.run.trial035.stderr:osd.3: osd_enable_op_tracker = '' 2026-02-24T07:52:14.993 INFO:tasks.radosbench.radosbench.0.trial035.stdout:2026-02-24T07:52:14.992353+0000 min lat: 0.000706575 max lat: 0.0436993 avg lat: 0.00242602 2026-02-24T07:52:14.993 INFO:tasks.radosbench.radosbench.0.trial035.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2026-02-24T07:52:14.993 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 20 16 131864 131848 411.993 295.25 0.00176449 0.00242602 2026-02-24T07:52:15.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:52:14 trial035 ceph-mon[39284]: from='client.15174 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2026-02-24T07:52:15.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:52:14 trial035 ceph-mon[39284]: pgmap v327: 17 pgs: 17 active+clean; 6.6 GiB data, 23 GiB used, 2.7 TiB / 2.7 TiB avail; 347 MiB/s wr, 5.55k op/s 2026-02-24T07:52:15.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:52:14 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/1003779004' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2026-02-24T07:52:15.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:52:14 trial035 ceph-mon[39284]: from='mgr.14284 10.20.193.35:0/3681373222' entity='mgr.x' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2026-02-24T07:52:15.261 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:52:15.892 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:52:15.827+0000 7f5f1d1cd640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:52:15.892 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:52:15.827+0000 7f5f1d1cd640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:52:15.974 INFO:teuthology.orchestra.run.trial035.stderr:Error EPERM: crimson_allow_pg_split is false; pg_num increase denied for crimson pool 2026-02-24T07:52:15.992 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 21 16 136937 136921 407.471 317.062 0.000923067 0.00245289 2026-02-24T07:52:16.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:52:15 trial035 ceph-mon[39284]: purged_snaps scrub starts 2026-02-24T07:52:16.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:52:15 trial035 ceph-mon[39284]: purged_snaps scrub ok 2026-02-24T07:52:16.660 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-24T07:52:16.993 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 22 15 142214 142199 403.943 329.875 0.00226581 0.00247445 2026-02-24T07:52:17.204 INFO:journalctl@ceph.mgr.x.trial035.stdout:Feb 24 07:52:16 trial035 ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-mgr-x[39621]: ::ffff:10.20.193.91 - - [24/Feb/2026:07:52:16] "GET /metrics HTTP/1.1" 200 4217 "" "Prometheus/3.6.0" 2026-02-24T07:52:17.205 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:52:16 trial035 ceph-mon[39284]: pgmap v328: 17 pgs: 17 active+clean; 6.6 GiB data, 23 GiB used, 2.7 TiB / 2.7 TiB avail; 347 MiB/s wr, 5.55k op/s 2026-02-24T07:52:17.205 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:52:16 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/1806768802' entity='client.admin' cmd={"prefix": "osd pool set", "pool": ".mgr", "var": "pg_num", "val": "321"} : dispatch 2026-02-24T07:52:17.205 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:52:16 trial035 ceph-mon[39284]: purged_snaps scrub starts 2026-02-24T07:52:17.205 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:52:16 trial035 ceph-mon[39284]: purged_snaps scrub ok 2026-02-24T07:52:17.264 INFO:journalctl@ceph.prometheus.a.trial091.stdout:Feb 24 07:52:16 trial091 ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-prometheus-a[66007]: time=2026-02-24T07:52:16.974Z level=ERROR source=scrape.go:1630 msg="Failed to determine correct type of scrape target." component="scrape manager" scrape_pool=ceph target=http://trial035:9283/metrics content_type="text/html;charset=utf-8" fallback_media_type="" err="received unsupported Content-Type \"text/html;charset=utf-8\" and no fallback_scrape_protocol specified for target" 2026-02-24T07:52:17.393 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2026-02-24T07:52:17.533 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:52:17.915 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:52:17.865+0000 7f7ebac4a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:52:17.915 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:52:17.866+0000 7f7ebac4a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:52:17.916 INFO:teuthology.orchestra.run.trial035.stderr:osd.0: osd_enable_op_tracker = '' 2026-02-24T07:52:17.916 INFO:teuthology.orchestra.run.trial035.stderr:osd.1: osd_enable_op_tracker = '' 2026-02-24T07:52:17.916 INFO:teuthology.orchestra.run.trial035.stdout:osd.0: {} 2026-02-24T07:52:17.916 INFO:teuthology.orchestra.run.trial035.stdout:osd.1: {} 2026-02-24T07:52:17.916 INFO:teuthology.orchestra.run.trial035.stdout:osd.2: {} 2026-02-24T07:52:17.917 INFO:teuthology.orchestra.run.trial035.stderr:osd.2: osd_enable_op_tracker = '' 2026-02-24T07:52:17.929 INFO:teuthology.orchestra.run.trial035.stdout:osd.3: {} 2026-02-24T07:52:17.929 INFO:teuthology.orchestra.run.trial035.stderr:osd.3: osd_enable_op_tracker = '' 2026-02-24T07:52:17.954 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:52:17 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/2427299254' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2026-02-24T07:52:17.993 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 23 16 148487 148471 403.423 392 0.00141462 0.00247545 2026-02-24T07:52:18.993 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 24 16 153979 153963 400.914 343.25 0.00243046 0.0024931 2026-02-24T07:52:19.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:52:18 trial035 ceph-mon[39284]: pgmap v329: 17 pgs: 17 active+clean; 8.3 GiB data, 29 GiB used, 2.7 TiB / 2.7 TiB avail; 492 MiB/s wr, 7.88k op/s 2026-02-24T07:52:19.993 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 25 16 160956 160940 402.319 436.062 0.00162219 0.00248435 2026-02-24T07:52:20.449 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2026-02-24T07:52:20.593 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:52:20.726 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:52:20.724+0000 7fb052d8b640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:52:20.726 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:52:20.725+0000 7fb052d8b640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:52:20.741 INFO:teuthology.orchestra.run.trial035.stdout:osd.0: {} 2026-02-24T07:52:20.741 INFO:teuthology.orchestra.run.trial035.stderr:osd.0: osd_enable_op_tracker = '' 2026-02-24T07:52:20.756 INFO:teuthology.orchestra.run.trial035.stdout:osd.1: {} 2026-02-24T07:52:20.756 INFO:teuthology.orchestra.run.trial035.stderr:osd.1: osd_enable_op_tracker = '' 2026-02-24T07:52:20.769 INFO:teuthology.orchestra.run.trial035.stdout:osd.2: {} 2026-02-24T07:52:20.770 INFO:teuthology.orchestra.run.trial035.stderr:osd.2: osd_enable_op_tracker = '' 2026-02-24T07:52:20.782 INFO:teuthology.orchestra.run.trial035.stdout:osd.3: {} 2026-02-24T07:52:20.783 INFO:teuthology.orchestra.run.trial035.stderr:osd.3: osd_enable_op_tracker = '' 2026-02-24T07:52:20.993 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 26 16 167994 167978 403.762 439.875 0.00128683 0.00247547 2026-02-24T07:52:21.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:52:20 trial035 ceph-mon[39284]: pgmap v330: 17 pgs: 17 active+clean; 8.3 GiB data, 29 GiB used, 2.7 TiB / 2.7 TiB avail; 314 MiB/s wr, 5.03k op/s 2026-02-24T07:52:21.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:52:20 trial035 ceph-mon[39284]: purged_snaps scrub starts 2026-02-24T07:52:21.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:52:20 trial035 ceph-mon[39284]: purged_snaps scrub ok 2026-02-24T07:52:21.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:52:20 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/1833881133' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2026-02-24T07:52:21.662 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2026-02-24T07:52:21.662 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2026-02-24T07:52:21.662 INFO:tasks.thrashosds.thrasher:fixing pg num pool .mgr 2026-02-24T07:52:21.663 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph pg dump --format=json 2026-02-24T07:52:21.803 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:52:21.993 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 27 16 174040 174024 402.802 377.875 0.00267612 0.00247885 2026-02-24T07:52:22.254 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:52:22.234+0000 7fb820693640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:52:22.255 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:52:22.235+0000 7fb820693640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:52:22.370 INFO:teuthology.orchestra.run.trial035.stdout: 2026-02-24T07:52:22.371 INFO:teuthology.orchestra.run.trial035.stderr:dumped all 2026-02-24T07:52:22.828 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2026-02-24T07:52:22.829 INFO:teuthology.orchestra.run.trial035.stdout:{"pg_ready":true,"pg_map":{"version":331,"stamp":"2026-02-24T07:52:21.733533+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":10901586464,"num_objects":166338,"num_object_clones":0,"num_object_copies":499014,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":166338,"num_whiteouts":0,"num_read":136,"num_read_kb":240,"num_write":166471,"num_write_kb":10647654,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":52,"ondisk_log_size":52,"up":51,"acting":51,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":51,"num_osds":4,"num_per_pool_osds":4,"num_per_pool_omap_osds":4,"kb":2936012800,"kb_used":35985964,"kb_used_data":31330012,"kb_used_omap":75,"kb_used_meta":4655796,"kb_avail":2900026836,"statfs":{"total":3006477107200,"available":2969627480064,"internally_reserved":0,"allocated":32081932288,"data_stored":32079813292,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":76909,"internal_metadata":4767536019},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[6],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":5970722816,"num_objects":91106,"num_object_clones":0,"num_object_copies":273318,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":91106,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":91106,"num_write_kb":5830784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001593"},"pg_stats":[{"pgid":"5.f","version":"76'10233","reported_seq":10247,"reported_epoch":76,"state":"active+clean","last_fresh":"2026-02-24T07:52:20.740728+0000","last_change":"2026-02-24T07:51:53.900797+0000","last_active":"2026-02-24T07:52:20.740728+0000","last_peered":"2026-02-24T07:52:20.740728+0000","last_clean":"2026-02-24T07:52:20.740728+0000","last_became_active":"2026-02-24T07:51:53.900431+0000","last_became_peered":"2026-02-24T07:51:53.900431+0000","last_unstale":"2026-02-24T07:52:20.740728+0000","last_undegraded":"2026-02-24T07:52:20.740728+0000","last_fullsized":"2026-02-24T07:52:20.740728+0000","mapping_epoch":74,"log_start":"76'10230","ondisk_log_start":"76'10230","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_clean_scrub_stamp":"2026-02-24T07:51:52.890170+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:52:57.614871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":670629888,"num_objects":10233,"num_object_clones":0,"num_object_copies":30699,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10233,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10233,"num_write_kb":654912,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.e","version":"76'10369","reported_seq":10383,"reported_epoch":76,"state":"active+clean","last_fresh":"2026-02-24T07:52:20.749799+0000","last_change":"2026-02-24T07:51:53.899686+0000","last_active":"2026-02-24T07:52:20.749799+0000","last_peered":"2026-02-24T07:52:20.749799+0000","last_clean":"2026-02-24T07:52:20.749799+0000","last_became_active":"2026-02-24T07:51:53.899294+0000","last_became_peered":"2026-02-24T07:51:53.899294+0000","last_unstale":"2026-02-24T07:52:20.749799+0000","last_undegraded":"2026-02-24T07:52:20.749799+0000","last_fullsized":"2026-02-24T07:52:20.749799+0000","mapping_epoch":74,"log_start":"76'10366","ondisk_log_start":"76'10366","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_clean_scrub_stamp":"2026-02-24T07:51:52.890170+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:52:53.084042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":679542784,"num_objects":10369,"num_object_clones":0,"num_object_copies":31107,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10369,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10369,"num_write_kb":663616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"76'10367","reported_seq":10381,"reported_epoch":76,"state":"active+clean","last_fresh":"2026-02-24T07:52:20.761177+0000","last_change":"2026-02-24T07:51:53.900129+0000","last_active":"2026-02-24T07:52:20.761177+0000","last_peered":"2026-02-24T07:52:20.761177+0000","last_clean":"2026-02-24T07:52:20.761177+0000","last_became_active":"2026-02-24T07:51:53.899772+0000","last_became_peered":"2026-02-24T07:51:53.899772+0000","last_unstale":"2026-02-24T07:52:20.761177+0000","last_undegraded":"2026-02-24T07:52:20.761177+0000","last_fullsized":"2026-02-24T07:52:20.761177+0000","mapping_epoch":74,"log_start":"76'10364","ondisk_log_start":"76'10364","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_clean_scrub_stamp":"2026-02-24T07:51:52.890170+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:53:19.516698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":679411712,"num_objects":10367,"num_object_clones":0,"num_object_copies":31101,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10367,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10367,"num_write_kb":663488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,3],"acting":[0,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.c","version":"76'10404","reported_seq":10418,"reported_epoch":76,"state":"active+clean","last_fresh":"2026-02-24T07:52:20.844747+0000","last_change":"2026-02-24T07:51:53.898484+0000","last_active":"2026-02-24T07:52:20.844747+0000","last_peered":"2026-02-24T07:52:20.844747+0000","last_clean":"2026-02-24T07:52:20.844747+0000","last_became_active":"2026-02-24T07:51:53.898285+0000","last_became_peered":"2026-02-24T07:51:53.898285+0000","last_unstale":"2026-02-24T07:52:20.844747+0000","last_undegraded":"2026-02-24T07:52:20.844747+0000","last_fullsized":"2026-02-24T07:52:20.844747+0000","mapping_epoch":74,"log_start":"76'10401","ondisk_log_start":"76'10401","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_clean_scrub_stamp":"2026-02-24T07:51:52.890170+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:53:11.947668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":681836544,"num_objects":10404,"num_object_clones":0,"num_object_copies":31212,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10404,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10404,"num_write_kb":665856,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.b","version":"76'10500","reported_seq":10512,"reported_epoch":76,"state":"active+clean","last_fresh":"2026-02-24T07:52:20.761228+0000","last_change":"2026-02-24T07:51:53.900298+0000","last_active":"2026-02-24T07:52:20.761228+0000","last_peered":"2026-02-24T07:52:20.761228+0000","last_clean":"2026-02-24T07:52:20.761228+0000","last_became_active":"2026-02-24T07:51:53.899901+0000","last_became_peered":"2026-02-24T07:51:53.899901+0000","last_unstale":"2026-02-24T07:52:20.761228+0000","last_undegraded":"2026-02-24T07:52:20.761228+0000","last_fullsized":"2026-02-24T07:52:20.761228+0000","mapping_epoch":74,"log_start":"76'10497","ondisk_log_start":"76'10497","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_clean_scrub_stamp":"2026-02-24T07:51:52.890170+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:53:12.321784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":688128000,"num_objects":10500,"num_object_clones":0,"num_object_copies":31500,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10500,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10500,"num_write_kb":672000,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"76'10349","reported_seq":10362,"reported_epoch":76,"state":"active+clean","last_fresh":"2026-02-24T07:52:20.763203+0000","last_change":"2026-02-24T07:51:53.899828+0000","last_active":"2026-02-24T07:52:20.763203+0000","last_peered":"2026-02-24T07:52:20.763203+0000","last_clean":"2026-02-24T07:52:20.763203+0000","last_became_active":"2026-02-24T07:51:53.899549+0000","last_became_peered":"2026-02-24T07:51:53.899549+0000","last_unstale":"2026-02-24T07:52:20.763203+0000","last_undegraded":"2026-02-24T07:52:20.763203+0000","last_fullsized":"2026-02-24T07:52:20.763203+0000","mapping_epoch":74,"log_start":"76'10346","ondisk_log_start":"76'10346","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_clean_scrub_stamp":"2026-02-24T07:51:52.890170+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:53:07.446496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":678232064,"num_objects":10349,"num_object_clones":0,"num_object_copies":31047,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10349,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10349,"num_write_kb":662336,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,1],"acting":[0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.9","version":"76'10581","reported_seq":10595,"reported_epoch":76,"state":"active+clean","last_fresh":"2026-02-24T07:52:20.755972+0000","last_change":"2026-02-24T07:51:53.901985+0000","last_active":"2026-02-24T07:52:20.755972+0000","last_peered":"2026-02-24T07:52:20.755972+0000","last_clean":"2026-02-24T07:52:20.755972+0000","last_became_active":"2026-02-24T07:51:53.901610+0000","last_became_peered":"2026-02-24T07:51:53.901610+0000","last_unstale":"2026-02-24T07:52:20.755972+0000","last_undegraded":"2026-02-24T07:52:20.755972+0000","last_fullsized":"2026-02-24T07:52:20.755972+0000","mapping_epoch":74,"log_start":"76'10578","ondisk_log_start":"76'10578","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_clean_scrub_stamp":"2026-02-24T07:51:52.890170+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:53:13.073570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":693436416,"num_objects":10581,"num_object_clones":0,"num_object_copies":31743,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10581,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10581,"num_write_kb":677184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.8","version":"76'10320","reported_seq":10334,"reported_epoch":76,"state":"active+clean","last_fresh":"2026-02-24T07:52:20.765214+0000","last_change":"2026-02-24T07:51:53.900477+0000","last_active":"2026-02-24T07:52:20.765214+0000","last_peered":"2026-02-24T07:52:20.765214+0000","last_clean":"2026-02-24T07:52:20.765214+0000","last_became_active":"2026-02-24T07:51:53.900207+0000","last_became_peered":"2026-02-24T07:51:53.900207+0000","last_unstale":"2026-02-24T07:52:20.765214+0000","last_undegraded":"2026-02-24T07:52:20.765214+0000","last_fullsized":"2026-02-24T07:52:20.765214+0000","mapping_epoch":74,"log_start":"76'10317","ondisk_log_start":"76'10317","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_clean_scrub_stamp":"2026-02-24T07:51:52.890170+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:52:57.062307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":676331520,"num_objects":10320,"num_object_clones":0,"num_object_copies":30960,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10320,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10320,"num_write_kb":660480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.7","version":"76'10310","reported_seq":10324,"reported_epoch":76,"state":"active+clean","last_fresh":"2026-02-24T07:52:20.757315+0000","last_change":"2026-02-24T07:51:53.899854+0000","last_active":"2026-02-24T07:52:20.757315+0000","last_peered":"2026-02-24T07:52:20.757315+0000","last_clean":"2026-02-24T07:52:20.757315+0000","last_became_active":"2026-02-24T07:51:53.899472+0000","last_became_peered":"2026-02-24T07:51:53.899472+0000","last_unstale":"2026-02-24T07:52:20.757315+0000","last_undegraded":"2026-02-24T07:52:20.757315+0000","last_fullsized":"2026-02-24T07:52:20.757315+0000","mapping_epoch":74,"log_start":"76'10307","ondisk_log_start":"76'10307","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_clean_scrub_stamp":"2026-02-24T07:51:52.890170+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:52:55.115725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":675676160,"num_objects":10310,"num_object_clones":0,"num_object_copies":30930,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10310,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10310,"num_write_kb":659840,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.6","version":"76'10246","reported_seq":10260,"reported_epoch":76,"state":"active+clean","last_fresh":"2026-02-24T07:52:20.840260+0000","last_change":"2026-02-24T07:51:53.898841+0000","last_active":"2026-02-24T07:52:20.840260+0000","last_peered":"2026-02-24T07:52:20.840260+0000","last_clean":"2026-02-24T07:52:20.840260+0000","last_became_active":"2026-02-24T07:51:53.898308+0000","last_became_peered":"2026-02-24T07:51:53.898308+0000","last_unstale":"2026-02-24T07:52:20.840260+0000","last_undegraded":"2026-02-24T07:52:20.840260+0000","last_fullsized":"2026-02-24T07:52:20.840260+0000","mapping_epoch":74,"log_start":"76'10243","ondisk_log_start":"76'10243","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_clean_scrub_stamp":"2026-02-24T07:51:52.890170+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:53:04.955186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":671481856,"num_objects":10246,"num_object_clones":0,"num_object_copies":30738,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10246,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10246,"num_write_kb":655744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.5","version":"76'10466","reported_seq":10480,"reported_epoch":76,"state":"active+clean","last_fresh":"2026-02-24T07:52:20.766690+0000","last_change":"2026-02-24T07:51:53.900139+0000","last_active":"2026-02-24T07:52:20.766690+0000","last_peered":"2026-02-24T07:52:20.766690+0000","last_clean":"2026-02-24T07:52:20.766690+0000","last_became_active":"2026-02-24T07:51:53.899749+0000","last_became_peered":"2026-02-24T07:51:53.899749+0000","last_unstale":"2026-02-24T07:52:20.766690+0000","last_undegraded":"2026-02-24T07:52:20.766690+0000","last_fullsized":"2026-02-24T07:52:20.766690+0000","mapping_epoch":74,"log_start":"76'10463","ondisk_log_start":"76'10463","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_clean_scrub_stamp":"2026-02-24T07:51:52.890170+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:53:09.489880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":685899776,"num_objects":10466,"num_object_clones":0,"num_object_copies":31398,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10466,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10466,"num_write_kb":669824,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"76'10319","reported_seq":10333,"reported_epoch":76,"state":"active+clean","last_fresh":"2026-02-24T07:52:20.763245+0000","last_change":"2026-02-24T07:51:53.900361+0000","last_active":"2026-02-24T07:52:20.763245+0000","last_peered":"2026-02-24T07:52:20.763245+0000","last_clean":"2026-02-24T07:52:20.763245+0000","last_became_active":"2026-02-24T07:51:53.900042+0000","last_became_peered":"2026-02-24T07:51:53.900042+0000","last_unstale":"2026-02-24T07:52:20.763245+0000","last_undegraded":"2026-02-24T07:52:20.763245+0000","last_fullsized":"2026-02-24T07:52:20.763245+0000","mapping_epoch":74,"log_start":"76'10316","ondisk_log_start":"76'10316","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_clean_scrub_stamp":"2026-02-24T07:51:52.890170+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:53:02.182725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":676265984,"num_objects":10319,"num_object_clones":0,"num_object_copies":30957,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10319,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10319,"num_write_kb":660416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.2","version":"76'10644","reported_seq":10658,"reported_epoch":76,"state":"active+clean","last_fresh":"2026-02-24T07:52:20.757993+0000","last_change":"2026-02-24T07:51:53.900424+0000","last_active":"2026-02-24T07:52:20.757993+0000","last_peered":"2026-02-24T07:52:20.757993+0000","last_clean":"2026-02-24T07:52:20.757993+0000","last_became_active":"2026-02-24T07:51:53.900096+0000","last_became_peered":"2026-02-24T07:51:53.900096+0000","last_unstale":"2026-02-24T07:52:20.757993+0000","last_undegraded":"2026-02-24T07:52:20.757993+0000","last_fullsized":"2026-02-24T07:52:20.757993+0000","mapping_epoch":74,"log_start":"76'10641","ondisk_log_start":"76'10641","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_clean_scrub_stamp":"2026-02-24T07:51:52.890170+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:53:10.802005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":697565184,"num_objects":10644,"num_object_clones":0,"num_object_copies":31932,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10644,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10644,"num_write_kb":681216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"76'10249","reported_seq":10263,"reported_epoch":76,"state":"active+clean","last_fresh":"2026-02-24T07:52:20.752049+0000","last_change":"2026-02-24T07:51:53.901421+0000","last_active":"2026-02-24T07:52:20.752049+0000","last_peered":"2026-02-24T07:52:20.752049+0000","last_clean":"2026-02-24T07:52:20.752049+0000","last_became_active":"2026-02-24T07:51:53.901027+0000","last_became_peered":"2026-02-24T07:51:53.901027+0000","last_unstale":"2026-02-24T07:52:20.752049+0000","last_undegraded":"2026-02-24T07:52:20.752049+0000","last_fullsized":"2026-02-24T07:52:20.752049+0000","mapping_epoch":74,"log_start":"76'10246","ondisk_log_start":"76'10246","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_clean_scrub_stamp":"2026-02-24T07:51:52.890170+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:53:04.436560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":671678464,"num_objects":10249,"num_object_clones":0,"num_object_copies":30747,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10249,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10249,"num_write_kb":655936,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.0","version":"76'10621","reported_seq":10632,"reported_epoch":76,"state":"active+clean","last_fresh":"2026-02-24T07:52:20.757702+0000","last_change":"2026-02-24T07:51:53.898740+0000","last_active":"2026-02-24T07:52:20.757702+0000","last_peered":"2026-02-24T07:52:20.757702+0000","last_clean":"2026-02-24T07:52:20.757702+0000","last_became_active":"2026-02-24T07:51:53.898551+0000","last_became_peered":"2026-02-24T07:51:53.898551+0000","last_unstale":"2026-02-24T07:52:20.757702+0000","last_undegraded":"2026-02-24T07:52:20.757702+0000","last_fullsized":"2026-02-24T07:52:20.757702+0000","mapping_epoch":74,"log_start":"76'10617","ondisk_log_start":"76'10617","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_clean_scrub_stamp":"2026-02-24T07:51:52.890170+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":3000,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:53:07.664864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":696057856,"num_objects":10621,"num_object_clones":0,"num_object_copies":31863,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10621,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10621,"num_write_kb":679744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.4","version":"76'10358","reported_seq":10372,"reported_epoch":76,"state":"active+clean","last_fresh":"2026-02-24T07:52:20.727879+0000","last_change":"2026-02-24T07:51:53.900179+0000","last_active":"2026-02-24T07:52:20.727879+0000","last_peered":"2026-02-24T07:52:20.727879+0000","last_clean":"2026-02-24T07:52:20.727879+0000","last_became_active":"2026-02-24T07:51:53.899675+0000","last_became_peered":"2026-02-24T07:51:53.899675+0000","last_unstale":"2026-02-24T07:52:20.727879+0000","last_undegraded":"2026-02-24T07:52:20.727879+0000","last_fullsized":"2026-02-24T07:52:20.727879+0000","mapping_epoch":74,"log_start":"76'10355","ondisk_log_start":"76'10355","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_clean_scrub_stamp":"2026-02-24T07:51:52.890170+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:53:06.339989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":678821888,"num_objects":10358,"num_object_clones":0,"num_object_copies":31074,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10358,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10358,"num_write_kb":662912,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"29'99","reported_seq":332,"reported_epoch":76,"state":"active+clean","last_fresh":"2026-02-24T07:51:54.993390+0000","last_change":"2026-02-24T07:51:54.101629+0000","last_active":"2026-02-24T07:51:54.993390+0000","last_peered":"2026-02-24T07:51:54.993390+0000","last_clean":"2026-02-24T07:51:54.993390+0000","last_became_active":"2026-02-24T07:42:54.186719+0000","last_became_peered":"2026-02-24T07:42:54.186719+0000","last_unstale":"2026-02-24T07:51:54.993390+0000","last_undegraded":"2026-02-24T07:51:54.993390+0000","last_fullsized":"2026-02-24T07:51:54.993390+0000","mapping_epoch":27,"log_start":"29'96","ondisk_log_start":"29'96","created":20,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'99","last_scrub_stamp":"2026-02-24T07:51:54.101595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:42:43.640776+0000","last_clean_scrub_stamp":"2026-02-24T07:51:54.101595+0000","objects_scrubbed":2,"log_size":3,"log_dups_size":96,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:52:54.995630+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":240,"num_write":135,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":16,"stat_sum":{"num_bytes":10900996096,"num_objects":166336,"num_object_clones":0,"num_object_copies":499008,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":166336,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":166336,"num_write_kb":10645504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32710787072,"data_stored":32710787072,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":49,"ondisk_log_size":49,"up":48,"acting":48,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":240,"num_write":135,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1682,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":3,"up_from":25,"seq":107374182516,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":10602876,"kb_used_data":9031912,"kb_used_omap":17,"kb_used_meta":1570926,"kb_avail":723400324,"statfs":{"total":751619276800,"available":740761931776,"internally_reserved":0,"allocated":9248677888,"data_stored":9248147251,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":17819,"internal_metadata":1608628837},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":19,"seq":81604378742,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":8198716,"kb_used_data":7155496,"kb_used_omap":21,"kb_used_meta":1043178,"kb_avail":725804484,"statfs":{"total":751619276800,"available":743223791616,"internally_reserved":0,"allocated":7327227904,"data_stored":7326697267,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":21939,"internal_metadata":1068214861},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542263,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":7378424,"kb_used_data":6540452,"kb_used_omap":18,"kb_used_meta":837933,"kb_avail":726624776,"statfs":{"total":751619276800,"available":744063770624,"internally_reserved":0,"allocated":6697422848,"data_stored":6696895763,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":18821,"internal_metadata":858044027},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[4],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705786,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":9805948,"kb_used_data":8602152,"kb_used_omap":17,"kb_used_meta":1203758,"kb_avail":724197252,"statfs":{"total":751619276800,"available":741577986048,"internally_reserved":0,"allocated":8808603648,"data_stored":8808073011,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":18330,"internal_metadata":1232648294},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1682,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8857190400,"data_stored":8857190400,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":6839009280,"data_stored":6839009280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":7495221248,"data_stored":7495221248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":9519366144,"data_stored":9519366144,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-24T07:52:22.830 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph osd pool set .mgr pgp_num 321 2026-02-24T07:52:22.954 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:52:22 trial035 ceph-mon[39284]: pgmap v331: 17 pgs: 17 active+clean; 10 GiB data, 34 GiB used, 2.7 TiB / 2.7 TiB avail; 474 MiB/s wr, 7.59k op/s 2026-02-24T07:52:22.993 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 28 16 179424 179408 400.433 336.5 0.00194576 0.00249604 2026-02-24T07:52:23.008 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:52:23.009 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:52:23.127 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:52:23.125+0000 7fb5da7dc640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:52:23.127 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:52:23.125+0000 7fb5da7dc640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:52:23.129 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:52:23.128+0000 7fc1e02c9640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:52:23.130 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:52:23.129+0000 7fc1e02c9640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:52:23.140 INFO:teuthology.orchestra.run.trial035.stdout:osd.0: {} 2026-02-24T07:52:23.141 INFO:teuthology.orchestra.run.trial035.stderr:osd.0: osd_enable_op_tracker = '' 2026-02-24T07:52:23.153 INFO:teuthology.orchestra.run.trial035.stdout:osd.1: {} 2026-02-24T07:52:23.153 INFO:teuthology.orchestra.run.trial035.stderr:osd.1: osd_enable_op_tracker = '' 2026-02-24T07:52:23.166 INFO:teuthology.orchestra.run.trial035.stdout:osd.2: {} 2026-02-24T07:52:23.166 INFO:teuthology.orchestra.run.trial035.stderr:osd.2: osd_enable_op_tracker = '' 2026-02-24T07:52:23.178 INFO:teuthology.orchestra.run.trial035.stdout:osd.3: {} 2026-02-24T07:52:23.179 INFO:teuthology.orchestra.run.trial035.stderr:osd.3: osd_enable_op_tracker = '' 2026-02-24T07:52:23.269 INFO:teuthology.orchestra.run.trial035.stderr:Error EPERM: crimson_allow_pg_split is false; pgp_num increase denied 2026-02-24T07:52:23.308 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-24T07:52:23.993 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 29 16 186052 186036 400.909 414.25 0.00221743 0.00249314 2026-02-24T07:52:24.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:52:23 trial035 ceph-mon[39284]: from='client.15184 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2026-02-24T07:52:24.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:52:23 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/943765456' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2026-02-24T07:52:24.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:52:23 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/3063760072' entity='client.admin' cmd={"prefix": "osd pool set", "pool": ".mgr", "var": "pgp_num", "val": "321"} : dispatch 2026-02-24T07:52:24.993 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 30 16 192794 192778 401.59 421.375 0.00390458 0.00248892 2026-02-24T07:52:25.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:52:24 trial035 ceph-mon[39284]: pgmap v332: 17 pgs: 17 active+clean; 10 GiB data, 34 GiB used, 2.7 TiB / 2.7 TiB avail; 305 MiB/s wr, 4.88k op/s 2026-02-24T07:52:25.223 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2026-02-24T07:52:25.360 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:52:25.475 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:52:25.473+0000 7fe83d455640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:52:25.476 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:52:25.474+0000 7fe83d455640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:52:25.489 INFO:teuthology.orchestra.run.trial035.stdout:osd.0: {} 2026-02-24T07:52:25.489 INFO:teuthology.orchestra.run.trial035.stderr:osd.0: osd_enable_op_tracker = '' 2026-02-24T07:52:25.502 INFO:teuthology.orchestra.run.trial035.stdout:osd.1: {} 2026-02-24T07:52:25.503 INFO:teuthology.orchestra.run.trial035.stderr:osd.1: osd_enable_op_tracker = '' 2026-02-24T07:52:25.515 INFO:teuthology.orchestra.run.trial035.stdout:osd.2: {} 2026-02-24T07:52:25.515 INFO:teuthology.orchestra.run.trial035.stderr:osd.2: osd_enable_op_tracker = '' 2026-02-24T07:52:25.528 INFO:teuthology.orchestra.run.trial035.stdout:osd.3: {} 2026-02-24T07:52:25.528 INFO:teuthology.orchestra.run.trial035.stderr:osd.3: osd_enable_op_tracker = '' 2026-02-24T07:52:25.993 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 31 16 199918 199902 402.997 445.25 0.000974212 0.00248024 2026-02-24T07:52:26.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:52:25 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/2138437108' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2026-02-24T07:52:26.993 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 32 16 206827 206811 403.896 431.812 0.00123937 0.00247473 2026-02-24T07:52:27.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:52:26 trial035 ceph-mon[39284]: pgmap v333: 17 pgs: 17 active+clean; 10 GiB data, 34 GiB used, 2.7 TiB / 2.7 TiB avail; 305 MiB/s wr, 4.88k op/s 2026-02-24T07:52:27.204 INFO:journalctl@ceph.mgr.x.trial035.stdout:Feb 24 07:52:26 trial035 ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-mgr-x[39621]: ::ffff:10.20.193.91 - - [24/Feb/2026:07:52:26] "GET /metrics HTTP/1.1" 200 4217 "" "Prometheus/3.6.0" 2026-02-24T07:52:27.356 INFO:journalctl@ceph.prometheus.a.trial091.stdout:Feb 24 07:52:26 trial091 ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-prometheus-a[66007]: time=2026-02-24T07:52:26.973Z level=ERROR source=scrape.go:1630 msg="Failed to determine correct type of scrape target." component="scrape manager" scrape_pool=ceph target=http://trial035:9283/metrics content_type="text/html;charset=utf-8" fallback_media_type="" err="received unsupported Content-Type \"text/html;charset=utf-8\" and no fallback_scrape_protocol specified for target" 2026-02-24T07:52:27.570 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2026-02-24T07:52:27.712 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:52:27.830 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:52:27.829+0000 7fedff3f3640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:52:27.830 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:52:27.829+0000 7fedff3f3640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:52:27.849 INFO:teuthology.orchestra.run.trial035.stdout:osd.0: {} 2026-02-24T07:52:27.850 INFO:teuthology.orchestra.run.trial035.stderr:osd.0: osd_enable_op_tracker = '' 2026-02-24T07:52:27.862 INFO:teuthology.orchestra.run.trial035.stdout:osd.1: {} 2026-02-24T07:52:27.862 INFO:teuthology.orchestra.run.trial035.stderr:osd.1: osd_enable_op_tracker = '' 2026-02-24T07:52:27.876 INFO:teuthology.orchestra.run.trial035.stdout:osd.2: {} 2026-02-24T07:52:27.877 INFO:teuthology.orchestra.run.trial035.stderr:osd.2: osd_enable_op_tracker = '' 2026-02-24T07:52:27.888 INFO:teuthology.orchestra.run.trial035.stdout:osd.3: {} 2026-02-24T07:52:27.889 INFO:teuthology.orchestra.run.trial035.stderr:osd.3: osd_enable_op_tracker = '' 2026-02-24T07:52:27.993 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 33 15 213893 213878 405.041 441.688 0.00141725 0.00246774 2026-02-24T07:52:28.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:52:27 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/2735012220' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2026-02-24T07:52:28.310 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2026-02-24T07:52:28.310 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2026-02-24T07:52:28.310 INFO:tasks.thrashosds.thrasher:Growing pool .mgr 2026-02-24T07:52:28.310 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph pg dump --format=json 2026-02-24T07:52:28.452 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:52:28.858 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:52:28.847+0000 7f4af9d28640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:52:28.858 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:52:28.847+0000 7f4af9d28640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:52:28.990 INFO:teuthology.orchestra.run.trial035.stdout: 2026-02-24T07:52:28.991 INFO:teuthology.orchestra.run.trial035.stderr:dumped all 2026-02-24T07:52:28.993 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 34 16 219588 219572 403.594 355.875 0.00171784 0.0024766 2026-02-24T07:52:29.158 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:52:28 trial035 ceph-mon[39284]: pgmap v334: 17 pgs: 17 active+clean; 12 GiB data, 40 GiB used, 2.7 TiB / 2.7 TiB avail; 471 MiB/s wr, 7.54k op/s 2026-02-24T07:52:29.179 INFO:teuthology.orchestra.run.trial035.stdout:{"pg_ready":true,"pg_map":{"version":335,"stamp":"2026-02-24T07:52:28.906166+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":12994085408,"num_objects":198267,"num_object_clones":0,"num_object_copies":594801,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":198267,"num_whiteouts":0,"num_read":136,"num_read_kb":240,"num_write":198400,"num_write_kb":12691110,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":51,"ondisk_log_size":51,"up":51,"acting":51,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":51,"num_osds":4,"num_per_pool_osds":4,"num_per_pool_omap_osds":4,"kb":2936012800,"kb_used":41901228,"kb_used_data":37455644,"kb_used_omap":76,"kb_used_meta":4445427,"kb_avail":2894111572,"statfs":{"total":3006477107200,"available":2963570249728,"internally_reserved":0,"allocated":38354579456,"data_stored":38352460460,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":77998,"internal_metadata":4552118098},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[9,3],"upper_bound":4},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":4105830400,"num_objects":62650,"num_object_clones":0,"num_object_copies":187950,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62650,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":62650,"num_write_kb":4009600,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"11.173177"},"pg_stats":[{"pgid":"5.f","version":"76'12207","reported_seq":12221,"reported_epoch":76,"state":"active+clean","last_fresh":"2026-02-24T07:52:25.741623+0000","last_change":"2026-02-24T07:51:53.900797+0000","last_active":"2026-02-24T07:52:25.741623+0000","last_peered":"2026-02-24T07:52:25.741623+0000","last_clean":"2026-02-24T07:52:25.741623+0000","last_became_active":"2026-02-24T07:51:53.900431+0000","last_became_peered":"2026-02-24T07:51:53.900431+0000","last_unstale":"2026-02-24T07:52:25.741623+0000","last_undegraded":"2026-02-24T07:52:25.741623+0000","last_fullsized":"2026-02-24T07:52:25.741623+0000","mapping_epoch":74,"log_start":"76'12204","ondisk_log_start":"76'12204","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_clean_scrub_stamp":"2026-02-24T07:51:52.890170+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:52:57.614871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":799997952,"num_objects":12207,"num_object_clones":0,"num_object_copies":36621,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12207,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12207,"num_write_kb":781248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.e","version":"76'12373","reported_seq":12387,"reported_epoch":76,"state":"active+clean","last_fresh":"2026-02-24T07:52:25.752379+0000","last_change":"2026-02-24T07:51:53.899686+0000","last_active":"2026-02-24T07:52:25.752379+0000","last_peered":"2026-02-24T07:52:25.752379+0000","last_clean":"2026-02-24T07:52:25.752379+0000","last_became_active":"2026-02-24T07:51:53.899294+0000","last_became_peered":"2026-02-24T07:51:53.899294+0000","last_unstale":"2026-02-24T07:52:25.752379+0000","last_undegraded":"2026-02-24T07:52:25.752379+0000","last_fullsized":"2026-02-24T07:52:25.752379+0000","mapping_epoch":74,"log_start":"76'12370","ondisk_log_start":"76'12370","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_clean_scrub_stamp":"2026-02-24T07:51:52.890170+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:52:53.084042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":810876928,"num_objects":12373,"num_object_clones":0,"num_object_copies":37119,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12373,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12373,"num_write_kb":791872,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"76'12375","reported_seq":12389,"reported_epoch":76,"state":"active+clean","last_fresh":"2026-02-24T07:52:25.762975+0000","last_change":"2026-02-24T07:51:53.900129+0000","last_active":"2026-02-24T07:52:25.762975+0000","last_peered":"2026-02-24T07:52:25.762975+0000","last_clean":"2026-02-24T07:52:25.762975+0000","last_became_active":"2026-02-24T07:51:53.899772+0000","last_became_peered":"2026-02-24T07:51:53.899772+0000","last_unstale":"2026-02-24T07:52:25.762975+0000","last_undegraded":"2026-02-24T07:52:25.762975+0000","last_fullsized":"2026-02-24T07:52:25.762975+0000","mapping_epoch":74,"log_start":"76'12372","ondisk_log_start":"76'12372","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_clean_scrub_stamp":"2026-02-24T07:51:52.890170+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:53:19.516698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":811008000,"num_objects":12375,"num_object_clones":0,"num_object_copies":37125,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12375,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12375,"num_write_kb":792000,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,3],"acting":[0,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.c","version":"76'12411","reported_seq":12425,"reported_epoch":76,"state":"active+clean","last_fresh":"2026-02-24T07:52:25.846924+0000","last_change":"2026-02-24T07:51:53.898484+0000","last_active":"2026-02-24T07:52:25.846924+0000","last_peered":"2026-02-24T07:52:25.846924+0000","last_clean":"2026-02-24T07:52:25.846924+0000","last_became_active":"2026-02-24T07:51:53.898285+0000","last_became_peered":"2026-02-24T07:51:53.898285+0000","last_unstale":"2026-02-24T07:52:25.846924+0000","last_undegraded":"2026-02-24T07:52:25.846924+0000","last_fullsized":"2026-02-24T07:52:25.846924+0000","mapping_epoch":74,"log_start":"76'12408","ondisk_log_start":"76'12408","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_clean_scrub_stamp":"2026-02-24T07:51:52.890170+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:53:11.947668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":813367296,"num_objects":12411,"num_object_clones":0,"num_object_copies":37233,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12411,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12411,"num_write_kb":794304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.b","version":"76'12494","reported_seq":12508,"reported_epoch":76,"state":"active+clean","last_fresh":"2026-02-24T07:52:25.765810+0000","last_change":"2026-02-24T07:51:53.900298+0000","last_active":"2026-02-24T07:52:25.765810+0000","last_peered":"2026-02-24T07:52:25.765810+0000","last_clean":"2026-02-24T07:52:25.765810+0000","last_became_active":"2026-02-24T07:51:53.899901+0000","last_became_peered":"2026-02-24T07:51:53.899901+0000","last_unstale":"2026-02-24T07:52:25.765810+0000","last_undegraded":"2026-02-24T07:52:25.765810+0000","last_fullsized":"2026-02-24T07:52:25.765810+0000","mapping_epoch":74,"log_start":"76'12491","ondisk_log_start":"76'12491","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_clean_scrub_stamp":"2026-02-24T07:51:52.890170+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:53:12.321784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":818806784,"num_objects":12494,"num_object_clones":0,"num_object_copies":37482,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12494,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12494,"num_write_kb":799616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"76'12293","reported_seq":12307,"reported_epoch":76,"state":"active+clean","last_fresh":"2026-02-24T07:52:25.762643+0000","last_change":"2026-02-24T07:51:53.899828+0000","last_active":"2026-02-24T07:52:25.762643+0000","last_peered":"2026-02-24T07:52:25.762643+0000","last_clean":"2026-02-24T07:52:25.762643+0000","last_became_active":"2026-02-24T07:51:53.899549+0000","last_became_peered":"2026-02-24T07:51:53.899549+0000","last_unstale":"2026-02-24T07:52:25.762643+0000","last_undegraded":"2026-02-24T07:52:25.762643+0000","last_fullsized":"2026-02-24T07:52:25.762643+0000","mapping_epoch":74,"log_start":"76'12290","ondisk_log_start":"76'12290","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_clean_scrub_stamp":"2026-02-24T07:51:52.890170+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:53:07.446496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":805634048,"num_objects":12293,"num_object_clones":0,"num_object_copies":36879,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12293,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12293,"num_write_kb":786752,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,1],"acting":[0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.9","version":"76'12549","reported_seq":12563,"reported_epoch":76,"state":"active+clean","last_fresh":"2026-02-24T07:52:25.755519+0000","last_change":"2026-02-24T07:51:53.901985+0000","last_active":"2026-02-24T07:52:25.755519+0000","last_peered":"2026-02-24T07:52:25.755519+0000","last_clean":"2026-02-24T07:52:25.755519+0000","last_became_active":"2026-02-24T07:51:53.901610+0000","last_became_peered":"2026-02-24T07:51:53.901610+0000","last_unstale":"2026-02-24T07:52:25.755519+0000","last_undegraded":"2026-02-24T07:52:25.755519+0000","last_fullsized":"2026-02-24T07:52:25.755519+0000","mapping_epoch":74,"log_start":"76'12546","ondisk_log_start":"76'12546","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_clean_scrub_stamp":"2026-02-24T07:51:52.890170+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:53:13.073570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":822411264,"num_objects":12549,"num_object_clones":0,"num_object_copies":37647,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12549,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12549,"num_write_kb":803136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.8","version":"76'12367","reported_seq":12381,"reported_epoch":76,"state":"active+clean","last_fresh":"2026-02-24T07:52:25.765428+0000","last_change":"2026-02-24T07:51:53.900477+0000","last_active":"2026-02-24T07:52:25.765428+0000","last_peered":"2026-02-24T07:52:25.765428+0000","last_clean":"2026-02-24T07:52:25.765428+0000","last_became_active":"2026-02-24T07:51:53.900207+0000","last_became_peered":"2026-02-24T07:51:53.900207+0000","last_unstale":"2026-02-24T07:52:25.765428+0000","last_undegraded":"2026-02-24T07:52:25.765428+0000","last_fullsized":"2026-02-24T07:52:25.765428+0000","mapping_epoch":74,"log_start":"76'12364","ondisk_log_start":"76'12364","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_clean_scrub_stamp":"2026-02-24T07:51:52.890170+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:52:57.062307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":810483712,"num_objects":12367,"num_object_clones":0,"num_object_copies":37101,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12367,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12367,"num_write_kb":791488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.7","version":"76'12313","reported_seq":12327,"reported_epoch":76,"state":"active+clean","last_fresh":"2026-02-24T07:52:25.758523+0000","last_change":"2026-02-24T07:51:53.899854+0000","last_active":"2026-02-24T07:52:25.758523+0000","last_peered":"2026-02-24T07:52:25.758523+0000","last_clean":"2026-02-24T07:52:25.758523+0000","last_became_active":"2026-02-24T07:51:53.899472+0000","last_became_peered":"2026-02-24T07:51:53.899472+0000","last_unstale":"2026-02-24T07:52:25.758523+0000","last_undegraded":"2026-02-24T07:52:25.758523+0000","last_fullsized":"2026-02-24T07:52:25.758523+0000","mapping_epoch":74,"log_start":"76'12310","ondisk_log_start":"76'12310","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_clean_scrub_stamp":"2026-02-24T07:51:52.890170+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:52:55.115725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":806944768,"num_objects":12313,"num_object_clones":0,"num_object_copies":36939,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12313,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12313,"num_write_kb":788032,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.6","version":"76'12204","reported_seq":12218,"reported_epoch":76,"state":"active+clean","last_fresh":"2026-02-24T07:52:25.846398+0000","last_change":"2026-02-24T07:51:53.898841+0000","last_active":"2026-02-24T07:52:25.846398+0000","last_peered":"2026-02-24T07:52:25.846398+0000","last_clean":"2026-02-24T07:52:25.846398+0000","last_became_active":"2026-02-24T07:51:53.898308+0000","last_became_peered":"2026-02-24T07:51:53.898308+0000","last_unstale":"2026-02-24T07:52:25.846398+0000","last_undegraded":"2026-02-24T07:52:25.846398+0000","last_fullsized":"2026-02-24T07:52:25.846398+0000","mapping_epoch":74,"log_start":"76'12201","ondisk_log_start":"76'12201","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_clean_scrub_stamp":"2026-02-24T07:51:52.890170+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:53:04.955186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":799801344,"num_objects":12204,"num_object_clones":0,"num_object_copies":36612,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12204,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12204,"num_write_kb":781056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.5","version":"76'12520","reported_seq":12534,"reported_epoch":76,"state":"active+clean","last_fresh":"2026-02-24T07:52:25.765296+0000","last_change":"2026-02-24T07:51:53.900139+0000","last_active":"2026-02-24T07:52:25.765296+0000","last_peered":"2026-02-24T07:52:25.765296+0000","last_clean":"2026-02-24T07:52:25.765296+0000","last_became_active":"2026-02-24T07:51:53.899749+0000","last_became_peered":"2026-02-24T07:51:53.899749+0000","last_unstale":"2026-02-24T07:52:25.765296+0000","last_undegraded":"2026-02-24T07:52:25.765296+0000","last_fullsized":"2026-02-24T07:52:25.765296+0000","mapping_epoch":74,"log_start":"76'12517","ondisk_log_start":"76'12517","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_clean_scrub_stamp":"2026-02-24T07:51:52.890170+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:53:09.489880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":820510720,"num_objects":12520,"num_object_clones":0,"num_object_copies":37560,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12520,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12520,"num_write_kb":801280,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"76'12342","reported_seq":12356,"reported_epoch":76,"state":"active+clean","last_fresh":"2026-02-24T07:52:25.767859+0000","last_change":"2026-02-24T07:51:53.900361+0000","last_active":"2026-02-24T07:52:25.767859+0000","last_peered":"2026-02-24T07:52:25.767859+0000","last_clean":"2026-02-24T07:52:25.767859+0000","last_became_active":"2026-02-24T07:51:53.900042+0000","last_became_peered":"2026-02-24T07:51:53.900042+0000","last_unstale":"2026-02-24T07:52:25.767859+0000","last_undegraded":"2026-02-24T07:52:25.767859+0000","last_fullsized":"2026-02-24T07:52:25.767859+0000","mapping_epoch":74,"log_start":"76'12339","ondisk_log_start":"76'12339","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_clean_scrub_stamp":"2026-02-24T07:51:52.890170+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:53:02.182725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":808845312,"num_objects":12342,"num_object_clones":0,"num_object_copies":37026,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12342,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12342,"num_write_kb":789888,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.2","version":"76'12635","reported_seq":12649,"reported_epoch":76,"state":"active+clean","last_fresh":"2026-02-24T07:52:25.757088+0000","last_change":"2026-02-24T07:51:53.900424+0000","last_active":"2026-02-24T07:52:25.757088+0000","last_peered":"2026-02-24T07:52:25.757088+0000","last_clean":"2026-02-24T07:52:25.757088+0000","last_became_active":"2026-02-24T07:51:53.900096+0000","last_became_peered":"2026-02-24T07:51:53.900096+0000","last_unstale":"2026-02-24T07:52:25.757088+0000","last_undegraded":"2026-02-24T07:52:25.757088+0000","last_fullsized":"2026-02-24T07:52:25.757088+0000","mapping_epoch":74,"log_start":"76'12632","ondisk_log_start":"76'12632","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_clean_scrub_stamp":"2026-02-24T07:51:52.890170+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:53:10.802005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":828047360,"num_objects":12635,"num_object_clones":0,"num_object_copies":37905,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12635,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12635,"num_write_kb":808640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"76'12191","reported_seq":12205,"reported_epoch":76,"state":"active+clean","last_fresh":"2026-02-24T07:52:25.758022+0000","last_change":"2026-02-24T07:51:53.901421+0000","last_active":"2026-02-24T07:52:25.758022+0000","last_peered":"2026-02-24T07:52:25.758022+0000","last_clean":"2026-02-24T07:52:25.758022+0000","last_became_active":"2026-02-24T07:51:53.901027+0000","last_became_peered":"2026-02-24T07:51:53.901027+0000","last_unstale":"2026-02-24T07:52:25.758022+0000","last_undegraded":"2026-02-24T07:52:25.758022+0000","last_fullsized":"2026-02-24T07:52:25.758022+0000","mapping_epoch":74,"log_start":"76'12188","ondisk_log_start":"76'12188","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_clean_scrub_stamp":"2026-02-24T07:51:52.890170+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:53:04.436560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":798949376,"num_objects":12191,"num_object_clones":0,"num_object_copies":36573,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12191,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12191,"num_write_kb":780224,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.0","version":"76'12664","reported_seq":12678,"reported_epoch":76,"state":"active+clean","last_fresh":"2026-02-24T07:52:25.765348+0000","last_change":"2026-02-24T07:51:53.898740+0000","last_active":"2026-02-24T07:52:25.765348+0000","last_peered":"2026-02-24T07:52:25.765348+0000","last_clean":"2026-02-24T07:52:25.765348+0000","last_became_active":"2026-02-24T07:51:53.898551+0000","last_became_peered":"2026-02-24T07:51:53.898551+0000","last_unstale":"2026-02-24T07:52:25.765348+0000","last_undegraded":"2026-02-24T07:52:25.765348+0000","last_fullsized":"2026-02-24T07:52:25.765348+0000","mapping_epoch":74,"log_start":"76'12661","ondisk_log_start":"76'12661","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_clean_scrub_stamp":"2026-02-24T07:51:52.890170+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:53:07.664864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":829947904,"num_objects":12664,"num_object_clones":0,"num_object_copies":37992,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12664,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12664,"num_write_kb":810496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.4","version":"76'12327","reported_seq":12341,"reported_epoch":76,"state":"active+clean","last_fresh":"2026-02-24T07:52:25.741484+0000","last_change":"2026-02-24T07:51:53.900179+0000","last_active":"2026-02-24T07:52:25.741484+0000","last_peered":"2026-02-24T07:52:25.741484+0000","last_clean":"2026-02-24T07:52:25.741484+0000","last_became_active":"2026-02-24T07:51:53.899675+0000","last_became_peered":"2026-02-24T07:51:53.899675+0000","last_unstale":"2026-02-24T07:52:25.741484+0000","last_undegraded":"2026-02-24T07:52:25.741484+0000","last_fullsized":"2026-02-24T07:52:25.741484+0000","mapping_epoch":74,"log_start":"76'12324","ondisk_log_start":"76'12324","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_clean_scrub_stamp":"2026-02-24T07:51:52.890170+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:53:06.339989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":807862272,"num_objects":12327,"num_object_clones":0,"num_object_copies":36981,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12327,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12327,"num_write_kb":788928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"29'99","reported_seq":332,"reported_epoch":76,"state":"active+clean","last_fresh":"2026-02-24T07:51:54.993390+0000","last_change":"2026-02-24T07:51:54.101629+0000","last_active":"2026-02-24T07:51:54.993390+0000","last_peered":"2026-02-24T07:51:54.993390+0000","last_clean":"2026-02-24T07:51:54.993390+0000","last_became_active":"2026-02-24T07:42:54.186719+0000","last_became_peered":"2026-02-24T07:42:54.186719+0000","last_unstale":"2026-02-24T07:51:54.993390+0000","last_undegraded":"2026-02-24T07:51:54.993390+0000","last_fullsized":"2026-02-24T07:51:54.993390+0000","mapping_epoch":27,"log_start":"29'96","ondisk_log_start":"29'96","created":20,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'99","last_scrub_stamp":"2026-02-24T07:51:54.101595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:42:43.640776+0000","last_clean_scrub_stamp":"2026-02-24T07:51:54.101595+0000","objects_scrubbed":2,"log_size":3,"log_dups_size":96,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:52:54.995630+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":240,"num_write":135,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":16,"stat_sum":{"num_bytes":12993495040,"num_objects":198265,"num_object_clones":0,"num_object_copies":594795,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":198265,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":198265,"num_write_kb":12688960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":38989201408,"data_stored":38989201408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":48,"ondisk_log_size":48,"up":48,"acting":48,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":240,"num_write":135,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1682,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":3,"up_from":25,"seq":107374182517,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":11908028,"kb_used_data":10800104,"kb_used_omap":18,"kb_used_meta":1107885,"kb_avail":722095172,"statfs":{"total":751619276800,"available":739425456128,"internally_reserved":0,"allocated":11059306496,"data_stored":11058775859,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":18908,"internal_metadata":1134474788},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,2],"upper_bound":4},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":19,"seq":81604378743,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":9673020,"kb_used_data":8539560,"kb_used_omap":21,"kb_used_meta":1133418,"kb_avail":724330180,"statfs":{"total":751619276800,"available":741714104320,"internally_reserved":0,"allocated":8744509440,"data_stored":8743978803,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":21939,"internal_metadata":1160620621},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[4,1],"upper_bound":4},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542264,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":8750712,"kb_used_data":7840548,"kb_used_omap":18,"kb_used_meta":910125,"kb_avail":725252488,"statfs":{"total":751619276800,"available":742658547712,"internally_reserved":0,"allocated":8028721152,"data_stored":8028194067,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":18821,"internal_metadata":931968635},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[4],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705787,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":11569468,"kb_used_data":10275432,"kb_used_omap":17,"kb_used_meta":1293998,"kb_avail":722433732,"statfs":{"total":751619276800,"available":739772141568,"internally_reserved":0,"allocated":10522042368,"data_stored":10521511731,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":18330,"internal_metadata":1325054054},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1682,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":10563485696,"data_stored":10563485696,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8149925888,"data_stored":8149925888,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8928296960,"data_stored":8928296960,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":11347492864,"data_stored":11347492864,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-24T07:52:29.180 INFO:tasks.cephadm.ceph_manager.ceph:increase pool size by 10 2026-02-24T07:52:29.180 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph osd pool set .mgr pg_num 331 2026-02-24T07:52:29.323 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:52:29.456 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:52:29.454+0000 7fa3865e2640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:52:29.456 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:52:29.454+0000 7fa3865e2640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:52:29.595 INFO:teuthology.orchestra.run.trial035.stderr:Error EPERM: crimson_allow_pg_split is false; pg_num increase denied for crimson pool 2026-02-24T07:52:29.635 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-24T07:52:29.931 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2026-02-24T07:52:29.994 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 35 16 225615 225599 402.824 376.688 0.00141231 0.0024813 2026-02-24T07:52:30.063 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:52:30.181 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:52:29 trial035 ceph-mon[39284]: osdmap e77: 4 total, 4 up, 4 in 2026-02-24T07:52:30.181 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:52:29 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/689022359' entity='client.admin' cmd={"prefix": "osd pool set", "pool": ".mgr", "var": "pg_num", "val": "331"} : dispatch 2026-02-24T07:52:30.181 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:52:29 trial035 ceph-mon[39284]: from='mgr.14284 10.20.193.35:0/3681373222' entity='mgr.x' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2026-02-24T07:52:30.194 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:52:30.192+0000 7f1ab3dd6640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:52:30.195 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:52:30.192+0000 7f1ab3dd6640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:52:30.209 INFO:teuthology.orchestra.run.trial035.stdout:osd.0: {} 2026-02-24T07:52:30.209 INFO:teuthology.orchestra.run.trial035.stderr:osd.0: osd_enable_op_tracker = '' 2026-02-24T07:52:30.223 INFO:teuthology.orchestra.run.trial035.stdout:osd.1: {} 2026-02-24T07:52:30.223 INFO:teuthology.orchestra.run.trial035.stderr:osd.1: osd_enable_op_tracker = '' 2026-02-24T07:52:30.237 INFO:teuthology.orchestra.run.trial035.stdout:osd.2: {} 2026-02-24T07:52:30.237 INFO:teuthology.orchestra.run.trial035.stderr:osd.2: osd_enable_op_tracker = '' 2026-02-24T07:52:30.250 INFO:teuthology.orchestra.run.trial035.stdout:osd.3: {} 2026-02-24T07:52:30.250 INFO:teuthology.orchestra.run.trial035.stderr:osd.3: osd_enable_op_tracker = '' 2026-02-24T07:52:30.993 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 36 15 232250 232235 403.154 414.75 0.00163748 0.00247926 2026-02-24T07:52:31.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:52:30 trial035 ceph-mon[39284]: from='client.15194 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2026-02-24T07:52:31.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:52:30 trial035 ceph-mon[39284]: pgmap v336: 17 pgs: 17 active+clean; 12 GiB data, 40 GiB used, 2.7 TiB / 2.7 TiB avail; 392 MiB/s wr, 6.26k op/s 2026-02-24T07:52:31.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:52:30 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/1278741341' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2026-02-24T07:52:31.994 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 37 16 238937 238921 403.551 417.875 0.00238396 0.00247683 2026-02-24T07:52:32.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:52:31 trial035 ceph-mon[39284]: pgmap v337: 17 pgs: 17 active+clean; 14 GiB data, 46 GiB used, 2.7 TiB / 2.7 TiB avail; 403 MiB/s wr, 6.44k op/s 2026-02-24T07:52:32.292 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2026-02-24T07:52:32.432 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:52:32.564 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:52:32.562+0000 7fd807bd0640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:52:32.564 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:52:32.562+0000 7fd807bd0640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:52:32.578 INFO:teuthology.orchestra.run.trial035.stdout:osd.0: {} 2026-02-24T07:52:32.578 INFO:teuthology.orchestra.run.trial035.stderr:osd.0: osd_enable_op_tracker = '' 2026-02-24T07:52:32.592 INFO:teuthology.orchestra.run.trial035.stdout:osd.1: {} 2026-02-24T07:52:32.592 INFO:teuthology.orchestra.run.trial035.stderr:osd.1: osd_enable_op_tracker = '' 2026-02-24T07:52:32.604 INFO:teuthology.orchestra.run.trial035.stdout:osd.2: {} 2026-02-24T07:52:32.604 INFO:teuthology.orchestra.run.trial035.stderr:osd.2: osd_enable_op_tracker = '' 2026-02-24T07:52:32.617 INFO:teuthology.orchestra.run.trial035.stdout:osd.3: {} 2026-02-24T07:52:32.617 INFO:teuthology.orchestra.run.trial035.stderr:osd.3: osd_enable_op_tracker = '' 2026-02-24T07:52:32.994 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 38 16 246175 246159 404.835 452.375 0.00157215 0.00246899 2026-02-24T07:52:33.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:52:32 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/3715610488' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2026-02-24T07:52:33.994 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 39 16 252848 252832 405.148 417.062 0.00324425 0.00246705 2026-02-24T07:52:34.637 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2026-02-24T07:52:34.637 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2026-02-24T07:52:34.637 INFO:tasks.thrashosds.thrasher:Growing pool .mgr 2026-02-24T07:52:34.637 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph pg dump --format=json 2026-02-24T07:52:34.662 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2026-02-24T07:52:34.781 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:52:34.835 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:52:34.954 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:52:34 trial035 ceph-mon[39284]: pgmap v338: 17 pgs: 17 active+clean; 14 GiB data, 46 GiB used, 2.7 TiB / 2.7 TiB avail; 403 MiB/s wr, 6.44k op/s 2026-02-24T07:52:34.954 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:52:34 trial035 ceph-mon[39284]: osdmap e78: 4 total, 4 up, 4 in 2026-02-24T07:52:34.994 INFO:tasks.radosbench.radosbench.0.trial035.stdout:2026-02-24T07:52:34.993915+0000 min lat: 0.000706575 max lat: 0.0442884 avg lat: 0.00246972 2026-02-24T07:52:34.994 INFO:tasks.radosbench.radosbench.0.trial035.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2026-02-24T07:52:34.994 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 40 16 259053 259037 404.714 387.812 0.00165528 0.00246972 2026-02-24T07:52:35.253 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:52:35.190+0000 7fccecb7f640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:52:35.253 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:52:35.190+0000 7fccecb7f640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:52:35.330 INFO:teuthology.orchestra.run.trial035.stdout: 2026-02-24T07:52:35.331 INFO:teuthology.orchestra.run.trial035.stderr:dumped all 2026-02-24T07:52:35.738 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:52:35.669+0000 7f3a20b08640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:52:35.738 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:52:35.669+0000 7f3a20b08640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:52:35.738 INFO:teuthology.orchestra.run.trial035.stderr:osd.0: osd_enable_op_tracker = '' 2026-02-24T07:52:35.739 INFO:teuthology.orchestra.run.trial035.stderr:osd.1: osd_enable_op_tracker = '' 2026-02-24T07:52:35.739 INFO:teuthology.orchestra.run.trial035.stdout:osd.0: {} 2026-02-24T07:52:35.739 INFO:teuthology.orchestra.run.trial035.stdout:osd.1: {} 2026-02-24T07:52:35.739 INFO:teuthology.orchestra.run.trial035.stdout:osd.2: {} 2026-02-24T07:52:35.740 INFO:teuthology.orchestra.run.trial035.stderr:osd.2: osd_enable_op_tracker = '' 2026-02-24T07:52:35.746 INFO:teuthology.orchestra.run.trial035.stdout:osd.3: {} 2026-02-24T07:52:35.746 INFO:teuthology.orchestra.run.trial035.stderr:osd.3: osd_enable_op_tracker = '' 2026-02-24T07:52:35.857 INFO:teuthology.orchestra.run.trial035.stdout:{"pg_ready":true,"pg_map":{"version":339,"stamp":"2026-02-24T07:52:33.888002+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":15123481120,"num_objects":230759,"num_object_clones":0,"num_object_copies":692277,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":230759,"num_whiteouts":0,"num_read":136,"num_read_kb":240,"num_write":230892,"num_write_kb":14770598,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":51,"ondisk_log_size":51,"up":51,"acting":51,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":51,"num_osds":4,"num_per_pool_osds":4,"num_per_pool_omap_osds":4,"kb":2936012800,"kb_used":48484956,"kb_used_data":43678412,"kb_used_omap":79,"kb_used_meta":4806384,"kb_avail":2887527844,"statfs":{"total":3006477107200,"available":2956828512256,"internally_reserved":0,"allocated":44726693888,"data_stored":44724543544,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":81417,"internal_metadata":4921737719},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[8,4,0,0,0,2],"upper_bound":64},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":4221894656,"num_objects":64421,"num_object_clones":0,"num_object_copies":193263,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64421,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":64421,"num_write_kb":4122944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.153946"},"pg_stats":[{"pgid":"5.f","version":"77'14205","reported_seq":14221,"reported_epoch":77,"state":"active+clean","last_fresh":"2026-02-24T07:52:30.741114+0000","last_change":"2026-02-24T07:51:53.900797+0000","last_active":"2026-02-24T07:52:30.741114+0000","last_peered":"2026-02-24T07:52:30.741114+0000","last_clean":"2026-02-24T07:52:30.741114+0000","last_became_active":"2026-02-24T07:51:53.900431+0000","last_became_peered":"2026-02-24T07:51:53.900431+0000","last_unstale":"2026-02-24T07:52:30.741114+0000","last_undegraded":"2026-02-24T07:52:30.741114+0000","last_fullsized":"2026-02-24T07:52:30.741114+0000","mapping_epoch":74,"log_start":"77'14202","ondisk_log_start":"77'14202","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_clean_scrub_stamp":"2026-02-24T07:51:52.890170+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:52:57.614871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":930938880,"num_objects":14205,"num_object_clones":0,"num_object_copies":42615,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14205,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14205,"num_write_kb":909120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.e","version":"77'14357","reported_seq":14373,"reported_epoch":77,"state":"active+clean","last_fresh":"2026-02-24T07:52:30.756463+0000","last_change":"2026-02-24T07:51:53.899686+0000","last_active":"2026-02-24T07:52:30.756463+0000","last_peered":"2026-02-24T07:52:30.756463+0000","last_clean":"2026-02-24T07:52:30.756463+0000","last_became_active":"2026-02-24T07:51:53.899294+0000","last_became_peered":"2026-02-24T07:51:53.899294+0000","last_unstale":"2026-02-24T07:52:30.756463+0000","last_undegraded":"2026-02-24T07:52:30.756463+0000","last_fullsized":"2026-02-24T07:52:30.756463+0000","mapping_epoch":74,"log_start":"77'14354","ondisk_log_start":"77'14354","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_clean_scrub_stamp":"2026-02-24T07:51:52.890170+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:52:53.084042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":940900352,"num_objects":14357,"num_object_clones":0,"num_object_copies":43071,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14357,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14357,"num_write_kb":918848,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"77'14368","reported_seq":14384,"reported_epoch":77,"state":"active+clean","last_fresh":"2026-02-24T07:52:30.768193+0000","last_change":"2026-02-24T07:51:53.900129+0000","last_active":"2026-02-24T07:52:30.768193+0000","last_peered":"2026-02-24T07:52:30.768193+0000","last_clean":"2026-02-24T07:52:30.768193+0000","last_became_active":"2026-02-24T07:51:53.899772+0000","last_became_peered":"2026-02-24T07:51:53.899772+0000","last_unstale":"2026-02-24T07:52:30.768193+0000","last_undegraded":"2026-02-24T07:52:30.768193+0000","last_fullsized":"2026-02-24T07:52:30.768193+0000","mapping_epoch":74,"log_start":"77'14365","ondisk_log_start":"77'14365","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_clean_scrub_stamp":"2026-02-24T07:51:52.890170+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:53:19.516698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":941621248,"num_objects":14368,"num_object_clones":0,"num_object_copies":43104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14368,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14368,"num_write_kb":919552,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,3],"acting":[0,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.c","version":"77'14532","reported_seq":14548,"reported_epoch":77,"state":"active+clean","last_fresh":"2026-02-24T07:52:30.845990+0000","last_change":"2026-02-24T07:51:53.898484+0000","last_active":"2026-02-24T07:52:30.845990+0000","last_peered":"2026-02-24T07:52:30.845990+0000","last_clean":"2026-02-24T07:52:30.845990+0000","last_became_active":"2026-02-24T07:51:53.898285+0000","last_became_peered":"2026-02-24T07:51:53.898285+0000","last_unstale":"2026-02-24T07:52:30.845990+0000","last_undegraded":"2026-02-24T07:52:30.845990+0000","last_fullsized":"2026-02-24T07:52:30.845990+0000","mapping_epoch":74,"log_start":"77'14529","ondisk_log_start":"77'14529","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_clean_scrub_stamp":"2026-02-24T07:51:52.890170+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:53:11.947668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":952369152,"num_objects":14532,"num_object_clones":0,"num_object_copies":43596,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14532,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14532,"num_write_kb":930048,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.b","version":"77'14526","reported_seq":14542,"reported_epoch":77,"state":"active+clean","last_fresh":"2026-02-24T07:52:30.768102+0000","last_change":"2026-02-24T07:51:53.900298+0000","last_active":"2026-02-24T07:52:30.768102+0000","last_peered":"2026-02-24T07:52:30.768102+0000","last_clean":"2026-02-24T07:52:30.768102+0000","last_became_active":"2026-02-24T07:51:53.899901+0000","last_became_peered":"2026-02-24T07:51:53.899901+0000","last_unstale":"2026-02-24T07:52:30.768102+0000","last_undegraded":"2026-02-24T07:52:30.768102+0000","last_fullsized":"2026-02-24T07:52:30.768102+0000","mapping_epoch":74,"log_start":"77'14523","ondisk_log_start":"77'14523","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_clean_scrub_stamp":"2026-02-24T07:51:52.890170+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:53:12.321784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":951975936,"num_objects":14526,"num_object_clones":0,"num_object_copies":43578,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14526,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14526,"num_write_kb":929664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"77'14337","reported_seq":14353,"reported_epoch":77,"state":"active+clean","last_fresh":"2026-02-24T07:52:30.768253+0000","last_change":"2026-02-24T07:51:53.899828+0000","last_active":"2026-02-24T07:52:30.768253+0000","last_peered":"2026-02-24T07:52:30.768253+0000","last_clean":"2026-02-24T07:52:30.768253+0000","last_became_active":"2026-02-24T07:51:53.899549+0000","last_became_peered":"2026-02-24T07:51:53.899549+0000","last_unstale":"2026-02-24T07:52:30.768253+0000","last_undegraded":"2026-02-24T07:52:30.768253+0000","last_fullsized":"2026-02-24T07:52:30.768253+0000","mapping_epoch":74,"log_start":"77'14334","ondisk_log_start":"77'14334","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_clean_scrub_stamp":"2026-02-24T07:51:52.890170+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:53:07.446496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":939589632,"num_objects":14337,"num_object_clones":0,"num_object_copies":43011,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14337,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14337,"num_write_kb":917568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,1],"acting":[0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.9","version":"77'14653","reported_seq":14669,"reported_epoch":77,"state":"active+clean","last_fresh":"2026-02-24T07:52:30.758259+0000","last_change":"2026-02-24T07:51:53.901985+0000","last_active":"2026-02-24T07:52:30.758259+0000","last_peered":"2026-02-24T07:52:30.758259+0000","last_clean":"2026-02-24T07:52:30.758259+0000","last_became_active":"2026-02-24T07:51:53.901610+0000","last_became_peered":"2026-02-24T07:51:53.901610+0000","last_unstale":"2026-02-24T07:52:30.758259+0000","last_undegraded":"2026-02-24T07:52:30.758259+0000","last_fullsized":"2026-02-24T07:52:30.758259+0000","mapping_epoch":74,"log_start":"77'14650","ondisk_log_start":"77'14650","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_clean_scrub_stamp":"2026-02-24T07:51:52.890170+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:53:13.073570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":960299008,"num_objects":14653,"num_object_clones":0,"num_object_copies":43959,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14653,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14653,"num_write_kb":937792,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.8","version":"77'14300","reported_seq":14316,"reported_epoch":77,"state":"active+clean","last_fresh":"2026-02-24T07:52:30.763393+0000","last_change":"2026-02-24T07:51:53.900477+0000","last_active":"2026-02-24T07:52:30.763393+0000","last_peered":"2026-02-24T07:52:30.763393+0000","last_clean":"2026-02-24T07:52:30.763393+0000","last_became_active":"2026-02-24T07:51:53.900207+0000","last_became_peered":"2026-02-24T07:51:53.900207+0000","last_unstale":"2026-02-24T07:52:30.763393+0000","last_undegraded":"2026-02-24T07:52:30.763393+0000","last_fullsized":"2026-02-24T07:52:30.763393+0000","mapping_epoch":74,"log_start":"77'14297","ondisk_log_start":"77'14297","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_clean_scrub_stamp":"2026-02-24T07:51:52.890170+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:52:57.062307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":937164800,"num_objects":14300,"num_object_clones":0,"num_object_copies":42900,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14300,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14300,"num_write_kb":915200,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.7","version":"77'14389","reported_seq":14405,"reported_epoch":77,"state":"active+clean","last_fresh":"2026-02-24T07:52:30.758262+0000","last_change":"2026-02-24T07:51:53.899854+0000","last_active":"2026-02-24T07:52:30.758262+0000","last_peered":"2026-02-24T07:52:30.758262+0000","last_clean":"2026-02-24T07:52:30.758262+0000","last_became_active":"2026-02-24T07:51:53.899472+0000","last_became_peered":"2026-02-24T07:51:53.899472+0000","last_unstale":"2026-02-24T07:52:30.758262+0000","last_undegraded":"2026-02-24T07:52:30.758262+0000","last_fullsized":"2026-02-24T07:52:30.758262+0000","mapping_epoch":74,"log_start":"77'14386","ondisk_log_start":"77'14386","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_clean_scrub_stamp":"2026-02-24T07:51:52.890170+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:52:55.115725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":942997504,"num_objects":14389,"num_object_clones":0,"num_object_copies":43167,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14389,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14389,"num_write_kb":920896,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.6","version":"77'14328","reported_seq":14344,"reported_epoch":77,"state":"active+clean","last_fresh":"2026-02-24T07:52:30.846129+0000","last_change":"2026-02-24T07:51:53.898841+0000","last_active":"2026-02-24T07:52:30.846129+0000","last_peered":"2026-02-24T07:52:30.846129+0000","last_clean":"2026-02-24T07:52:30.846129+0000","last_became_active":"2026-02-24T07:51:53.898308+0000","last_became_peered":"2026-02-24T07:51:53.898308+0000","last_unstale":"2026-02-24T07:52:30.846129+0000","last_undegraded":"2026-02-24T07:52:30.846129+0000","last_fullsized":"2026-02-24T07:52:30.846129+0000","mapping_epoch":74,"log_start":"77'14325","ondisk_log_start":"77'14325","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_clean_scrub_stamp":"2026-02-24T07:51:52.890170+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:53:04.955186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":938999808,"num_objects":14328,"num_object_clones":0,"num_object_copies":42984,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14328,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14328,"num_write_kb":916992,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.5","version":"77'14530","reported_seq":14546,"reported_epoch":77,"state":"active+clean","last_fresh":"2026-02-24T07:52:30.755474+0000","last_change":"2026-02-24T07:51:53.900139+0000","last_active":"2026-02-24T07:52:30.755474+0000","last_peered":"2026-02-24T07:52:30.755474+0000","last_clean":"2026-02-24T07:52:30.755474+0000","last_became_active":"2026-02-24T07:51:53.899749+0000","last_became_peered":"2026-02-24T07:51:53.899749+0000","last_unstale":"2026-02-24T07:52:30.755474+0000","last_undegraded":"2026-02-24T07:52:30.755474+0000","last_fullsized":"2026-02-24T07:52:30.755474+0000","mapping_epoch":74,"log_start":"77'14527","ondisk_log_start":"77'14527","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_clean_scrub_stamp":"2026-02-24T07:51:52.890170+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:53:09.489880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":952238080,"num_objects":14530,"num_object_clones":0,"num_object_copies":43590,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14530,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14530,"num_write_kb":929920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"77'14381","reported_seq":14397,"reported_epoch":77,"state":"active+clean","last_fresh":"2026-02-24T07:52:30.764972+0000","last_change":"2026-02-24T07:51:53.900361+0000","last_active":"2026-02-24T07:52:30.764972+0000","last_peered":"2026-02-24T07:52:30.764972+0000","last_clean":"2026-02-24T07:52:30.764972+0000","last_became_active":"2026-02-24T07:51:53.900042+0000","last_became_peered":"2026-02-24T07:51:53.900042+0000","last_unstale":"2026-02-24T07:52:30.764972+0000","last_undegraded":"2026-02-24T07:52:30.764972+0000","last_fullsized":"2026-02-24T07:52:30.764972+0000","mapping_epoch":74,"log_start":"77'14378","ondisk_log_start":"77'14378","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_clean_scrub_stamp":"2026-02-24T07:51:52.890170+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:53:02.182725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":942473216,"num_objects":14381,"num_object_clones":0,"num_object_copies":43143,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14381,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14381,"num_write_kb":920384,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.2","version":"77'14620","reported_seq":14636,"reported_epoch":77,"state":"active+clean","last_fresh":"2026-02-24T07:52:30.747229+0000","last_change":"2026-02-24T07:51:53.900424+0000","last_active":"2026-02-24T07:52:30.747229+0000","last_peered":"2026-02-24T07:52:30.747229+0000","last_clean":"2026-02-24T07:52:30.747229+0000","last_became_active":"2026-02-24T07:51:53.900096+0000","last_became_peered":"2026-02-24T07:51:53.900096+0000","last_unstale":"2026-02-24T07:52:30.747229+0000","last_undegraded":"2026-02-24T07:52:30.747229+0000","last_fullsized":"2026-02-24T07:52:30.747229+0000","mapping_epoch":74,"log_start":"77'14617","ondisk_log_start":"77'14617","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_clean_scrub_stamp":"2026-02-24T07:51:52.890170+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:53:10.802005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":958136320,"num_objects":14620,"num_object_clones":0,"num_object_copies":43860,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14620,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14620,"num_write_kb":935680,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"77'14204","reported_seq":14220,"reported_epoch":77,"state":"active+clean","last_fresh":"2026-02-24T07:52:30.758220+0000","last_change":"2026-02-24T07:51:53.901421+0000","last_active":"2026-02-24T07:52:30.758220+0000","last_peered":"2026-02-24T07:52:30.758220+0000","last_clean":"2026-02-24T07:52:30.758220+0000","last_became_active":"2026-02-24T07:51:53.901027+0000","last_became_peered":"2026-02-24T07:51:53.901027+0000","last_unstale":"2026-02-24T07:52:30.758220+0000","last_undegraded":"2026-02-24T07:52:30.758220+0000","last_fullsized":"2026-02-24T07:52:30.758220+0000","mapping_epoch":74,"log_start":"77'14201","ondisk_log_start":"77'14201","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_clean_scrub_stamp":"2026-02-24T07:51:52.890170+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:53:04.436560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":930873344,"num_objects":14204,"num_object_clones":0,"num_object_copies":42612,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14204,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14204,"num_write_kb":909056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.0","version":"77'14591","reported_seq":14607,"reported_epoch":77,"state":"active+clean","last_fresh":"2026-02-24T07:52:30.765686+0000","last_change":"2026-02-24T07:51:53.898740+0000","last_active":"2026-02-24T07:52:30.765686+0000","last_peered":"2026-02-24T07:52:30.765686+0000","last_clean":"2026-02-24T07:52:30.765686+0000","last_became_active":"2026-02-24T07:51:53.898551+0000","last_became_peered":"2026-02-24T07:51:53.898551+0000","last_unstale":"2026-02-24T07:52:30.765686+0000","last_undegraded":"2026-02-24T07:52:30.765686+0000","last_fullsized":"2026-02-24T07:52:30.765686+0000","mapping_epoch":74,"log_start":"77'14588","ondisk_log_start":"77'14588","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_clean_scrub_stamp":"2026-02-24T07:51:52.890170+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:53:07.664864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":956235776,"num_objects":14591,"num_object_clones":0,"num_object_copies":43773,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14591,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14591,"num_write_kb":933824,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.4","version":"77'14436","reported_seq":14452,"reported_epoch":77,"state":"active+clean","last_fresh":"2026-02-24T07:52:30.744420+0000","last_change":"2026-02-24T07:51:53.900179+0000","last_active":"2026-02-24T07:52:30.744420+0000","last_peered":"2026-02-24T07:52:30.744420+0000","last_clean":"2026-02-24T07:52:30.744420+0000","last_became_active":"2026-02-24T07:51:53.899675+0000","last_became_peered":"2026-02-24T07:51:53.899675+0000","last_unstale":"2026-02-24T07:52:30.744420+0000","last_undegraded":"2026-02-24T07:52:30.744420+0000","last_fullsized":"2026-02-24T07:52:30.744420+0000","mapping_epoch":74,"log_start":"77'14433","ondisk_log_start":"77'14433","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_clean_scrub_stamp":"2026-02-24T07:51:52.890170+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:53:06.339989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":946077696,"num_objects":14436,"num_object_clones":0,"num_object_copies":43308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14436,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14436,"num_write_kb":923904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"29'99","reported_seq":334,"reported_epoch":77,"state":"active+clean","last_fresh":"2026-02-24T07:52:28.905040+0000","last_change":"2026-02-24T07:51:54.101629+0000","last_active":"2026-02-24T07:52:28.905040+0000","last_peered":"2026-02-24T07:52:28.905040+0000","last_clean":"2026-02-24T07:52:28.905040+0000","last_became_active":"2026-02-24T07:42:54.186719+0000","last_became_peered":"2026-02-24T07:42:54.186719+0000","last_unstale":"2026-02-24T07:52:28.905040+0000","last_undegraded":"2026-02-24T07:52:28.905040+0000","last_fullsized":"2026-02-24T07:52:28.905040+0000","mapping_epoch":27,"log_start":"29'96","ondisk_log_start":"29'96","created":20,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'99","last_scrub_stamp":"2026-02-24T07:51:54.101595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:42:43.640776+0000","last_clean_scrub_stamp":"2026-02-24T07:51:54.101595+0000","objects_scrubbed":2,"log_size":3,"log_dups_size":96,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:52:54.995630+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":240,"num_write":135,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":16,"stat_sum":{"num_bytes":15122890752,"num_objects":230757,"num_object_clones":0,"num_object_copies":692271,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":230757,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":230757,"num_write_kb":14768448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":45373259776,"data_stored":45373259776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":48,"ondisk_log_size":48,"up":48,"acting":48,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":240,"num_write":135,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1682,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":3,"up_from":25,"seq":107374182518,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":13829192,"kb_used_data":12612980,"kb_used_omap":20,"kb_used_meta":1216171,"kb_avail":720174008,"statfs":{"total":751619276800,"available":737458184192,"internally_reserved":0,"allocated":12915691520,"data_stored":12915153046,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":20848,"internal_metadata":1245359760},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[6,2],"upper_bound":4},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":19,"seq":81604378744,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":11196488,"kb_used_data":9990836,"kb_used_omap":22,"kb_used_meta":1205609,"kb_avail":722806712,"statfs":{"total":751619276800,"available":740154073088,"internally_reserved":0,"allocated":10230616064,"data_stored":10230077590,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":22606,"internal_metadata":1234544562},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,2],"upper_bound":64},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542265,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":10135812,"kb_used_data":9153456,"kb_used_omap":19,"kb_used_meta":982316,"kb_avail":723867388,"statfs":{"total":751619276800,"available":741240205312,"internally_reserved":0,"allocated":9373138944,"data_stored":9372604022,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19473,"internal_metadata":1005892591},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,2],"upper_bound":4},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705788,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":13323464,"kb_used_data":11921140,"kb_used_omap":18,"kb_used_meta":1402285,"kb_avail":720679736,"statfs":{"total":751619276800,"available":737976049664,"internally_reserved":0,"allocated":12207247360,"data_stored":12206708886,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":18490,"internal_metadata":1435940806},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1682,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12277645312,"data_stored":12277645312,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":9484238848,"data_stored":9484238848,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":10400432128,"data_stored":10400432128,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":13210943488,"data_stored":13210943488,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-24T07:52:35.859 INFO:tasks.cephadm.ceph_manager.ceph:increase pool size by 10 2026-02-24T07:52:35.859 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph osd pool set .mgr pg_num 341 2026-02-24T07:52:35.994 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 41 16 263730 263714 401.972 292.312 0.0014374 0.00248652 2026-02-24T07:52:36.000 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:52:36.135 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:52:36.133+0000 7f39db071640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:52:36.135 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:52:36.133+0000 7f39db071640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:52:36.135 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:52:35 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/3852182871' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2026-02-24T07:52:36.265 INFO:teuthology.orchestra.run.trial035.stderr:Error EPERM: crimson_allow_pg_split is false; pg_num increase denied for crimson pool 2026-02-24T07:52:36.305 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-24T07:52:36.994 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 42 15 270246 270231 402.098 407.312 0.00122045 0.00248579 2026-02-24T07:52:37.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:52:36 trial035 ceph-mon[39284]: from='client.15202 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2026-02-24T07:52:37.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:52:36 trial035 ceph-mon[39284]: pgmap v340: 17 pgs: 17 active+clean; 14 GiB data, 46 GiB used, 2.7 TiB / 2.7 TiB avail; 254 MiB/s wr, 4.06k op/s 2026-02-24T07:52:37.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:52:36 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/3843167384' entity='client.admin' cmd={"prefix": "osd pool set", "pool": ".mgr", "var": "pg_num", "val": "341"} : dispatch 2026-02-24T07:52:37.205 INFO:journalctl@ceph.mgr.x.trial035.stdout:Feb 24 07:52:36 trial035 ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-mgr-x[39621]: ::ffff:10.20.193.91 - - [24/Feb/2026:07:52:36] "GET /metrics HTTP/1.1" 200 4223 "" "Prometheus/3.6.0" 2026-02-24T07:52:37.356 INFO:journalctl@ceph.prometheus.a.trial091.stdout:Feb 24 07:52:36 trial091 ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-prometheus-a[66007]: time=2026-02-24T07:52:36.973Z level=ERROR source=scrape.go:1630 msg="Failed to determine correct type of scrape target." component="scrape manager" scrape_pool=ceph target=http://trial035:9283/metrics content_type="text/html;charset=utf-8" fallback_media_type="" err="received unsupported Content-Type \"text/html;charset=utf-8\" and no fallback_scrape_protocol specified for target" 2026-02-24T07:52:37.868 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2026-02-24T07:52:37.994 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 43 16 276790 276774 402.256 408.938 0.00282322 0.00248483 2026-02-24T07:52:38.008 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:52:38.129 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:52:38.126+0000 7fd236bc3640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:52:38.129 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:52:38.127+0000 7fd236bc3640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:52:38.142 INFO:teuthology.orchestra.run.trial035.stdout:osd.0: {} 2026-02-24T07:52:38.143 INFO:teuthology.orchestra.run.trial035.stderr:osd.0: osd_enable_op_tracker = '' 2026-02-24T07:52:38.155 INFO:teuthology.orchestra.run.trial035.stdout:osd.1: {} 2026-02-24T07:52:38.156 INFO:teuthology.orchestra.run.trial035.stderr:osd.1: osd_enable_op_tracker = '' 2026-02-24T07:52:38.168 INFO:teuthology.orchestra.run.trial035.stdout:osd.2: {} 2026-02-24T07:52:38.169 INFO:teuthology.orchestra.run.trial035.stderr:osd.2: osd_enable_op_tracker = '' 2026-02-24T07:52:38.180 INFO:teuthology.orchestra.run.trial035.stdout:osd.3: {} 2026-02-24T07:52:38.181 INFO:teuthology.orchestra.run.trial035.stderr:osd.3: osd_enable_op_tracker = '' 2026-02-24T07:52:38.994 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 44 16 282342 282326 401 347 0.00145498 0.00249259 2026-02-24T07:52:39.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:52:38 trial035 ceph-mon[39284]: pgmap v341: 17 pgs: 17 active+clean; 16 GiB data, 53 GiB used, 2.7 TiB / 2.7 TiB avail; 454 MiB/s wr, 7.27k op/s 2026-02-24T07:52:39.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:52:38 trial035 ceph-mon[39284]: osdmap e79: 4 total, 4 up, 4 in 2026-02-24T07:52:39.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:52:38 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/4122587190' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2026-02-24T07:52:39.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:52:38 trial035 ceph-mon[39284]: osdmap e80: 4 total, 4 up, 4 in 2026-02-24T07:52:39.994 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 45 16 289279 289263 401.723 433.562 0.000959975 0.00248775 2026-02-24T07:52:40.635 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2026-02-24T07:52:40.779 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:52:40.902 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:52:40.899+0000 7fe906416640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:52:40.902 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:52:40.899+0000 7fe906416640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:52:40.917 INFO:teuthology.orchestra.run.trial035.stdout:osd.0: {} 2026-02-24T07:52:40.918 INFO:teuthology.orchestra.run.trial035.stderr:osd.0: osd_enable_op_tracker = '' 2026-02-24T07:52:40.918 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:52:40 trial035 ceph-mon[39284]: pgmap v344: 17 pgs: 17 active+clean; 16 GiB data, 53 GiB used, 2.7 TiB / 2.7 TiB avail; 330 MiB/s wr, 5.28k op/s 2026-02-24T07:52:40.930 INFO:teuthology.orchestra.run.trial035.stdout:osd.1: {} 2026-02-24T07:52:40.930 INFO:teuthology.orchestra.run.trial035.stderr:osd.1: osd_enable_op_tracker = '' 2026-02-24T07:52:40.943 INFO:teuthology.orchestra.run.trial035.stdout:osd.2: {} 2026-02-24T07:52:40.943 INFO:teuthology.orchestra.run.trial035.stderr:osd.2: osd_enable_op_tracker = '' 2026-02-24T07:52:40.956 INFO:teuthology.orchestra.run.trial035.stdout:osd.3: {} 2026-02-24T07:52:40.957 INFO:teuthology.orchestra.run.trial035.stderr:osd.3: osd_enable_op_tracker = '' 2026-02-24T07:52:40.994 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 46 16 296197 296181 402.389 432.375 0.00137846 0.00248401 2026-02-24T07:52:41.306 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2026-02-24T07:52:41.307 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2026-02-24T07:52:41.307 INFO:tasks.thrashosds.thrasher:Growing pool unique_pool_3 2026-02-24T07:52:41.307 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph pg dump --format=json 2026-02-24T07:52:41.444 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:52:41.575 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:52:41.572+0000 7f0f99ef6640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:52:41.575 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:52:41.573+0000 7f0f99ef6640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:52:41.718 INFO:teuthology.orchestra.run.trial035.stdout: 2026-02-24T07:52:41.718 INFO:teuthology.orchestra.run.trial035.stderr:dumped all 2026-02-24T07:52:41.765 INFO:teuthology.orchestra.run.trial035.stdout:{"pg_ready":true,"pg_map":{"version":344,"stamp":"2026-02-24T07:52:39.736021+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":17199464992,"num_objects":262436,"num_object_clones":0,"num_object_copies":787308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":262436,"num_whiteouts":0,"num_read":136,"num_read_kb":240,"num_write":262569,"num_write_kb":16797926,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":52,"ondisk_log_size":52,"up":51,"acting":51,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":51,"num_osds":4,"num_per_pool_osds":4,"num_per_pool_omap_osds":4,"kb":2936012800,"kb_used":55211020,"kb_used_data":50043516,"kb_used_omap":84,"kb_used_meta":5167339,"kb_avail":2880801780,"statfs":{"total":3006477107200,"available":2949941022720,"internally_reserved":0,"allocated":51244560384,"data_stored":51242378492,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":86344,"internal_metadata":5291355832},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[8,10,2],"upper_bound":8},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2075983872,"num_objects":31677,"num_object_clones":0,"num_object_copies":95031,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31677,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":31677,"num_write_kb":2027328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.000856"},"pg_stats":[{"pgid":"5.f","version":"78'16188","reported_seq":16206,"reported_epoch":78,"state":"active+clean","last_fresh":"2026-02-24T07:52:35.744733+0000","last_change":"2026-02-24T07:51:53.900797+0000","last_active":"2026-02-24T07:52:35.744733+0000","last_peered":"2026-02-24T07:52:35.744733+0000","last_clean":"2026-02-24T07:52:35.744733+0000","last_became_active":"2026-02-24T07:51:53.900431+0000","last_became_peered":"2026-02-24T07:51:53.900431+0000","last_unstale":"2026-02-24T07:52:35.744733+0000","last_undegraded":"2026-02-24T07:52:35.744733+0000","last_fullsized":"2026-02-24T07:52:35.744733+0000","mapping_epoch":74,"log_start":"78'16185","ondisk_log_start":"78'16185","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_clean_scrub_stamp":"2026-02-24T07:51:52.890170+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:52:57.614871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1060896768,"num_objects":16188,"num_object_clones":0,"num_object_copies":48564,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16188,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16188,"num_write_kb":1036032,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.e","version":"78'16377","reported_seq":16395,"reported_epoch":78,"state":"active+clean","last_fresh":"2026-02-24T07:52:35.754780+0000","last_change":"2026-02-24T07:51:53.899686+0000","last_active":"2026-02-24T07:52:35.754780+0000","last_peered":"2026-02-24T07:52:35.754780+0000","last_clean":"2026-02-24T07:52:35.754780+0000","last_became_active":"2026-02-24T07:51:53.899294+0000","last_became_peered":"2026-02-24T07:51:53.899294+0000","last_unstale":"2026-02-24T07:52:35.754780+0000","last_undegraded":"2026-02-24T07:52:35.754780+0000","last_fullsized":"2026-02-24T07:52:35.754780+0000","mapping_epoch":74,"log_start":"78'16374","ondisk_log_start":"78'16374","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_clean_scrub_stamp":"2026-02-24T07:51:52.890170+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:52:53.084042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1073283072,"num_objects":16377,"num_object_clones":0,"num_object_copies":49131,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16377,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16377,"num_write_kb":1048128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"78'16418","reported_seq":16436,"reported_epoch":78,"state":"active+clean","last_fresh":"2026-02-24T07:52:35.764123+0000","last_change":"2026-02-24T07:51:53.900129+0000","last_active":"2026-02-24T07:52:35.764123+0000","last_peered":"2026-02-24T07:52:35.764123+0000","last_clean":"2026-02-24T07:52:35.764123+0000","last_became_active":"2026-02-24T07:51:53.899772+0000","last_became_peered":"2026-02-24T07:51:53.899772+0000","last_unstale":"2026-02-24T07:52:35.764123+0000","last_undegraded":"2026-02-24T07:52:35.764123+0000","last_fullsized":"2026-02-24T07:52:35.764123+0000","mapping_epoch":74,"log_start":"78'16414","ondisk_log_start":"78'16414","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_clean_scrub_stamp":"2026-02-24T07:51:52.890170+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":3000,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:53:19.516698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1075970048,"num_objects":16418,"num_object_clones":0,"num_object_copies":49254,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16418,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16418,"num_write_kb":1050752,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,3],"acting":[0,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.c","version":"78'16384","reported_seq":16401,"reported_epoch":78,"state":"active+clean","last_fresh":"2026-02-24T07:52:35.847482+0000","last_change":"2026-02-24T07:51:53.898484+0000","last_active":"2026-02-24T07:52:35.847482+0000","last_peered":"2026-02-24T07:52:35.847482+0000","last_clean":"2026-02-24T07:52:35.847482+0000","last_became_active":"2026-02-24T07:51:53.898285+0000","last_became_peered":"2026-02-24T07:51:53.898285+0000","last_unstale":"2026-02-24T07:52:35.847482+0000","last_undegraded":"2026-02-24T07:52:35.847482+0000","last_fullsized":"2026-02-24T07:52:35.847482+0000","mapping_epoch":74,"log_start":"78'16381","ondisk_log_start":"78'16381","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_clean_scrub_stamp":"2026-02-24T07:51:52.890170+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:53:11.947668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1073741824,"num_objects":16384,"num_object_clones":0,"num_object_copies":49152,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16384,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16384,"num_write_kb":1048576,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.b","version":"78'16531","reported_seq":16549,"reported_epoch":78,"state":"active+clean","last_fresh":"2026-02-24T07:52:35.766885+0000","last_change":"2026-02-24T07:51:53.900298+0000","last_active":"2026-02-24T07:52:35.766885+0000","last_peered":"2026-02-24T07:52:35.766885+0000","last_clean":"2026-02-24T07:52:35.766885+0000","last_became_active":"2026-02-24T07:51:53.899901+0000","last_became_peered":"2026-02-24T07:51:53.899901+0000","last_unstale":"2026-02-24T07:52:35.766885+0000","last_undegraded":"2026-02-24T07:52:35.766885+0000","last_fullsized":"2026-02-24T07:52:35.766885+0000","mapping_epoch":74,"log_start":"78'16528","ondisk_log_start":"78'16528","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_clean_scrub_stamp":"2026-02-24T07:51:52.890170+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:53:12.321784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1083375616,"num_objects":16531,"num_object_clones":0,"num_object_copies":49593,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16531,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16531,"num_write_kb":1057984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"78'16329","reported_seq":16347,"reported_epoch":78,"state":"active+clean","last_fresh":"2026-02-24T07:52:35.767191+0000","last_change":"2026-02-24T07:51:53.899828+0000","last_active":"2026-02-24T07:52:35.767191+0000","last_peered":"2026-02-24T07:52:35.767191+0000","last_clean":"2026-02-24T07:52:35.767191+0000","last_became_active":"2026-02-24T07:51:53.899549+0000","last_became_peered":"2026-02-24T07:51:53.899549+0000","last_unstale":"2026-02-24T07:52:35.767191+0000","last_undegraded":"2026-02-24T07:52:35.767191+0000","last_fullsized":"2026-02-24T07:52:35.767191+0000","mapping_epoch":74,"log_start":"78'16326","ondisk_log_start":"78'16326","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_clean_scrub_stamp":"2026-02-24T07:51:52.890170+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:53:07.446496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1070137344,"num_objects":16329,"num_object_clones":0,"num_object_copies":48987,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16329,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16329,"num_write_kb":1045056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,1],"acting":[0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.9","version":"78'16633","reported_seq":16651,"reported_epoch":78,"state":"active+clean","last_fresh":"2026-02-24T07:52:35.756299+0000","last_change":"2026-02-24T07:51:53.901985+0000","last_active":"2026-02-24T07:52:35.756299+0000","last_peered":"2026-02-24T07:52:35.756299+0000","last_clean":"2026-02-24T07:52:35.756299+0000","last_became_active":"2026-02-24T07:51:53.901610+0000","last_became_peered":"2026-02-24T07:51:53.901610+0000","last_unstale":"2026-02-24T07:52:35.756299+0000","last_undegraded":"2026-02-24T07:52:35.756299+0000","last_fullsized":"2026-02-24T07:52:35.756299+0000","mapping_epoch":74,"log_start":"78'16630","ondisk_log_start":"78'16630","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_clean_scrub_stamp":"2026-02-24T07:51:52.890170+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:53:13.073570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1090060288,"num_objects":16633,"num_object_clones":0,"num_object_copies":49899,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16633,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16633,"num_write_kb":1064512,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.8","version":"78'16332","reported_seq":16350,"reported_epoch":78,"state":"active+clean","last_fresh":"2026-02-24T07:52:35.764069+0000","last_change":"2026-02-24T07:51:53.900477+0000","last_active":"2026-02-24T07:52:35.764069+0000","last_peered":"2026-02-24T07:52:35.764069+0000","last_clean":"2026-02-24T07:52:35.764069+0000","last_became_active":"2026-02-24T07:51:53.900207+0000","last_became_peered":"2026-02-24T07:51:53.900207+0000","last_unstale":"2026-02-24T07:52:35.764069+0000","last_undegraded":"2026-02-24T07:52:35.764069+0000","last_fullsized":"2026-02-24T07:52:35.764069+0000","mapping_epoch":74,"log_start":"78'16329","ondisk_log_start":"78'16329","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_clean_scrub_stamp":"2026-02-24T07:51:52.890170+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:52:57.062307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1070333952,"num_objects":16332,"num_object_clones":0,"num_object_copies":48996,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16332,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16332,"num_write_kb":1045248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.7","version":"78'16406","reported_seq":16424,"reported_epoch":78,"state":"active+clean","last_fresh":"2026-02-24T07:52:35.757681+0000","last_change":"2026-02-24T07:51:53.899854+0000","last_active":"2026-02-24T07:52:35.757681+0000","last_peered":"2026-02-24T07:52:35.757681+0000","last_clean":"2026-02-24T07:52:35.757681+0000","last_became_active":"2026-02-24T07:51:53.899472+0000","last_became_peered":"2026-02-24T07:51:53.899472+0000","last_unstale":"2026-02-24T07:52:35.757681+0000","last_undegraded":"2026-02-24T07:52:35.757681+0000","last_fullsized":"2026-02-24T07:52:35.757681+0000","mapping_epoch":74,"log_start":"78'16403","ondisk_log_start":"78'16403","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_clean_scrub_stamp":"2026-02-24T07:51:52.890170+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:52:55.115725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1075183616,"num_objects":16406,"num_object_clones":0,"num_object_copies":49218,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16406,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16406,"num_write_kb":1049984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.6","version":"78'16371","reported_seq":16389,"reported_epoch":78,"state":"active+clean","last_fresh":"2026-02-24T07:52:35.843955+0000","last_change":"2026-02-24T07:51:53.898841+0000","last_active":"2026-02-24T07:52:35.843955+0000","last_peered":"2026-02-24T07:52:35.843955+0000","last_clean":"2026-02-24T07:52:35.843955+0000","last_became_active":"2026-02-24T07:51:53.898308+0000","last_became_peered":"2026-02-24T07:51:53.898308+0000","last_unstale":"2026-02-24T07:52:35.843955+0000","last_undegraded":"2026-02-24T07:52:35.843955+0000","last_fullsized":"2026-02-24T07:52:35.843955+0000","mapping_epoch":74,"log_start":"78'16368","ondisk_log_start":"78'16368","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_clean_scrub_stamp":"2026-02-24T07:51:52.890170+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:53:04.955186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1072889856,"num_objects":16371,"num_object_clones":0,"num_object_copies":49113,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16371,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16371,"num_write_kb":1047744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.5","version":"78'16496","reported_seq":16514,"reported_epoch":78,"state":"active+clean","last_fresh":"2026-02-24T07:52:35.761009+0000","last_change":"2026-02-24T07:51:53.900139+0000","last_active":"2026-02-24T07:52:35.761009+0000","last_peered":"2026-02-24T07:52:35.761009+0000","last_clean":"2026-02-24T07:52:35.761009+0000","last_became_active":"2026-02-24T07:51:53.899749+0000","last_became_peered":"2026-02-24T07:51:53.899749+0000","last_unstale":"2026-02-24T07:52:35.761009+0000","last_undegraded":"2026-02-24T07:52:35.761009+0000","last_fullsized":"2026-02-24T07:52:35.761009+0000","mapping_epoch":74,"log_start":"78'16493","ondisk_log_start":"78'16493","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_clean_scrub_stamp":"2026-02-24T07:51:52.890170+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:53:09.489880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1081081856,"num_objects":16496,"num_object_clones":0,"num_object_copies":49488,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16496,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16496,"num_write_kb":1055744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"78'16333","reported_seq":16351,"reported_epoch":78,"state":"active+clean","last_fresh":"2026-02-24T07:52:35.762155+0000","last_change":"2026-02-24T07:51:53.900361+0000","last_active":"2026-02-24T07:52:35.762155+0000","last_peered":"2026-02-24T07:52:35.762155+0000","last_clean":"2026-02-24T07:52:35.762155+0000","last_became_active":"2026-02-24T07:51:53.900042+0000","last_became_peered":"2026-02-24T07:51:53.900042+0000","last_unstale":"2026-02-24T07:52:35.762155+0000","last_undegraded":"2026-02-24T07:52:35.762155+0000","last_fullsized":"2026-02-24T07:52:35.762155+0000","mapping_epoch":74,"log_start":"78'16330","ondisk_log_start":"78'16330","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_clean_scrub_stamp":"2026-02-24T07:51:52.890170+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:53:02.182725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1070399488,"num_objects":16333,"num_object_clones":0,"num_object_copies":48999,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16333,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16333,"num_write_kb":1045312,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.2","version":"78'16569","reported_seq":16587,"reported_epoch":78,"state":"active+clean","last_fresh":"2026-02-24T07:52:35.754658+0000","last_change":"2026-02-24T07:51:53.900424+0000","last_active":"2026-02-24T07:52:35.754658+0000","last_peered":"2026-02-24T07:52:35.754658+0000","last_clean":"2026-02-24T07:52:35.754658+0000","last_became_active":"2026-02-24T07:51:53.900096+0000","last_became_peered":"2026-02-24T07:51:53.900096+0000","last_unstale":"2026-02-24T07:52:35.754658+0000","last_undegraded":"2026-02-24T07:52:35.754658+0000","last_fullsized":"2026-02-24T07:52:35.754658+0000","mapping_epoch":74,"log_start":"78'16566","ondisk_log_start":"78'16566","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_clean_scrub_stamp":"2026-02-24T07:51:52.890170+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:53:10.802005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1085865984,"num_objects":16569,"num_object_clones":0,"num_object_copies":49707,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16569,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16569,"num_write_kb":1060416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"78'16092","reported_seq":16110,"reported_epoch":78,"state":"active+clean","last_fresh":"2026-02-24T07:52:35.757467+0000","last_change":"2026-02-24T07:51:53.901421+0000","last_active":"2026-02-24T07:52:35.757467+0000","last_peered":"2026-02-24T07:52:35.757467+0000","last_clean":"2026-02-24T07:52:35.757467+0000","last_became_active":"2026-02-24T07:51:53.901027+0000","last_became_peered":"2026-02-24T07:51:53.901027+0000","last_unstale":"2026-02-24T07:52:35.757467+0000","last_undegraded":"2026-02-24T07:52:35.757467+0000","last_fullsized":"2026-02-24T07:52:35.757467+0000","mapping_epoch":74,"log_start":"78'16089","ondisk_log_start":"78'16089","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_clean_scrub_stamp":"2026-02-24T07:51:52.890170+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:53:04.436560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1054605312,"num_objects":16092,"num_object_clones":0,"num_object_copies":48276,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16092,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16092,"num_write_kb":1029888,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.0","version":"78'16642","reported_seq":16659,"reported_epoch":78,"state":"active+clean","last_fresh":"2026-02-24T07:52:35.768086+0000","last_change":"2026-02-24T07:51:53.898740+0000","last_active":"2026-02-24T07:52:35.768086+0000","last_peered":"2026-02-24T07:52:35.768086+0000","last_clean":"2026-02-24T07:52:35.768086+0000","last_became_active":"2026-02-24T07:51:53.898551+0000","last_became_peered":"2026-02-24T07:51:53.898551+0000","last_unstale":"2026-02-24T07:52:35.768086+0000","last_undegraded":"2026-02-24T07:52:35.768086+0000","last_fullsized":"2026-02-24T07:52:35.768086+0000","mapping_epoch":74,"log_start":"78'16639","ondisk_log_start":"78'16639","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_clean_scrub_stamp":"2026-02-24T07:51:52.890170+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:53:07.664864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1090650112,"num_objects":16642,"num_object_clones":0,"num_object_copies":49926,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16642,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16642,"num_write_kb":1065088,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.4","version":"78'16333","reported_seq":16351,"reported_epoch":78,"state":"active+clean","last_fresh":"2026-02-24T07:52:35.727559+0000","last_change":"2026-02-24T07:51:53.900179+0000","last_active":"2026-02-24T07:52:35.727559+0000","last_peered":"2026-02-24T07:52:35.727559+0000","last_clean":"2026-02-24T07:52:35.727559+0000","last_became_active":"2026-02-24T07:51:53.899675+0000","last_became_peered":"2026-02-24T07:51:53.899675+0000","last_unstale":"2026-02-24T07:52:35.727559+0000","last_undegraded":"2026-02-24T07:52:35.727559+0000","last_fullsized":"2026-02-24T07:52:35.727559+0000","mapping_epoch":74,"log_start":"78'16330","ondisk_log_start":"78'16330","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_clean_scrub_stamp":"2026-02-24T07:51:52.890170+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:53:06.339989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1070399488,"num_objects":16333,"num_object_clones":0,"num_object_copies":48999,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16333,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16333,"num_write_kb":1045312,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"29'99","reported_seq":336,"reported_epoch":78,"state":"active+clean","last_fresh":"2026-02-24T07:52:33.888598+0000","last_change":"2026-02-24T07:51:54.101629+0000","last_active":"2026-02-24T07:52:33.888598+0000","last_peered":"2026-02-24T07:52:33.888598+0000","last_clean":"2026-02-24T07:52:33.888598+0000","last_became_active":"2026-02-24T07:42:54.186719+0000","last_became_peered":"2026-02-24T07:42:54.186719+0000","last_unstale":"2026-02-24T07:52:33.888598+0000","last_undegraded":"2026-02-24T07:52:33.888598+0000","last_fullsized":"2026-02-24T07:52:33.888598+0000","mapping_epoch":27,"log_start":"29'96","ondisk_log_start":"29'96","created":20,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'99","last_scrub_stamp":"2026-02-24T07:51:54.101595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:42:43.640776+0000","last_clean_scrub_stamp":"2026-02-24T07:51:54.101595+0000","objects_scrubbed":2,"log_size":3,"log_dups_size":96,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:52:54.995630+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":240,"num_write":135,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":16,"stat_sum":{"num_bytes":17198874624,"num_objects":262434,"num_object_clones":0,"num_object_copies":787302,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":262434,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":262434,"num_write_kb":16795776,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":51599966208,"data_stored":51599966208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":49,"ondisk_log_size":49,"up":48,"acting":48,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":240,"num_write":135,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1682,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":3,"up_from":25,"seq":107374182519,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":15853396,"kb_used_data":14528896,"kb_used_omap":22,"kb_used_meta":1324457,"kb_avail":718149804,"statfs":{"total":751619276800,"available":735385399296,"internally_reserved":0,"allocated":14877589504,"data_stored":14877043143,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":22868,"internal_metadata":1356244652},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":19,"seq":81604378745,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":12721748,"kb_used_data":11443904,"kb_used_omap":22,"kb_used_meta":1277801,"kb_avail":721281452,"statfs":{"total":751619276800,"available":738592206848,"internally_reserved":0,"allocated":11718557696,"data_stored":11718011335,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":23253,"internal_metadata":1308468523},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1],"upper_bound":4},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542266,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":11556368,"kb_used_data":10465724,"kb_used_omap":19,"kb_used_meta":1090604,"kb_avail":722446832,"statfs":{"total":751619276800,"available":739785555968,"internally_reserved":0,"allocated":10716901376,"data_stored":10716358567,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":20125,"internal_metadata":1116778851},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705789,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":15079508,"kb_used_data":13604992,"kb_used_omap":19,"kb_used_meta":1474476,"kb_avail":718923692,"statfs":{"total":751619276800,"available":736177860608,"internally_reserved":0,"allocated":13931511808,"data_stored":13930965447,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":20098,"internal_metadata":1509863806},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[6,9,2],"upper_bound":8},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1682,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":13966770176,"data_stored":13966770176,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":10779820032,"data_stored":10779820032,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":11821252608,"data_stored":11821252608,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":15032123392,"data_stored":15032123392,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-24T07:52:41.767 INFO:tasks.cephadm.ceph_manager.ceph:increase pool size by 10 2026-02-24T07:52:41.768 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph osd pool set unique_pool_3 pg_num 26 2026-02-24T07:52:41.908 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:52:41.972 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:52:41 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/1425666017' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2026-02-24T07:52:41.994 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 47 16 302885 302869 402.72 418 0.00111746 0.0024819 2026-02-24T07:52:42.027 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:52:42.025+0000 7fe4877a2640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:52:42.028 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:52:42.026+0000 7fe4877a2640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:52:42.157 INFO:teuthology.orchestra.run.trial035.stderr:Error EPERM: crimson_allow_pg_split is false; pg_num increase denied for crimson pool 2026-02-24T07:52:42.199 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-24T07:52:42.995 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 48 16 309503 309487 402.947 413.625 0.00133066 0.00248055 2026-02-24T07:52:43.016 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2026-02-24T07:52:43.154 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:52:43.179 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:52:42 trial035 ceph-mon[39284]: from='client.15212 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2026-02-24T07:52:43.179 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:52:42 trial035 ceph-mon[39284]: pgmap v345: 17 pgs: 17 active+clean; 18 GiB data, 59 GiB used, 2.7 TiB / 2.7 TiB avail; 510 MiB/s wr, 8.15k op/s 2026-02-24T07:52:43.179 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:52:42 trial035 ceph-mon[39284]: purged_snaps scrub starts 2026-02-24T07:52:43.179 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:52:42 trial035 ceph-mon[39284]: purged_snaps scrub ok 2026-02-24T07:52:43.179 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:52:42 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/3621648851' entity='client.admin' cmd={"prefix": "osd pool set", "pool": "unique_pool_3", "var": "pg_num", "val": "26"} : dispatch 2026-02-24T07:52:43.276 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:52:43.273+0000 7fc7a6ab9640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:52:43.276 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:52:43.273+0000 7fc7a6ab9640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:52:43.291 INFO:teuthology.orchestra.run.trial035.stdout:osd.0: {} 2026-02-24T07:52:43.291 INFO:teuthology.orchestra.run.trial035.stderr:osd.0: osd_enable_op_tracker = '' 2026-02-24T07:52:43.305 INFO:teuthology.orchestra.run.trial035.stdout:osd.1: {} 2026-02-24T07:52:43.305 INFO:teuthology.orchestra.run.trial035.stderr:osd.1: osd_enable_op_tracker = '' 2026-02-24T07:52:43.319 INFO:teuthology.orchestra.run.trial035.stdout:osd.2: {} 2026-02-24T07:52:43.319 INFO:teuthology.orchestra.run.trial035.stderr:osd.2: osd_enable_op_tracker = '' 2026-02-24T07:52:43.332 INFO:teuthology.orchestra.run.trial035.stdout:osd.3: {} 2026-02-24T07:52:43.332 INFO:teuthology.orchestra.run.trial035.stderr:osd.3: osd_enable_op_tracker = '' 2026-02-24T07:52:43.994 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 49 15 316531 316516 403.688 439.312 0.00211894 0.00247603 2026-02-24T07:52:44.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:52:43 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/2802644867' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2026-02-24T07:52:44.995 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 50 15 323184 323169 403.93 415.812 0.00198847 0.00247454 2026-02-24T07:52:45.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:52:44 trial035 ceph-mon[39284]: pgmap v346: 17 pgs: 17 active+clean; 18 GiB data, 59 GiB used, 2.7 TiB / 2.7 TiB avail; 500 MiB/s wr, 8.00k op/s 2026-02-24T07:52:45.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:52:44 trial035 ceph-mon[39284]: from='mgr.14284 10.20.193.35:0/3681373222' entity='mgr.x' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2026-02-24T07:52:45.376 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2026-02-24T07:52:45.515 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:52:45.642 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:52:45.639+0000 7fb55bdf8640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:52:45.642 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:52:45.640+0000 7fb55bdf8640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:52:45.656 INFO:teuthology.orchestra.run.trial035.stdout:osd.0: {} 2026-02-24T07:52:45.656 INFO:teuthology.orchestra.run.trial035.stderr:osd.0: osd_enable_op_tracker = '' 2026-02-24T07:52:45.669 INFO:teuthology.orchestra.run.trial035.stdout:osd.1: {} 2026-02-24T07:52:45.669 INFO:teuthology.orchestra.run.trial035.stderr:osd.1: osd_enable_op_tracker = '' 2026-02-24T07:52:45.681 INFO:teuthology.orchestra.run.trial035.stdout:osd.2: {} 2026-02-24T07:52:45.681 INFO:teuthology.orchestra.run.trial035.stderr:osd.2: osd_enable_op_tracker = '' 2026-02-24T07:52:45.694 INFO:teuthology.orchestra.run.trial035.stdout:osd.3: {} 2026-02-24T07:52:45.695 INFO:teuthology.orchestra.run.trial035.stderr:osd.3: osd_enable_op_tracker = '' 2026-02-24T07:52:45.954 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:52:45 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/2438578431' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2026-02-24T07:52:45.995 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 51 16 329828 329812 404.15 415.188 0.00198388 0.0024732 2026-02-24T07:52:46.995 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 52 16 336711 336695 404.65 430.188 0.00170077 0.00247014 2026-02-24T07:52:47.200 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2026-02-24T07:52:47.200 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2026-02-24T07:52:47.200 INFO:tasks.thrashosds.thrasher:fixing pg num pool unique_pool_3 2026-02-24T07:52:47.201 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph pg dump --format=json 2026-02-24T07:52:47.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:52:46 trial035 ceph-mon[39284]: pgmap v347: 17 pgs: 17 active+clean; 18 GiB data, 59 GiB used, 2.7 TiB / 2.7 TiB avail; 252 MiB/s wr, 4.04k op/s 2026-02-24T07:52:47.204 INFO:journalctl@ceph.mgr.x.trial035.stdout:Feb 24 07:52:46 trial035 ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-mgr-x[39621]: ::ffff:10.20.193.91 - - [24/Feb/2026:07:52:46] "GET /metrics HTTP/1.1" 200 4214 "" "Prometheus/3.6.0" 2026-02-24T07:52:47.265 INFO:journalctl@ceph.prometheus.a.trial091.stdout:Feb 24 07:52:46 trial091 ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-prometheus-a[66007]: time=2026-02-24T07:52:46.973Z level=ERROR source=scrape.go:1630 msg="Failed to determine correct type of scrape target." component="scrape manager" scrape_pool=ceph target=http://trial035:9283/metrics content_type="text/html;charset=utf-8" fallback_media_type="" err="received unsupported Content-Type \"text/html;charset=utf-8\" and no fallback_scrape_protocol specified for target" 2026-02-24T07:52:47.340 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:52:47.463 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:52:47.460+0000 7f3f1eeea640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:52:47.463 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:52:47.461+0000 7f3f1eeea640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:52:47.601 INFO:teuthology.orchestra.run.trial035.stdout: 2026-02-24T07:52:47.602 INFO:teuthology.orchestra.run.trial035.stderr:dumped all 2026-02-24T07:52:47.644 INFO:teuthology.orchestra.run.trial035.stdout:{"pg_ready":true,"pg_map":{"version":348,"stamp":"2026-02-24T07:52:46.904698+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":21517042208,"num_objects":328317,"num_object_clones":0,"num_object_copies":984951,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":328317,"num_whiteouts":0,"num_read":136,"num_read_kb":240,"num_write":328450,"num_write_kb":21014310,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":51,"ondisk_log_size":51,"up":51,"acting":51,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":51,"num_osds":4,"num_per_pool_osds":4,"num_per_pool_omap_osds":4,"kb":2936012800,"kb_used":68104236,"kb_used_data":62178908,"kb_used_omap":88,"kb_used_meta":5925159,"kb_avail":2867908564,"statfs":{"total":3006477107200,"available":2936738369536,"internally_reserved":0,"allocated":63671201792,"data_stored":63668957004,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":90402,"internal_metadata":6067363550},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[11,0,2],"upper_bound":8},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":4317577216,"num_objects":65881,"num_object_clones":0,"num_object_copies":197643,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":65881,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":65881,"num_write_kb":4216384,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.010989"},"pg_stats":[{"pgid":"5.f","version":"80'20339","reported_seq":20361,"reported_epoch":80,"state":"active+clean","last_fresh":"2026-02-24T07:52:45.746256+0000","last_change":"2026-02-24T07:51:53.900797+0000","last_active":"2026-02-24T07:52:45.746256+0000","last_peered":"2026-02-24T07:52:45.746256+0000","last_clean":"2026-02-24T07:52:45.746256+0000","last_became_active":"2026-02-24T07:51:53.900431+0000","last_became_peered":"2026-02-24T07:51:53.900431+0000","last_unstale":"2026-02-24T07:52:45.746256+0000","last_undegraded":"2026-02-24T07:52:45.746256+0000","last_fullsized":"2026-02-24T07:52:45.746256+0000","mapping_epoch":74,"log_start":"80'20336","ondisk_log_start":"80'20336","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_clean_scrub_stamp":"2026-02-24T07:51:52.890170+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:52:57.614871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1332936704,"num_objects":20339,"num_object_clones":0,"num_object_copies":61017,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20339,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20339,"num_write_kb":1301696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.e","version":"80'20430","reported_seq":20452,"reported_epoch":80,"state":"active+clean","last_fresh":"2026-02-24T07:52:45.755777+0000","last_change":"2026-02-24T07:51:53.899686+0000","last_active":"2026-02-24T07:52:45.755777+0000","last_peered":"2026-02-24T07:52:45.755777+0000","last_clean":"2026-02-24T07:52:45.755777+0000","last_became_active":"2026-02-24T07:51:53.899294+0000","last_became_peered":"2026-02-24T07:51:53.899294+0000","last_unstale":"2026-02-24T07:52:45.755777+0000","last_undegraded":"2026-02-24T07:52:45.755777+0000","last_fullsized":"2026-02-24T07:52:45.755777+0000","mapping_epoch":74,"log_start":"80'20427","ondisk_log_start":"80'20427","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_clean_scrub_stamp":"2026-02-24T07:51:52.890170+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:52:53.084042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1338900480,"num_objects":20430,"num_object_clones":0,"num_object_copies":61290,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20430,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20430,"num_write_kb":1307520,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"80'20508","reported_seq":20530,"reported_epoch":80,"state":"active+clean","last_fresh":"2026-02-24T07:52:45.760734+0000","last_change":"2026-02-24T07:51:53.900129+0000","last_active":"2026-02-24T07:52:45.760734+0000","last_peered":"2026-02-24T07:52:45.760734+0000","last_clean":"2026-02-24T07:52:45.760734+0000","last_became_active":"2026-02-24T07:51:53.899772+0000","last_became_peered":"2026-02-24T07:51:53.899772+0000","last_unstale":"2026-02-24T07:52:45.760734+0000","last_undegraded":"2026-02-24T07:52:45.760734+0000","last_fullsized":"2026-02-24T07:52:45.760734+0000","mapping_epoch":74,"log_start":"80'20505","ondisk_log_start":"80'20505","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_clean_scrub_stamp":"2026-02-24T07:51:52.890170+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:53:19.516698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1344012288,"num_objects":20508,"num_object_clones":0,"num_object_copies":61524,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20508,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20508,"num_write_kb":1312512,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,3],"acting":[0,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.c","version":"80'20468","reported_seq":20490,"reported_epoch":80,"state":"active+clean","last_fresh":"2026-02-24T07:52:45.847588+0000","last_change":"2026-02-24T07:51:53.898484+0000","last_active":"2026-02-24T07:52:45.847588+0000","last_peered":"2026-02-24T07:52:45.847588+0000","last_clean":"2026-02-24T07:52:45.847588+0000","last_became_active":"2026-02-24T07:51:53.898285+0000","last_became_peered":"2026-02-24T07:51:53.898285+0000","last_unstale":"2026-02-24T07:52:45.847588+0000","last_undegraded":"2026-02-24T07:52:45.847588+0000","last_fullsized":"2026-02-24T07:52:45.847588+0000","mapping_epoch":74,"log_start":"80'20465","ondisk_log_start":"80'20465","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_clean_scrub_stamp":"2026-02-24T07:51:52.890170+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:53:11.947668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1341390848,"num_objects":20468,"num_object_clones":0,"num_object_copies":61404,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20468,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20468,"num_write_kb":1309952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.b","version":"80'20703","reported_seq":20725,"reported_epoch":80,"state":"active+clean","last_fresh":"2026-02-24T07:52:45.763951+0000","last_change":"2026-02-24T07:51:53.900298+0000","last_active":"2026-02-24T07:52:45.763951+0000","last_peered":"2026-02-24T07:52:45.763951+0000","last_clean":"2026-02-24T07:52:45.763951+0000","last_became_active":"2026-02-24T07:51:53.899901+0000","last_became_peered":"2026-02-24T07:51:53.899901+0000","last_unstale":"2026-02-24T07:52:45.763951+0000","last_undegraded":"2026-02-24T07:52:45.763951+0000","last_fullsized":"2026-02-24T07:52:45.763951+0000","mapping_epoch":74,"log_start":"80'20700","ondisk_log_start":"80'20700","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_clean_scrub_stamp":"2026-02-24T07:51:52.890170+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:53:12.321784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1356791808,"num_objects":20703,"num_object_clones":0,"num_object_copies":62109,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20703,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20703,"num_write_kb":1324992,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"80'20457","reported_seq":20479,"reported_epoch":80,"state":"active+clean","last_fresh":"2026-02-24T07:52:45.760609+0000","last_change":"2026-02-24T07:51:53.899828+0000","last_active":"2026-02-24T07:52:45.760609+0000","last_peered":"2026-02-24T07:52:45.760609+0000","last_clean":"2026-02-24T07:52:45.760609+0000","last_became_active":"2026-02-24T07:51:53.899549+0000","last_became_peered":"2026-02-24T07:51:53.899549+0000","last_unstale":"2026-02-24T07:52:45.760609+0000","last_undegraded":"2026-02-24T07:52:45.760609+0000","last_fullsized":"2026-02-24T07:52:45.760609+0000","mapping_epoch":74,"log_start":"80'20454","ondisk_log_start":"80'20454","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_clean_scrub_stamp":"2026-02-24T07:51:52.890170+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:53:07.446496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1340669952,"num_objects":20457,"num_object_clones":0,"num_object_copies":61371,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20457,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20457,"num_write_kb":1309248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,1],"acting":[0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.9","version":"80'20780","reported_seq":20802,"reported_epoch":80,"state":"active+clean","last_fresh":"2026-02-24T07:52:45.754937+0000","last_change":"2026-02-24T07:51:53.901985+0000","last_active":"2026-02-24T07:52:45.754937+0000","last_peered":"2026-02-24T07:52:45.754937+0000","last_clean":"2026-02-24T07:52:45.754937+0000","last_became_active":"2026-02-24T07:51:53.901610+0000","last_became_peered":"2026-02-24T07:51:53.901610+0000","last_unstale":"2026-02-24T07:52:45.754937+0000","last_undegraded":"2026-02-24T07:52:45.754937+0000","last_fullsized":"2026-02-24T07:52:45.754937+0000","mapping_epoch":74,"log_start":"80'20777","ondisk_log_start":"80'20777","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_clean_scrub_stamp":"2026-02-24T07:51:52.890170+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:53:13.073570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1361838080,"num_objects":20780,"num_object_clones":0,"num_object_copies":62340,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20780,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20780,"num_write_kb":1329920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.8","version":"80'20495","reported_seq":20517,"reported_epoch":80,"state":"active+clean","last_fresh":"2026-02-24T07:52:45.766312+0000","last_change":"2026-02-24T07:51:53.900477+0000","last_active":"2026-02-24T07:52:45.766312+0000","last_peered":"2026-02-24T07:52:45.766312+0000","last_clean":"2026-02-24T07:52:45.766312+0000","last_became_active":"2026-02-24T07:51:53.900207+0000","last_became_peered":"2026-02-24T07:51:53.900207+0000","last_unstale":"2026-02-24T07:52:45.766312+0000","last_undegraded":"2026-02-24T07:52:45.766312+0000","last_fullsized":"2026-02-24T07:52:45.766312+0000","mapping_epoch":74,"log_start":"80'20492","ondisk_log_start":"80'20492","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_clean_scrub_stamp":"2026-02-24T07:51:52.890170+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:52:57.062307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1343160320,"num_objects":20495,"num_object_clones":0,"num_object_copies":61485,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20495,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20495,"num_write_kb":1311680,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.7","version":"80'20484","reported_seq":20506,"reported_epoch":80,"state":"active+clean","last_fresh":"2026-02-24T07:52:45.760548+0000","last_change":"2026-02-24T07:51:53.899854+0000","last_active":"2026-02-24T07:52:45.760548+0000","last_peered":"2026-02-24T07:52:45.760548+0000","last_clean":"2026-02-24T07:52:45.760548+0000","last_became_active":"2026-02-24T07:51:53.899472+0000","last_became_peered":"2026-02-24T07:51:53.899472+0000","last_unstale":"2026-02-24T07:52:45.760548+0000","last_undegraded":"2026-02-24T07:52:45.760548+0000","last_fullsized":"2026-02-24T07:52:45.760548+0000","mapping_epoch":74,"log_start":"80'20481","ondisk_log_start":"80'20481","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_clean_scrub_stamp":"2026-02-24T07:51:52.890170+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:52:55.115725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1342439424,"num_objects":20484,"num_object_clones":0,"num_object_copies":61452,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20484,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20484,"num_write_kb":1310976,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.6","version":"80'20424","reported_seq":20446,"reported_epoch":80,"state":"active+clean","last_fresh":"2026-02-24T07:52:45.848708+0000","last_change":"2026-02-24T07:51:53.898841+0000","last_active":"2026-02-24T07:52:45.848708+0000","last_peered":"2026-02-24T07:52:45.848708+0000","last_clean":"2026-02-24T07:52:45.848708+0000","last_became_active":"2026-02-24T07:51:53.898308+0000","last_became_peered":"2026-02-24T07:51:53.898308+0000","last_unstale":"2026-02-24T07:52:45.848708+0000","last_undegraded":"2026-02-24T07:52:45.848708+0000","last_fullsized":"2026-02-24T07:52:45.848708+0000","mapping_epoch":74,"log_start":"80'20421","ondisk_log_start":"80'20421","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_clean_scrub_stamp":"2026-02-24T07:51:52.890170+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:53:04.955186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1338507264,"num_objects":20424,"num_object_clones":0,"num_object_copies":61272,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20424,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20424,"num_write_kb":1307136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.5","version":"80'20555","reported_seq":20577,"reported_epoch":80,"state":"active+clean","last_fresh":"2026-02-24T07:52:45.768309+0000","last_change":"2026-02-24T07:51:53.900139+0000","last_active":"2026-02-24T07:52:45.768309+0000","last_peered":"2026-02-24T07:52:45.768309+0000","last_clean":"2026-02-24T07:52:45.768309+0000","last_became_active":"2026-02-24T07:51:53.899749+0000","last_became_peered":"2026-02-24T07:51:53.899749+0000","last_unstale":"2026-02-24T07:52:45.768309+0000","last_undegraded":"2026-02-24T07:52:45.768309+0000","last_fullsized":"2026-02-24T07:52:45.768309+0000","mapping_epoch":74,"log_start":"80'20552","ondisk_log_start":"80'20552","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_clean_scrub_stamp":"2026-02-24T07:51:52.890170+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:53:09.489880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1347092480,"num_objects":20555,"num_object_clones":0,"num_object_copies":61665,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20555,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20555,"num_write_kb":1315520,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"80'20421","reported_seq":20443,"reported_epoch":80,"state":"active+clean","last_fresh":"2026-02-24T07:52:45.763897+0000","last_change":"2026-02-24T07:51:53.900361+0000","last_active":"2026-02-24T07:52:45.763897+0000","last_peered":"2026-02-24T07:52:45.763897+0000","last_clean":"2026-02-24T07:52:45.763897+0000","last_became_active":"2026-02-24T07:51:53.900042+0000","last_became_peered":"2026-02-24T07:51:53.900042+0000","last_unstale":"2026-02-24T07:52:45.763897+0000","last_undegraded":"2026-02-24T07:52:45.763897+0000","last_fullsized":"2026-02-24T07:52:45.763897+0000","mapping_epoch":74,"log_start":"80'20418","ondisk_log_start":"80'20418","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_clean_scrub_stamp":"2026-02-24T07:51:52.890170+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:53:02.182725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1338310656,"num_objects":20421,"num_object_clones":0,"num_object_copies":61263,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20421,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20421,"num_write_kb":1306944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.2","version":"80'20691","reported_seq":20713,"reported_epoch":80,"state":"active+clean","last_fresh":"2026-02-24T07:52:45.755619+0000","last_change":"2026-02-24T07:51:53.900424+0000","last_active":"2026-02-24T07:52:45.755619+0000","last_peered":"2026-02-24T07:52:45.755619+0000","last_clean":"2026-02-24T07:52:45.755619+0000","last_became_active":"2026-02-24T07:51:53.900096+0000","last_became_peered":"2026-02-24T07:51:53.900096+0000","last_unstale":"2026-02-24T07:52:45.755619+0000","last_undegraded":"2026-02-24T07:52:45.755619+0000","last_fullsized":"2026-02-24T07:52:45.755619+0000","mapping_epoch":74,"log_start":"80'20688","ondisk_log_start":"80'20688","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_clean_scrub_stamp":"2026-02-24T07:51:52.890170+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:53:10.802005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1356005376,"num_objects":20691,"num_object_clones":0,"num_object_copies":62073,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20691,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20691,"num_write_kb":1324224,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"80'20280","reported_seq":20302,"reported_epoch":80,"state":"active+clean","last_fresh":"2026-02-24T07:52:45.757992+0000","last_change":"2026-02-24T07:51:53.901421+0000","last_active":"2026-02-24T07:52:45.757992+0000","last_peered":"2026-02-24T07:52:45.757992+0000","last_clean":"2026-02-24T07:52:45.757992+0000","last_became_active":"2026-02-24T07:51:53.901027+0000","last_became_peered":"2026-02-24T07:51:53.901027+0000","last_unstale":"2026-02-24T07:52:45.757992+0000","last_undegraded":"2026-02-24T07:52:45.757992+0000","last_fullsized":"2026-02-24T07:52:45.757992+0000","mapping_epoch":74,"log_start":"80'20277","ondisk_log_start":"80'20277","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_clean_scrub_stamp":"2026-02-24T07:51:52.890170+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:53:04.436560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1329070080,"num_objects":20280,"num_object_clones":0,"num_object_copies":60840,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20280,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20280,"num_write_kb":1297920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.0","version":"80'20798","reported_seq":20820,"reported_epoch":80,"state":"active+clean","last_fresh":"2026-02-24T07:52:45.769788+0000","last_change":"2026-02-24T07:51:53.898740+0000","last_active":"2026-02-24T07:52:45.769788+0000","last_peered":"2026-02-24T07:52:45.769788+0000","last_clean":"2026-02-24T07:52:45.769788+0000","last_became_active":"2026-02-24T07:51:53.898551+0000","last_became_peered":"2026-02-24T07:51:53.898551+0000","last_unstale":"2026-02-24T07:52:45.769788+0000","last_undegraded":"2026-02-24T07:52:45.769788+0000","last_fullsized":"2026-02-24T07:52:45.769788+0000","mapping_epoch":74,"log_start":"80'20795","ondisk_log_start":"80'20795","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_clean_scrub_stamp":"2026-02-24T07:51:52.890170+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:53:07.664864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1363017728,"num_objects":20798,"num_object_clones":0,"num_object_copies":62394,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20798,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20798,"num_write_kb":1331072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.4","version":"80'20482","reported_seq":20503,"reported_epoch":80,"state":"active+clean","last_fresh":"2026-02-24T07:52:45.745360+0000","last_change":"2026-02-24T07:51:53.900179+0000","last_active":"2026-02-24T07:52:45.745360+0000","last_peered":"2026-02-24T07:52:45.745360+0000","last_clean":"2026-02-24T07:52:45.745360+0000","last_became_active":"2026-02-24T07:51:53.899675+0000","last_became_peered":"2026-02-24T07:51:53.899675+0000","last_unstale":"2026-02-24T07:52:45.745360+0000","last_undegraded":"2026-02-24T07:52:45.745360+0000","last_fullsized":"2026-02-24T07:52:45.745360+0000","mapping_epoch":74,"log_start":"80'20479","ondisk_log_start":"80'20479","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_clean_scrub_stamp":"2026-02-24T07:51:52.890170+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:53:06.339989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1342308352,"num_objects":20482,"num_object_clones":0,"num_object_copies":61446,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20482,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20482,"num_write_kb":1310848,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"29'99","reported_seq":340,"reported_epoch":80,"state":"active+clean","last_fresh":"2026-02-24T07:52:38.889411+0000","last_change":"2026-02-24T07:51:54.101629+0000","last_active":"2026-02-24T07:52:38.889411+0000","last_peered":"2026-02-24T07:52:38.889411+0000","last_clean":"2026-02-24T07:52:38.889411+0000","last_became_active":"2026-02-24T07:42:54.186719+0000","last_became_peered":"2026-02-24T07:42:54.186719+0000","last_unstale":"2026-02-24T07:52:38.889411+0000","last_undegraded":"2026-02-24T07:52:38.889411+0000","last_fullsized":"2026-02-24T07:52:38.889411+0000","mapping_epoch":27,"log_start":"29'96","ondisk_log_start":"29'96","created":20,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'99","last_scrub_stamp":"2026-02-24T07:51:54.101595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:42:43.640776+0000","last_clean_scrub_stamp":"2026-02-24T07:51:54.101595+0000","objects_scrubbed":2,"log_size":3,"log_dups_size":96,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:52:54.995630+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":240,"num_write":135,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":16,"stat_sum":{"num_bytes":21516451840,"num_objects":328315,"num_object_clones":0,"num_object_copies":984945,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":328315,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":328315,"num_write_kb":21012160,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":64559841280,"data_stored":64559841280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":48,"ondisk_log_size":48,"up":48,"acting":48,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":240,"num_write":135,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1682,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":3,"up_from":25,"seq":107374182521,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":19677868,"kb_used_data":18136792,"kb_used_omap":23,"kb_used_meta":1541032,"kb_avail":714325332,"statfs":{"total":751619276800,"available":731469139968,"internally_reserved":0,"allocated":18572075008,"data_stored":18571512923,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":24172,"internal_metadata":1578017172},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[8],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":19,"seq":81604378747,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":15725676,"kb_used_data":14267544,"kb_used_omap":23,"kb_used_meta":1458088,"kb_avail":718277524,"statfs":{"total":751619276800,"available":735516184576,"internally_reserved":0,"allocated":14609965056,"data_stored":14609402971,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":24547,"internal_metadata":1493082141},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542268,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":14264040,"kb_used_data":13029012,"kb_used_omap":20,"kb_used_meta":1234987,"kb_avail":719739160,"statfs":{"total":751619276800,"available":737012899840,"internally_reserved":0,"allocated":13341708288,"data_stored":13341149755,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":20777,"internal_metadata":1264627415},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,0,2],"upper_bound":8},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705791,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":18436652,"kb_used_data":16745560,"kb_used_omap":20,"kb_used_meta":1691051,"kb_avail":715566548,"statfs":{"total":751619276800,"available":732740145152,"internally_reserved":0,"allocated":17147453440,"data_stored":17146891355,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":20906,"internal_metadata":1731636822},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1682,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":17474650112,"data_stored":17474650112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":13478264832,"data_stored":13478264832,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":14796718080,"data_stored":14796718080,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":18810208256,"data_stored":18810208256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-24T07:52:47.646 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph osd pool set unique_pool_3 pgp_num 26 2026-02-24T07:52:47.739 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2026-02-24T07:52:47.782 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:52:47.883 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:52:47.913 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:52:47.911+0000 7f7f0d650640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:52:47.914 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:52:47.912+0000 7f7f0d650640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:52:47.995 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 53 16 343130 343114 404.584 401.188 0.00368036 0.00247052 2026-02-24T07:52:48.006 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:52:48.004+0000 7f84714a4640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:52:48.006 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:52:48.004+0000 7f84714a4640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:52:48.022 INFO:teuthology.orchestra.run.trial035.stdout:osd.0: {} 2026-02-24T07:52:48.022 INFO:teuthology.orchestra.run.trial035.stderr:osd.0: osd_enable_op_tracker = '' 2026-02-24T07:52:48.035 INFO:teuthology.orchestra.run.trial035.stdout:osd.1: {} 2026-02-24T07:52:48.035 INFO:teuthology.orchestra.run.trial035.stderr:osd.1: osd_enable_op_tracker = '' 2026-02-24T07:52:48.046 INFO:teuthology.orchestra.run.trial035.stderr:Error EPERM: crimson_allow_pg_split is false; pgp_num increase denied 2026-02-24T07:52:48.052 INFO:teuthology.orchestra.run.trial035.stdout:osd.2: {} 2026-02-24T07:52:48.052 INFO:teuthology.orchestra.run.trial035.stderr:osd.2: osd_enable_op_tracker = '' 2026-02-24T07:52:48.067 INFO:teuthology.orchestra.run.trial035.stdout:osd.3: {} 2026-02-24T07:52:48.068 INFO:teuthology.orchestra.run.trial035.stderr:osd.3: osd_enable_op_tracker = '' 2026-02-24T07:52:48.091 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-24T07:52:48.112 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:52:47 trial035 ceph-mon[39284]: osdmap e81: 4 total, 4 up, 4 in 2026-02-24T07:52:48.995 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 54 16 349398 349382 404.346 391.75 0.002251 0.00247203 2026-02-24T07:52:49.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:52:48 trial035 ceph-mon[39284]: from='client.15220 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2026-02-24T07:52:49.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:52:48 trial035 ceph-mon[39284]: pgmap v349: 17 pgs: 17 active+clean; 20 GiB data, 65 GiB used, 2.7 TiB / 2.7 TiB avail; 465 MiB/s wr, 7.45k op/s 2026-02-24T07:52:49.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:52:48 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/2495584808' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2026-02-24T07:52:49.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:52:48 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/3596823396' entity='client.admin' cmd={"prefix": "osd pool set", "pool": "unique_pool_3", "var": "pgp_num", "val": "26"} : dispatch 2026-02-24T07:52:49.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:52:48 trial035 ceph-mon[39284]: osdmap e82: 4 total, 4 up, 4 in 2026-02-24T07:52:49.995 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 55 16 355859 355843 404.336 403.812 0.00206311 0.00247204 2026-02-24T07:52:50.114 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2026-02-24T07:52:50.253 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:52:50.375 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:52:50.374+0000 7fa2b3cf8640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:52:50.376 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:52:50.375+0000 7fa2b3cf8640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:52:50.392 INFO:teuthology.orchestra.run.trial035.stdout:osd.0: {} 2026-02-24T07:52:50.392 INFO:teuthology.orchestra.run.trial035.stderr:osd.0: osd_enable_op_tracker = '' 2026-02-24T07:52:50.406 INFO:teuthology.orchestra.run.trial035.stdout:osd.1: {} 2026-02-24T07:52:50.406 INFO:teuthology.orchestra.run.trial035.stderr:osd.1: osd_enable_op_tracker = '' 2026-02-24T07:52:50.418 INFO:teuthology.orchestra.run.trial035.stdout:osd.2: {} 2026-02-24T07:52:50.418 INFO:teuthology.orchestra.run.trial035.stderr:osd.2: osd_enable_op_tracker = '' 2026-02-24T07:52:50.431 INFO:teuthology.orchestra.run.trial035.stdout:osd.3: {} 2026-02-24T07:52:50.431 INFO:teuthology.orchestra.run.trial035.stderr:osd.3: osd_enable_op_tracker = '' 2026-02-24T07:52:50.995 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 56 16 362624 362608 404.665 422.812 0.00236419 0.00247004 2026-02-24T07:52:51.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:52:50 trial035 ceph-mon[39284]: pgmap v351: 17 pgs: 17 active+clean; 20 GiB data, 65 GiB used, 2.7 TiB / 2.7 TiB avail; 262 MiB/s wr, 4.20k op/s 2026-02-24T07:52:51.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:52:50 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/3120526161' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2026-02-24T07:52:51.995 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 57 16 369536 369520 405.144 432 0.00158205 0.00246713 2026-02-24T07:52:52.475 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2026-02-24T07:52:52.617 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:52:52.996 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 58 16 375314 375298 404.385 361.125 0.00208268 0.00247172 2026-02-24T07:52:53.008 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:52:52.966+0000 7f1e2dc15640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:52:53.008 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:52:52.966+0000 7f1e2dc15640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:52:53.008 INFO:teuthology.orchestra.run.trial035.stderr:osd.0: osd_enable_op_tracker = '' 2026-02-24T07:52:53.008 INFO:teuthology.orchestra.run.trial035.stderr:osd.1: osd_enable_op_tracker = '' 2026-02-24T07:52:53.008 INFO:teuthology.orchestra.run.trial035.stdout:osd.0: {} 2026-02-24T07:52:53.008 INFO:teuthology.orchestra.run.trial035.stdout:osd.1: {} 2026-02-24T07:52:53.012 INFO:teuthology.orchestra.run.trial035.stdout:osd.2: {} 2026-02-24T07:52:53.012 INFO:teuthology.orchestra.run.trial035.stderr:osd.2: osd_enable_op_tracker = '' 2026-02-24T07:52:53.029 INFO:teuthology.orchestra.run.trial035.stdout:osd.3: {} 2026-02-24T07:52:53.030 INFO:teuthology.orchestra.run.trial035.stderr:osd.3: osd_enable_op_tracker = '' 2026-02-24T07:52:53.092 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2026-02-24T07:52:53.092 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2026-02-24T07:52:53.092 INFO:tasks.thrashosds.thrasher:fixing pg num pool .mgr 2026-02-24T07:52:53.092 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph pg dump --format=json 2026-02-24T07:52:53.116 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:52:52 trial035 ceph-mon[39284]: pgmap v352: 17 pgs: 17 active+clean; 22 GiB data, 71 GiB used, 2.7 TiB / 2.7 TiB avail; 518 MiB/s wr, 8.29k op/s 2026-02-24T07:52:53.218 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:52:53.983 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:52:53.934+0000 7fc302aa8640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:52:53.983 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:52:53.934+0000 7fc302aa8640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:52:53.995 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 59 15 379739 379724 402.217 276.625 0.00432811 0.0024851 2026-02-24T07:52:54.072 INFO:teuthology.orchestra.run.trial035.stdout: 2026-02-24T07:52:54.072 INFO:teuthology.orchestra.run.trial035.stderr:dumped all 2026-02-24T07:52:54.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:52:53 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/3680644698' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2026-02-24T07:52:54.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:52:53 trial035 ceph-mon[39284]: pgmap v353: 17 pgs: 17 active+clean; 22 GiB data, 71 GiB used, 2.7 TiB / 2.7 TiB avail; 518 MiB/s wr, 8.29k op/s 2026-02-24T07:52:54.489 INFO:teuthology.orchestra.run.trial035.stdout:{"pg_ready":true,"pg_map":{"version":353,"stamp":"2026-02-24T07:52:53.738025+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":23663739424,"num_objects":361073,"num_object_clones":0,"num_object_copies":1083219,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":361073,"num_whiteouts":0,"num_read":136,"num_read_kb":240,"num_write":361206,"num_write_kb":23110694,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":55,"ondisk_log_size":55,"up":51,"acting":51,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":51,"num_osds":4,"num_per_pool_osds":4,"num_per_pool_omap_osds":4,"kb":2936012800,"kb_used":74789132,"kb_used_data":68466684,"kb_used_omap":92,"kb_used_meta":6322211,"kb_avail":2861223668,"statfs":{"total":3006477107200,"available":2929893036032,"internally_reserved":0,"allocated":70109884416,"data_stored":70107576732,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":94832,"internal_metadata":6473944464},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[6,2],"upper_bound":4},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":4346544128,"num_objects":66323,"num_object_clones":0,"num_object_copies":198969,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66323,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":66323,"num_write_kb":4244672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001146"},"pg_stats":[{"pgid":"5.f","version":"82'22396","reported_seq":22422,"reported_epoch":82,"state":"active+clean","last_fresh":"2026-02-24T07:52:50.741236+0000","last_change":"2026-02-24T07:51:53.900797+0000","last_active":"2026-02-24T07:52:50.741236+0000","last_peered":"2026-02-24T07:52:50.741236+0000","last_clean":"2026-02-24T07:52:50.741236+0000","last_became_active":"2026-02-24T07:51:53.900431+0000","last_became_peered":"2026-02-24T07:51:53.900431+0000","last_unstale":"2026-02-24T07:52:50.741236+0000","last_undegraded":"2026-02-24T07:52:50.741236+0000","last_fullsized":"2026-02-24T07:52:50.741236+0000","mapping_epoch":74,"log_start":"82'22393","ondisk_log_start":"82'22393","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_clean_scrub_stamp":"2026-02-24T07:51:52.890170+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:52:57.614871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1467744256,"num_objects":22396,"num_object_clones":0,"num_object_copies":67188,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22396,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22396,"num_write_kb":1433344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.e","version":"82'22521","reported_seq":22547,"reported_epoch":82,"state":"active+clean","last_fresh":"2026-02-24T07:52:50.757505+0000","last_change":"2026-02-24T07:51:53.899686+0000","last_active":"2026-02-24T07:52:50.757505+0000","last_peered":"2026-02-24T07:52:50.757505+0000","last_clean":"2026-02-24T07:52:50.757505+0000","last_became_active":"2026-02-24T07:51:53.899294+0000","last_became_peered":"2026-02-24T07:51:53.899294+0000","last_unstale":"2026-02-24T07:52:50.757505+0000","last_undegraded":"2026-02-24T07:52:50.757505+0000","last_fullsized":"2026-02-24T07:52:50.757505+0000","mapping_epoch":74,"log_start":"82'22518","ondisk_log_start":"82'22518","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_clean_scrub_stamp":"2026-02-24T07:51:52.890170+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:52:53.084042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1475936256,"num_objects":22521,"num_object_clones":0,"num_object_copies":67563,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22521,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22521,"num_write_kb":1441344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"82'22607","reported_seq":22633,"reported_epoch":82,"state":"active+clean","last_fresh":"2026-02-24T07:52:50.762637+0000","last_change":"2026-02-24T07:51:53.900129+0000","last_active":"2026-02-24T07:52:50.762637+0000","last_peered":"2026-02-24T07:52:50.762637+0000","last_clean":"2026-02-24T07:52:50.762637+0000","last_became_active":"2026-02-24T07:51:53.899772+0000","last_became_peered":"2026-02-24T07:51:53.899772+0000","last_unstale":"2026-02-24T07:52:50.762637+0000","last_undegraded":"2026-02-24T07:52:50.762637+0000","last_fullsized":"2026-02-24T07:52:50.762637+0000","mapping_epoch":74,"log_start":"82'22604","ondisk_log_start":"82'22604","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_clean_scrub_stamp":"2026-02-24T07:51:52.890170+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:53:19.516698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1481572352,"num_objects":22607,"num_object_clones":0,"num_object_copies":67821,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22607,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22607,"num_write_kb":1446848,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,3],"acting":[0,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.c","version":"82'22553","reported_seq":22579,"reported_epoch":82,"state":"active+clean","last_fresh":"2026-02-24T07:52:50.843900+0000","last_change":"2026-02-24T07:51:53.898484+0000","last_active":"2026-02-24T07:52:50.843900+0000","last_peered":"2026-02-24T07:52:50.843900+0000","last_clean":"2026-02-24T07:52:50.843900+0000","last_became_active":"2026-02-24T07:51:53.898285+0000","last_became_peered":"2026-02-24T07:51:53.898285+0000","last_unstale":"2026-02-24T07:52:50.843900+0000","last_undegraded":"2026-02-24T07:52:50.843900+0000","last_fullsized":"2026-02-24T07:52:50.843900+0000","mapping_epoch":74,"log_start":"82'22550","ondisk_log_start":"82'22550","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_clean_scrub_stamp":"2026-02-24T07:51:52.890170+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:53:11.947668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1478033408,"num_objects":22553,"num_object_clones":0,"num_object_copies":67659,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22553,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22553,"num_write_kb":1443392,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.b","version":"82'22734","reported_seq":22759,"reported_epoch":82,"state":"active+clean","last_fresh":"2026-02-24T07:52:50.770365+0000","last_change":"2026-02-24T07:51:53.900298+0000","last_active":"2026-02-24T07:52:50.770365+0000","last_peered":"2026-02-24T07:52:50.770365+0000","last_clean":"2026-02-24T07:52:50.770365+0000","last_became_active":"2026-02-24T07:51:53.899901+0000","last_became_peered":"2026-02-24T07:51:53.899901+0000","last_unstale":"2026-02-24T07:52:50.770365+0000","last_undegraded":"2026-02-24T07:52:50.770365+0000","last_fullsized":"2026-02-24T07:52:50.770365+0000","mapping_epoch":74,"log_start":"82'22731","ondisk_log_start":"82'22731","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_clean_scrub_stamp":"2026-02-24T07:51:52.890170+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:53:12.321784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1489895424,"num_objects":22734,"num_object_clones":0,"num_object_copies":68202,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22734,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22734,"num_write_kb":1454976,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"82'22533","reported_seq":22559,"reported_epoch":82,"state":"active+clean","last_fresh":"2026-02-24T07:52:50.762912+0000","last_change":"2026-02-24T07:51:53.899828+0000","last_active":"2026-02-24T07:52:50.762912+0000","last_peered":"2026-02-24T07:52:50.762912+0000","last_clean":"2026-02-24T07:52:50.762912+0000","last_became_active":"2026-02-24T07:51:53.899549+0000","last_became_peered":"2026-02-24T07:51:53.899549+0000","last_unstale":"2026-02-24T07:52:50.762912+0000","last_undegraded":"2026-02-24T07:52:50.762912+0000","last_fullsized":"2026-02-24T07:52:50.762912+0000","mapping_epoch":74,"log_start":"82'22529","ondisk_log_start":"82'22529","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_clean_scrub_stamp":"2026-02-24T07:51:52.890170+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":3000,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:53:07.446496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1476722688,"num_objects":22533,"num_object_clones":0,"num_object_copies":67599,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22533,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22533,"num_write_kb":1442112,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,1],"acting":[0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.9","version":"82'22855","reported_seq":22881,"reported_epoch":82,"state":"active+clean","last_fresh":"2026-02-24T07:52:50.755989+0000","last_change":"2026-02-24T07:51:53.901985+0000","last_active":"2026-02-24T07:52:50.755989+0000","last_peered":"2026-02-24T07:52:50.755989+0000","last_clean":"2026-02-24T07:52:50.755989+0000","last_became_active":"2026-02-24T07:51:53.901610+0000","last_became_peered":"2026-02-24T07:51:53.901610+0000","last_unstale":"2026-02-24T07:52:50.755989+0000","last_undegraded":"2026-02-24T07:52:50.755989+0000","last_fullsized":"2026-02-24T07:52:50.755989+0000","mapping_epoch":74,"log_start":"82'22852","ondisk_log_start":"82'22852","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_clean_scrub_stamp":"2026-02-24T07:51:52.890170+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:53:13.073570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1497825280,"num_objects":22855,"num_object_clones":0,"num_object_copies":68565,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22855,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22855,"num_write_kb":1462720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.8","version":"82'22529","reported_seq":22555,"reported_epoch":82,"state":"active+clean","last_fresh":"2026-02-24T07:52:50.768190+0000","last_change":"2026-02-24T07:51:53.900477+0000","last_active":"2026-02-24T07:52:50.768190+0000","last_peered":"2026-02-24T07:52:50.768190+0000","last_clean":"2026-02-24T07:52:50.768190+0000","last_became_active":"2026-02-24T07:51:53.900207+0000","last_became_peered":"2026-02-24T07:51:53.900207+0000","last_unstale":"2026-02-24T07:52:50.768190+0000","last_undegraded":"2026-02-24T07:52:50.768190+0000","last_fullsized":"2026-02-24T07:52:50.768190+0000","mapping_epoch":74,"log_start":"82'22525","ondisk_log_start":"82'22525","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_clean_scrub_stamp":"2026-02-24T07:51:52.890170+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":3000,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:52:57.062307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1476460544,"num_objects":22529,"num_object_clones":0,"num_object_copies":67587,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22529,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22529,"num_write_kb":1441856,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.7","version":"82'22516","reported_seq":22542,"reported_epoch":82,"state":"active+clean","last_fresh":"2026-02-24T07:52:50.759651+0000","last_change":"2026-02-24T07:51:53.899854+0000","last_active":"2026-02-24T07:52:50.759651+0000","last_peered":"2026-02-24T07:52:50.759651+0000","last_clean":"2026-02-24T07:52:50.759651+0000","last_became_active":"2026-02-24T07:51:53.899472+0000","last_became_peered":"2026-02-24T07:51:53.899472+0000","last_unstale":"2026-02-24T07:52:50.759651+0000","last_undegraded":"2026-02-24T07:52:50.759651+0000","last_fullsized":"2026-02-24T07:52:50.759651+0000","mapping_epoch":74,"log_start":"82'22513","ondisk_log_start":"82'22513","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_clean_scrub_stamp":"2026-02-24T07:51:52.890170+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:52:55.115725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1475608576,"num_objects":22516,"num_object_clones":0,"num_object_copies":67548,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22516,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22516,"num_write_kb":1441024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.6","version":"82'22465","reported_seq":22491,"reported_epoch":82,"state":"active+clean","last_fresh":"2026-02-24T07:52:50.849001+0000","last_change":"2026-02-24T07:51:53.898841+0000","last_active":"2026-02-24T07:52:50.849001+0000","last_peered":"2026-02-24T07:52:50.849001+0000","last_clean":"2026-02-24T07:52:50.849001+0000","last_became_active":"2026-02-24T07:51:53.898308+0000","last_became_peered":"2026-02-24T07:51:53.898308+0000","last_unstale":"2026-02-24T07:52:50.849001+0000","last_undegraded":"2026-02-24T07:52:50.849001+0000","last_fullsized":"2026-02-24T07:52:50.849001+0000","mapping_epoch":74,"log_start":"82'22462","ondisk_log_start":"82'22462","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_clean_scrub_stamp":"2026-02-24T07:51:52.890170+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:53:04.955186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1472266240,"num_objects":22465,"num_object_clones":0,"num_object_copies":67395,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22465,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22465,"num_write_kb":1437760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.5","version":"82'22517","reported_seq":22543,"reported_epoch":82,"state":"active+clean","last_fresh":"2026-02-24T07:52:50.768132+0000","last_change":"2026-02-24T07:51:53.900139+0000","last_active":"2026-02-24T07:52:50.768132+0000","last_peered":"2026-02-24T07:52:50.768132+0000","last_clean":"2026-02-24T07:52:50.768132+0000","last_became_active":"2026-02-24T07:51:53.899749+0000","last_became_peered":"2026-02-24T07:51:53.899749+0000","last_unstale":"2026-02-24T07:52:50.768132+0000","last_undegraded":"2026-02-24T07:52:50.768132+0000","last_fullsized":"2026-02-24T07:52:50.768132+0000","mapping_epoch":74,"log_start":"82'22513","ondisk_log_start":"82'22513","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_clean_scrub_stamp":"2026-02-24T07:51:52.890170+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":3000,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:53:09.489880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1475674112,"num_objects":22517,"num_object_clones":0,"num_object_copies":67551,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22517,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22517,"num_write_kb":1441088,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"82'22460","reported_seq":22484,"reported_epoch":82,"state":"active+clean","last_fresh":"2026-02-24T07:52:50.771519+0000","last_change":"2026-02-24T07:51:53.900361+0000","last_active":"2026-02-24T07:52:50.771519+0000","last_peered":"2026-02-24T07:52:50.771519+0000","last_clean":"2026-02-24T07:52:50.771519+0000","last_became_active":"2026-02-24T07:51:53.900042+0000","last_became_peered":"2026-02-24T07:51:53.900042+0000","last_unstale":"2026-02-24T07:52:50.771519+0000","last_undegraded":"2026-02-24T07:52:50.771519+0000","last_fullsized":"2026-02-24T07:52:50.771519+0000","mapping_epoch":74,"log_start":"82'22456","ondisk_log_start":"82'22456","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_clean_scrub_stamp":"2026-02-24T07:51:52.890170+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":3000,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:53:02.182725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1471938560,"num_objects":22460,"num_object_clones":0,"num_object_copies":67380,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22460,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22460,"num_write_kb":1437440,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.2","version":"82'22788","reported_seq":22814,"reported_epoch":82,"state":"active+clean","last_fresh":"2026-02-24T07:52:50.755467+0000","last_change":"2026-02-24T07:51:53.900424+0000","last_active":"2026-02-24T07:52:50.755467+0000","last_peered":"2026-02-24T07:52:50.755467+0000","last_clean":"2026-02-24T07:52:50.755467+0000","last_became_active":"2026-02-24T07:51:53.900096+0000","last_became_peered":"2026-02-24T07:51:53.900096+0000","last_unstale":"2026-02-24T07:52:50.755467+0000","last_undegraded":"2026-02-24T07:52:50.755467+0000","last_fullsized":"2026-02-24T07:52:50.755467+0000","mapping_epoch":74,"log_start":"82'22785","ondisk_log_start":"82'22785","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_clean_scrub_stamp":"2026-02-24T07:51:52.890170+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:53:10.802005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1493434368,"num_objects":22788,"num_object_clones":0,"num_object_copies":68364,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22788,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22788,"num_write_kb":1458432,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"82'22256","reported_seq":22282,"reported_epoch":82,"state":"active+clean","last_fresh":"2026-02-24T07:52:50.761458+0000","last_change":"2026-02-24T07:51:53.901421+0000","last_active":"2026-02-24T07:52:50.761458+0000","last_peered":"2026-02-24T07:52:50.761458+0000","last_clean":"2026-02-24T07:52:50.761458+0000","last_became_active":"2026-02-24T07:51:53.901027+0000","last_became_peered":"2026-02-24T07:51:53.901027+0000","last_unstale":"2026-02-24T07:52:50.761458+0000","last_undegraded":"2026-02-24T07:52:50.761458+0000","last_fullsized":"2026-02-24T07:52:50.761458+0000","mapping_epoch":74,"log_start":"82'22253","ondisk_log_start":"82'22253","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_clean_scrub_stamp":"2026-02-24T07:51:52.890170+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:53:04.436560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1458569216,"num_objects":22256,"num_object_clones":0,"num_object_copies":66768,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22256,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22256,"num_write_kb":1424384,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.0","version":"82'22804","reported_seq":22830,"reported_epoch":82,"state":"active+clean","last_fresh":"2026-02-24T07:52:50.768813+0000","last_change":"2026-02-24T07:51:53.898740+0000","last_active":"2026-02-24T07:52:50.768813+0000","last_peered":"2026-02-24T07:52:50.768813+0000","last_clean":"2026-02-24T07:52:50.768813+0000","last_became_active":"2026-02-24T07:51:53.898551+0000","last_became_peered":"2026-02-24T07:51:53.898551+0000","last_unstale":"2026-02-24T07:52:50.768813+0000","last_undegraded":"2026-02-24T07:52:50.768813+0000","last_fullsized":"2026-02-24T07:52:50.768813+0000","mapping_epoch":74,"log_start":"82'22801","ondisk_log_start":"82'22801","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_clean_scrub_stamp":"2026-02-24T07:51:52.890170+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:53:07.664864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1494482944,"num_objects":22804,"num_object_clones":0,"num_object_copies":68412,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22804,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22804,"num_write_kb":1459456,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.4","version":"82'22537","reported_seq":22563,"reported_epoch":82,"state":"active+clean","last_fresh":"2026-02-24T07:52:50.738427+0000","last_change":"2026-02-24T07:51:53.900179+0000","last_active":"2026-02-24T07:52:50.738427+0000","last_peered":"2026-02-24T07:52:50.738427+0000","last_clean":"2026-02-24T07:52:50.738427+0000","last_became_active":"2026-02-24T07:51:53.899675+0000","last_became_peered":"2026-02-24T07:51:53.899675+0000","last_unstale":"2026-02-24T07:52:50.738427+0000","last_undegraded":"2026-02-24T07:52:50.738427+0000","last_fullsized":"2026-02-24T07:52:50.738427+0000","mapping_epoch":74,"log_start":"82'22534","ondisk_log_start":"82'22534","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_clean_scrub_stamp":"2026-02-24T07:51:52.890170+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:53:06.339989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1476984832,"num_objects":22537,"num_object_clones":0,"num_object_copies":67611,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22537,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22537,"num_write_kb":1442368,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"29'99","reported_seq":344,"reported_epoch":82,"state":"active+clean","last_fresh":"2026-02-24T07:52:48.892469+0000","last_change":"2026-02-24T07:51:54.101629+0000","last_active":"2026-02-24T07:52:48.892469+0000","last_peered":"2026-02-24T07:52:48.892469+0000","last_clean":"2026-02-24T07:52:48.892469+0000","last_became_active":"2026-02-24T07:42:54.186719+0000","last_became_peered":"2026-02-24T07:42:54.186719+0000","last_unstale":"2026-02-24T07:52:48.892469+0000","last_undegraded":"2026-02-24T07:52:48.892469+0000","last_fullsized":"2026-02-24T07:52:48.892469+0000","mapping_epoch":27,"log_start":"29'96","ondisk_log_start":"29'96","created":20,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'99","last_scrub_stamp":"2026-02-24T07:51:54.101595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:42:43.640776+0000","last_clean_scrub_stamp":"2026-02-24T07:51:54.101595+0000","objects_scrubbed":2,"log_size":3,"log_dups_size":96,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:52:54.995630+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":240,"num_write":135,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":16,"stat_sum":{"num_bytes":23663149056,"num_objects":361071,"num_object_clones":0,"num_object_copies":1083213,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":361071,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":361071,"num_write_kb":23108544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":70995935232,"data_stored":70995935232,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":52,"ondisk_log_size":52,"up":48,"acting":48,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":240,"num_write":135,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1682,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":3,"up_from":25,"seq":107374182522,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":21577988,"kb_used_data":19928560,"kb_used_omap":24,"kb_used_meta":1649319,"kb_avail":712425212,"statfs":{"total":751619276800,"available":729523417088,"internally_reserved":0,"allocated":20406845440,"data_stored":20406267631,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":25472,"internal_metadata":1688902784},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":19,"seq":81604378748,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":17286020,"kb_used_data":15719600,"kb_used_omap":25,"kb_used_meta":1566374,"kb_avail":716717180,"statfs":{"total":751619276800,"available":733918392320,"internally_reserved":0,"allocated":16096870400,"data_stored":16096292591,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":25866,"internal_metadata":1603967734},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1],"upper_bound":4},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542269,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":15644672,"kb_used_data":14337452,"kb_used_omap":20,"kb_used_meta":1307179,"kb_avail":718358528,"statfs":{"total":751619276800,"available":735599132672,"internally_reserved":0,"allocated":14681550848,"data_stored":14680976591,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":20777,"internal_metadata":1338552023},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705792,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":20280452,"kb_used_data":18481072,"kb_used_omap":22,"kb_used_meta":1799337,"kb_avail":713722748,"statfs":{"total":751619276800,"available":730852093952,"internally_reserved":0,"allocated":18924617728,"data_stored":18924039919,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":22717,"internal_metadata":1842521923},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[5,1],"upper_bound":4},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1682,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":19216465920,"data_stored":19216465920,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":14818410496,"data_stored":14818410496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16273440768,"data_stored":16273440768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":20687618048,"data_stored":20687618048,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-24T07:52:54.491 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph osd pool set .mgr pgp_num 341 2026-02-24T07:52:54.626 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:52:54.996 INFO:tasks.radosbench.radosbench.0.trial035.stdout:2026-02-24T07:52:54.995792+0000 min lat: 0.000691325 max lat: 0.0442884 avg lat: 0.00249595 2026-02-24T07:52:54.996 INFO:tasks.radosbench.radosbench.0.trial035.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2026-02-24T07:52:54.996 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 60 15 384493 384478 400.465 297.125 0.00226901 0.00249595 2026-02-24T07:52:55.031 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:52:55.029+0000 7fd537717640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:52:55.031 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:52:55.030+0000 7fd537717640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:52:55.031 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:52:54 trial035 ceph-mon[39284]: 5.e scrub starts 2026-02-24T07:52:55.031 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:52:54 trial035 ceph-mon[39284]: from='client.15230 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2026-02-24T07:52:55.167 INFO:teuthology.orchestra.run.trial035.stderr:Error EPERM: crimson_allow_pg_split is false; pgp_num increase denied 2026-02-24T07:52:55.570 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-24T07:52:55.670 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2026-02-24T07:52:55.817 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:52:55.996 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 61 15 389664 389649 399.197 323.188 0.0022549 0.00250391 2026-02-24T07:52:56.183 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:52:55 trial035 ceph-mon[39284]: osdmap e83: 4 total, 4 up, 4 in 2026-02-24T07:52:56.183 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:52:55 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/1842029909' entity='client.admin' cmd={"prefix": "osd pool set", "pool": ".mgr", "var": "pgp_num", "val": "341"} : dispatch 2026-02-24T07:52:56.183 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:52:55 trial035 ceph-mon[39284]: pgmap v355: 17 pgs: 17 active+clean; 22 GiB data, 71 GiB used, 2.7 TiB / 2.7 TiB avail; 256 MiB/s wr, 4.09k op/s 2026-02-24T07:52:56.200 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:52:56.182+0000 7f5be7f6b640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:52:56.200 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:52:56.182+0000 7f5be7f6b640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:52:56.212 INFO:teuthology.orchestra.run.trial035.stdout:osd.0: {} 2026-02-24T07:52:56.212 INFO:teuthology.orchestra.run.trial035.stderr:osd.0: osd_enable_op_tracker = '' 2026-02-24T07:52:56.224 INFO:teuthology.orchestra.run.trial035.stdout:osd.1: {} 2026-02-24T07:52:56.225 INFO:teuthology.orchestra.run.trial035.stderr:osd.1: osd_enable_op_tracker = '' 2026-02-24T07:52:56.237 INFO:teuthology.orchestra.run.trial035.stdout:osd.2: {} 2026-02-24T07:52:56.237 INFO:teuthology.orchestra.run.trial035.stderr:osd.2: osd_enable_op_tracker = '' 2026-02-24T07:52:56.250 INFO:teuthology.orchestra.run.trial035.stdout:osd.3: {} 2026-02-24T07:52:56.250 INFO:teuthology.orchestra.run.trial035.stderr:osd.3: osd_enable_op_tracker = '' 2026-02-24T07:52:56.996 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 62 16 394489 394473 397.621 301.5 0.00326913 0.0025138 2026-02-24T07:52:57.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:52:56 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/2700562190' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2026-02-24T07:52:57.205 INFO:journalctl@ceph.mgr.x.trial035.stdout:Feb 24 07:52:56 trial035 ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-mgr-x[39621]: ::ffff:10.20.193.91 - - [24/Feb/2026:07:52:56] "GET /metrics HTTP/1.1" 200 4214 "" "Prometheus/3.6.0" 2026-02-24T07:52:57.356 INFO:journalctl@ceph.prometheus.a.trial091.stdout:Feb 24 07:52:56 trial091 ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-prometheus-a[66007]: time=2026-02-24T07:52:56.973Z level=ERROR source=scrape.go:1630 msg="Failed to determine correct type of scrape target." component="scrape manager" scrape_pool=ceph target=http://trial035:9283/metrics content_type="text/html;charset=utf-8" fallback_media_type="" err="received unsupported Content-Type \"text/html;charset=utf-8\" and no fallback_scrape_protocol specified for target" 2026-02-24T07:52:57.996 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 63 16 400853 400837 397.621 397.75 0.00103106 0.00251378 2026-02-24T07:52:58.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:52:57 trial035 ceph-mon[39284]: 5.e scrub ok 2026-02-24T07:52:58.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:52:57 trial035 ceph-mon[39284]: 1.0 scrub starts 2026-02-24T07:52:58.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:52:57 trial035 ceph-mon[39284]: 1.0 scrub ok 2026-02-24T07:52:58.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:52:57 trial035 ceph-mon[39284]: pgmap v356: 17 pgs: 1 active+clean+scrubbing, 16 active+clean; 24 GiB data, 76 GiB used, 2.7 TiB / 2.7 TiB avail; 425 MiB/s wr, 6.80k op/s 2026-02-24T07:52:58.812 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2026-02-24T07:52:58.946 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:52:58.997 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 64 16 406265 406249 396.693 338.25 0.0013119 0.00251968 2026-02-24T07:52:59.073 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:52:59.071+0000 7f954585f640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:52:59.073 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:52:59.072+0000 7f954585f640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:52:59.087 INFO:teuthology.orchestra.run.trial035.stdout:osd.0: {} 2026-02-24T07:52:59.088 INFO:teuthology.orchestra.run.trial035.stderr:osd.0: osd_enable_op_tracker = '' 2026-02-24T07:52:59.088 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:52:58 trial035 ceph-mon[39284]: 5.8 scrub starts 2026-02-24T07:52:59.088 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:52:58 trial035 ceph-mon[39284]: 5.7 scrub starts 2026-02-24T07:52:59.089 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:52:58 trial035 ceph-mon[39284]: osdmap e84: 4 total, 4 up, 4 in 2026-02-24T07:52:59.100 INFO:teuthology.orchestra.run.trial035.stdout:osd.1: {} 2026-02-24T07:52:59.101 INFO:teuthology.orchestra.run.trial035.stderr:osd.1: osd_enable_op_tracker = '' 2026-02-24T07:52:59.113 INFO:teuthology.orchestra.run.trial035.stdout:osd.2: {} 2026-02-24T07:52:59.113 INFO:teuthology.orchestra.run.trial035.stderr:osd.2: osd_enable_op_tracker = '' 2026-02-24T07:52:59.125 INFO:teuthology.orchestra.run.trial035.stdout:osd.3: {} 2026-02-24T07:52:59.125 INFO:teuthology.orchestra.run.trial035.stderr:osd.3: osd_enable_op_tracker = '' 2026-02-24T07:52:59.996 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 65 15 411481 411466 395.606 326.062 0.00309797 0.00252663 2026-02-24T07:53:00.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:52:59 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/3675690643' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2026-02-24T07:53:00.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:52:59 trial035 ceph-mon[39284]: 5.f scrub starts 2026-02-24T07:53:00.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:52:59 trial035 ceph-mon[39284]: pgmap v358: 17 pgs: 1 active+clean+scrubbing, 16 active+clean; 24 GiB data, 76 GiB used, 2.7 TiB / 2.7 TiB avail; 214 MiB/s wr, 3.43k op/s 2026-02-24T07:53:00.205 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:52:59 trial035 ceph-mon[39284]: from='mgr.14284 10.20.193.35:0/3681373222' entity='mgr.x' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2026-02-24T07:53:00.571 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2026-02-24T07:53:00.571 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2026-02-24T07:53:00.572 INFO:tasks.thrashosds.thrasher:Growing pool unique_pool_3 2026-02-24T07:53:00.572 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph pg dump --format=json 2026-02-24T07:53:00.710 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:53:00.826 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:53:00.825+0000 7f918c590640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:53:00.827 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:53:00.825+0000 7f918c590640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:53:00.963 INFO:teuthology.orchestra.run.trial035.stdout: 2026-02-24T07:53:00.963 INFO:teuthology.orchestra.run.trial035.stderr:dumped all 2026-02-24T07:53:00.997 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 66 15 417066 417051 394.901 349.062 0.00202042 0.00253117 2026-02-24T07:53:01.005 INFO:teuthology.orchestra.run.trial035.stdout:{"pg_ready":true,"pg_map":{"version":358,"stamp":"2026-02-24T07:52:59.738839+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":25461326368,"num_objects":388502,"num_object_clones":0,"num_object_copies":1165506,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":388502,"num_whiteouts":0,"num_read":136,"num_read_kb":240,"num_write":388635,"num_write_kb":24866150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":51,"ondisk_log_size":51,"up":51,"acting":51,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":51,"num_osds":4,"num_per_pool_osds":4,"num_per_pool_omap_osds":4,"kb":2936012800,"kb_used":80093232,"kb_used_data":73852768,"kb_used_omap":94,"kb_used_meta":6240289,"kb_avail":2855919568,"statfs":{"total":3006477107200,"available":2924461637632,"internally_reserved":0,"allocated":75625234432,"data_stored":75622903237,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":97207,"internal_metadata":6390056009},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[6,1,1,0,3,2,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1797586944,"num_objects":27429,"num_object_clones":0,"num_object_copies":82287,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27429,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":27429,"num_write_kb":1755456,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001139"},"pg_stats":[{"pgid":"5.f","version":"83'24168","reported_seq":24195,"reported_epoch":83,"state":"active+clean","last_fresh":"2026-02-24T07:52:55.745161+0000","last_change":"2026-02-24T07:51:53.900797+0000","last_active":"2026-02-24T07:52:55.745161+0000","last_peered":"2026-02-24T07:52:55.745161+0000","last_clean":"2026-02-24T07:52:55.745161+0000","last_became_active":"2026-02-24T07:51:53.900431+0000","last_became_peered":"2026-02-24T07:51:53.900431+0000","last_unstale":"2026-02-24T07:52:55.745161+0000","last_undegraded":"2026-02-24T07:52:55.745161+0000","last_fullsized":"2026-02-24T07:52:55.745161+0000","mapping_epoch":74,"log_start":"83'24165","ondisk_log_start":"83'24165","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_clean_scrub_stamp":"2026-02-24T07:51:52.890170+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:52:57.614871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1583874048,"num_objects":24168,"num_object_clones":0,"num_object_copies":72504,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24168,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24168,"num_write_kb":1546752,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.e","version":"83'24245","reported_seq":24431,"reported_epoch":83,"state":"active+clean+scrubbing","last_fresh":"2026-02-24T07:52:55.759705+0000","last_change":"2026-02-24T07:52:53.961844+0000","last_active":"2026-02-24T07:52:55.759705+0000","last_peered":"2026-02-24T07:52:55.759705+0000","last_clean":"2026-02-24T07:52:55.759705+0000","last_became_active":"2026-02-24T07:51:53.899294+0000","last_became_peered":"2026-02-24T07:51:53.899294+0000","last_unstale":"2026-02-24T07:52:55.759705+0000","last_undegraded":"2026-02-24T07:52:55.759705+0000","last_fullsized":"2026-02-24T07:52:55.759705+0000","mapping_epoch":74,"log_start":"83'24242","ondisk_log_start":"83'24242","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_clean_scrub_stamp":"2026-02-24T07:51:52.890170+0000","objects_scrubbed":15449,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"scrubbing for 2s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1588920320,"num_objects":24245,"num_object_clones":0,"num_object_copies":72735,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24245,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24245,"num_write_kb":1551680,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"83'24296","reported_seq":24324,"reported_epoch":83,"state":"active+clean","last_fresh":"2026-02-24T07:52:55.767816+0000","last_change":"2026-02-24T07:51:53.900129+0000","last_active":"2026-02-24T07:52:55.767816+0000","last_peered":"2026-02-24T07:52:55.767816+0000","last_clean":"2026-02-24T07:52:55.767816+0000","last_became_active":"2026-02-24T07:51:53.899772+0000","last_became_peered":"2026-02-24T07:51:53.899772+0000","last_unstale":"2026-02-24T07:52:55.767816+0000","last_undegraded":"2026-02-24T07:52:55.767816+0000","last_fullsized":"2026-02-24T07:52:55.767816+0000","mapping_epoch":74,"log_start":"83'24293","ondisk_log_start":"83'24293","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_clean_scrub_stamp":"2026-02-24T07:51:52.890170+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:53:19.516698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1592262656,"num_objects":24296,"num_object_clones":0,"num_object_copies":72888,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24296,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24296,"num_write_kb":1554944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,3],"acting":[0,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.c","version":"83'24302","reported_seq":24330,"reported_epoch":83,"state":"active+clean","last_fresh":"2026-02-24T07:52:55.849228+0000","last_change":"2026-02-24T07:51:53.898484+0000","last_active":"2026-02-24T07:52:55.849228+0000","last_peered":"2026-02-24T07:52:55.849228+0000","last_clean":"2026-02-24T07:52:55.849228+0000","last_became_active":"2026-02-24T07:51:53.898285+0000","last_became_peered":"2026-02-24T07:51:53.898285+0000","last_unstale":"2026-02-24T07:52:55.849228+0000","last_undegraded":"2026-02-24T07:52:55.849228+0000","last_fullsized":"2026-02-24T07:52:55.849228+0000","mapping_epoch":74,"log_start":"83'24299","ondisk_log_start":"83'24299","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_clean_scrub_stamp":"2026-02-24T07:51:52.890170+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:53:11.947668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1592655872,"num_objects":24302,"num_object_clones":0,"num_object_copies":72906,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24302,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24302,"num_write_kb":1555328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.b","version":"83'24380","reported_seq":24408,"reported_epoch":83,"state":"active+clean","last_fresh":"2026-02-24T07:52:55.769909+0000","last_change":"2026-02-24T07:51:53.900298+0000","last_active":"2026-02-24T07:52:55.769909+0000","last_peered":"2026-02-24T07:52:55.769909+0000","last_clean":"2026-02-24T07:52:55.769909+0000","last_became_active":"2026-02-24T07:51:53.899901+0000","last_became_peered":"2026-02-24T07:51:53.899901+0000","last_unstale":"2026-02-24T07:52:55.769909+0000","last_undegraded":"2026-02-24T07:52:55.769909+0000","last_fullsized":"2026-02-24T07:52:55.769909+0000","mapping_epoch":74,"log_start":"83'24377","ondisk_log_start":"83'24377","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_clean_scrub_stamp":"2026-02-24T07:51:52.890170+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:53:12.321784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1597767680,"num_objects":24380,"num_object_clones":0,"num_object_copies":73140,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24380,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24380,"num_write_kb":1560320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"83'24266","reported_seq":24294,"reported_epoch":83,"state":"active+clean","last_fresh":"2026-02-24T07:52:55.770621+0000","last_change":"2026-02-24T07:51:53.899828+0000","last_active":"2026-02-24T07:52:55.770621+0000","last_peered":"2026-02-24T07:52:55.770621+0000","last_clean":"2026-02-24T07:52:55.770621+0000","last_became_active":"2026-02-24T07:51:53.899549+0000","last_became_peered":"2026-02-24T07:51:53.899549+0000","last_unstale":"2026-02-24T07:52:55.770621+0000","last_undegraded":"2026-02-24T07:52:55.770621+0000","last_fullsized":"2026-02-24T07:52:55.770621+0000","mapping_epoch":74,"log_start":"83'24263","ondisk_log_start":"83'24263","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_clean_scrub_stamp":"2026-02-24T07:51:52.890170+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:53:07.446496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1590296576,"num_objects":24266,"num_object_clones":0,"num_object_copies":72798,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24266,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24266,"num_write_kb":1553024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,1],"acting":[0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.9","version":"83'24576","reported_seq":24604,"reported_epoch":83,"state":"active+clean","last_fresh":"2026-02-24T07:52:55.760069+0000","last_change":"2026-02-24T07:51:53.901985+0000","last_active":"2026-02-24T07:52:55.760069+0000","last_peered":"2026-02-24T07:52:55.760069+0000","last_clean":"2026-02-24T07:52:55.760069+0000","last_became_active":"2026-02-24T07:51:53.901610+0000","last_became_peered":"2026-02-24T07:51:53.901610+0000","last_unstale":"2026-02-24T07:52:55.760069+0000","last_undegraded":"2026-02-24T07:52:55.760069+0000","last_fullsized":"2026-02-24T07:52:55.760069+0000","mapping_epoch":74,"log_start":"83'24573","ondisk_log_start":"83'24573","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_clean_scrub_stamp":"2026-02-24T07:51:52.890170+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:53:13.073570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1610612736,"num_objects":24576,"num_object_clones":0,"num_object_copies":73728,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24576,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24576,"num_write_kb":1572864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.8","version":"83'24258","reported_seq":24286,"reported_epoch":83,"state":"active+clean","last_fresh":"2026-02-24T07:52:55.770666+0000","last_change":"2026-02-24T07:51:53.900477+0000","last_active":"2026-02-24T07:52:55.770666+0000","last_peered":"2026-02-24T07:52:55.770666+0000","last_clean":"2026-02-24T07:52:55.770666+0000","last_became_active":"2026-02-24T07:51:53.900207+0000","last_became_peered":"2026-02-24T07:51:53.900207+0000","last_unstale":"2026-02-24T07:52:55.770666+0000","last_undegraded":"2026-02-24T07:52:55.770666+0000","last_fullsized":"2026-02-24T07:52:55.770666+0000","mapping_epoch":74,"log_start":"83'24255","ondisk_log_start":"83'24255","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_clean_scrub_stamp":"2026-02-24T07:51:52.890170+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:52:57.062307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1589772288,"num_objects":24258,"num_object_clones":0,"num_object_copies":72774,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24258,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24258,"num_write_kb":1552512,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.7","version":"83'24165","reported_seq":24193,"reported_epoch":83,"state":"active+clean","last_fresh":"2026-02-24T07:52:55.759827+0000","last_change":"2026-02-24T07:51:53.899854+0000","last_active":"2026-02-24T07:52:55.759827+0000","last_peered":"2026-02-24T07:52:55.759827+0000","last_clean":"2026-02-24T07:52:55.759827+0000","last_became_active":"2026-02-24T07:51:53.899472+0000","last_became_peered":"2026-02-24T07:51:53.899472+0000","last_unstale":"2026-02-24T07:52:55.759827+0000","last_undegraded":"2026-02-24T07:52:55.759827+0000","last_fullsized":"2026-02-24T07:52:55.759827+0000","mapping_epoch":74,"log_start":"83'24162","ondisk_log_start":"83'24162","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_clean_scrub_stamp":"2026-02-24T07:51:52.890170+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1583677440,"num_objects":24165,"num_object_clones":0,"num_object_copies":72495,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24165,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24165,"num_write_kb":1546560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.6","version":"83'24201","reported_seq":24228,"reported_epoch":83,"state":"active+clean","last_fresh":"2026-02-24T07:52:55.851127+0000","last_change":"2026-02-24T07:51:53.898841+0000","last_active":"2026-02-24T07:52:55.851127+0000","last_peered":"2026-02-24T07:52:55.851127+0000","last_clean":"2026-02-24T07:52:55.851127+0000","last_became_active":"2026-02-24T07:51:53.898308+0000","last_became_peered":"2026-02-24T07:51:53.898308+0000","last_unstale":"2026-02-24T07:52:55.851127+0000","last_undegraded":"2026-02-24T07:52:55.851127+0000","last_fullsized":"2026-02-24T07:52:55.851127+0000","mapping_epoch":74,"log_start":"83'24198","ondisk_log_start":"83'24198","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_clean_scrub_stamp":"2026-02-24T07:51:52.890170+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:53:04.955186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1586036736,"num_objects":24201,"num_object_clones":0,"num_object_copies":72603,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24201,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24201,"num_write_kb":1548864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.5","version":"83'24229","reported_seq":24257,"reported_epoch":83,"state":"active+clean","last_fresh":"2026-02-24T07:52:55.770791+0000","last_change":"2026-02-24T07:51:53.900139+0000","last_active":"2026-02-24T07:52:55.770791+0000","last_peered":"2026-02-24T07:52:55.770791+0000","last_clean":"2026-02-24T07:52:55.770791+0000","last_became_active":"2026-02-24T07:51:53.899749+0000","last_became_peered":"2026-02-24T07:51:53.899749+0000","last_unstale":"2026-02-24T07:52:55.770791+0000","last_undegraded":"2026-02-24T07:52:55.770791+0000","last_fullsized":"2026-02-24T07:52:55.770791+0000","mapping_epoch":74,"log_start":"83'24226","ondisk_log_start":"83'24226","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_clean_scrub_stamp":"2026-02-24T07:51:52.890170+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:53:09.489880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1587871744,"num_objects":24229,"num_object_clones":0,"num_object_copies":72687,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24229,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24229,"num_write_kb":1550656,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"83'24163","reported_seq":24191,"reported_epoch":83,"state":"active+clean","last_fresh":"2026-02-24T07:52:55.767944+0000","last_change":"2026-02-24T07:51:53.900361+0000","last_active":"2026-02-24T07:52:55.767944+0000","last_peered":"2026-02-24T07:52:55.767944+0000","last_clean":"2026-02-24T07:52:55.767944+0000","last_became_active":"2026-02-24T07:51:53.900042+0000","last_became_peered":"2026-02-24T07:51:53.900042+0000","last_unstale":"2026-02-24T07:52:55.767944+0000","last_undegraded":"2026-02-24T07:52:55.767944+0000","last_fullsized":"2026-02-24T07:52:55.767944+0000","mapping_epoch":74,"log_start":"83'24160","ondisk_log_start":"83'24160","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_clean_scrub_stamp":"2026-02-24T07:51:52.890170+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:53:02.182725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1583546368,"num_objects":24163,"num_object_clones":0,"num_object_copies":72489,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24163,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24163,"num_write_kb":1546432,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.2","version":"83'24530","reported_seq":24558,"reported_epoch":83,"state":"active+clean","last_fresh":"2026-02-24T07:52:55.760420+0000","last_change":"2026-02-24T07:51:53.900424+0000","last_active":"2026-02-24T07:52:55.760420+0000","last_peered":"2026-02-24T07:52:55.760420+0000","last_clean":"2026-02-24T07:52:55.760420+0000","last_became_active":"2026-02-24T07:51:53.900096+0000","last_became_peered":"2026-02-24T07:51:53.900096+0000","last_unstale":"2026-02-24T07:52:55.760420+0000","last_undegraded":"2026-02-24T07:52:55.760420+0000","last_fullsized":"2026-02-24T07:52:55.760420+0000","mapping_epoch":74,"log_start":"83'24527","ondisk_log_start":"83'24527","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_clean_scrub_stamp":"2026-02-24T07:51:52.890170+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:53:10.802005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1607598080,"num_objects":24530,"num_object_clones":0,"num_object_copies":73590,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24530,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24530,"num_write_kb":1569920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"83'23943","reported_seq":23971,"reported_epoch":83,"state":"active+clean","last_fresh":"2026-02-24T07:52:55.757680+0000","last_change":"2026-02-24T07:51:53.901421+0000","last_active":"2026-02-24T07:52:55.757680+0000","last_peered":"2026-02-24T07:52:55.757680+0000","last_clean":"2026-02-24T07:52:55.757680+0000","last_became_active":"2026-02-24T07:51:53.901027+0000","last_became_peered":"2026-02-24T07:51:53.901027+0000","last_unstale":"2026-02-24T07:52:55.757680+0000","last_undegraded":"2026-02-24T07:52:55.757680+0000","last_fullsized":"2026-02-24T07:52:55.757680+0000","mapping_epoch":74,"log_start":"83'23940","ondisk_log_start":"83'23940","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_clean_scrub_stamp":"2026-02-24T07:51:52.890170+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:53:04.436560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1569128448,"num_objects":23943,"num_object_clones":0,"num_object_copies":71829,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23943,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":23943,"num_write_kb":1532352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.0","version":"83'24547","reported_seq":24575,"reported_epoch":83,"state":"active+clean","last_fresh":"2026-02-24T07:52:55.770553+0000","last_change":"2026-02-24T07:51:53.898740+0000","last_active":"2026-02-24T07:52:55.770553+0000","last_peered":"2026-02-24T07:52:55.770553+0000","last_clean":"2026-02-24T07:52:55.770553+0000","last_became_active":"2026-02-24T07:51:53.898551+0000","last_became_peered":"2026-02-24T07:51:53.898551+0000","last_unstale":"2026-02-24T07:52:55.770553+0000","last_undegraded":"2026-02-24T07:52:55.770553+0000","last_fullsized":"2026-02-24T07:52:55.770553+0000","mapping_epoch":74,"log_start":"83'24544","ondisk_log_start":"83'24544","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_clean_scrub_stamp":"2026-02-24T07:51:52.890170+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:53:07.664864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1608712192,"num_objects":24547,"num_object_clones":0,"num_object_copies":73641,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24547,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24547,"num_write_kb":1571008,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.4","version":"83'24231","reported_seq":24259,"reported_epoch":83,"state":"active+clean","last_fresh":"2026-02-24T07:52:55.740921+0000","last_change":"2026-02-24T07:51:53.900179+0000","last_active":"2026-02-24T07:52:55.740921+0000","last_peered":"2026-02-24T07:52:55.740921+0000","last_clean":"2026-02-24T07:52:55.740921+0000","last_became_active":"2026-02-24T07:51:53.899675+0000","last_became_peered":"2026-02-24T07:51:53.899675+0000","last_unstale":"2026-02-24T07:52:55.740921+0000","last_undegraded":"2026-02-24T07:52:55.740921+0000","last_fullsized":"2026-02-24T07:52:55.740921+0000","mapping_epoch":74,"log_start":"83'24228","ondisk_log_start":"83'24228","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_clean_scrub_stamp":"2026-02-24T07:51:52.890170+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:53:06.339989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1588002816,"num_objects":24231,"num_object_clones":0,"num_object_copies":72693,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24231,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24231,"num_write_kb":1550784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"29'99","reported_seq":347,"reported_epoch":83,"state":"active+clean","last_fresh":"2026-02-24T07:52:54.912910+0000","last_change":"2026-02-24T07:51:54.101629+0000","last_active":"2026-02-24T07:52:54.912910+0000","last_peered":"2026-02-24T07:52:54.912910+0000","last_clean":"2026-02-24T07:52:54.912910+0000","last_became_active":"2026-02-24T07:42:54.186719+0000","last_became_peered":"2026-02-24T07:42:54.186719+0000","last_unstale":"2026-02-24T07:52:54.912910+0000","last_undegraded":"2026-02-24T07:52:54.912910+0000","last_fullsized":"2026-02-24T07:52:54.912910+0000","mapping_epoch":27,"log_start":"29'96","ondisk_log_start":"29'96","created":20,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'99","last_scrub_stamp":"2026-02-24T07:51:54.101595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:42:43.640776+0000","last_clean_scrub_stamp":"2026-02-24T07:51:54.101595+0000","objects_scrubbed":2,"log_size":3,"log_dups_size":96,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":240,"num_write":135,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":16,"stat_sum":{"num_bytes":25460736000,"num_objects":388500,"num_object_clones":0,"num_object_copies":1165500,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":388500,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":388500,"num_write_kb":24864000,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":76387581952,"data_stored":76387581952,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":48,"ondisk_log_size":48,"up":48,"acting":48,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":240,"num_write":135,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1682,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":3,"up_from":25,"seq":107374182523,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":23216272,"kb_used_data":21494716,"kb_used_omap":25,"kb_used_meta":1721510,"kb_avail":710786928,"statfs":{"total":751619276800,"available":727845814272,"internally_reserved":0,"allocated":22010589184,"data_stored":22010003538,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":26118,"internal_metadata":1762826746},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,0,0,0,2,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":19,"seq":81604378749,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":18591824,"kb_used_data":16971260,"kb_used_omap":25,"kb_used_meta":1620518,"kb_avail":715411376,"statfs":{"total":751619276800,"available":732581249024,"internally_reserved":0,"allocated":17378570240,"data_stored":17377984594,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":26506,"internal_metadata":1659410550},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542270,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":16789260,"kb_used_data":15445944,"kb_used_omap":20,"kb_used_meta":1343275,"kb_avail":717213940,"statfs":{"total":751619276800,"available":734427074560,"internally_reserved":0,"allocated":15816646656,"data_stored":15816064562,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":20777,"internal_metadata":1375514327},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,1,1],"upper_bound":8},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705793,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":21495876,"kb_used_data":19940848,"kb_used_omap":23,"kb_used_meta":1554984,"kb_avail":712507324,"statfs":{"total":751619276800,"available":729607499776,"internally_reserved":0,"allocated":20419428352,"data_stored":20418850543,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":23806,"internal_metadata":1592304386},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,3],"upper_bound":32},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1682,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":20675428352,"data_stored":20675428352,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":15944908800,"data_stored":15944908800,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":17510629376,"data_stored":17510629376,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":22256615424,"data_stored":22256615424,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-24T07:53:01.007 INFO:tasks.cephadm.ceph_manager.ceph:increase pool size by 10 2026-02-24T07:53:01.007 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph osd pool set unique_pool_3 pg_num 36 2026-02-24T07:53:01.150 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:53:01.168 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2026-02-24T07:53:01.175 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:53:00 trial035 ceph-mon[39284]: 5.7 scrub ok 2026-02-24T07:53:01.276 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:53:01.275+0000 7f6cfd2b0640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:53:01.276 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:53:01.276+0000 7f6cfd2b0640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:53:01.319 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:53:01.423 INFO:teuthology.orchestra.run.trial035.stderr:Error EPERM: crimson_allow_pg_split is false; pg_num increase denied for crimson pool 2026-02-24T07:53:01.437 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:53:01.436+0000 7f0e0d52c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:53:01.437 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:53:01.436+0000 7f0e0d52c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:53:01.457 INFO:teuthology.orchestra.run.trial035.stdout:osd.0: {} 2026-02-24T07:53:01.457 INFO:teuthology.orchestra.run.trial035.stderr:osd.0: osd_enable_op_tracker = '' 2026-02-24T07:53:01.470 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-24T07:53:01.471 INFO:teuthology.orchestra.run.trial035.stdout:osd.1: {} 2026-02-24T07:53:01.471 INFO:teuthology.orchestra.run.trial035.stderr:osd.1: osd_enable_op_tracker = '' 2026-02-24T07:53:01.485 INFO:teuthology.orchestra.run.trial035.stdout:osd.2: {} 2026-02-24T07:53:01.485 INFO:teuthology.orchestra.run.trial035.stderr:osd.2: osd_enable_op_tracker = '' 2026-02-24T07:53:01.499 INFO:teuthology.orchestra.run.trial035.stdout:osd.3: {} 2026-02-24T07:53:01.499 INFO:teuthology.orchestra.run.trial035.stderr:osd.3: osd_enable_op_tracker = '' 2026-02-24T07:53:01.997 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 67 16 423504 423488 395.011 402.312 0.000943264 0.00253045 2026-02-24T07:53:02.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:53:01 trial035 ceph-mon[39284]: 5.8 scrub ok 2026-02-24T07:53:02.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:53:01 trial035 ceph-mon[39284]: from='client.15238 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2026-02-24T07:53:02.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:53:01 trial035 ceph-mon[39284]: 5.f scrub ok 2026-02-24T07:53:02.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:53:01 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/2694257568' entity='client.admin' cmd={"prefix": "osd pool set", "pool": "unique_pool_3", "var": "pg_num", "val": "36"} : dispatch 2026-02-24T07:53:02.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:53:01 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/2361156613' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2026-02-24T07:53:02.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:53:01 trial035 ceph-mon[39284]: pgmap v359: 17 pgs: 1 active+clean+scrubbing, 16 active+clean; 25 GiB data, 82 GiB used, 2.7 TiB / 2.7 TiB avail; 427 MiB/s wr, 6.83k op/s 2026-02-24T07:53:02.997 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 68 16 429949 429933 395.125 402.812 0.00161188 0.00252971 2026-02-24T07:53:03.543 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2026-02-24T07:53:03.699 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:53:03.704 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:53:03 trial035 ceph-mon[39284]: 5.3 scrub starts 2026-02-24T07:53:03.820 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:53:03.817+0000 7f60dc53b640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:53:03.820 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:53:03.817+0000 7f60dc53b640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:53:03.834 INFO:teuthology.orchestra.run.trial035.stdout:osd.0: {} 2026-02-24T07:53:03.834 INFO:teuthology.orchestra.run.trial035.stderr:osd.0: osd_enable_op_tracker = '' 2026-02-24T07:53:03.846 INFO:teuthology.orchestra.run.trial035.stdout:osd.1: {} 2026-02-24T07:53:03.846 INFO:teuthology.orchestra.run.trial035.stderr:osd.1: osd_enable_op_tracker = '' 2026-02-24T07:53:03.858 INFO:teuthology.orchestra.run.trial035.stdout:osd.2: {} 2026-02-24T07:53:03.859 INFO:teuthology.orchestra.run.trial035.stderr:osd.2: osd_enable_op_tracker = '' 2026-02-24T07:53:03.871 INFO:teuthology.orchestra.run.trial035.stdout:osd.3: {} 2026-02-24T07:53:03.871 INFO:teuthology.orchestra.run.trial035.stderr:osd.3: osd_enable_op_tracker = '' 2026-02-24T07:53:03.997 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 69 16 436613 436597 395.434 416.5 0.00143253 0.0025277 2026-02-24T07:53:04.704 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:53:04 trial035 ceph-mon[39284]: pgmap v360: 17 pgs: 1 active+clean+scrubbing, 16 active+clean; 25 GiB data, 82 GiB used, 2.7 TiB / 2.7 TiB avail; 387 MiB/s wr, 6.19k op/s 2026-02-24T07:53:04.704 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:53:04 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/3353799729' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2026-02-24T07:53:04.997 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 70 16 442658 442642 395.182 377.812 0.00287523 0.00252931 2026-02-24T07:53:05.704 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:53:05 trial035 ceph-mon[39284]: 5.3 scrub ok 2026-02-24T07:53:05.913 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2026-02-24T07:53:05.997 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 71 16 449107 449091 395.293 403.062 0.00163882 0.00252864 2026-02-24T07:53:06.058 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:53:06.177 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:53:06.174+0000 7f4bd797b640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:53:06.177 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:53:06.174+0000 7f4bd797b640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:53:06.191 INFO:teuthology.orchestra.run.trial035.stdout:osd.0: {} 2026-02-24T07:53:06.191 INFO:teuthology.orchestra.run.trial035.stderr:osd.0: osd_enable_op_tracker = '' 2026-02-24T07:53:06.207 INFO:teuthology.orchestra.run.trial035.stdout:osd.1: {} 2026-02-24T07:53:06.207 INFO:teuthology.orchestra.run.trial035.stderr:osd.1: osd_enable_op_tracker = '' 2026-02-24T07:53:06.219 INFO:teuthology.orchestra.run.trial035.stdout:osd.2: {} 2026-02-24T07:53:06.219 INFO:teuthology.orchestra.run.trial035.stderr:osd.2: osd_enable_op_tracker = '' 2026-02-24T07:53:06.231 INFO:teuthology.orchestra.run.trial035.stdout:osd.3: {} 2026-02-24T07:53:06.232 INFO:teuthology.orchestra.run.trial035.stderr:osd.3: osd_enable_op_tracker = '' 2026-02-24T07:53:06.454 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:53:06 trial035 ceph-mon[39284]: pgmap v361: 17 pgs: 1 active+clean+scrubbing, 16 active+clean; 25 GiB data, 82 GiB used, 2.7 TiB / 2.7 TiB avail; 341 MiB/s wr, 5.46k op/s 2026-02-24T07:53:06.454 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:53:06 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/3439605743' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2026-02-24T07:53:06.471 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2026-02-24T07:53:06.471 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2026-02-24T07:53:06.471 INFO:tasks.thrashosds.thrasher:fixing pg num pool unique_pool_3 2026-02-24T07:53:06.471 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph pg dump --format=json 2026-02-24T07:53:06.611 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:53:06.740 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:53:06.738+0000 7fb861199640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:53:06.740 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:53:06.739+0000 7fb861199640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:53:06.878 INFO:teuthology.orchestra.run.trial035.stdout: 2026-02-24T07:53:06.879 INFO:teuthology.orchestra.run.trial035.stderr:dumped all 2026-02-24T07:53:06.921 INFO:teuthology.orchestra.run.trial035.stdout:{"pg_ready":true,"pg_map":{"version":361,"stamp":"2026-02-24T07:53:05.739642+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":27245085216,"num_objects":415720,"num_object_clones":0,"num_object_copies":1247160,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":415720,"num_whiteouts":0,"num_read":136,"num_read_kb":240,"num_write":415853,"num_write_kb":26608102,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":52,"ondisk_log_size":52,"up":51,"acting":51,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":51,"num_osds":4,"num_per_pool_osds":4,"num_per_pool_omap_osds":4,"kb":2936012800,"kb_used":85694188,"kb_used_data":79038620,"kb_used_omap":97,"kb_used_meta":6655390,"kb_avail":2850318612,"statfs":{"total":3006477107200,"available":2918726258688,"internally_reserved":0,"allocated":80935546880,"data_stored":80933176300,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":99861,"internal_metadata":6815119851},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[7,6,3,3],"upper_bound":16},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":3581345792,"num_objects":54647,"num_object_clones":0,"num_object_copies":163941,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54647,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":54647,"num_write_kb":3497408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.001356"},"pg_stats":[{"pgid":"5.f","version":"84'25863","reported_seq":26091,"reported_epoch":84,"state":"active+clean+scrubbing","last_fresh":"2026-02-24T07:53:00.746808+0000","last_change":"2026-02-24T07:52:59.113523+0000","last_active":"2026-02-24T07:53:00.746808+0000","last_peered":"2026-02-24T07:53:00.746808+0000","last_clean":"2026-02-24T07:53:00.746808+0000","last_became_active":"2026-02-24T07:51:53.900431+0000","last_became_peered":"2026-02-24T07:51:53.900431+0000","last_unstale":"2026-02-24T07:53:00.746808+0000","last_undegraded":"2026-02-24T07:53:00.746808+0000","last_fullsized":"2026-02-24T07:53:00.746808+0000","mapping_epoch":74,"log_start":"84'25859","ondisk_log_start":"84'25859","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_clean_scrub_stamp":"2026-02-24T07:51:52.890170+0000","objects_scrubbed":19599,"log_size":4,"log_dups_size":3000,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"scrubbing for 2s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1694957568,"num_objects":25863,"num_object_clones":0,"num_object_copies":77589,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25863,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":25863,"num_write_kb":1655232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.e","version":"84'25954","reported_seq":26236,"reported_epoch":84,"state":"active+clean","last_fresh":"2026-02-24T07:53:00.761613+0000","last_change":"2026-02-24T07:52:56.767335+0000","last_active":"2026-02-24T07:53:00.761613+0000","last_peered":"2026-02-24T07:53:00.761613+0000","last_clean":"2026-02-24T07:53:00.761613+0000","last_became_active":"2026-02-24T07:51:53.899294+0000","last_became_peered":"2026-02-24T07:51:53.899294+0000","last_unstale":"2026-02-24T07:53:00.761613+0000","last_undegraded":"2026-02-24T07:53:00.761613+0000","last_fullsized":"2026-02-24T07:53:00.761613+0000","mapping_epoch":74,"log_start":"84'25951","ondisk_log_start":"84'25951","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"83'24557","last_scrub_stamp":"2026-02-24T07:52:56.767267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_clean_scrub_stamp":"2026-02-24T07:52:56.767267+0000","objects_scrubbed":24118,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:54:24.913755+0000","scrub_duration":2806,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1700921344,"num_objects":25954,"num_object_clones":0,"num_object_copies":77862,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25954,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":25954,"num_write_kb":1661056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"84'26014","reported_seq":26044,"reported_epoch":84,"state":"active+clean","last_fresh":"2026-02-24T07:53:00.770210+0000","last_change":"2026-02-24T07:51:53.900129+0000","last_active":"2026-02-24T07:53:00.770210+0000","last_peered":"2026-02-24T07:53:00.770210+0000","last_clean":"2026-02-24T07:53:00.770210+0000","last_became_active":"2026-02-24T07:51:53.899772+0000","last_became_peered":"2026-02-24T07:51:53.899772+0000","last_unstale":"2026-02-24T07:53:00.770210+0000","last_undegraded":"2026-02-24T07:53:00.770210+0000","last_fullsized":"2026-02-24T07:53:00.770210+0000","mapping_epoch":74,"log_start":"84'26011","ondisk_log_start":"84'26011","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_clean_scrub_stamp":"2026-02-24T07:51:52.890170+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:53:19.516698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1704853504,"num_objects":26014,"num_object_clones":0,"num_object_copies":78042,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26014,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":26014,"num_write_kb":1664896,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,3],"acting":[0,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.c","version":"84'26045","reported_seq":26075,"reported_epoch":84,"state":"active+clean","last_fresh":"2026-02-24T07:53:00.844111+0000","last_change":"2026-02-24T07:51:53.898484+0000","last_active":"2026-02-24T07:53:00.844111+0000","last_peered":"2026-02-24T07:53:00.844111+0000","last_clean":"2026-02-24T07:53:00.844111+0000","last_became_active":"2026-02-24T07:51:53.898285+0000","last_became_peered":"2026-02-24T07:51:53.898285+0000","last_unstale":"2026-02-24T07:53:00.844111+0000","last_undegraded":"2026-02-24T07:53:00.844111+0000","last_fullsized":"2026-02-24T07:53:00.844111+0000","mapping_epoch":74,"log_start":"84'26042","ondisk_log_start":"84'26042","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_clean_scrub_stamp":"2026-02-24T07:51:52.890170+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:53:11.947668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1706885120,"num_objects":26045,"num_object_clones":0,"num_object_copies":78135,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26045,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":26045,"num_write_kb":1666880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.b","version":"84'25973","reported_seq":26003,"reported_epoch":84,"state":"active+clean","last_fresh":"2026-02-24T07:53:00.766504+0000","last_change":"2026-02-24T07:51:53.900298+0000","last_active":"2026-02-24T07:53:00.766504+0000","last_peered":"2026-02-24T07:53:00.766504+0000","last_clean":"2026-02-24T07:53:00.766504+0000","last_became_active":"2026-02-24T07:51:53.899901+0000","last_became_peered":"2026-02-24T07:51:53.899901+0000","last_unstale":"2026-02-24T07:53:00.766504+0000","last_undegraded":"2026-02-24T07:53:00.766504+0000","last_fullsized":"2026-02-24T07:53:00.766504+0000","mapping_epoch":74,"log_start":"84'25970","ondisk_log_start":"84'25970","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_clean_scrub_stamp":"2026-02-24T07:51:52.890170+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:53:12.321784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1702166528,"num_objects":25973,"num_object_clones":0,"num_object_copies":77919,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25973,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":25973,"num_write_kb":1662272,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"84'25909","reported_seq":25939,"reported_epoch":84,"state":"active+clean","last_fresh":"2026-02-24T07:53:00.760149+0000","last_change":"2026-02-24T07:51:53.899828+0000","last_active":"2026-02-24T07:53:00.760149+0000","last_peered":"2026-02-24T07:53:00.760149+0000","last_clean":"2026-02-24T07:53:00.760149+0000","last_became_active":"2026-02-24T07:51:53.899549+0000","last_became_peered":"2026-02-24T07:51:53.899549+0000","last_unstale":"2026-02-24T07:53:00.760149+0000","last_undegraded":"2026-02-24T07:53:00.760149+0000","last_fullsized":"2026-02-24T07:53:00.760149+0000","mapping_epoch":74,"log_start":"84'25906","ondisk_log_start":"84'25906","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_clean_scrub_stamp":"2026-02-24T07:51:52.890170+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:53:07.446496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1697972224,"num_objects":25909,"num_object_clones":0,"num_object_copies":77727,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25909,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":25909,"num_write_kb":1658176,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,1],"acting":[0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.9","version":"84'26234","reported_seq":26264,"reported_epoch":84,"state":"active+clean","last_fresh":"2026-02-24T07:53:00.757142+0000","last_change":"2026-02-24T07:51:53.901985+0000","last_active":"2026-02-24T07:53:00.757142+0000","last_peered":"2026-02-24T07:53:00.757142+0000","last_clean":"2026-02-24T07:53:00.757142+0000","last_became_active":"2026-02-24T07:51:53.901610+0000","last_became_peered":"2026-02-24T07:51:53.901610+0000","last_unstale":"2026-02-24T07:53:00.757142+0000","last_undegraded":"2026-02-24T07:53:00.757142+0000","last_fullsized":"2026-02-24T07:53:00.757142+0000","mapping_epoch":74,"log_start":"84'26231","ondisk_log_start":"84'26231","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_clean_scrub_stamp":"2026-02-24T07:51:52.890170+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:53:13.073570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1719271424,"num_objects":26234,"num_object_clones":0,"num_object_copies":78702,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26234,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":26234,"num_write_kb":1678976,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.8","version":"84'26037","reported_seq":26328,"reported_epoch":84,"state":"active+clean","last_fresh":"2026-02-24T07:53:00.771287+0000","last_change":"2026-02-24T07:53:00.549866+0000","last_active":"2026-02-24T07:53:00.771287+0000","last_peered":"2026-02-24T07:53:00.771287+0000","last_clean":"2026-02-24T07:53:00.771287+0000","last_became_active":"2026-02-24T07:51:53.900207+0000","last_became_peered":"2026-02-24T07:51:53.900207+0000","last_unstale":"2026-02-24T07:53:00.771287+0000","last_undegraded":"2026-02-24T07:53:00.771287+0000","last_fullsized":"2026-02-24T07:53:00.771287+0000","mapping_epoch":74,"log_start":"84'26034","ondisk_log_start":"84'26034","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"84'25943","last_scrub_stamp":"2026-02-24T07:53:00.549789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_clean_scrub_stamp":"2026-02-24T07:53:00.549789+0000","objects_scrubbed":25486,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:54:18.896412+0000","scrub_duration":2647,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1706360832,"num_objects":26037,"num_object_clones":0,"num_object_copies":78111,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26037,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":26037,"num_write_kb":1666368,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.7","version":"84'25822","reported_seq":26113,"reported_epoch":84,"state":"active+clean","last_fresh":"2026-02-24T07:53:00.759621+0000","last_change":"2026-02-24T07:53:00.188453+0000","last_active":"2026-02-24T07:53:00.759621+0000","last_peered":"2026-02-24T07:53:00.759621+0000","last_clean":"2026-02-24T07:53:00.759621+0000","last_became_active":"2026-02-24T07:51:53.899472+0000","last_became_peered":"2026-02-24T07:51:53.899472+0000","last_unstale":"2026-02-24T07:53:00.759621+0000","last_undegraded":"2026-02-24T07:53:00.759621+0000","last_fullsized":"2026-02-24T07:53:00.759621+0000","mapping_epoch":74,"log_start":"84'25819","ondisk_log_start":"84'25819","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"84'25630","last_scrub_stamp":"2026-02-24T07:53:00.188390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_clean_scrub_stamp":"2026-02-24T07:53:00.188390+0000","objects_scrubbed":25246,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:54:07.242502+0000","scrub_duration":2262,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1692270592,"num_objects":25822,"num_object_clones":0,"num_object_copies":77466,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25822,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":25822,"num_write_kb":1652608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.6","version":"84'25825","reported_seq":25855,"reported_epoch":84,"state":"active+clean","last_fresh":"2026-02-24T07:53:00.850480+0000","last_change":"2026-02-24T07:51:53.898841+0000","last_active":"2026-02-24T07:53:00.850480+0000","last_peered":"2026-02-24T07:53:00.850480+0000","last_clean":"2026-02-24T07:53:00.850480+0000","last_became_active":"2026-02-24T07:51:53.898308+0000","last_became_peered":"2026-02-24T07:51:53.898308+0000","last_unstale":"2026-02-24T07:53:00.850480+0000","last_undegraded":"2026-02-24T07:53:00.850480+0000","last_fullsized":"2026-02-24T07:53:00.850480+0000","mapping_epoch":74,"log_start":"84'25822","ondisk_log_start":"84'25822","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_clean_scrub_stamp":"2026-02-24T07:51:52.890170+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:53:04.955186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1692467200,"num_objects":25825,"num_object_clones":0,"num_object_copies":77475,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25825,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":25825,"num_write_kb":1652800,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.5","version":"84'25947","reported_seq":25977,"reported_epoch":84,"state":"active+clean","last_fresh":"2026-02-24T07:53:00.766056+0000","last_change":"2026-02-24T07:51:53.900139+0000","last_active":"2026-02-24T07:53:00.766056+0000","last_peered":"2026-02-24T07:53:00.766056+0000","last_clean":"2026-02-24T07:53:00.766056+0000","last_became_active":"2026-02-24T07:51:53.899749+0000","last_became_peered":"2026-02-24T07:51:53.899749+0000","last_unstale":"2026-02-24T07:53:00.766056+0000","last_undegraded":"2026-02-24T07:53:00.766056+0000","last_fullsized":"2026-02-24T07:53:00.766056+0000","mapping_epoch":74,"log_start":"84'25944","ondisk_log_start":"84'25944","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_clean_scrub_stamp":"2026-02-24T07:51:52.890170+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:53:09.489880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1700462592,"num_objects":25947,"num_object_clones":0,"num_object_copies":77841,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25947,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":25947,"num_write_kb":1660608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"84'25937","reported_seq":25967,"reported_epoch":84,"state":"active+clean","last_fresh":"2026-02-24T07:53:00.772465+0000","last_change":"2026-02-24T07:51:53.900361+0000","last_active":"2026-02-24T07:53:00.772465+0000","last_peered":"2026-02-24T07:53:00.772465+0000","last_clean":"2026-02-24T07:53:00.772465+0000","last_became_active":"2026-02-24T07:51:53.900042+0000","last_became_peered":"2026-02-24T07:51:53.900042+0000","last_unstale":"2026-02-24T07:53:00.772465+0000","last_undegraded":"2026-02-24T07:53:00.772465+0000","last_fullsized":"2026-02-24T07:53:00.772465+0000","mapping_epoch":74,"log_start":"84'25934","ondisk_log_start":"84'25934","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_clean_scrub_stamp":"2026-02-24T07:51:52.890170+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:53:02.182725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1699807232,"num_objects":25937,"num_object_clones":0,"num_object_copies":77811,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25937,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":25937,"num_write_kb":1659968,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.2","version":"84'26269","reported_seq":26299,"reported_epoch":84,"state":"active+clean","last_fresh":"2026-02-24T07:53:00.763567+0000","last_change":"2026-02-24T07:51:53.900424+0000","last_active":"2026-02-24T07:53:00.763567+0000","last_peered":"2026-02-24T07:53:00.763567+0000","last_clean":"2026-02-24T07:53:00.763567+0000","last_became_active":"2026-02-24T07:51:53.900096+0000","last_became_peered":"2026-02-24T07:51:53.900096+0000","last_unstale":"2026-02-24T07:53:00.763567+0000","last_undegraded":"2026-02-24T07:53:00.763567+0000","last_fullsized":"2026-02-24T07:53:00.763567+0000","mapping_epoch":74,"log_start":"84'26266","ondisk_log_start":"84'26266","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_clean_scrub_stamp":"2026-02-24T07:51:52.890170+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:53:10.802005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1721565184,"num_objects":26269,"num_object_clones":0,"num_object_copies":78807,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26269,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":26269,"num_write_kb":1681216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"84'25659","reported_seq":25689,"reported_epoch":84,"state":"active+clean","last_fresh":"2026-02-24T07:53:00.763510+0000","last_change":"2026-02-24T07:51:53.901421+0000","last_active":"2026-02-24T07:53:00.763510+0000","last_peered":"2026-02-24T07:53:00.763510+0000","last_clean":"2026-02-24T07:53:00.763510+0000","last_became_active":"2026-02-24T07:51:53.901027+0000","last_became_peered":"2026-02-24T07:51:53.901027+0000","last_unstale":"2026-02-24T07:53:00.763510+0000","last_undegraded":"2026-02-24T07:53:00.763510+0000","last_fullsized":"2026-02-24T07:53:00.763510+0000","mapping_epoch":74,"log_start":"84'25656","ondisk_log_start":"84'25656","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_clean_scrub_stamp":"2026-02-24T07:51:52.890170+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:53:04.436560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1681588224,"num_objects":25659,"num_object_clones":0,"num_object_copies":76977,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25659,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":25659,"num_write_kb":1642176,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.0","version":"84'26346","reported_seq":26376,"reported_epoch":84,"state":"active+clean","last_fresh":"2026-02-24T07:53:00.764032+0000","last_change":"2026-02-24T07:51:53.898740+0000","last_active":"2026-02-24T07:53:00.764032+0000","last_peered":"2026-02-24T07:53:00.764032+0000","last_clean":"2026-02-24T07:53:00.764032+0000","last_became_active":"2026-02-24T07:51:53.898551+0000","last_became_peered":"2026-02-24T07:51:53.898551+0000","last_unstale":"2026-02-24T07:53:00.764032+0000","last_undegraded":"2026-02-24T07:53:00.764032+0000","last_fullsized":"2026-02-24T07:53:00.764032+0000","mapping_epoch":74,"log_start":"84'26343","ondisk_log_start":"84'26343","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_clean_scrub_stamp":"2026-02-24T07:51:52.890170+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:53:07.664864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1726611456,"num_objects":26346,"num_object_clones":0,"num_object_copies":79038,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26346,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":26346,"num_write_kb":1686144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.4","version":"84'25884","reported_seq":25914,"reported_epoch":84,"state":"active+clean","last_fresh":"2026-02-24T07:53:00.742865+0000","last_change":"2026-02-24T07:51:53.900179+0000","last_active":"2026-02-24T07:53:00.742865+0000","last_peered":"2026-02-24T07:53:00.742865+0000","last_clean":"2026-02-24T07:53:00.742865+0000","last_became_active":"2026-02-24T07:51:53.899675+0000","last_became_peered":"2026-02-24T07:51:53.899675+0000","last_unstale":"2026-02-24T07:53:00.742865+0000","last_undegraded":"2026-02-24T07:53:00.742865+0000","last_fullsized":"2026-02-24T07:53:00.742865+0000","mapping_epoch":74,"log_start":"84'25881","ondisk_log_start":"84'25881","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_clean_scrub_stamp":"2026-02-24T07:51:52.890170+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:53:06.339989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1696333824,"num_objects":25884,"num_object_clones":0,"num_object_copies":77652,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25884,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":25884,"num_write_kb":1656576,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"29'99","reported_seq":357,"reported_epoch":84,"state":"active+clean","last_fresh":"2026-02-24T07:52:58.893578+0000","last_change":"2026-02-24T07:52:56.946143+0000","last_active":"2026-02-24T07:52:58.893578+0000","last_peered":"2026-02-24T07:52:58.893578+0000","last_clean":"2026-02-24T07:52:58.893578+0000","last_became_active":"2026-02-24T07:42:54.186719+0000","last_became_peered":"2026-02-24T07:42:54.186719+0000","last_unstale":"2026-02-24T07:52:58.893578+0000","last_undegraded":"2026-02-24T07:52:58.893578+0000","last_fullsized":"2026-02-24T07:52:58.893578+0000","mapping_epoch":27,"log_start":"29'96","ondisk_log_start":"29'96","created":20,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'99","last_scrub_stamp":"2026-02-24T07:52:56.946109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:42:43.640776+0000","last_clean_scrub_stamp":"2026-02-24T07:52:56.946109+0000","objects_scrubbed":2,"log_size":3,"log_dups_size":96,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:54:05.896060+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":240,"num_write":135,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":16,"stat_sum":{"num_bytes":27244494848,"num_objects":415718,"num_object_clones":0,"num_object_copies":1247154,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":415718,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":415718,"num_write_kb":26605952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":81740038144,"data_stored":81740038144,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":49,"ondisk_log_size":49,"up":48,"acting":48,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":240,"num_write":135,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1682,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":3,"up_from":25,"seq":107374182524,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":24878364,"kb_used_data":23012424,"kb_used_omap":26,"kb_used_meta":1865893,"kb_avail":709124836,"statfs":{"total":751619276800,"available":726143832064,"internally_reserved":0,"allocated":23564722176,"data_stored":23564128643,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":26840,"internal_metadata":1910675240},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,2],"upper_bound":4},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":19,"seq":81604378750,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":19853532,"kb_used_data":18142728,"kb_used_omap":26,"kb_used_meta":1710757,"kb_avail":714149668,"statfs":{"total":751619276800,"available":731289260032,"internally_reserved":0,"allocated":18578153472,"data_stored":18577559939,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":27154,"internal_metadata":1751815662},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542271,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":17976664,"kb_used_data":16525060,"kb_used_omap":20,"kb_used_meta":1451563,"kb_avail":716026536,"statfs":{"total":751619276800,"available":733211172864,"internally_reserved":0,"allocated":16921661440,"data_stored":16921071459,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":20777,"internal_metadata":1486401239},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,1,1],"upper_bound":8},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705794,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":22985628,"kb_used_data":21358408,"kb_used_omap":24,"kb_used_meta":1627175,"kb_avail":711017572,"statfs":{"total":751619276800,"available":728081993728,"internally_reserved":0,"allocated":21871009792,"data_stored":21870416259,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":25090,"internal_metadata":1666227710},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,3,2,3],"upper_bound":16},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1682,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":22136160256,"data_stored":22136160256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":17057775616,"data_stored":17057775616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":18737659904,"data_stored":18737659904,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":23808442368,"data_stored":23808442368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-24T07:53:06.923 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph osd pool set unique_pool_3 pgp_num 36 2026-02-24T07:53:06.997 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 72 15 455672 455657 395.502 410.375 0.00350389 0.00252729 2026-02-24T07:53:07.055 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:53:07.128 INFO:journalctl@ceph.mgr.x.trial035.stdout:Feb 24 07:53:06 trial035 ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-mgr-x[39621]: ::ffff:10.20.193.91 - - [24/Feb/2026:07:53:06] "GET /metrics HTTP/1.1" 200 4217 "" "Prometheus/3.6.0" 2026-02-24T07:53:07.187 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:53:07.185+0000 7f0285a0c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:53:07.188 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:53:07.185+0000 7f0285a0c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:53:07.323 INFO:teuthology.orchestra.run.trial035.stderr:Error EPERM: crimson_allow_pg_split is false; pgp_num increase denied 2026-02-24T07:53:07.356 INFO:journalctl@ceph.prometheus.a.trial091.stdout:Feb 24 07:53:06 trial091 ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-prometheus-a[66007]: time=2026-02-24T07:53:06.973Z level=ERROR source=scrape.go:1630 msg="Failed to determine correct type of scrape target." component="scrape manager" scrape_pool=ceph target=http://trial035:9283/metrics content_type="text/html;charset=utf-8" fallback_media_type="" err="received unsupported Content-Type \"text/html;charset=utf-8\" and no fallback_scrape_protocol specified for target" 2026-02-24T07:53:07.362 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-24T07:53:07.704 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:53:07 trial035 ceph-mon[39284]: from='client.15248 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2026-02-24T07:53:07.704 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:53:07 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/3874783092' entity='client.admin' cmd={"prefix": "osd pool set", "pool": "unique_pool_3", "var": "pgp_num", "val": "36"} : dispatch 2026-02-24T07:53:07.997 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 73 15 462766 462751 396.157 443.375 0.0022964 0.00252313 2026-02-24T07:53:08.273 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2026-02-24T07:53:08.419 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:53:08.581 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:53:08.579+0000 7fc764c45640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:53:08.581 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:53:08.579+0000 7fc764c45640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:53:08.581 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:53:08 trial035 ceph-mon[39284]: pgmap v362: 17 pgs: 17 active+clean; 27 GiB data, 88 GiB used, 2.6 TiB / 2.7 TiB avail; 370 MiB/s wr, 5.91k op/s 2026-02-24T07:53:08.596 INFO:teuthology.orchestra.run.trial035.stdout:osd.0: {} 2026-02-24T07:53:08.596 INFO:teuthology.orchestra.run.trial035.stderr:osd.0: osd_enable_op_tracker = '' 2026-02-24T07:53:08.609 INFO:teuthology.orchestra.run.trial035.stdout:osd.1: {} 2026-02-24T07:53:08.609 INFO:teuthology.orchestra.run.trial035.stderr:osd.1: osd_enable_op_tracker = '' 2026-02-24T07:53:08.625 INFO:teuthology.orchestra.run.trial035.stdout:osd.2: {} 2026-02-24T07:53:08.625 INFO:teuthology.orchestra.run.trial035.stderr:osd.2: osd_enable_op_tracker = '' 2026-02-24T07:53:08.641 INFO:teuthology.orchestra.run.trial035.stdout:osd.3: {} 2026-02-24T07:53:08.641 INFO:teuthology.orchestra.run.trial035.stderr:osd.3: osd_enable_op_tracker = '' 2026-02-24T07:53:08.997 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 74 16 469388 469372 396.395 413.812 0.00220511 0.00252161 2026-02-24T07:53:09.704 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:53:09 trial035 ceph-mon[39284]: 5.6 scrub starts 2026-02-24T07:53:09.704 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:53:09 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/305243831' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2026-02-24T07:53:09.704 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:53:09 trial035 ceph-mon[39284]: from='mgr.14284 10.20.193.35:0/3681373222' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2026-02-24T07:53:09.704 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:53:09 trial035 ceph-mon[39284]: from='mgr.14284 10.20.193.35:0/3681373222' entity='mgr.x' 2026-02-24T07:53:09.705 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:53:09 trial035 ceph-mon[39284]: from='mgr.14284 10.20.193.35:0/3681373222' entity='mgr.x' 2026-02-24T07:53:09.705 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:53:09 trial035 ceph-mon[39284]: from='mgr.14284 10.20.193.35:0/3681373222' entity='mgr.x' 2026-02-24T07:53:09.705 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:53:09 trial035 ceph-mon[39284]: from='mgr.14284 10.20.193.35:0/3681373222' entity='mgr.x' 2026-02-24T07:53:09.997 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 75 16 475253 475237 395.997 366.562 0.00164525 0.00252411 2026-02-24T07:53:10.681 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2026-02-24T07:53:10.704 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:53:10 trial035 ceph-mon[39284]: 5.4 scrub starts 2026-02-24T07:53:10.704 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:53:10 trial035 ceph-mon[39284]: from='mgr.14284 10.20.193.35:0/3681373222' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2026-02-24T07:53:10.704 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:53:10 trial035 ceph-mon[39284]: from='mgr.14284 10.20.193.35:0/3681373222' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2026-02-24T07:53:10.704 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:53:10 trial035 ceph-mon[39284]: Adjusting osd_memory_target on trial035 to 41511M 2026-02-24T07:53:10.705 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:53:10 trial035 ceph-mon[39284]: from='mgr.14284 10.20.193.35:0/3681373222' entity='mgr.x' 2026-02-24T07:53:10.705 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:53:10 trial035 ceph-mon[39284]: from='mgr.14284 10.20.193.35:0/3681373222' entity='mgr.x' 2026-02-24T07:53:10.705 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:53:10 trial035 ceph-mon[39284]: from='mgr.14284 10.20.193.35:0/3681373222' entity='mgr.x' 2026-02-24T07:53:10.705 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:53:10 trial035 ceph-mon[39284]: pgmap v363: 17 pgs: 17 active+clean; 27 GiB data, 88 GiB used, 2.6 TiB / 2.7 TiB avail; 341 MiB/s wr, 5.45k op/s 2026-02-24T07:53:10.705 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:53:10 trial035 ceph-mon[39284]: from='mgr.14284 10.20.193.35:0/3681373222' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2026-02-24T07:53:10.705 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:53:10 trial035 ceph-mon[39284]: from='mgr.14284 10.20.193.35:0/3681373222' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2026-02-24T07:53:10.705 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:53:10 trial035 ceph-mon[39284]: Adjusting osd_memory_target on trial091 to 43559M 2026-02-24T07:53:10.705 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:53:10 trial035 ceph-mon[39284]: from='mgr.14284 10.20.193.35:0/3681373222' entity='mgr.x' 2026-02-24T07:53:10.705 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:53:10 trial035 ceph-mon[39284]: from='mgr.14284 10.20.193.35:0/3681373222' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-24T07:53:10.705 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:53:10 trial035 ceph-mon[39284]: from='mgr.14284 10.20.193.35:0/3681373222' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2026-02-24T07:53:10.705 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:53:10 trial035 ceph-mon[39284]: from='mgr.14284 10.20.193.35:0/3681373222' entity='mgr.x' 2026-02-24T07:53:10.706 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:53:10 trial035 ceph-mon[39284]: from='mgr.14284 10.20.193.35:0/3681373222' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2026-02-24T07:53:10.815 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:53:10.998 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 76 16 480876 480860 395.41 351.438 0.00738112 0.00252788 2026-02-24T07:53:11.029 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:53:11.026+0000 7f231542a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:53:11.029 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:53:11.027+0000 7f231542a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:53:11.043 INFO:teuthology.orchestra.run.trial035.stdout:osd.0: {} 2026-02-24T07:53:11.043 INFO:teuthology.orchestra.run.trial035.stderr:osd.0: osd_enable_op_tracker = '' 2026-02-24T07:53:11.055 INFO:teuthology.orchestra.run.trial035.stdout:osd.1: {} 2026-02-24T07:53:11.055 INFO:teuthology.orchestra.run.trial035.stderr:osd.1: osd_enable_op_tracker = '' 2026-02-24T07:53:11.066 INFO:teuthology.orchestra.run.trial035.stdout:osd.2: {} 2026-02-24T07:53:11.067 INFO:teuthology.orchestra.run.trial035.stderr:osd.2: osd_enable_op_tracker = '' 2026-02-24T07:53:11.079 INFO:teuthology.orchestra.run.trial035.stdout:osd.3: {} 2026-02-24T07:53:11.080 INFO:teuthology.orchestra.run.trial035.stderr:osd.3: osd_enable_op_tracker = '' 2026-02-24T07:53:11.954 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:53:11 trial035 ceph-mon[39284]: 5.6 scrub ok 2026-02-24T07:53:11.954 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:53:11 trial035 ceph-mon[39284]: 5.a scrub starts 2026-02-24T07:53:11.954 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:53:11 trial035 ceph-mon[39284]: 5.1 scrub starts 2026-02-24T07:53:11.955 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:53:11 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/1588070464' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2026-02-24T07:53:11.997 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 77 16 486271 486255 394.654 337.188 0.00137329 0.00253271 2026-02-24T07:53:12.364 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2026-02-24T07:53:12.364 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2026-02-24T07:53:12.365 INFO:tasks.thrashosds.thrasher:Growing pool .mgr 2026-02-24T07:53:12.365 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph pg dump --format=json 2026-02-24T07:53:12.509 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:53:12.676 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:53:12 trial035 ceph-mon[39284]: 5.4 scrub ok 2026-02-24T07:53:12.676 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:53:12 trial035 ceph-mon[39284]: pgmap v364: 17 pgs: 1 active+clean+scrubbing, 16 active+clean; 29 GiB data, 94 GiB used, 2.6 TiB / 2.7 TiB avail; 475 MiB/s wr, 7.61k op/s 2026-02-24T07:53:12.818 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:53:12.805+0000 7ff19f58d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:53:12.818 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:53:12.805+0000 7ff19f58d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:53:12.940 INFO:teuthology.orchestra.run.trial035.stdout: 2026-02-24T07:53:12.941 INFO:teuthology.orchestra.run.trial035.stderr:dumped all 2026-02-24T07:53:12.997 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 78 16 491107 491091 393.469 302.25 0.00132945 0.00253991 2026-02-24T07:53:13.178 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2026-02-24T07:53:13.487 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:53:13.572 INFO:teuthology.orchestra.run.trial035.stdout:{"pg_ready":true,"pg_map":{"version":364,"stamp":"2026-02-24T07:53:11.740429+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":31444238880,"num_objects":479794,"num_object_clones":0,"num_object_copies":1439382,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":479794,"num_whiteouts":0,"num_read":136,"num_read_kb":240,"num_write":479927,"num_write_kb":30708838,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":52,"ondisk_log_size":52,"up":51,"acting":51,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":51,"num_osds":4,"num_per_pool_osds":4,"num_per_pool_omap_osds":4,"kb":2936012800,"kb_used":98553644,"kb_used_data":91298268,"kb_used_omap":98,"kb_used_meta":7255197,"kb_avail":2837459156,"statfs":{"total":3006477107200,"available":2905558175744,"internally_reserved":0,"allocated":93489426432,"data_stored":93487055852,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":100845,"internal_metadata":7429322259},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[5,1,0,1],"upper_bound":16},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":5982912512,"num_objects":91292,"num_object_clones":0,"num_object_copies":273876,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":91292,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":91292,"num_write_kb":5842688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001590"},"pg_stats":[{"pgid":"5.f","version":"84'29928","reported_seq":30223,"reported_epoch":84,"state":"active+clean","last_fresh":"2026-02-24T07:53:10.748945+0000","last_change":"2026-02-24T07:53:01.242353+0000","last_active":"2026-02-24T07:53:10.748945+0000","last_peered":"2026-02-24T07:53:10.748945+0000","last_clean":"2026-02-24T07:53:10.748945+0000","last_became_active":"2026-02-24T07:51:53.900431+0000","last_became_peered":"2026-02-24T07:51:53.900431+0000","last_unstale":"2026-02-24T07:53:10.748945+0000","last_undegraded":"2026-02-24T07:53:10.748945+0000","last_fullsized":"2026-02-24T07:53:10.748945+0000","mapping_epoch":74,"log_start":"84'29925","ondisk_log_start":"84'29925","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"84'26061","last_scrub_stamp":"2026-02-24T07:53:01.242306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_clean_scrub_stamp":"2026-02-24T07:53:01.242306+0000","objects_scrubbed":25674,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:54:22.699112+0000","scrub_duration":2129,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1961361408,"num_objects":29928,"num_object_clones":0,"num_object_copies":89784,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29928,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":29928,"num_write_kb":1915392,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.e","version":"84'29933","reported_seq":30215,"reported_epoch":84,"state":"active+clean","last_fresh":"2026-02-24T07:53:10.762412+0000","last_change":"2026-02-24T07:52:56.767335+0000","last_active":"2026-02-24T07:53:10.762412+0000","last_peered":"2026-02-24T07:53:10.762412+0000","last_clean":"2026-02-24T07:53:10.762412+0000","last_became_active":"2026-02-24T07:51:53.899294+0000","last_became_peered":"2026-02-24T07:51:53.899294+0000","last_unstale":"2026-02-24T07:53:10.762412+0000","last_undegraded":"2026-02-24T07:53:10.762412+0000","last_fullsized":"2026-02-24T07:53:10.762412+0000","mapping_epoch":74,"log_start":"84'29930","ondisk_log_start":"84'29930","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"83'24557","last_scrub_stamp":"2026-02-24T07:52:56.767267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_clean_scrub_stamp":"2026-02-24T07:52:56.767267+0000","objects_scrubbed":24118,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:54:24.913755+0000","scrub_duration":2806,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1961689088,"num_objects":29933,"num_object_clones":0,"num_object_copies":89799,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29933,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":29933,"num_write_kb":1915712,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"84'30025","reported_seq":30053,"reported_epoch":84,"state":"active+clean","last_fresh":"2026-02-24T07:53:10.763521+0000","last_change":"2026-02-24T07:51:53.900129+0000","last_active":"2026-02-24T07:53:10.763521+0000","last_peered":"2026-02-24T07:53:10.763521+0000","last_clean":"2026-02-24T07:53:10.763521+0000","last_became_active":"2026-02-24T07:51:53.899772+0000","last_became_peered":"2026-02-24T07:51:53.899772+0000","last_unstale":"2026-02-24T07:53:10.763521+0000","last_undegraded":"2026-02-24T07:53:10.763521+0000","last_fullsized":"2026-02-24T07:53:10.763521+0000","mapping_epoch":74,"log_start":"84'30022","ondisk_log_start":"84'30022","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_clean_scrub_stamp":"2026-02-24T07:51:52.890170+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:53:19.516698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1967718400,"num_objects":30025,"num_object_clones":0,"num_object_copies":90075,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30025,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":30025,"num_write_kb":1921600,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,3],"acting":[0,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.c","version":"84'30102","reported_seq":30132,"reported_epoch":84,"state":"active+clean","last_fresh":"2026-02-24T07:53:10.841926+0000","last_change":"2026-02-24T07:51:53.898484+0000","last_active":"2026-02-24T07:53:10.841926+0000","last_peered":"2026-02-24T07:53:10.841926+0000","last_clean":"2026-02-24T07:53:10.841926+0000","last_became_active":"2026-02-24T07:51:53.898285+0000","last_became_peered":"2026-02-24T07:51:53.898285+0000","last_unstale":"2026-02-24T07:53:10.841926+0000","last_undegraded":"2026-02-24T07:53:10.841926+0000","last_fullsized":"2026-02-24T07:53:10.841926+0000","mapping_epoch":74,"log_start":"84'30099","ondisk_log_start":"84'30099","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_clean_scrub_stamp":"2026-02-24T07:51:52.890170+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:53:11.947668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1972764672,"num_objects":30102,"num_object_clones":0,"num_object_copies":90306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30102,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":30102,"num_write_kb":1926528,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.b","version":"84'29952","reported_seq":29982,"reported_epoch":84,"state":"active+clean","last_fresh":"2026-02-24T07:53:10.769216+0000","last_change":"2026-02-24T07:51:53.900298+0000","last_active":"2026-02-24T07:53:10.769216+0000","last_peered":"2026-02-24T07:53:10.769216+0000","last_clean":"2026-02-24T07:53:10.769216+0000","last_became_active":"2026-02-24T07:51:53.899901+0000","last_became_peered":"2026-02-24T07:51:53.899901+0000","last_unstale":"2026-02-24T07:53:10.769216+0000","last_undegraded":"2026-02-24T07:53:10.769216+0000","last_fullsized":"2026-02-24T07:53:10.769216+0000","mapping_epoch":74,"log_start":"84'29949","ondisk_log_start":"84'29949","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_clean_scrub_stamp":"2026-02-24T07:51:52.890170+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:53:12.321784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1962934272,"num_objects":29952,"num_object_clones":0,"num_object_copies":89856,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29952,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":29952,"num_write_kb":1916928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"84'29979","reported_seq":30009,"reported_epoch":84,"state":"active+clean","last_fresh":"2026-02-24T07:53:10.758576+0000","last_change":"2026-02-24T07:51:53.899828+0000","last_active":"2026-02-24T07:53:10.758576+0000","last_peered":"2026-02-24T07:53:10.758576+0000","last_clean":"2026-02-24T07:53:10.758576+0000","last_became_active":"2026-02-24T07:51:53.899549+0000","last_became_peered":"2026-02-24T07:51:53.899549+0000","last_unstale":"2026-02-24T07:53:10.758576+0000","last_undegraded":"2026-02-24T07:53:10.758576+0000","last_fullsized":"2026-02-24T07:53:10.758576+0000","mapping_epoch":74,"log_start":"84'29976","ondisk_log_start":"84'29976","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_clean_scrub_stamp":"2026-02-24T07:51:52.890170+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1964703744,"num_objects":29979,"num_object_clones":0,"num_object_copies":89937,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29979,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":29979,"num_write_kb":1918656,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,1],"acting":[0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.9","version":"84'30272","reported_seq":30302,"reported_epoch":84,"state":"active+clean","last_fresh":"2026-02-24T07:53:10.754628+0000","last_change":"2026-02-24T07:51:53.901985+0000","last_active":"2026-02-24T07:53:10.754628+0000","last_peered":"2026-02-24T07:53:10.754628+0000","last_clean":"2026-02-24T07:53:10.754628+0000","last_became_active":"2026-02-24T07:51:53.901610+0000","last_became_peered":"2026-02-24T07:51:53.901610+0000","last_unstale":"2026-02-24T07:53:10.754628+0000","last_undegraded":"2026-02-24T07:53:10.754628+0000","last_fullsized":"2026-02-24T07:53:10.754628+0000","mapping_epoch":74,"log_start":"84'30269","ondisk_log_start":"84'30269","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_clean_scrub_stamp":"2026-02-24T07:51:52.890170+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:53:13.073570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1983905792,"num_objects":30272,"num_object_clones":0,"num_object_copies":90816,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30272,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":30272,"num_write_kb":1937408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.8","version":"84'30087","reported_seq":30380,"reported_epoch":84,"state":"active+clean","last_fresh":"2026-02-24T07:53:10.769032+0000","last_change":"2026-02-24T07:53:00.549866+0000","last_active":"2026-02-24T07:53:10.769032+0000","last_peered":"2026-02-24T07:53:10.769032+0000","last_clean":"2026-02-24T07:53:10.769032+0000","last_became_active":"2026-02-24T07:51:53.900207+0000","last_became_peered":"2026-02-24T07:51:53.900207+0000","last_unstale":"2026-02-24T07:53:10.769032+0000","last_undegraded":"2026-02-24T07:53:10.769032+0000","last_fullsized":"2026-02-24T07:53:10.769032+0000","mapping_epoch":74,"log_start":"84'30084","ondisk_log_start":"84'30084","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"84'25943","last_scrub_stamp":"2026-02-24T07:53:00.549789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_clean_scrub_stamp":"2026-02-24T07:53:00.549789+0000","objects_scrubbed":25486,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:54:18.896412+0000","scrub_duration":2647,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1971781632,"num_objects":30087,"num_object_clones":0,"num_object_copies":90261,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30087,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":30087,"num_write_kb":1925568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.7","version":"84'29814","reported_seq":30106,"reported_epoch":84,"state":"active+clean","last_fresh":"2026-02-24T07:53:10.760732+0000","last_change":"2026-02-24T07:53:00.188453+0000","last_active":"2026-02-24T07:53:10.760732+0000","last_peered":"2026-02-24T07:53:10.760732+0000","last_clean":"2026-02-24T07:53:10.760732+0000","last_became_active":"2026-02-24T07:51:53.899472+0000","last_became_peered":"2026-02-24T07:51:53.899472+0000","last_unstale":"2026-02-24T07:53:10.760732+0000","last_undegraded":"2026-02-24T07:53:10.760732+0000","last_fullsized":"2026-02-24T07:53:10.760732+0000","mapping_epoch":74,"log_start":"84'29811","ondisk_log_start":"84'29811","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"84'25630","last_scrub_stamp":"2026-02-24T07:53:00.188390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_clean_scrub_stamp":"2026-02-24T07:53:00.188390+0000","objects_scrubbed":25246,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:54:07.242502+0000","scrub_duration":2262,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1953890304,"num_objects":29814,"num_object_clones":0,"num_object_copies":89442,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29814,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":29814,"num_write_kb":1908096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.6","version":"84'29842","reported_seq":30175,"reported_epoch":84,"state":"active+clean","last_fresh":"2026-02-24T07:53:10.845525+0000","last_change":"2026-02-24T07:53:10.804133+0000","last_active":"2026-02-24T07:53:10.845525+0000","last_peered":"2026-02-24T07:53:10.845525+0000","last_clean":"2026-02-24T07:53:10.845525+0000","last_became_active":"2026-02-24T07:51:53.898308+0000","last_became_peered":"2026-02-24T07:51:53.898308+0000","last_unstale":"2026-02-24T07:53:10.845525+0000","last_undegraded":"2026-02-24T07:53:10.845525+0000","last_fullsized":"2026-02-24T07:53:10.845525+0000","mapping_epoch":74,"log_start":"84'29839","ondisk_log_start":"84'29839","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"84'29829","last_scrub_stamp":"2026-02-24T07:53:10.804104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_clean_scrub_stamp":"2026-02-24T07:53:10.804104+0000","objects_scrubbed":29351,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:54:13.621421+0000","scrub_duration":2501,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1955725312,"num_objects":29842,"num_object_clones":0,"num_object_copies":89526,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29842,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":29842,"num_write_kb":1909888,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.5","version":"84'29845","reported_seq":29875,"reported_epoch":84,"state":"active+clean","last_fresh":"2026-02-24T07:53:10.774710+0000","last_change":"2026-02-24T07:51:53.900139+0000","last_active":"2026-02-24T07:53:10.774710+0000","last_peered":"2026-02-24T07:53:10.774710+0000","last_clean":"2026-02-24T07:53:10.774710+0000","last_became_active":"2026-02-24T07:51:53.899749+0000","last_became_peered":"2026-02-24T07:51:53.899749+0000","last_unstale":"2026-02-24T07:53:10.774710+0000","last_undegraded":"2026-02-24T07:53:10.774710+0000","last_fullsized":"2026-02-24T07:53:10.774710+0000","mapping_epoch":74,"log_start":"84'29841","ondisk_log_start":"84'29841","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_clean_scrub_stamp":"2026-02-24T07:51:52.890170+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":3000,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1955921920,"num_objects":29845,"num_object_clones":0,"num_object_copies":89535,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29845,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":29845,"num_write_kb":1910080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"84'29942","reported_seq":30255,"reported_epoch":84,"state":"active+clean","last_fresh":"2026-02-24T07:53:10.766355+0000","last_change":"2026-02-24T07:53:05.312364+0000","last_active":"2026-02-24T07:53:10.766355+0000","last_peered":"2026-02-24T07:53:10.766355+0000","last_clean":"2026-02-24T07:53:10.766355+0000","last_became_active":"2026-02-24T07:51:53.900042+0000","last_became_peered":"2026-02-24T07:51:53.900042+0000","last_unstale":"2026-02-24T07:53:10.766355+0000","last_undegraded":"2026-02-24T07:53:10.766355+0000","last_fullsized":"2026-02-24T07:53:10.766355+0000","mapping_epoch":74,"log_start":"84'29939","ondisk_log_start":"84'29939","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"84'27738","last_scrub_stamp":"2026-02-24T07:53:05.312312+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_clean_scrub_stamp":"2026-02-24T07:53:05.312312+0000","objects_scrubbed":27245,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:54:32.166518+0000","scrub_duration":2472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1962278912,"num_objects":29942,"num_object_clones":0,"num_object_copies":89826,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29942,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":29942,"num_write_kb":1916288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.2","version":"84'30304","reported_seq":30333,"reported_epoch":84,"state":"active+clean","last_fresh":"2026-02-24T07:53:10.763856+0000","last_change":"2026-02-24T07:51:53.900424+0000","last_active":"2026-02-24T07:53:10.763856+0000","last_peered":"2026-02-24T07:53:10.763856+0000","last_clean":"2026-02-24T07:53:10.763856+0000","last_became_active":"2026-02-24T07:51:53.900096+0000","last_became_peered":"2026-02-24T07:51:53.900096+0000","last_unstale":"2026-02-24T07:53:10.763856+0000","last_undegraded":"2026-02-24T07:53:10.763856+0000","last_fullsized":"2026-02-24T07:53:10.763856+0000","mapping_epoch":74,"log_start":"84'30301","ondisk_log_start":"84'30301","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_clean_scrub_stamp":"2026-02-24T07:51:52.890170+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:53:10.802005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1986002944,"num_objects":30304,"num_object_clones":0,"num_object_copies":90912,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30304,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":30304,"num_write_kb":1939456,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"84'29636","reported_seq":29666,"reported_epoch":84,"state":"active+clean","last_fresh":"2026-02-24T07:53:10.761806+0000","last_change":"2026-02-24T07:51:53.901421+0000","last_active":"2026-02-24T07:53:10.761806+0000","last_peered":"2026-02-24T07:53:10.761806+0000","last_clean":"2026-02-24T07:53:10.761806+0000","last_became_active":"2026-02-24T07:51:53.901027+0000","last_became_peered":"2026-02-24T07:51:53.901027+0000","last_unstale":"2026-02-24T07:53:10.761806+0000","last_undegraded":"2026-02-24T07:53:10.761806+0000","last_fullsized":"2026-02-24T07:53:10.761806+0000","mapping_epoch":74,"log_start":"84'29633","ondisk_log_start":"84'29633","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_clean_scrub_stamp":"2026-02-24T07:51:52.890170+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1942224896,"num_objects":29636,"num_object_clones":0,"num_object_copies":88908,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29636,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":29636,"num_write_kb":1896704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.0","version":"84'30270","reported_seq":30300,"reported_epoch":84,"state":"active+clean","last_fresh":"2026-02-24T07:53:10.773598+0000","last_change":"2026-02-24T07:51:53.898740+0000","last_active":"2026-02-24T07:53:10.773598+0000","last_peered":"2026-02-24T07:53:10.773598+0000","last_clean":"2026-02-24T07:53:10.773598+0000","last_became_active":"2026-02-24T07:51:53.898551+0000","last_became_peered":"2026-02-24T07:51:53.898551+0000","last_unstale":"2026-02-24T07:53:10.773598+0000","last_undegraded":"2026-02-24T07:53:10.773598+0000","last_fullsized":"2026-02-24T07:53:10.773598+0000","mapping_epoch":74,"log_start":"84'30267","ondisk_log_start":"84'30267","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_clean_scrub_stamp":"2026-02-24T07:51:52.890170+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1983774720,"num_objects":30270,"num_object_clones":0,"num_object_copies":90810,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30270,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":30270,"num_write_kb":1937280,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.4","version":"84'29861","reported_seq":30085,"reported_epoch":84,"state":"active+clean+scrubbing","last_fresh":"2026-02-24T07:53:10.751995+0000","last_change":"2026-02-24T07:53:09.160541+0000","last_active":"2026-02-24T07:53:10.751995+0000","last_peered":"2026-02-24T07:53:10.751995+0000","last_clean":"2026-02-24T07:53:10.751995+0000","last_became_active":"2026-02-24T07:51:53.899675+0000","last_became_peered":"2026-02-24T07:51:53.899675+0000","last_unstale":"2026-02-24T07:53:10.751995+0000","last_undegraded":"2026-02-24T07:53:10.751995+0000","last_fullsized":"2026-02-24T07:53:10.751995+0000","mapping_epoch":74,"log_start":"84'29858","ondisk_log_start":"84'29858","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_clean_scrub_stamp":"2026-02-24T07:51:52.890170+0000","objects_scrubbed":19049,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"scrubbing for 2s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1956970496,"num_objects":29861,"num_object_clones":0,"num_object_copies":89583,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29861,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":29861,"num_write_kb":1911104,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"29'99","reported_seq":357,"reported_epoch":84,"state":"active+clean","last_fresh":"2026-02-24T07:52:58.893578+0000","last_change":"2026-02-24T07:52:56.946143+0000","last_active":"2026-02-24T07:52:58.893578+0000","last_peered":"2026-02-24T07:52:58.893578+0000","last_clean":"2026-02-24T07:52:58.893578+0000","last_became_active":"2026-02-24T07:42:54.186719+0000","last_became_peered":"2026-02-24T07:42:54.186719+0000","last_unstale":"2026-02-24T07:52:58.893578+0000","last_undegraded":"2026-02-24T07:52:58.893578+0000","last_fullsized":"2026-02-24T07:52:58.893578+0000","mapping_epoch":27,"log_start":"29'96","ondisk_log_start":"29'96","created":20,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'99","last_scrub_stamp":"2026-02-24T07:52:56.946109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:42:43.640776+0000","last_clean_scrub_stamp":"2026-02-24T07:52:56.946109+0000","objects_scrubbed":2,"log_size":3,"log_dups_size":96,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:54:05.896060+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":240,"num_write":135,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":16,"stat_sum":{"num_bytes":31443648512,"num_objects":479792,"num_object_clones":0,"num_object_copies":1439376,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":479792,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":479792,"num_write_kb":30706688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":94337236992,"data_stored":94337236992,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":49,"ondisk_log_size":49,"up":48,"acting":48,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":240,"num_write":135,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1682,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":3,"up_from":25,"seq":107374182526,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":28507932,"kb_used_data":26551688,"kb_used_omap":27,"kb_used_meta":1956196,"kb_avail":705495268,"statfs":{"total":751619276800,"available":722427154432,"internally_reserved":0,"allocated":27188928512,"data_stored":27188334979,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":27824,"internal_metadata":2003145552},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[5,1],"upper_bound":4},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":19,"seq":81604378752,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":22877852,"kb_used_data":20982408,"kb_used_omap":26,"kb_used_meta":1895397,"kb_avail":711125348,"statfs":{"total":751619276800,"available":728192356352,"internally_reserved":0,"allocated":21485985792,"data_stored":21485392259,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":27154,"internal_metadata":1940887022},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1],"upper_bound":16},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542273,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":20685848,"kb_used_data":19089860,"kb_used_omap":20,"kb_used_meta":1595947,"kb_avail":713317352,"statfs":{"total":751619276800,"available":730436968448,"internally_reserved":0,"allocated":19548016640,"data_stored":19547426659,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":20777,"internal_metadata":1634250455},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705796,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":26482012,"kb_used_data":24674312,"kb_used_omap":24,"kb_used_meta":1807655,"kb_avail":707521188,"statfs":{"total":751619276800,"available":724501696512,"internally_reserved":0,"allocated":25266495488,"data_stored":25265901955,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":25090,"internal_metadata":1851039230},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1682,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":25546653696,"data_stored":25546653696,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":19677577216,"data_stored":19677577216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":21627797504,"data_stored":21627797504,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":27485208576,"data_stored":27485208576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-24T07:53:13.574 INFO:tasks.cephadm.ceph_manager.ceph:increase pool size by 10 2026-02-24T07:53:13.574 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph osd pool set .mgr pg_num 351 2026-02-24T07:53:13.787 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:53:13.998 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 79 16 494750 494734 391.37 227.688 0.00785004 0.00255396 2026-02-24T07:53:14.021 INFO:teuthology.orchestra.run.trial035.stdout:osd.0: {} 2026-02-24T07:53:14.021 INFO:teuthology.orchestra.run.trial035.stdout:osd.1: {} 2026-02-24T07:53:14.022 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:53:13.964+0000 7f32990ab640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:53:14.022 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:53:13.964+0000 7f32990ab640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:53:14.022 INFO:teuthology.orchestra.run.trial035.stderr:osd.0: osd_enable_op_tracker = '' 2026-02-24T07:53:14.023 INFO:teuthology.orchestra.run.trial035.stderr:osd.1: osd_enable_op_tracker = '' 2026-02-24T07:53:14.023 INFO:teuthology.orchestra.run.trial035.stdout:osd.2: {} 2026-02-24T07:53:14.024 INFO:teuthology.orchestra.run.trial035.stderr:osd.2: osd_enable_op_tracker = '' 2026-02-24T07:53:14.035 INFO:teuthology.orchestra.run.trial035.stdout:osd.3: {} 2026-02-24T07:53:14.035 INFO:teuthology.orchestra.run.trial035.stderr:osd.3: osd_enable_op_tracker = '' 2026-02-24T07:53:14.998 INFO:tasks.radosbench.radosbench.0.trial035.stdout:2026-02-24T07:53:14.997609+0000 min lat: 0.000691325 max lat: 0.0517474 avg lat: 0.00256255 2026-02-24T07:53:14.998 INFO:tasks.radosbench.radosbench.0.trial035.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2026-02-24T07:53:14.998 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 80 15 499339 499324 390.064 286.875 0.00236095 0.00256255 2026-02-24T07:53:15.172 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:53:14 trial035 ceph-mon[39284]: from='client.15256 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2026-02-24T07:53:15.172 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:53:14 trial035 ceph-mon[39284]: pgmap v365: 17 pgs: 1 active+clean+scrubbing, 16 active+clean; 29 GiB data, 94 GiB used, 2.6 TiB / 2.7 TiB avail; 334 MiB/s wr, 5.34k op/s 2026-02-24T07:53:15.172 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:53:14 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/1066965290' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2026-02-24T07:53:15.172 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:53:14 trial035 ceph-mon[39284]: 5.1 scrub ok 2026-02-24T07:53:15.173 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:53:14 trial035 ceph-mon[39284]: 5.a scrub ok 2026-02-24T07:53:15.305 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:53:15.303+0000 7f0d9ea95640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:53:15.305 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:53:15.303+0000 7f0d9ea95640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:53:15.442 INFO:teuthology.orchestra.run.trial035.stderr:Error EPERM: crimson_allow_pg_split is false; pg_num increase denied for crimson pool 2026-02-24T07:53:15.479 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-24T07:53:15.998 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 81 16 504799 504783 389.46 341.188 0.00449144 0.00256653 2026-02-24T07:53:16.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:53:15 trial035 ceph-mon[39284]: 5.c scrub starts 2026-02-24T07:53:16.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:53:15 trial035 ceph-mon[39284]: from='mgr.14284 10.20.193.35:0/3681373222' entity='mgr.x' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2026-02-24T07:53:16.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:53:15 trial035 ceph-mon[39284]: 5.2 scrub starts 2026-02-24T07:53:16.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:53:15 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/1105832636' entity='client.admin' cmd={"prefix": "osd pool set", "pool": ".mgr", "var": "pg_num", "val": "351"} : dispatch 2026-02-24T07:53:16.852 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2026-02-24T07:53:16.989 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:53:16.998 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 82 16 510733 510717 389.233 370.875 0.00205728 0.002568 2026-02-24T07:53:17.052 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:53:16 trial035 ceph-mon[39284]: pgmap v366: 17 pgs: 1 active+clean+scrubbing, 16 active+clean; 29 GiB data, 94 GiB used, 2.6 TiB / 2.7 TiB avail; 334 MiB/s wr, 5.34k op/s 2026-02-24T07:53:17.052 INFO:journalctl@ceph.mgr.x.trial035.stdout:Feb 24 07:53:16 trial035 ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-mgr-x[39621]: ::ffff:10.20.193.91 - - [24/Feb/2026:07:53:16] "GET /metrics HTTP/1.1" 200 4214 "" "Prometheus/3.6.0" 2026-02-24T07:53:17.125 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:53:17.123+0000 7f72d2f53640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:53:17.126 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:53:17.123+0000 7f72d2f53640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:53:17.141 INFO:teuthology.orchestra.run.trial035.stdout:osd.0: {} 2026-02-24T07:53:17.141 INFO:teuthology.orchestra.run.trial035.stderr:osd.0: osd_enable_op_tracker = '' 2026-02-24T07:53:17.153 INFO:teuthology.orchestra.run.trial035.stdout:osd.1: {} 2026-02-24T07:53:17.154 INFO:teuthology.orchestra.run.trial035.stderr:osd.1: osd_enable_op_tracker = '' 2026-02-24T07:53:17.167 INFO:teuthology.orchestra.run.trial035.stdout:osd.2: {} 2026-02-24T07:53:17.167 INFO:teuthology.orchestra.run.trial035.stderr:osd.2: osd_enable_op_tracker = '' 2026-02-24T07:53:17.183 INFO:teuthology.orchestra.run.trial035.stdout:osd.3: {} 2026-02-24T07:53:17.183 INFO:teuthology.orchestra.run.trial035.stderr:osd.3: osd_enable_op_tracker = '' 2026-02-24T07:53:17.356 INFO:journalctl@ceph.prometheus.a.trial091.stdout:Feb 24 07:53:16 trial091 ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-prometheus-a[66007]: time=2026-02-24T07:53:16.973Z level=ERROR source=scrape.go:1630 msg="Failed to determine correct type of scrape target." component="scrape manager" scrape_pool=ceph target=http://trial035:9283/metrics content_type="text/html;charset=utf-8" fallback_media_type="" err="received unsupported Content-Type \"text/html;charset=utf-8\" and no fallback_scrape_protocol specified for target" 2026-02-24T07:53:17.998 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 83 16 516449 516433 388.847 357.25 0.00145635 0.00257055 2026-02-24T07:53:18.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:53:17 trial035 ceph-mon[39284]: 5.0 scrub starts 2026-02-24T07:53:18.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:53:17 trial035 ceph-mon[39284]: 5.c scrub ok 2026-02-24T07:53:18.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:53:17 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/1853036598' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2026-02-24T07:53:18.998 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 84 16 522260 522244 388.541 363.188 0.001085 0.00257258 2026-02-24T07:53:19.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:53:18 trial035 ceph-mon[39284]: pgmap v367: 17 pgs: 2 active+clean+scrubbing, 15 active+clean; 31 GiB data, 99 GiB used, 2.6 TiB / 2.7 TiB avail; 457 MiB/s wr, 7.32k op/s 2026-02-24T07:53:19.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:53:18 trial035 ceph-mon[39284]: 5.2 scrub ok 2026-02-24T07:53:19.234 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2026-02-24T07:53:19.371 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:53:19.490 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:53:19.488+0000 7fd426d69640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:53:19.491 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:53:19.488+0000 7fd426d69640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:53:19.506 INFO:teuthology.orchestra.run.trial035.stdout:osd.0: {} 2026-02-24T07:53:19.507 INFO:teuthology.orchestra.run.trial035.stderr:osd.0: osd_enable_op_tracker = '' 2026-02-24T07:53:19.520 INFO:teuthology.orchestra.run.trial035.stdout:osd.1: {} 2026-02-24T07:53:19.520 INFO:teuthology.orchestra.run.trial035.stderr:osd.1: osd_enable_op_tracker = '' 2026-02-24T07:53:19.532 INFO:teuthology.orchestra.run.trial035.stdout:osd.2: {} 2026-02-24T07:53:19.533 INFO:teuthology.orchestra.run.trial035.stderr:osd.2: osd_enable_op_tracker = '' 2026-02-24T07:53:19.547 INFO:teuthology.orchestra.run.trial035.stdout:osd.3: {} 2026-02-24T07:53:19.547 INFO:teuthology.orchestra.run.trial035.stderr:osd.3: osd_enable_op_tracker = '' 2026-02-24T07:53:19.998 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 85 16 527791 527775 388.037 345.688 0.00720708 0.00257594 2026-02-24T07:53:20.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:53:19 trial035 ceph-mon[39284]: 5.9 scrub starts 2026-02-24T07:53:20.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:53:19 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/4001596781' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2026-02-24T07:53:20.481 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2026-02-24T07:53:20.481 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2026-02-24T07:53:20.481 INFO:tasks.thrashosds.thrasher:fixing pg num pool unique_pool_3 2026-02-24T07:53:20.482 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph pg dump --format=json 2026-02-24T07:53:20.624 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:53:20.857 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:53:20.833+0000 7f11e56a8640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:53:20.857 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:53:20.833+0000 7f11e56a8640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:53:20.963 INFO:teuthology.orchestra.run.trial035.stdout: 2026-02-24T07:53:20.964 INFO:teuthology.orchestra.run.trial035.stderr:dumped all 2026-02-24T07:53:20.998 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 86 16 533490 533474 387.666 356.188 0.00199523 0.00257839 2026-02-24T07:53:21.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:53:20 trial035 ceph-mon[39284]: pgmap v368: 17 pgs: 2 active+clean+scrubbing, 15 active+clean; 31 GiB data, 99 GiB used, 2.6 TiB / 2.7 TiB avail; 291 MiB/s wr, 4.66k op/s 2026-02-24T07:53:21.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:53:20 trial035 ceph-mon[39284]: 5.0 scrub ok 2026-02-24T07:53:21.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:53:20 trial035 ceph-mon[39284]: purged_snaps scrub starts 2026-02-24T07:53:21.205 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:53:20 trial035 ceph-mon[39284]: purged_snaps scrub ok 2026-02-24T07:53:21.591 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2026-02-24T07:53:21.726 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:53:21.744 INFO:teuthology.orchestra.run.trial035.stdout:{"pg_ready":true,"pg_map":{"version":368,"stamp":"2026-02-24T07:53:19.741526+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":32999604768,"num_objects":503527,"num_object_clones":0,"num_object_copies":1510581,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":503527,"num_whiteouts":0,"num_read":136,"num_read_kb":240,"num_write":503660,"num_write_kb":32227750,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":51,"ondisk_log_size":51,"up":51,"acting":51,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":51,"num_osds":4,"num_per_pool_osds":4,"num_per_pool_omap_osds":4,"kb":2936012800,"kb_used":103510252,"kb_used_data":95930012,"kb_used_omap":98,"kb_used_meta":7580061,"kb_avail":2832502548,"statfs":{"total":3006477107200,"available":2900482609152,"internally_reserved":0,"allocated":98232332288,"data_stored":98229961708,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":100845,"internal_metadata":7761982995},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[4,3,0,1],"upper_bound":16},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":3661430784,"num_objects":55869,"num_object_clones":0,"num_object_copies":167607,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":55869,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":55869,"num_write_kb":3575616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001615"},"pg_stats":[{"pgid":"5.f","version":"84'31440","reported_seq":31735,"reported_epoch":84,"state":"active+clean","last_fresh":"2026-02-24T07:53:15.746198+0000","last_change":"2026-02-24T07:53:01.242353+0000","last_active":"2026-02-24T07:53:15.746198+0000","last_peered":"2026-02-24T07:53:15.746198+0000","last_clean":"2026-02-24T07:53:15.746198+0000","last_became_active":"2026-02-24T07:51:53.900431+0000","last_became_peered":"2026-02-24T07:51:53.900431+0000","last_unstale":"2026-02-24T07:53:15.746198+0000","last_undegraded":"2026-02-24T07:53:15.746198+0000","last_fullsized":"2026-02-24T07:53:15.746198+0000","mapping_epoch":74,"log_start":"84'31437","ondisk_log_start":"84'31437","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"84'26061","last_scrub_stamp":"2026-02-24T07:53:01.242306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_clean_scrub_stamp":"2026-02-24T07:53:01.242306+0000","objects_scrubbed":25674,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:54:22.699112+0000","scrub_duration":2129,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2060451840,"num_objects":31440,"num_object_clones":0,"num_object_copies":94320,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31440,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":31440,"num_write_kb":2012160,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.e","version":"84'31332","reported_seq":31614,"reported_epoch":84,"state":"active+clean","last_fresh":"2026-02-24T07:53:15.763033+0000","last_change":"2026-02-24T07:52:56.767335+0000","last_active":"2026-02-24T07:53:15.763033+0000","last_peered":"2026-02-24T07:53:15.763033+0000","last_clean":"2026-02-24T07:53:15.763033+0000","last_became_active":"2026-02-24T07:51:53.899294+0000","last_became_peered":"2026-02-24T07:51:53.899294+0000","last_unstale":"2026-02-24T07:53:15.763033+0000","last_undegraded":"2026-02-24T07:53:15.763033+0000","last_fullsized":"2026-02-24T07:53:15.763033+0000","mapping_epoch":74,"log_start":"84'31329","ondisk_log_start":"84'31329","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"83'24557","last_scrub_stamp":"2026-02-24T07:52:56.767267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_clean_scrub_stamp":"2026-02-24T07:52:56.767267+0000","objects_scrubbed":24118,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:54:24.913755+0000","scrub_duration":2806,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2053373952,"num_objects":31332,"num_object_clones":0,"num_object_copies":93996,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31332,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":31332,"num_write_kb":2005248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"84'31538","reported_seq":31568,"reported_epoch":84,"state":"active+clean","last_fresh":"2026-02-24T07:53:15.774254+0000","last_change":"2026-02-24T07:51:53.900129+0000","last_active":"2026-02-24T07:53:15.774254+0000","last_peered":"2026-02-24T07:53:15.774254+0000","last_clean":"2026-02-24T07:53:15.774254+0000","last_became_active":"2026-02-24T07:51:53.899772+0000","last_became_peered":"2026-02-24T07:51:53.899772+0000","last_unstale":"2026-02-24T07:53:15.774254+0000","last_undegraded":"2026-02-24T07:53:15.774254+0000","last_fullsized":"2026-02-24T07:53:15.774254+0000","mapping_epoch":74,"log_start":"84'31535","ondisk_log_start":"84'31535","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_clean_scrub_stamp":"2026-02-24T07:51:52.890170+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:53:19.516698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2066874368,"num_objects":31538,"num_object_clones":0,"num_object_copies":94614,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31538,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":31538,"num_write_kb":2018432,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,3],"acting":[0,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.c","version":"84'31614","reported_seq":31807,"reported_epoch":84,"state":"active+clean+scrubbing","last_fresh":"2026-02-24T07:53:15.855964+0000","last_change":"2026-02-24T07:53:14.227569+0000","last_active":"2026-02-24T07:53:15.855964+0000","last_peered":"2026-02-24T07:53:15.855964+0000","last_clean":"2026-02-24T07:53:15.855964+0000","last_became_active":"2026-02-24T07:51:53.898285+0000","last_became_peered":"2026-02-24T07:51:53.898285+0000","last_unstale":"2026-02-24T07:53:15.855964+0000","last_undegraded":"2026-02-24T07:53:15.855964+0000","last_fullsized":"2026-02-24T07:53:15.855964+0000","mapping_epoch":74,"log_start":"84'31611","ondisk_log_start":"84'31611","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_clean_scrub_stamp":"2026-02-24T07:51:52.890170+0000","objects_scrubbed":15949,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"scrubbing for 2s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2071855104,"num_objects":31614,"num_object_clones":0,"num_object_copies":94842,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31614,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":31614,"num_write_kb":2023296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.b","version":"84'31409","reported_seq":31439,"reported_epoch":84,"state":"active+clean","last_fresh":"2026-02-24T07:53:15.774230+0000","last_change":"2026-02-24T07:51:53.900298+0000","last_active":"2026-02-24T07:53:15.774230+0000","last_peered":"2026-02-24T07:53:15.774230+0000","last_clean":"2026-02-24T07:53:15.774230+0000","last_became_active":"2026-02-24T07:51:53.899901+0000","last_became_peered":"2026-02-24T07:51:53.899901+0000","last_unstale":"2026-02-24T07:53:15.774230+0000","last_undegraded":"2026-02-24T07:53:15.774230+0000","last_fullsized":"2026-02-24T07:53:15.774230+0000","mapping_epoch":74,"log_start":"84'31406","ondisk_log_start":"84'31406","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_clean_scrub_stamp":"2026-02-24T07:51:52.890170+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2058420224,"num_objects":31409,"num_object_clones":0,"num_object_copies":94227,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31409,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":31409,"num_write_kb":2010176,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"84'31472","reported_seq":31815,"reported_epoch":84,"state":"active+clean","last_fresh":"2026-02-24T07:53:15.770605+0000","last_change":"2026-02-24T07:53:14.212339+0000","last_active":"2026-02-24T07:53:15.770605+0000","last_peered":"2026-02-24T07:53:15.770605+0000","last_clean":"2026-02-24T07:53:15.770605+0000","last_became_active":"2026-02-24T07:51:53.899549+0000","last_became_peered":"2026-02-24T07:51:53.899549+0000","last_unstale":"2026-02-24T07:53:15.770605+0000","last_undegraded":"2026-02-24T07:53:15.770605+0000","last_fullsized":"2026-02-24T07:53:15.770605+0000","mapping_epoch":74,"log_start":"84'31469","ondisk_log_start":"84'31469","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"84'30986","last_scrub_stamp":"2026-02-24T07:53:14.212265+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_clean_scrub_stamp":"2026-02-24T07:53:14.212265+0000","objects_scrubbed":30490,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:54:35.341423+0000","scrub_duration":3381,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2062548992,"num_objects":31472,"num_object_clones":0,"num_object_copies":94416,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31472,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":31472,"num_write_kb":2014208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,1],"acting":[0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.9","version":"84'31806","reported_seq":31836,"reported_epoch":84,"state":"active+clean","last_fresh":"2026-02-24T07:53:15.762404+0000","last_change":"2026-02-24T07:51:53.901985+0000","last_active":"2026-02-24T07:53:15.762404+0000","last_peered":"2026-02-24T07:53:15.762404+0000","last_clean":"2026-02-24T07:53:15.762404+0000","last_became_active":"2026-02-24T07:51:53.901610+0000","last_became_peered":"2026-02-24T07:51:53.901610+0000","last_unstale":"2026-02-24T07:53:15.762404+0000","last_undegraded":"2026-02-24T07:53:15.762404+0000","last_fullsized":"2026-02-24T07:53:15.762404+0000","mapping_epoch":74,"log_start":"84'31803","ondisk_log_start":"84'31803","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_clean_scrub_stamp":"2026-02-24T07:51:52.890170+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2084438016,"num_objects":31806,"num_object_clones":0,"num_object_copies":95418,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31806,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":31806,"num_write_kb":2035584,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.8","version":"84'31634","reported_seq":31927,"reported_epoch":84,"state":"active+clean","last_fresh":"2026-02-24T07:53:15.770553+0000","last_change":"2026-02-24T07:53:00.549866+0000","last_active":"2026-02-24T07:53:15.770553+0000","last_peered":"2026-02-24T07:53:15.770553+0000","last_clean":"2026-02-24T07:53:15.770553+0000","last_became_active":"2026-02-24T07:51:53.900207+0000","last_became_peered":"2026-02-24T07:51:53.900207+0000","last_unstale":"2026-02-24T07:53:15.770553+0000","last_undegraded":"2026-02-24T07:53:15.770553+0000","last_fullsized":"2026-02-24T07:53:15.770553+0000","mapping_epoch":74,"log_start":"84'31631","ondisk_log_start":"84'31631","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"84'25943","last_scrub_stamp":"2026-02-24T07:53:00.549789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_clean_scrub_stamp":"2026-02-24T07:53:00.549789+0000","objects_scrubbed":25486,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:54:18.896412+0000","scrub_duration":2647,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2073165824,"num_objects":31634,"num_object_clones":0,"num_object_copies":94902,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31634,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":31634,"num_write_kb":2024576,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.7","version":"84'31230","reported_seq":31521,"reported_epoch":84,"state":"active+clean","last_fresh":"2026-02-24T07:53:15.763841+0000","last_change":"2026-02-24T07:53:00.188453+0000","last_active":"2026-02-24T07:53:15.763841+0000","last_peered":"2026-02-24T07:53:15.763841+0000","last_clean":"2026-02-24T07:53:15.763841+0000","last_became_active":"2026-02-24T07:51:53.899472+0000","last_became_peered":"2026-02-24T07:51:53.899472+0000","last_unstale":"2026-02-24T07:53:15.763841+0000","last_undegraded":"2026-02-24T07:53:15.763841+0000","last_fullsized":"2026-02-24T07:53:15.763841+0000","mapping_epoch":74,"log_start":"84'31227","ondisk_log_start":"84'31227","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"84'25630","last_scrub_stamp":"2026-02-24T07:53:00.188390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_clean_scrub_stamp":"2026-02-24T07:53:00.188390+0000","objects_scrubbed":25246,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:54:07.242502+0000","scrub_duration":2262,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2046689280,"num_objects":31230,"num_object_clones":0,"num_object_copies":93690,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31230,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":31230,"num_write_kb":1998720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.6","version":"84'31324","reported_seq":31657,"reported_epoch":84,"state":"active+clean","last_fresh":"2026-02-24T07:53:15.850911+0000","last_change":"2026-02-24T07:53:10.804133+0000","last_active":"2026-02-24T07:53:15.850911+0000","last_peered":"2026-02-24T07:53:15.850911+0000","last_clean":"2026-02-24T07:53:15.850911+0000","last_became_active":"2026-02-24T07:51:53.898308+0000","last_became_peered":"2026-02-24T07:51:53.898308+0000","last_unstale":"2026-02-24T07:53:15.850911+0000","last_undegraded":"2026-02-24T07:53:15.850911+0000","last_fullsized":"2026-02-24T07:53:15.850911+0000","mapping_epoch":74,"log_start":"84'31321","ondisk_log_start":"84'31321","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"84'29829","last_scrub_stamp":"2026-02-24T07:53:10.804104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_clean_scrub_stamp":"2026-02-24T07:53:10.804104+0000","objects_scrubbed":29351,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:54:13.621421+0000","scrub_duration":2501,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2052849664,"num_objects":31324,"num_object_clones":0,"num_object_copies":93972,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31324,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":31324,"num_write_kb":2004736,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.5","version":"84'31333","reported_seq":31363,"reported_epoch":84,"state":"active+clean","last_fresh":"2026-02-24T07:53:15.772604+0000","last_change":"2026-02-24T07:51:53.900139+0000","last_active":"2026-02-24T07:53:15.772604+0000","last_peered":"2026-02-24T07:53:15.772604+0000","last_clean":"2026-02-24T07:53:15.772604+0000","last_became_active":"2026-02-24T07:51:53.899749+0000","last_became_peered":"2026-02-24T07:51:53.899749+0000","last_unstale":"2026-02-24T07:53:15.772604+0000","last_undegraded":"2026-02-24T07:53:15.772604+0000","last_fullsized":"2026-02-24T07:53:15.772604+0000","mapping_epoch":74,"log_start":"84'31330","ondisk_log_start":"84'31330","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_clean_scrub_stamp":"2026-02-24T07:51:52.890170+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2053439488,"num_objects":31333,"num_object_clones":0,"num_object_copies":93999,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31333,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":31333,"num_write_kb":2005312,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"84'31428","reported_seq":31741,"reported_epoch":84,"state":"active+clean","last_fresh":"2026-02-24T07:53:15.769720+0000","last_change":"2026-02-24T07:53:05.312364+0000","last_active":"2026-02-24T07:53:15.769720+0000","last_peered":"2026-02-24T07:53:15.769720+0000","last_clean":"2026-02-24T07:53:15.769720+0000","last_became_active":"2026-02-24T07:51:53.900042+0000","last_became_peered":"2026-02-24T07:51:53.900042+0000","last_unstale":"2026-02-24T07:53:15.769720+0000","last_undegraded":"2026-02-24T07:53:15.769720+0000","last_fullsized":"2026-02-24T07:53:15.769720+0000","mapping_epoch":74,"log_start":"84'31425","ondisk_log_start":"84'31425","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"84'27738","last_scrub_stamp":"2026-02-24T07:53:05.312312+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_clean_scrub_stamp":"2026-02-24T07:53:05.312312+0000","objects_scrubbed":27245,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:54:32.166518+0000","scrub_duration":2472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2059665408,"num_objects":31428,"num_object_clones":0,"num_object_copies":94284,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31428,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":31428,"num_write_kb":2011392,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.2","version":"84'31762","reported_seq":31871,"reported_epoch":84,"state":"active+clean+scrubbing","last_fresh":"2026-02-24T07:53:15.759093+0000","last_change":"2026-02-24T07:53:15.039521+0000","last_active":"2026-02-24T07:53:15.759093+0000","last_peered":"2026-02-24T07:53:15.759093+0000","last_clean":"2026-02-24T07:53:15.759093+0000","last_became_active":"2026-02-24T07:51:53.900096+0000","last_became_peered":"2026-02-24T07:51:53.900096+0000","last_unstale":"2026-02-24T07:53:15.759093+0000","last_undegraded":"2026-02-24T07:53:15.759093+0000","last_fullsized":"2026-02-24T07:53:15.759093+0000","mapping_epoch":74,"log_start":"84'31759","ondisk_log_start":"84'31759","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_clean_scrub_stamp":"2026-02-24T07:51:52.890170+0000","objects_scrubbed":7449,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"scrubbing for 1s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2081554432,"num_objects":31762,"num_object_clones":0,"num_object_copies":95286,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31762,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":31762,"num_write_kb":2032768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"84'31086","reported_seq":31427,"reported_epoch":84,"state":"active+clean","last_fresh":"2026-02-24T07:53:15.764046+0000","last_change":"2026-02-24T07:53:14.051194+0000","last_active":"2026-02-24T07:53:15.764046+0000","last_peered":"2026-02-24T07:53:15.764046+0000","last_clean":"2026-02-24T07:53:15.764046+0000","last_became_active":"2026-02-24T07:51:53.901027+0000","last_became_peered":"2026-02-24T07:51:53.901027+0000","last_unstale":"2026-02-24T07:53:15.764046+0000","last_undegraded":"2026-02-24T07:53:15.764046+0000","last_fullsized":"2026-02-24T07:53:15.764046+0000","mapping_epoch":74,"log_start":"84'31083","ondisk_log_start":"84'31083","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"84'30565","last_scrub_stamp":"2026-02-24T07:53:14.051154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_clean_scrub_stamp":"2026-02-24T07:53:14.051154+0000","objects_scrubbed":30140,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:54:38.448076+0000","scrub_duration":3147,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2037252096,"num_objects":31086,"num_object_clones":0,"num_object_copies":93258,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31086,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":31086,"num_write_kb":1989504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.0","version":"84'31757","reported_seq":31787,"reported_epoch":84,"state":"active+clean","last_fresh":"2026-02-24T07:53:15.771216+0000","last_change":"2026-02-24T07:51:53.898740+0000","last_active":"2026-02-24T07:53:15.771216+0000","last_peered":"2026-02-24T07:53:15.771216+0000","last_clean":"2026-02-24T07:53:15.771216+0000","last_became_active":"2026-02-24T07:51:53.898551+0000","last_became_peered":"2026-02-24T07:51:53.898551+0000","last_unstale":"2026-02-24T07:53:15.771216+0000","last_undegraded":"2026-02-24T07:53:15.771216+0000","last_fullsized":"2026-02-24T07:53:15.771216+0000","mapping_epoch":74,"log_start":"84'31754","ondisk_log_start":"84'31754","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_clean_scrub_stamp":"2026-02-24T07:51:52.890170+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2081226752,"num_objects":31757,"num_object_clones":0,"num_object_copies":95271,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31757,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":31757,"num_write_kb":2032448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.4","version":"84'31360","reported_seq":31697,"reported_epoch":84,"state":"active+clean","last_fresh":"2026-02-24T07:53:15.745607+0000","last_change":"2026-02-24T07:53:11.592191+0000","last_active":"2026-02-24T07:53:15.745607+0000","last_peered":"2026-02-24T07:53:15.745607+0000","last_clean":"2026-02-24T07:53:15.745607+0000","last_became_active":"2026-02-24T07:51:53.899675+0000","last_became_peered":"2026-02-24T07:51:53.899675+0000","last_unstale":"2026-02-24T07:53:15.745607+0000","last_undegraded":"2026-02-24T07:53:15.745607+0000","last_fullsized":"2026-02-24T07:53:15.745607+0000","mapping_epoch":74,"log_start":"84'31357","ondisk_log_start":"84'31357","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"84'30127","last_scrub_stamp":"2026-02-24T07:53:11.592155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_clean_scrub_stamp":"2026-02-24T07:53:11.592155+0000","objects_scrubbed":29709,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:54:23.794472+0000","scrub_duration":2432,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2055208960,"num_objects":31360,"num_object_clones":0,"num_object_copies":94080,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31360,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":31360,"num_write_kb":2007040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"29'99","reported_seq":357,"reported_epoch":84,"state":"active+clean","last_fresh":"2026-02-24T07:52:58.893578+0000","last_change":"2026-02-24T07:52:56.946143+0000","last_active":"2026-02-24T07:52:58.893578+0000","last_peered":"2026-02-24T07:52:58.893578+0000","last_clean":"2026-02-24T07:52:58.893578+0000","last_became_active":"2026-02-24T07:42:54.186719+0000","last_became_peered":"2026-02-24T07:42:54.186719+0000","last_unstale":"2026-02-24T07:52:58.893578+0000","last_undegraded":"2026-02-24T07:52:58.893578+0000","last_fullsized":"2026-02-24T07:52:58.893578+0000","mapping_epoch":27,"log_start":"29'96","ondisk_log_start":"29'96","created":20,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'99","last_scrub_stamp":"2026-02-24T07:52:56.946109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:42:43.640776+0000","last_clean_scrub_stamp":"2026-02-24T07:52:56.946109+0000","objects_scrubbed":2,"log_size":3,"log_dups_size":96,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:54:05.896060+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":240,"num_write":135,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":16,"stat_sum":{"num_bytes":32999014400,"num_objects":503525,"num_object_clones":0,"num_object_copies":1510575,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":503525,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":503525,"num_write_kb":32225600,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":99003727872,"data_stored":99003727872,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":48,"ondisk_log_size":48,"up":48,"acting":48,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":240,"num_write":135,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1682,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":3,"up_from":25,"seq":107374182527,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":30032860,"kb_used_data":27950280,"kb_used_omap":27,"kb_used_meta":2082532,"kb_avail":703970340,"statfs":{"total":751619276800,"available":720865628160,"internally_reserved":0,"allocated":28621086720,"data_stored":28620493187,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":27824,"internal_metadata":2132513616},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":19,"seq":81604378753,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":23969372,"kb_used_data":22037832,"kb_used_omap":26,"kb_used_meta":1931493,"kb_avail":710033828,"statfs":{"total":751619276800,"available":727074639872,"internally_reserved":0,"allocated":22566739968,"data_stored":22566146435,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":27154,"internal_metadata":1977849326},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542274,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":21655832,"kb_used_data":20023748,"kb_used_omap":20,"kb_used_meta":1632043,"kb_avail":712347368,"statfs":{"total":751619276800,"available":729443704832,"internally_reserved":0,"allocated":20504317952,"data_stored":20503727971,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":20777,"internal_metadata":1671212759},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,2,0,1],"upper_bound":16},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705797,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":27852188,"kb_used_data":25918152,"kb_used_omap":24,"kb_used_meta":1933991,"kb_avail":706151012,"statfs":{"total":751619276800,"available":723098636288,"internally_reserved":0,"allocated":26540187648,"data_stored":26539594115,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":25090,"internal_metadata":1980407294},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1],"upper_bound":4},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1682,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":26806386688,"data_stored":26806386688,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":20652294144,"data_stored":20652294144,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":22701801472,"data_stored":22701801472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28843245568,"data_stored":28843245568,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-24T07:53:21.747 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph osd pool set unique_pool_3 pgp_num 36 2026-02-24T07:53:21.959 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:53:21.998 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 87 16 537775 537759 386.288 267.812 0.00876731 0.00258759 2026-02-24T07:53:22.201 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:53:22.137+0000 7f824ec6f640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:53:22.201 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:53:22.137+0000 7f824ec6f640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:53:22.201 INFO:teuthology.orchestra.run.trial035.stderr:osd.0: osd_enable_op_tracker = '' 2026-02-24T07:53:22.202 INFO:teuthology.orchestra.run.trial035.stdout:osd.0: {} 2026-02-24T07:53:22.202 INFO:teuthology.orchestra.run.trial035.stdout:osd.1: {} 2026-02-24T07:53:22.202 INFO:teuthology.orchestra.run.trial035.stdout:osd.2: {} 2026-02-24T07:53:22.202 INFO:teuthology.orchestra.run.trial035.stdout:osd.3: {} 2026-02-24T07:53:22.202 INFO:teuthology.orchestra.run.trial035.stderr:osd.1: osd_enable_op_tracker = '' 2026-02-24T07:53:22.202 INFO:teuthology.orchestra.run.trial035.stderr:osd.2: osd_enable_op_tracker = '' 2026-02-24T07:53:22.202 INFO:teuthology.orchestra.run.trial035.stderr:osd.3: osd_enable_op_tracker = '' 2026-02-24T07:53:22.998 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 88 16 542479 542463 385.24 294 0.00205736 0.00259465 2026-02-24T07:53:23.056 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:53:23.054+0000 7f2a11261640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:53:23.056 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:53:23.055+0000 7f2a11261640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:53:23.193 INFO:teuthology.orchestra.run.trial035.stderr:Error EPERM: crimson_allow_pg_split is false; pgp_num increase denied 2026-02-24T07:53:23.194 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:53:22 trial035 ceph-mon[39284]: from='client.15266 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2026-02-24T07:53:23.194 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:53:22 trial035 ceph-mon[39284]: pgmap v369: 17 pgs: 1 active+clean+scrubbing, 16 active+clean; 32 GiB data, 104 GiB used, 2.6 TiB / 2.7 TiB avail; 441 MiB/s wr, 7.06k op/s 2026-02-24T07:53:23.194 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:53:22 trial035 ceph-mon[39284]: purged_snaps scrub starts 2026-02-24T07:53:23.194 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:53:22 trial035 ceph-mon[39284]: purged_snaps scrub ok 2026-02-24T07:53:23.195 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:53:22 trial035 ceph-mon[39284]: 5.9 scrub ok 2026-02-24T07:53:23.195 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:53:22 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/2471851649' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2026-02-24T07:53:23.232 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-24T07:53:23.999 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 89 16 548971 548955 385.47 405.75 0.00192054 0.0025931 2026-02-24T07:53:24.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:53:23 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/3879415684' entity='client.admin' cmd={"prefix": "osd pool set", "pool": "unique_pool_3", "var": "pgp_num", "val": "36"} : dispatch 2026-02-24T07:53:24.751 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2026-02-24T07:53:24.892 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:53:24.999 INFO:tasks.radosbench.radosbench.0.trial035.stdout:Total time run: 90.0058 2026-02-24T07:53:24.999 INFO:tasks.radosbench.radosbench.0.trial035.stdout:Total writes made: 555569 2026-02-24T07:53:24.999 INFO:tasks.radosbench.radosbench.0.trial035.stdout:Write size: 65536 2026-02-24T07:53:24.999 INFO:tasks.radosbench.radosbench.0.trial035.stdout:Object size: 65536 2026-02-24T07:53:24.999 INFO:tasks.radosbench.radosbench.0.trial035.stdout:Bandwidth (MB/sec): 385.787 2026-02-24T07:53:24.999 INFO:tasks.radosbench.radosbench.0.trial035.stdout:Stddev Bandwidth: 51.847 2026-02-24T07:53:24.999 INFO:tasks.radosbench.radosbench.0.trial035.stdout:Max bandwidth (MB/sec): 495.312 2026-02-24T07:53:24.999 INFO:tasks.radosbench.radosbench.0.trial035.stdout:Min bandwidth (MB/sec): 227.688 2026-02-24T07:53:25.000 INFO:tasks.radosbench.radosbench.0.trial035.stdout:Average IOPS: 6172 2026-02-24T07:53:25.000 INFO:tasks.radosbench.radosbench.0.trial035.stdout:Stddev IOPS: 829.551 2026-02-24T07:53:25.000 INFO:tasks.radosbench.radosbench.0.trial035.stdout:Max IOPS: 7925 2026-02-24T07:53:25.000 INFO:tasks.radosbench.radosbench.0.trial035.stdout:Min IOPS: 3643 2026-02-24T07:53:25.000 INFO:tasks.radosbench.radosbench.0.trial035.stdout:Average Latency(s): 0.00259088 2026-02-24T07:53:25.000 INFO:tasks.radosbench.radosbench.0.trial035.stdout:Stddev Latency(s): 0.00233118 2026-02-24T07:53:25.000 INFO:tasks.radosbench.radosbench.0.trial035.stdout:Max latency(s): 0.0517474 2026-02-24T07:53:25.000 INFO:tasks.radosbench.radosbench.0.trial035.stdout:Min latency(s): 0.000691325 2026-02-24T07:53:25.001 INFO:tasks.radosbench.radosbench.0.trial035.stdout:Cleaning up (deleting benchmark objects) 2026-02-24T07:53:25.012 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:53:25.011+0000 7f05add6b640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:53:25.013 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:53:25.012+0000 7f05add6b640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:53:25.028 INFO:teuthology.orchestra.run.trial035.stdout:osd.0: {} 2026-02-24T07:53:25.028 INFO:teuthology.orchestra.run.trial035.stderr:osd.0: osd_enable_op_tracker = '' 2026-02-24T07:53:25.028 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:53:24 trial035 ceph-mon[39284]: pgmap v370: 17 pgs: 1 active+clean+scrubbing, 16 active+clean; 32 GiB data, 104 GiB used, 2.6 TiB / 2.7 TiB avail; 274 MiB/s wr, 4.38k op/s 2026-02-24T07:53:25.029 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:53:24 trial035 ceph-mon[39284]: 5.5 scrub starts 2026-02-24T07:53:25.041 INFO:teuthology.orchestra.run.trial035.stdout:osd.1: {} 2026-02-24T07:53:25.041 INFO:teuthology.orchestra.run.trial035.stderr:osd.1: osd_enable_op_tracker = '' 2026-02-24T07:53:25.053 INFO:teuthology.orchestra.run.trial035.stdout:osd.2: {} 2026-02-24T07:53:25.053 INFO:teuthology.orchestra.run.trial035.stderr:osd.2: osd_enable_op_tracker = '' 2026-02-24T07:53:25.065 INFO:teuthology.orchestra.run.trial035.stdout:osd.3: {} 2026-02-24T07:53:25.065 INFO:teuthology.orchestra.run.trial035.stderr:osd.3: osd_enable_op_tracker = '' 2026-02-24T07:53:26.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:53:25 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/399288508' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2026-02-24T07:53:27.105 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2026-02-24T07:53:27.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:53:26 trial035 ceph-mon[39284]: pgmap v371: 17 pgs: 1 active+clean+scrubbing, 16 active+clean; 32 GiB data, 104 GiB used, 2.6 TiB / 2.7 TiB avail; 274 MiB/s wr, 4.38k op/s 2026-02-24T07:53:27.205 INFO:journalctl@ceph.mgr.x.trial035.stdout:Feb 24 07:53:26 trial035 ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-mgr-x[39621]: ::ffff:10.20.193.91 - - [24/Feb/2026:07:53:26] "GET /metrics HTTP/1.1" 200 4214 "" "Prometheus/3.6.0" 2026-02-24T07:53:27.242 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:53:27.356 INFO:journalctl@ceph.prometheus.a.trial091.stdout:Feb 24 07:53:26 trial091 ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-prometheus-a[66007]: time=2026-02-24T07:53:26.973Z level=ERROR source=scrape.go:1630 msg="Failed to determine correct type of scrape target." component="scrape manager" scrape_pool=ceph target=http://trial035:9283/metrics content_type="text/html;charset=utf-8" fallback_media_type="" err="received unsupported Content-Type \"text/html;charset=utf-8\" and no fallback_scrape_protocol specified for target" 2026-02-24T07:53:27.364 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:53:27.363+0000 7f5d460e5640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:53:27.364 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:53:27.363+0000 7f5d460e5640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:53:27.378 INFO:teuthology.orchestra.run.trial035.stdout:osd.0: {} 2026-02-24T07:53:27.379 INFO:teuthology.orchestra.run.trial035.stderr:osd.0: osd_enable_op_tracker = '' 2026-02-24T07:53:27.391 INFO:teuthology.orchestra.run.trial035.stdout:osd.1: {} 2026-02-24T07:53:27.391 INFO:teuthology.orchestra.run.trial035.stderr:osd.1: osd_enable_op_tracker = '' 2026-02-24T07:53:27.406 INFO:teuthology.orchestra.run.trial035.stdout:osd.2: {} 2026-02-24T07:53:27.406 INFO:teuthology.orchestra.run.trial035.stderr:osd.2: osd_enable_op_tracker = '' 2026-02-24T07:53:27.419 INFO:teuthology.orchestra.run.trial035.stdout:osd.3: {} 2026-02-24T07:53:27.419 INFO:teuthology.orchestra.run.trial035.stderr:osd.3: osd_enable_op_tracker = '' 2026-02-24T07:53:27.954 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:53:27 trial035 ceph-mon[39284]: 5.b scrub starts 2026-02-24T07:53:27.954 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:53:27 trial035 ceph-mon[39284]: 5.5 scrub ok 2026-02-24T07:53:27.954 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:53:27 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/2951572363' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2026-02-24T07:53:28.233 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2026-02-24T07:53:28.233 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2026-02-24T07:53:28.234 INFO:tasks.thrashosds.thrasher:trim_stale_osdmaps 2026-02-24T07:53:28.234 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph tell osd.0 'trim stale osdmaps' 2026-02-24T07:53:28.372 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:53:28.732 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:53:28.731+0000 7f412c3ff640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:53:28.733 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:53:28.731+0000 7f412c3ff640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:53:28.757 INFO:teuthology.orchestra.run.trial035.stderr:no valid command found; 10 closest matches: 2026-02-24T07:53:28.758 INFO:teuthology.orchestra.run.trial035.stderr:0 2026-02-24T07:53:28.758 INFO:teuthology.orchestra.run.trial035.stderr:1 2026-02-24T07:53:28.758 INFO:teuthology.orchestra.run.trial035.stderr:2 2026-02-24T07:53:28.758 INFO:teuthology.orchestra.run.trial035.stderr:abort 2026-02-24T07:53:28.758 INFO:teuthology.orchestra.run.trial035.stderr:assert 2026-02-24T07:53:28.758 INFO:teuthology.orchestra.run.trial035.stderr:bench [] [] [] [] 2026-02-24T07:53:28.758 INFO:teuthology.orchestra.run.trial035.stderr:bluefs debug_inject_read_zeros 2026-02-24T07:53:28.758 INFO:teuthology.orchestra.run.trial035.stderr:bluefs files list 2026-02-24T07:53:28.758 INFO:teuthology.orchestra.run.trial035.stderr:bluefs stats 2026-02-24T07:53:28.759 INFO:teuthology.orchestra.run.trial035.stderr:bluestore allocator dump block 2026-02-24T07:53:28.759 INFO:teuthology.orchestra.run.trial035.stderr:Error EINVAL: invalid command 2026-02-24T07:53:29.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:53:28 trial035 ceph-mon[39284]: pgmap v372: 17 pgs: 1 active+clean+scrubbing, 16 active+clean; 34 GiB data, 109 GiB used, 2.6 TiB / 2.7 TiB avail; 85 B/s rd, 395 MiB/s wr, 6.80k op/s 2026-02-24T07:53:29.433 DEBUG:teuthology.orchestra.run:got remote process result: 22 2026-02-24T07:53:29.434 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2026-02-24T07:53:29.434 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph tell osd.1 'trim stale osdmaps' 2026-02-24T07:53:29.568 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:53:29.944 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2026-02-24T07:53:29.954 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:53:29 trial035 ceph-mon[39284]: from='mgr.14284 10.20.193.35:0/3681373222' entity='mgr.x' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2026-02-24T07:53:29.994 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:53:29.981+0000 7f2e1d31e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:53:29.994 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:53:29.982+0000 7f2e1d31e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:53:30.001 INFO:teuthology.orchestra.run.trial035.stderr:no valid command found; 10 closest matches: 2026-02-24T07:53:30.001 INFO:teuthology.orchestra.run.trial035.stderr:0 2026-02-24T07:53:30.001 INFO:teuthology.orchestra.run.trial035.stderr:1 2026-02-24T07:53:30.002 INFO:teuthology.orchestra.run.trial035.stderr:2 2026-02-24T07:53:30.002 INFO:teuthology.orchestra.run.trial035.stderr:abort 2026-02-24T07:53:30.002 INFO:teuthology.orchestra.run.trial035.stderr:assert 2026-02-24T07:53:30.002 INFO:teuthology.orchestra.run.trial035.stderr:bench [] [] [] [] 2026-02-24T07:53:30.002 INFO:teuthology.orchestra.run.trial035.stderr:bluefs debug_inject_read_zeros 2026-02-24T07:53:30.002 INFO:teuthology.orchestra.run.trial035.stderr:bluefs files list 2026-02-24T07:53:30.002 INFO:teuthology.orchestra.run.trial035.stderr:bluefs stats 2026-02-24T07:53:30.002 INFO:teuthology.orchestra.run.trial035.stderr:bluestore allocator dump block 2026-02-24T07:53:30.002 INFO:teuthology.orchestra.run.trial035.stderr:Error EINVAL: invalid command 2026-02-24T07:53:30.127 DEBUG:teuthology.orchestra.run:got remote process result: 22 2026-02-24T07:53:30.128 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2026-02-24T07:53:30.128 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph tell osd.2 'trim stale osdmaps' 2026-02-24T07:53:30.131 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:53:30.259 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:53:30.257+0000 7f90c4260640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:53:30.259 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:53:30.258+0000 7f90c4260640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:53:30.275 INFO:teuthology.orchestra.run.trial035.stdout:osd.0: {} 2026-02-24T07:53:30.276 INFO:teuthology.orchestra.run.trial035.stderr:osd.0: osd_enable_op_tracker = '' 2026-02-24T07:53:30.279 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:53:30.289 INFO:teuthology.orchestra.run.trial035.stdout:osd.1: {} 2026-02-24T07:53:30.289 INFO:teuthology.orchestra.run.trial035.stderr:osd.1: osd_enable_op_tracker = '' 2026-02-24T07:53:30.304 INFO:teuthology.orchestra.run.trial035.stdout:osd.2: {} 2026-02-24T07:53:30.304 INFO:teuthology.orchestra.run.trial035.stderr:osd.2: osd_enable_op_tracker = '' 2026-02-24T07:53:30.318 INFO:teuthology.orchestra.run.trial035.stdout:osd.3: {} 2026-02-24T07:53:30.318 INFO:teuthology.orchestra.run.trial035.stderr:osd.3: osd_enable_op_tracker = '' 2026-02-24T07:53:30.392 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:53:30.391+0000 7f09a6cc6640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:53:30.393 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:53:30.391+0000 7f09a6cc6640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:53:30.414 INFO:teuthology.orchestra.run.trial035.stderr:no valid command found; 10 closest matches: 2026-02-24T07:53:30.414 INFO:teuthology.orchestra.run.trial035.stderr:0 2026-02-24T07:53:30.414 INFO:teuthology.orchestra.run.trial035.stderr:1 2026-02-24T07:53:30.414 INFO:teuthology.orchestra.run.trial035.stderr:2 2026-02-24T07:53:30.414 INFO:teuthology.orchestra.run.trial035.stderr:abort 2026-02-24T07:53:30.414 INFO:teuthology.orchestra.run.trial035.stderr:assert 2026-02-24T07:53:30.415 INFO:teuthology.orchestra.run.trial035.stderr:bench [] [] [] [] 2026-02-24T07:53:30.415 INFO:teuthology.orchestra.run.trial035.stderr:bluefs debug_inject_read_zeros 2026-02-24T07:53:30.415 INFO:teuthology.orchestra.run.trial035.stderr:bluefs files list 2026-02-24T07:53:30.415 INFO:teuthology.orchestra.run.trial035.stderr:bluefs stats 2026-02-24T07:53:30.415 INFO:teuthology.orchestra.run.trial035.stderr:bluestore allocator dump block 2026-02-24T07:53:30.415 INFO:teuthology.orchestra.run.trial035.stderr:Error EINVAL: invalid command 2026-02-24T07:53:30.448 DEBUG:teuthology.orchestra.run:got remote process result: 22 2026-02-24T07:53:30.449 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2026-02-24T07:53:30.449 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph tell osd.3 'trim stale osdmaps' 2026-02-24T07:53:30.628 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:53:30.746 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:53:30.744+0000 7f0aed1d2640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:53:30.746 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:53:30.745+0000 7f0aed1d2640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:53:30.767 INFO:teuthology.orchestra.run.trial035.stderr:no valid command found; 10 closest matches: 2026-02-24T07:53:30.767 INFO:teuthology.orchestra.run.trial035.stderr:0 2026-02-24T07:53:30.767 INFO:teuthology.orchestra.run.trial035.stderr:1 2026-02-24T07:53:30.767 INFO:teuthology.orchestra.run.trial035.stderr:2 2026-02-24T07:53:30.768 INFO:teuthology.orchestra.run.trial035.stderr:abort 2026-02-24T07:53:30.768 INFO:teuthology.orchestra.run.trial035.stderr:assert 2026-02-24T07:53:30.768 INFO:teuthology.orchestra.run.trial035.stderr:bench [] [] [] [] 2026-02-24T07:53:30.768 INFO:teuthology.orchestra.run.trial035.stderr:bluefs debug_inject_read_zeros 2026-02-24T07:53:30.768 INFO:teuthology.orchestra.run.trial035.stderr:bluefs files list 2026-02-24T07:53:30.768 INFO:teuthology.orchestra.run.trial035.stderr:bluefs stats 2026-02-24T07:53:30.768 INFO:teuthology.orchestra.run.trial035.stderr:bluestore allocator dump block 2026-02-24T07:53:30.768 INFO:teuthology.orchestra.run.trial035.stderr:Error EINVAL: invalid command 2026-02-24T07:53:30.800 DEBUG:teuthology.orchestra.run:got remote process result: 22 2026-02-24T07:53:30.801 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2026-02-24T07:53:31.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:53:30 trial035 ceph-mon[39284]: 5.d scrub starts 2026-02-24T07:53:31.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:53:30 trial035 ceph-mon[39284]: pgmap v373: 17 pgs: 1 active+clean+scrubbing, 16 active+clean; 34 GiB data, 109 GiB used, 2.6 TiB / 2.7 TiB avail; 85 B/s rd, 271 MiB/s wr, 4.82k op/s 2026-02-24T07:53:31.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:53:30 trial035 ceph-mon[39284]: 5.b scrub ok 2026-02-24T07:53:31.205 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:53:30 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/536679086' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2026-02-24T07:53:32.364 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2026-02-24T07:53:32.507 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:53:32.621 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:53:32.619+0000 7f4f9f76b640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:53:32.621 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:53:32.619+0000 7f4f9f76b640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:53:32.635 INFO:teuthology.orchestra.run.trial035.stdout:osd.0: {} 2026-02-24T07:53:32.635 INFO:teuthology.orchestra.run.trial035.stderr:osd.0: osd_enable_op_tracker = '' 2026-02-24T07:53:32.648 INFO:teuthology.orchestra.run.trial035.stdout:osd.1: {} 2026-02-24T07:53:32.648 INFO:teuthology.orchestra.run.trial035.stderr:osd.1: osd_enable_op_tracker = '' 2026-02-24T07:53:32.662 INFO:teuthology.orchestra.run.trial035.stdout:osd.2: {} 2026-02-24T07:53:32.662 INFO:teuthology.orchestra.run.trial035.stderr:osd.2: osd_enable_op_tracker = '' 2026-02-24T07:53:32.674 INFO:teuthology.orchestra.run.trial035.stdout:osd.3: {} 2026-02-24T07:53:32.674 INFO:teuthology.orchestra.run.trial035.stderr:osd.3: osd_enable_op_tracker = '' 2026-02-24T07:53:33.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:53:32 trial035 ceph-mon[39284]: pgmap v374: 17 pgs: 1 active+clean+scrubbing, 16 active+clean; 32 GiB data, 102 GiB used, 2.6 TiB / 2.7 TiB avail; 85 B/s rd, 271 MiB/s wr, 7.49k op/s 2026-02-24T07:53:33.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:53:32 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/4147833622' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2026-02-24T07:53:34.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:53:33 trial035 ceph-mon[39284]: 5.d scrub ok 2026-02-24T07:53:34.714 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2026-02-24T07:53:34.857 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:53:34.983 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:53:34.981+0000 7f83f9a41640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:53:34.983 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:53:34.981+0000 7f83f9a41640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:53:34.997 INFO:teuthology.orchestra.run.trial035.stdout:osd.0: {} 2026-02-24T07:53:34.998 INFO:teuthology.orchestra.run.trial035.stderr:osd.0: osd_enable_op_tracker = '' 2026-02-24T07:53:34.998 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:53:34 trial035 ceph-mon[39284]: pgmap v375: 17 pgs: 1 active+clean+scrubbing, 16 active+clean; 32 GiB data, 102 GiB used, 2.6 TiB / 2.7 TiB avail; 85 B/s rd, 121 MiB/s wr, 5.09k op/s 2026-02-24T07:53:35.010 INFO:teuthology.orchestra.run.trial035.stdout:osd.1: {} 2026-02-24T07:53:35.011 INFO:teuthology.orchestra.run.trial035.stderr:osd.1: osd_enable_op_tracker = '' 2026-02-24T07:53:35.022 INFO:teuthology.orchestra.run.trial035.stdout:osd.2: {} 2026-02-24T07:53:35.022 INFO:teuthology.orchestra.run.trial035.stderr:osd.2: osd_enable_op_tracker = '' 2026-02-24T07:53:35.037 INFO:teuthology.orchestra.run.trial035.stdout:osd.3: {} 2026-02-24T07:53:35.037 INFO:teuthology.orchestra.run.trial035.stderr:osd.3: osd_enable_op_tracker = '' 2026-02-24T07:53:35.802 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2026-02-24T07:53:35.803 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph osd reweight 0 1 2026-02-24T07:53:35.940 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:53:36.059 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:53:36.057+0000 7fef72b77640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:53:36.060 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:53:36.057+0000 7fef72b77640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:53:36.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:53:35 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/1544556160' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2026-02-24T07:53:36.205 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:53:35 trial035 ceph-mon[39284]: pgmap v376: 17 pgs: 1 active+clean+scrubbing, 16 active+clean; 32 GiB data, 102 GiB used, 2.6 TiB / 2.7 TiB avail; 85 B/s rd, 121 MiB/s wr, 5.09k op/s 2026-02-24T07:53:36.917 INFO:teuthology.orchestra.run.trial035.stderr:reweighted osd.0 to 1 (10000) 2026-02-24T07:53:36.964 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph osd reweight 1 1 2026-02-24T07:53:37.077 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2026-02-24T07:53:37.100 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:53:37.168 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:53:36 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/1172147145' entity='client.admin' cmd={"prefix": "osd reweight", "id": 0, "weight": 1.0} : dispatch 2026-02-24T07:53:37.168 INFO:journalctl@ceph.mgr.x.trial035.stdout:Feb 24 07:53:36 trial035 ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-mgr-x[39621]: ::ffff:10.20.193.91 - - [24/Feb/2026:07:53:36] "GET /metrics HTTP/1.1" 200 4235 "" "Prometheus/3.6.0" 2026-02-24T07:53:37.216 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:53:37.228 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:53:37.226+0000 7f759bd5b640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:53:37.228 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:53:37.226+0000 7f759bd5b640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:53:37.333 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:53:37.331+0000 7fafa1213640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:53:37.333 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:53:37.331+0000 7fafa1213640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:53:37.347 INFO:teuthology.orchestra.run.trial035.stdout:osd.0: {} 2026-02-24T07:53:37.347 INFO:teuthology.orchestra.run.trial035.stderr:osd.0: osd_enable_op_tracker = '' 2026-02-24T07:53:37.356 INFO:journalctl@ceph.prometheus.a.trial091.stdout:Feb 24 07:53:36 trial091 ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-prometheus-a[66007]: time=2026-02-24T07:53:36.973Z level=ERROR source=scrape.go:1630 msg="Failed to determine correct type of scrape target." component="scrape manager" scrape_pool=ceph target=http://trial035:9283/metrics content_type="text/html;charset=utf-8" fallback_media_type="" err="received unsupported Content-Type \"text/html;charset=utf-8\" and no fallback_scrape_protocol specified for target" 2026-02-24T07:53:37.360 INFO:teuthology.orchestra.run.trial035.stdout:osd.1: {} 2026-02-24T07:53:37.360 INFO:teuthology.orchestra.run.trial035.stderr:osd.1: osd_enable_op_tracker = '' 2026-02-24T07:53:37.376 INFO:teuthology.orchestra.run.trial035.stdout:osd.2: {} 2026-02-24T07:53:37.377 INFO:teuthology.orchestra.run.trial035.stderr:osd.2: osd_enable_op_tracker = '' 2026-02-24T07:53:37.388 INFO:teuthology.orchestra.run.trial035.stdout:osd.3: {} 2026-02-24T07:53:37.389 INFO:teuthology.orchestra.run.trial035.stderr:osd.3: osd_enable_op_tracker = '' 2026-02-24T07:53:37.921 INFO:teuthology.orchestra.run.trial035.stderr:reweighted osd.1 to 1 (10000) 2026-02-24T07:53:37.965 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph osd reweight 2 1 2026-02-24T07:53:38.099 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:53:38.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:53:37 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/1172147145' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 0, "weight": 1.0}]': finished 2026-02-24T07:53:38.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:53:37 trial035 ceph-mon[39284]: osdmap e85: 4 total, 4 up, 4 in 2026-02-24T07:53:38.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:53:37 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/2740647135' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2026-02-24T07:53:38.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:53:37 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/270446946' entity='client.admin' cmd={"prefix": "osd reweight", "id": 1, "weight": 1.0} : dispatch 2026-02-24T07:53:38.205 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:53:37 trial035 ceph-mon[39284]: pgmap v378: 17 pgs: 17 active+clean; 29 GiB data, 94 GiB used, 2.6 TiB / 2.7 TiB avail; 0 B/s wr, 7.25k op/s 2026-02-24T07:53:38.210 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:53:38.207+0000 7ffb3c521640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:53:38.210 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:53:38.208+0000 7ffb3c521640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:53:38.922 INFO:teuthology.orchestra.run.trial035.stderr:reweighted osd.2 to 1 (10000) 2026-02-24T07:53:39.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:53:38 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/270446946' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 1, "weight": 1.0}]': finished 2026-02-24T07:53:39.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:53:38 trial035 ceph-mon[39284]: osdmap e86: 4 total, 4 up, 4 in 2026-02-24T07:53:39.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:53:38 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/1159017323' entity='client.admin' cmd={"prefix": "osd reweight", "id": 2, "weight": 1.0} : dispatch 2026-02-24T07:53:39.311 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph osd reweight 3 1 2026-02-24T07:53:39.429 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2026-02-24T07:53:39.447 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:53:39.565 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:53:39.563+0000 7f2ef3414640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:53:39.566 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:53:39.563+0000 7f2ef3414640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:53:39.571 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:53:39.697 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:53:39.695+0000 7f99e54e1640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:53:39.697 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:53:39.695+0000 7f99e54e1640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:53:39.711 INFO:teuthology.orchestra.run.trial035.stdout:osd.0: {} 2026-02-24T07:53:39.711 INFO:teuthology.orchestra.run.trial035.stderr:osd.0: osd_enable_op_tracker = '' 2026-02-24T07:53:39.723 INFO:teuthology.orchestra.run.trial035.stdout:osd.1: {} 2026-02-24T07:53:39.723 INFO:teuthology.orchestra.run.trial035.stderr:osd.1: osd_enable_op_tracker = '' 2026-02-24T07:53:39.735 INFO:teuthology.orchestra.run.trial035.stdout:osd.2: {} 2026-02-24T07:53:39.735 INFO:teuthology.orchestra.run.trial035.stderr:osd.2: osd_enable_op_tracker = '' 2026-02-24T07:53:39.746 INFO:teuthology.orchestra.run.trial035.stdout:osd.3: {} 2026-02-24T07:53:39.747 INFO:teuthology.orchestra.run.trial035.stderr:osd.3: osd_enable_op_tracker = '' 2026-02-24T07:53:39.924 INFO:teuthology.orchestra.run.trial035.stderr:reweighted osd.3 to 1 (10000) 2026-02-24T07:53:39.969 INFO:tasks.cephadm.ceph_manager.ceph:waiting for recovery to complete 2026-02-24T07:53:39.970 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph pg dump --format=json 2026-02-24T07:53:40.107 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:53:40.176 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:53:39 trial035 ceph-mon[39284]: purged_snaps scrub starts 2026-02-24T07:53:40.176 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:53:39 trial035 ceph-mon[39284]: purged_snaps scrub ok 2026-02-24T07:53:40.176 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:53:39 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/1159017323' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 2, "weight": 1.0}]': finished 2026-02-24T07:53:40.176 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:53:39 trial035 ceph-mon[39284]: osdmap e87: 4 total, 4 up, 4 in 2026-02-24T07:53:40.176 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:53:39 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/2544725774' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2026-02-24T07:53:40.176 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:53:39 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/389092402' entity='client.admin' cmd={"prefix": "osd reweight", "id": 3, "weight": 1.0} : dispatch 2026-02-24T07:53:40.176 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:53:39 trial035 ceph-mon[39284]: pgmap v381: 17 pgs: 17 active+clean; 29 GiB data, 94 GiB used, 2.6 TiB / 2.7 TiB avail; 0 B/s wr, 6.75k op/s 2026-02-24T07:53:40.229 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:53:40.227+0000 7f9b1aea5640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:53:40.230 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:53:40.228+0000 7f9b1aea5640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:53:40.360 INFO:teuthology.orchestra.run.trial035.stdout: 2026-02-24T07:53:40.360 INFO:teuthology.orchestra.run.trial035.stderr:dumped all 2026-02-24T07:53:40.402 INFO:teuthology.orchestra.run.trial035.stdout:{"pg_ready":true,"pg_map":{"version":382,"stamp":"2026-02-24T07:53:39.923727+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":31273255480,"num_objects":477186,"num_object_clones":0,"num_object_copies":1431558,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":477186,"num_whiteouts":0,"num_read":137,"num_read_kb":241,"num_write":634091,"num_write_kb":35558567,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":53,"ondisk_log_size":53,"up":51,"acting":51,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":51,"num_osds":4,"num_per_pool_osds":4,"num_per_pool_omap_osds":4,"kb":2936012800,"kb_used":98702264,"kb_used_data":92357992,"kb_used_omap":65,"kb_used_meta":6343806,"kb_avail":2837310536,"statfs":{"total":3006477107200,"available":2905405988864,"internally_reserved":0,"allocated":94574583808,"data_stored":94572201012,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":67025,"internal_metadata":6496057903},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[8],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-2654797824,"num_objects":-40509,"num_object_clones":0,"num_object_copies":-121527,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-40509,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":40509,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.180003"},"pg_stats":[{"pgid":"5.f","version":"84'39489","reported_seq":39784,"reported_epoch":84,"state":"active+clean","last_fresh":"2026-02-24T07:53:35.751195+0000","last_change":"2026-02-24T07:53:01.242353+0000","last_active":"2026-02-24T07:53:35.751195+0000","last_peered":"2026-02-24T07:53:35.751195+0000","last_clean":"2026-02-24T07:53:35.751195+0000","last_became_active":"2026-02-24T07:51:53.900431+0000","last_became_peered":"2026-02-24T07:51:53.900431+0000","last_unstale":"2026-02-24T07:53:35.751195+0000","last_undegraded":"2026-02-24T07:53:35.751195+0000","last_fullsized":"2026-02-24T07:53:35.751195+0000","mapping_epoch":74,"log_start":"84'39486","ondisk_log_start":"84'39486","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"84'26061","last_scrub_stamp":"2026-02-24T07:53:01.242306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_clean_scrub_stamp":"2026-02-24T07:53:01.242306+0000","objects_scrubbed":25674,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:54:22.699112+0000","scrub_duration":2129,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1955790848,"num_objects":29843,"num_object_clones":0,"num_object_copies":89529,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29843,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":39489,"num_write_kb":2218624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.e","version":"84'39369","reported_seq":39651,"reported_epoch":84,"state":"active+clean","last_fresh":"2026-02-24T07:53:35.763308+0000","last_change":"2026-02-24T07:52:56.767335+0000","last_active":"2026-02-24T07:53:35.763308+0000","last_peered":"2026-02-24T07:53:35.763308+0000","last_clean":"2026-02-24T07:53:35.763308+0000","last_became_active":"2026-02-24T07:51:53.899294+0000","last_became_peered":"2026-02-24T07:51:53.899294+0000","last_unstale":"2026-02-24T07:53:35.763308+0000","last_undegraded":"2026-02-24T07:53:35.763308+0000","last_fullsized":"2026-02-24T07:53:35.763308+0000","mapping_epoch":74,"log_start":"84'39366","ondisk_log_start":"84'39366","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"83'24557","last_scrub_stamp":"2026-02-24T07:52:56.767267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_clean_scrub_stamp":"2026-02-24T07:52:56.767267+0000","objects_scrubbed":24118,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:54:24.913755+0000","scrub_duration":2806,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1944125440,"num_objects":29665,"num_object_clones":0,"num_object_copies":88995,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29665,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":39369,"num_write_kb":2209088,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"84'39608","reported_seq":39970,"reported_epoch":84,"state":"active+clean","last_fresh":"2026-02-24T07:53:35.778110+0000","last_change":"2026-02-24T07:53:32.480551+0000","last_active":"2026-02-24T07:53:35.778110+0000","last_peered":"2026-02-24T07:53:35.778110+0000","last_clean":"2026-02-24T07:53:35.778110+0000","last_became_active":"2026-02-24T07:51:53.899772+0000","last_became_peered":"2026-02-24T07:51:53.899772+0000","last_unstale":"2026-02-24T07:53:35.778110+0000","last_undegraded":"2026-02-24T07:53:35.778110+0000","last_fullsized":"2026-02-24T07:53:35.778110+0000","mapping_epoch":74,"log_start":"84'39605","ondisk_log_start":"84'39605","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"84'37904","last_scrub_stamp":"2026-02-24T07:53:32.480499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_clean_scrub_stamp":"2026-02-24T07:53:32.480499+0000","objects_scrubbed":32149,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:54:51.939533+0000","scrub_duration":2742,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1956446232,"num_objects":29854,"num_object_clones":0,"num_object_copies":89562,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29854,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":39608,"num_write_kb":2222721,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,3],"acting":[0,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.c","version":"84'39740","reported_seq":40094,"reported_epoch":84,"state":"active+clean","last_fresh":"2026-02-24T07:53:35.854338+0000","last_change":"2026-02-24T07:53:17.083868+0000","last_active":"2026-02-24T07:53:35.854338+0000","last_peered":"2026-02-24T07:53:35.854338+0000","last_clean":"2026-02-24T07:53:35.854338+0000","last_became_active":"2026-02-24T07:51:53.898285+0000","last_became_peered":"2026-02-24T07:51:53.898285+0000","last_unstale":"2026-02-24T07:53:35.854338+0000","last_undegraded":"2026-02-24T07:53:35.854338+0000","last_fullsized":"2026-02-24T07:53:35.854338+0000","mapping_epoch":74,"log_start":"84'39737","ondisk_log_start":"84'39737","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"84'32067","last_scrub_stamp":"2026-02-24T07:53:17.083821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_clean_scrub_stamp":"2026-02-24T07:53:17.083821+0000","objects_scrubbed":31581,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:54:37.993683+0000","scrub_duration":2856,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1946812416,"num_objects":29706,"num_object_clones":0,"num_object_copies":89118,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29706,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":39740,"num_write_kb":2222272,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.b","version":"84'39625","reported_seq":39996,"reported_epoch":84,"state":"active+clean","last_fresh":"2026-02-24T07:53:35.778024+0000","last_change":"2026-02-24T07:53:30.225127+0000","last_active":"2026-02-24T07:53:35.778024+0000","last_peered":"2026-02-24T07:53:35.778024+0000","last_clean":"2026-02-24T07:53:35.778024+0000","last_became_active":"2026-02-24T07:51:53.899901+0000","last_became_peered":"2026-02-24T07:51:53.899901+0000","last_unstale":"2026-02-24T07:53:35.778024+0000","last_undegraded":"2026-02-24T07:53:35.778024+0000","last_fullsized":"2026-02-24T07:53:35.778024+0000","mapping_epoch":74,"log_start":"84'39622","ondisk_log_start":"84'39622","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"84'36879","last_scrub_stamp":"2026-02-24T07:53:30.225075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_clean_scrub_stamp":"2026-02-24T07:53:30.225075+0000","objects_scrubbed":33279,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:54:35.718496+0000","scrub_duration":3524,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1957756928,"num_objects":29873,"num_object_clones":0,"num_object_copies":89619,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29873,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":39625,"num_write_kb":2223936,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"84'39620","reported_seq":39963,"reported_epoch":84,"state":"active+clean","last_fresh":"2026-02-24T07:53:35.778175+0000","last_change":"2026-02-24T07:53:14.212339+0000","last_active":"2026-02-24T07:53:35.778175+0000","last_peered":"2026-02-24T07:53:35.778175+0000","last_clean":"2026-02-24T07:53:35.778175+0000","last_became_active":"2026-02-24T07:51:53.899549+0000","last_became_peered":"2026-02-24T07:51:53.899549+0000","last_unstale":"2026-02-24T07:53:35.778175+0000","last_undegraded":"2026-02-24T07:53:35.778175+0000","last_fullsized":"2026-02-24T07:53:35.778175+0000","mapping_epoch":74,"log_start":"84'39617","ondisk_log_start":"84'39617","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"84'30986","last_scrub_stamp":"2026-02-24T07:53:14.212265+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_clean_scrub_stamp":"2026-02-24T07:53:14.212265+0000","objects_scrubbed":30490,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:54:35.341423+0000","scrub_duration":3381,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1954545664,"num_objects":29824,"num_object_clones":0,"num_object_copies":89472,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29824,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":39620,"num_write_kb":2222208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,1],"acting":[0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.9","version":"84'40146","reported_seq":40521,"reported_epoch":84,"state":"active+clean","last_fresh":"2026-02-24T07:53:35.765690+0000","last_change":"2026-02-24T07:53:22.113835+0000","last_active":"2026-02-24T07:53:35.765690+0000","last_peered":"2026-02-24T07:53:35.765690+0000","last_clean":"2026-02-24T07:53:35.765690+0000","last_became_active":"2026-02-24T07:51:53.901610+0000","last_became_peered":"2026-02-24T07:51:53.901610+0000","last_unstale":"2026-02-24T07:53:35.765690+0000","last_undegraded":"2026-02-24T07:53:35.765690+0000","last_fullsized":"2026-02-24T07:53:35.765690+0000","mapping_epoch":74,"log_start":"84'40143","ondisk_log_start":"84'40143","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"84'34029","last_scrub_stamp":"2026-02-24T07:53:22.113774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_clean_scrub_stamp":"2026-02-24T07:53:22.113774+0000","objects_scrubbed":33520,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:54:41.903404+0000","scrub_duration":3005,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1976958976,"num_objects":30166,"num_object_clones":0,"num_object_copies":90498,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30166,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":40146,"num_write_kb":2249984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.8","version":"84'39828","reported_seq":40120,"reported_epoch":84,"state":"active+clean","last_fresh":"2026-02-24T07:53:35.778220+0000","last_change":"2026-02-24T07:53:00.549866+0000","last_active":"2026-02-24T07:53:35.778220+0000","last_peered":"2026-02-24T07:53:35.778220+0000","last_clean":"2026-02-24T07:53:35.778220+0000","last_became_active":"2026-02-24T07:51:53.900207+0000","last_became_peered":"2026-02-24T07:51:53.900207+0000","last_unstale":"2026-02-24T07:53:35.778220+0000","last_undegraded":"2026-02-24T07:53:35.778220+0000","last_fullsized":"2026-02-24T07:53:35.778220+0000","mapping_epoch":74,"log_start":"84'39825","ondisk_log_start":"84'39825","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"84'25943","last_scrub_stamp":"2026-02-24T07:53:00.549789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_clean_scrub_stamp":"2026-02-24T07:53:00.549789+0000","objects_scrubbed":25486,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:54:18.896412+0000","scrub_duration":2647,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1965686784,"num_objects":29994,"num_object_clones":0,"num_object_copies":89982,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29994,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":39828,"num_write_kb":2234304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.7","version":"84'39255","reported_seq":39547,"reported_epoch":84,"state":"active+clean","last_fresh":"2026-02-24T07:53:35.764526+0000","last_change":"2026-02-24T07:53:00.188453+0000","last_active":"2026-02-24T07:53:35.764526+0000","last_peered":"2026-02-24T07:53:35.764526+0000","last_clean":"2026-02-24T07:53:35.764526+0000","last_became_active":"2026-02-24T07:51:53.899472+0000","last_became_peered":"2026-02-24T07:51:53.899472+0000","last_unstale":"2026-02-24T07:53:35.764526+0000","last_undegraded":"2026-02-24T07:53:35.764526+0000","last_fullsized":"2026-02-24T07:53:35.764526+0000","mapping_epoch":74,"log_start":"84'39252","ondisk_log_start":"84'39252","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"84'25630","last_scrub_stamp":"2026-02-24T07:53:00.188390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_clean_scrub_stamp":"2026-02-24T07:53:00.188390+0000","objects_scrubbed":25246,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:54:07.242502+0000","scrub_duration":2262,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1936130048,"num_objects":29543,"num_object_clones":0,"num_object_copies":88629,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29543,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":39255,"num_write_kb":2201536,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.6","version":"84'39298","reported_seq":39631,"reported_epoch":84,"state":"active+clean","last_fresh":"2026-02-24T07:53:35.857374+0000","last_change":"2026-02-24T07:53:10.804133+0000","last_active":"2026-02-24T07:53:35.857374+0000","last_peered":"2026-02-24T07:53:35.857374+0000","last_clean":"2026-02-24T07:53:35.857374+0000","last_became_active":"2026-02-24T07:51:53.898308+0000","last_became_peered":"2026-02-24T07:51:53.898308+0000","last_unstale":"2026-02-24T07:53:35.857374+0000","last_undegraded":"2026-02-24T07:53:35.857374+0000","last_fullsized":"2026-02-24T07:53:35.857374+0000","mapping_epoch":74,"log_start":"84'39295","ondisk_log_start":"84'39295","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"84'29829","last_scrub_stamp":"2026-02-24T07:53:10.804104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_clean_scrub_stamp":"2026-02-24T07:53:10.804104+0000","objects_scrubbed":29351,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:54:13.621421+0000","scrub_duration":2501,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1943011328,"num_objects":29648,"num_object_clones":0,"num_object_copies":88944,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29648,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":39298,"num_write_kb":2206272,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.5","version":"84'39622","reported_seq":40007,"reported_epoch":84,"state":"active+clean","last_fresh":"2026-02-24T07:53:35.769242+0000","last_change":"2026-02-24T07:53:26.781324+0000","last_active":"2026-02-24T07:53:35.769242+0000","last_peered":"2026-02-24T07:53:35.769242+0000","last_clean":"2026-02-24T07:53:35.769242+0000","last_became_active":"2026-02-24T07:51:53.899749+0000","last_became_peered":"2026-02-24T07:51:53.899749+0000","last_unstale":"2026-02-24T07:53:35.769242+0000","last_undegraded":"2026-02-24T07:53:35.769242+0000","last_fullsized":"2026-02-24T07:53:35.769242+0000","mapping_epoch":74,"log_start":"84'39617","ondisk_log_start":"84'39617","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"84'35538","last_scrub_stamp":"2026-02-24T07:53:26.781276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_clean_scrub_stamp":"2026-02-24T07:53:26.781276+0000","objects_scrubbed":34358,"log_size":5,"log_dups_size":3000,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:54:37.440161+0000","scrub_duration":2982,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1952055296,"num_objects":29786,"num_object_clones":0,"num_object_copies":89358,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29786,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":39622,"num_write_kb":2221056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"84'39672","reported_seq":39985,"reported_epoch":84,"state":"active+clean","last_fresh":"2026-02-24T07:53:35.778266+0000","last_change":"2026-02-24T07:53:05.312364+0000","last_active":"2026-02-24T07:53:35.778266+0000","last_peered":"2026-02-24T07:53:35.778266+0000","last_clean":"2026-02-24T07:53:35.778266+0000","last_became_active":"2026-02-24T07:51:53.900042+0000","last_became_peered":"2026-02-24T07:51:53.900042+0000","last_unstale":"2026-02-24T07:53:35.778266+0000","last_undegraded":"2026-02-24T07:53:35.778266+0000","last_fullsized":"2026-02-24T07:53:35.778266+0000","mapping_epoch":74,"log_start":"84'39669","ondisk_log_start":"84'39669","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"84'27738","last_scrub_stamp":"2026-02-24T07:53:05.312312+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_clean_scrub_stamp":"2026-02-24T07:53:05.312312+0000","objects_scrubbed":27245,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:54:32.166518+0000","scrub_duration":2472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1952710656,"num_objects":29796,"num_object_clones":0,"num_object_copies":89388,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29796,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":39672,"num_write_kb":2222976,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.2","version":"84'40008","reported_seq":40368,"reported_epoch":84,"state":"active+clean","last_fresh":"2026-02-24T07:53:35.758753+0000","last_change":"2026-02-24T07:53:17.780799+0000","last_active":"2026-02-24T07:53:35.758753+0000","last_peered":"2026-02-24T07:53:35.758753+0000","last_clean":"2026-02-24T07:53:35.758753+0000","last_became_active":"2026-02-24T07:51:53.900096+0000","last_became_peered":"2026-02-24T07:51:53.900096+0000","last_unstale":"2026-02-24T07:53:35.758753+0000","last_undegraded":"2026-02-24T07:53:35.758753+0000","last_fullsized":"2026-02-24T07:53:35.758753+0000","mapping_epoch":74,"log_start":"84'40005","ondisk_log_start":"84'40005","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"84'32548","last_scrub_stamp":"2026-02-24T07:53:17.780705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_clean_scrub_stamp":"2026-02-24T07:53:17.780705+0000","objects_scrubbed":32030,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:54:35.344780+0000","scrub_duration":2741,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1973551104,"num_objects":30114,"num_object_clones":0,"num_object_copies":90342,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30114,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":40008,"num_write_kb":2243904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"84'39200","reported_seq":39541,"reported_epoch":84,"state":"active+clean","last_fresh":"2026-02-24T07:53:35.766622+0000","last_change":"2026-02-24T07:53:14.051194+0000","last_active":"2026-02-24T07:53:35.766622+0000","last_peered":"2026-02-24T07:53:35.766622+0000","last_clean":"2026-02-24T07:53:35.766622+0000","last_became_active":"2026-02-24T07:51:53.901027+0000","last_became_peered":"2026-02-24T07:51:53.901027+0000","last_unstale":"2026-02-24T07:53:35.766622+0000","last_undegraded":"2026-02-24T07:53:35.766622+0000","last_fullsized":"2026-02-24T07:53:35.766622+0000","mapping_epoch":74,"log_start":"84'39197","ondisk_log_start":"84'39197","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"84'30565","last_scrub_stamp":"2026-02-24T07:53:14.051154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_clean_scrub_stamp":"2026-02-24T07:53:14.051154+0000","objects_scrubbed":30140,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:54:38.448076+0000","scrub_duration":3147,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1938292736,"num_objects":29576,"num_object_clones":0,"num_object_copies":88728,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29576,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":39200,"num_write_kb":2200832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.0","version":"84'39974","reported_seq":40340,"reported_epoch":84,"state":"active+clean","last_fresh":"2026-02-24T07:53:35.774498+0000","last_change":"2026-02-24T07:53:20.014077+0000","last_active":"2026-02-24T07:53:35.774498+0000","last_peered":"2026-02-24T07:53:35.774498+0000","last_clean":"2026-02-24T07:53:35.774498+0000","last_became_active":"2026-02-24T07:51:53.898551+0000","last_became_peered":"2026-02-24T07:51:53.898551+0000","last_unstale":"2026-02-24T07:53:35.774498+0000","last_undegraded":"2026-02-24T07:53:35.774498+0000","last_fullsized":"2026-02-24T07:53:35.774498+0000","mapping_epoch":74,"log_start":"84'39971","ondisk_log_start":"84'39971","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"84'33261","last_scrub_stamp":"2026-02-24T07:53:20.014009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_clean_scrub_stamp":"2026-02-24T07:53:20.014009+0000","objects_scrubbed":32679,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:54:37.178341+0000","scrub_duration":3163,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1969225728,"num_objects":30048,"num_object_clones":0,"num_object_copies":90144,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30048,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":39974,"num_write_kb":2240704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.4","version":"84'39502","reported_seq":39839,"reported_epoch":84,"state":"active+clean","last_fresh":"2026-02-24T07:53:35.753777+0000","last_change":"2026-02-24T07:53:11.592191+0000","last_active":"2026-02-24T07:53:35.753777+0000","last_peered":"2026-02-24T07:53:35.753777+0000","last_clean":"2026-02-24T07:53:35.753777+0000","last_became_active":"2026-02-24T07:51:53.899675+0000","last_became_peered":"2026-02-24T07:51:53.899675+0000","last_unstale":"2026-02-24T07:53:35.753777+0000","last_undegraded":"2026-02-24T07:53:35.753777+0000","last_fullsized":"2026-02-24T07:53:35.753777+0000","mapping_epoch":74,"log_start":"84'39499","ondisk_log_start":"84'39499","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"84'30127","last_scrub_stamp":"2026-02-24T07:53:11.592155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_clean_scrub_stamp":"2026-02-24T07:53:11.592155+0000","objects_scrubbed":29709,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:54:23.794472+0000","scrub_duration":2432,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1949564928,"num_objects":29748,"num_object_clones":0,"num_object_copies":89244,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29748,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":39502,"num_write_kb":2216000,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"29'99","reported_seq":357,"reported_epoch":84,"state":"active+clean","last_fresh":"2026-02-24T07:52:58.893578+0000","last_change":"2026-02-24T07:52:56.946143+0000","last_active":"2026-02-24T07:52:58.893578+0000","last_peered":"2026-02-24T07:52:58.893578+0000","last_clean":"2026-02-24T07:52:58.893578+0000","last_became_active":"2026-02-24T07:42:54.186719+0000","last_became_peered":"2026-02-24T07:42:54.186719+0000","last_unstale":"2026-02-24T07:52:58.893578+0000","last_undegraded":"2026-02-24T07:52:58.893578+0000","last_fullsized":"2026-02-24T07:52:58.893578+0000","mapping_epoch":27,"log_start":"29'96","ondisk_log_start":"29'96","created":20,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'99","last_scrub_stamp":"2026-02-24T07:52:56.946109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:42:43.640776+0000","last_clean_scrub_stamp":"2026-02-24T07:52:56.946109+0000","objects_scrubbed":2,"log_size":3,"log_dups_size":96,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:54:05.896060+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":240,"num_write":135,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":16,"stat_sum":{"num_bytes":31272665112,"num_objects":477184,"num_object_clones":0,"num_object_copies":1431552,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":477184,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":633956,"num_write_kb":35556417,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":93812502528,"data_stored":93812490312,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":50,"ondisk_log_size":50,"up":48,"acting":48,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":240,"num_write":135,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1682,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":3,"up_from":25,"seq":107374182531,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":28578400,"kb_used_data":26996428,"kb_used_omap":16,"kb_used_meta":1581743,"kb_avail":705424800,"statfs":{"total":751619276800,"available":722354995200,"internally_reserved":0,"allocated":27644342272,"data_stored":27643744667,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16915,"internal_metadata":1619705325},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[7],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":19,"seq":81604378757,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":23225056,"kb_used_data":21257228,"kb_used_omap":15,"kb_used_meta":1967664,"kb_avail":710778144,"statfs":{"total":751619276800,"available":727836819456,"internally_reserved":0,"allocated":21767401472,"data_stored":21766803867,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":15701,"internal_metadata":2014888619},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542278,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":21124120,"kb_used_data":19239364,"kb_used_omap":20,"kb_used_meta":1884715,"kb_avail":712879080,"statfs":{"total":751619276800,"available":729988177920,"internally_reserved":0,"allocated":19701108736,"data_stored":19700518755,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":20777,"internal_metadata":1929948887},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705801,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":25774688,"kb_used_data":24864972,"kb_used_omap":13,"kb_used_meta":909682,"kb_avail":708228512,"statfs":{"total":751619276800,"available":725225996288,"internally_reserved":0,"allocated":25461731328,"data_stored":25461133723,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13632,"internal_metadata":931515072},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1682,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":25403265024,"data_stored":25403260952,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":19517276160,"data_stored":19517276160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":21531398144,"data_stored":21531394072,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":27360563200,"data_stored":27360559128,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-24T07:53:40.404 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph pg dump --format=json 2026-02-24T07:53:40.551 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:53:40.670 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:53:40.667+0000 7f14c0806640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:53:40.670 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:53:40.668+0000 7f14c0806640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:53:40.809 INFO:teuthology.orchestra.run.trial035.stdout: 2026-02-24T07:53:40.809 INFO:teuthology.orchestra.run.trial035.stderr:dumped all 2026-02-24T07:53:40.856 INFO:teuthology.orchestra.run.trial035.stdout:{"pg_ready":true,"pg_map":{"version":382,"stamp":"2026-02-24T07:53:39.923727+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":31273255480,"num_objects":477186,"num_object_clones":0,"num_object_copies":1431558,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":477186,"num_whiteouts":0,"num_read":137,"num_read_kb":241,"num_write":634091,"num_write_kb":35558567,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":53,"ondisk_log_size":53,"up":51,"acting":51,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":51,"num_osds":4,"num_per_pool_osds":4,"num_per_pool_omap_osds":4,"kb":2936012800,"kb_used":98702264,"kb_used_data":92357992,"kb_used_omap":65,"kb_used_meta":6343806,"kb_avail":2837310536,"statfs":{"total":3006477107200,"available":2905405988864,"internally_reserved":0,"allocated":94574583808,"data_stored":94572201012,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":67025,"internal_metadata":6496057903},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[8],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-2654797824,"num_objects":-40509,"num_object_clones":0,"num_object_copies":-121527,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-40509,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":40509,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.180003"},"pg_stats":[{"pgid":"5.f","version":"84'39489","reported_seq":39784,"reported_epoch":84,"state":"active+clean","last_fresh":"2026-02-24T07:53:35.751195+0000","last_change":"2026-02-24T07:53:01.242353+0000","last_active":"2026-02-24T07:53:35.751195+0000","last_peered":"2026-02-24T07:53:35.751195+0000","last_clean":"2026-02-24T07:53:35.751195+0000","last_became_active":"2026-02-24T07:51:53.900431+0000","last_became_peered":"2026-02-24T07:51:53.900431+0000","last_unstale":"2026-02-24T07:53:35.751195+0000","last_undegraded":"2026-02-24T07:53:35.751195+0000","last_fullsized":"2026-02-24T07:53:35.751195+0000","mapping_epoch":74,"log_start":"84'39486","ondisk_log_start":"84'39486","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"84'26061","last_scrub_stamp":"2026-02-24T07:53:01.242306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_clean_scrub_stamp":"2026-02-24T07:53:01.242306+0000","objects_scrubbed":25674,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:54:22.699112+0000","scrub_duration":2129,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1955790848,"num_objects":29843,"num_object_clones":0,"num_object_copies":89529,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29843,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":39489,"num_write_kb":2218624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.e","version":"84'39369","reported_seq":39651,"reported_epoch":84,"state":"active+clean","last_fresh":"2026-02-24T07:53:35.763308+0000","last_change":"2026-02-24T07:52:56.767335+0000","last_active":"2026-02-24T07:53:35.763308+0000","last_peered":"2026-02-24T07:53:35.763308+0000","last_clean":"2026-02-24T07:53:35.763308+0000","last_became_active":"2026-02-24T07:51:53.899294+0000","last_became_peered":"2026-02-24T07:51:53.899294+0000","last_unstale":"2026-02-24T07:53:35.763308+0000","last_undegraded":"2026-02-24T07:53:35.763308+0000","last_fullsized":"2026-02-24T07:53:35.763308+0000","mapping_epoch":74,"log_start":"84'39366","ondisk_log_start":"84'39366","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"83'24557","last_scrub_stamp":"2026-02-24T07:52:56.767267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_clean_scrub_stamp":"2026-02-24T07:52:56.767267+0000","objects_scrubbed":24118,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:54:24.913755+0000","scrub_duration":2806,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1944125440,"num_objects":29665,"num_object_clones":0,"num_object_copies":88995,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29665,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":39369,"num_write_kb":2209088,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"84'39608","reported_seq":39970,"reported_epoch":84,"state":"active+clean","last_fresh":"2026-02-24T07:53:35.778110+0000","last_change":"2026-02-24T07:53:32.480551+0000","last_active":"2026-02-24T07:53:35.778110+0000","last_peered":"2026-02-24T07:53:35.778110+0000","last_clean":"2026-02-24T07:53:35.778110+0000","last_became_active":"2026-02-24T07:51:53.899772+0000","last_became_peered":"2026-02-24T07:51:53.899772+0000","last_unstale":"2026-02-24T07:53:35.778110+0000","last_undegraded":"2026-02-24T07:53:35.778110+0000","last_fullsized":"2026-02-24T07:53:35.778110+0000","mapping_epoch":74,"log_start":"84'39605","ondisk_log_start":"84'39605","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"84'37904","last_scrub_stamp":"2026-02-24T07:53:32.480499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_clean_scrub_stamp":"2026-02-24T07:53:32.480499+0000","objects_scrubbed":32149,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:54:51.939533+0000","scrub_duration":2742,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1956446232,"num_objects":29854,"num_object_clones":0,"num_object_copies":89562,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29854,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":39608,"num_write_kb":2222721,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,3],"acting":[0,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.c","version":"84'39740","reported_seq":40094,"reported_epoch":84,"state":"active+clean","last_fresh":"2026-02-24T07:53:35.854338+0000","last_change":"2026-02-24T07:53:17.083868+0000","last_active":"2026-02-24T07:53:35.854338+0000","last_peered":"2026-02-24T07:53:35.854338+0000","last_clean":"2026-02-24T07:53:35.854338+0000","last_became_active":"2026-02-24T07:51:53.898285+0000","last_became_peered":"2026-02-24T07:51:53.898285+0000","last_unstale":"2026-02-24T07:53:35.854338+0000","last_undegraded":"2026-02-24T07:53:35.854338+0000","last_fullsized":"2026-02-24T07:53:35.854338+0000","mapping_epoch":74,"log_start":"84'39737","ondisk_log_start":"84'39737","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"84'32067","last_scrub_stamp":"2026-02-24T07:53:17.083821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_clean_scrub_stamp":"2026-02-24T07:53:17.083821+0000","objects_scrubbed":31581,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:54:37.993683+0000","scrub_duration":2856,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1946812416,"num_objects":29706,"num_object_clones":0,"num_object_copies":89118,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29706,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":39740,"num_write_kb":2222272,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.b","version":"84'39625","reported_seq":39996,"reported_epoch":84,"state":"active+clean","last_fresh":"2026-02-24T07:53:35.778024+0000","last_change":"2026-02-24T07:53:30.225127+0000","last_active":"2026-02-24T07:53:35.778024+0000","last_peered":"2026-02-24T07:53:35.778024+0000","last_clean":"2026-02-24T07:53:35.778024+0000","last_became_active":"2026-02-24T07:51:53.899901+0000","last_became_peered":"2026-02-24T07:51:53.899901+0000","last_unstale":"2026-02-24T07:53:35.778024+0000","last_undegraded":"2026-02-24T07:53:35.778024+0000","last_fullsized":"2026-02-24T07:53:35.778024+0000","mapping_epoch":74,"log_start":"84'39622","ondisk_log_start":"84'39622","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"84'36879","last_scrub_stamp":"2026-02-24T07:53:30.225075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_clean_scrub_stamp":"2026-02-24T07:53:30.225075+0000","objects_scrubbed":33279,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:54:35.718496+0000","scrub_duration":3524,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1957756928,"num_objects":29873,"num_object_clones":0,"num_object_copies":89619,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29873,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":39625,"num_write_kb":2223936,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"84'39620","reported_seq":39963,"reported_epoch":84,"state":"active+clean","last_fresh":"2026-02-24T07:53:35.778175+0000","last_change":"2026-02-24T07:53:14.212339+0000","last_active":"2026-02-24T07:53:35.778175+0000","last_peered":"2026-02-24T07:53:35.778175+0000","last_clean":"2026-02-24T07:53:35.778175+0000","last_became_active":"2026-02-24T07:51:53.899549+0000","last_became_peered":"2026-02-24T07:51:53.899549+0000","last_unstale":"2026-02-24T07:53:35.778175+0000","last_undegraded":"2026-02-24T07:53:35.778175+0000","last_fullsized":"2026-02-24T07:53:35.778175+0000","mapping_epoch":74,"log_start":"84'39617","ondisk_log_start":"84'39617","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"84'30986","last_scrub_stamp":"2026-02-24T07:53:14.212265+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_clean_scrub_stamp":"2026-02-24T07:53:14.212265+0000","objects_scrubbed":30490,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:54:35.341423+0000","scrub_duration":3381,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1954545664,"num_objects":29824,"num_object_clones":0,"num_object_copies":89472,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29824,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":39620,"num_write_kb":2222208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,1],"acting":[0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.9","version":"84'40146","reported_seq":40521,"reported_epoch":84,"state":"active+clean","last_fresh":"2026-02-24T07:53:35.765690+0000","last_change":"2026-02-24T07:53:22.113835+0000","last_active":"2026-02-24T07:53:35.765690+0000","last_peered":"2026-02-24T07:53:35.765690+0000","last_clean":"2026-02-24T07:53:35.765690+0000","last_became_active":"2026-02-24T07:51:53.901610+0000","last_became_peered":"2026-02-24T07:51:53.901610+0000","last_unstale":"2026-02-24T07:53:35.765690+0000","last_undegraded":"2026-02-24T07:53:35.765690+0000","last_fullsized":"2026-02-24T07:53:35.765690+0000","mapping_epoch":74,"log_start":"84'40143","ondisk_log_start":"84'40143","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"84'34029","last_scrub_stamp":"2026-02-24T07:53:22.113774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_clean_scrub_stamp":"2026-02-24T07:53:22.113774+0000","objects_scrubbed":33520,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:54:41.903404+0000","scrub_duration":3005,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1976958976,"num_objects":30166,"num_object_clones":0,"num_object_copies":90498,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30166,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":40146,"num_write_kb":2249984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.8","version":"84'39828","reported_seq":40120,"reported_epoch":84,"state":"active+clean","last_fresh":"2026-02-24T07:53:35.778220+0000","last_change":"2026-02-24T07:53:00.549866+0000","last_active":"2026-02-24T07:53:35.778220+0000","last_peered":"2026-02-24T07:53:35.778220+0000","last_clean":"2026-02-24T07:53:35.778220+0000","last_became_active":"2026-02-24T07:51:53.900207+0000","last_became_peered":"2026-02-24T07:51:53.900207+0000","last_unstale":"2026-02-24T07:53:35.778220+0000","last_undegraded":"2026-02-24T07:53:35.778220+0000","last_fullsized":"2026-02-24T07:53:35.778220+0000","mapping_epoch":74,"log_start":"84'39825","ondisk_log_start":"84'39825","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"84'25943","last_scrub_stamp":"2026-02-24T07:53:00.549789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_clean_scrub_stamp":"2026-02-24T07:53:00.549789+0000","objects_scrubbed":25486,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:54:18.896412+0000","scrub_duration":2647,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1965686784,"num_objects":29994,"num_object_clones":0,"num_object_copies":89982,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29994,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":39828,"num_write_kb":2234304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.7","version":"84'39255","reported_seq":39547,"reported_epoch":84,"state":"active+clean","last_fresh":"2026-02-24T07:53:35.764526+0000","last_change":"2026-02-24T07:53:00.188453+0000","last_active":"2026-02-24T07:53:35.764526+0000","last_peered":"2026-02-24T07:53:35.764526+0000","last_clean":"2026-02-24T07:53:35.764526+0000","last_became_active":"2026-02-24T07:51:53.899472+0000","last_became_peered":"2026-02-24T07:51:53.899472+0000","last_unstale":"2026-02-24T07:53:35.764526+0000","last_undegraded":"2026-02-24T07:53:35.764526+0000","last_fullsized":"2026-02-24T07:53:35.764526+0000","mapping_epoch":74,"log_start":"84'39252","ondisk_log_start":"84'39252","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"84'25630","last_scrub_stamp":"2026-02-24T07:53:00.188390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_clean_scrub_stamp":"2026-02-24T07:53:00.188390+0000","objects_scrubbed":25246,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:54:07.242502+0000","scrub_duration":2262,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1936130048,"num_objects":29543,"num_object_clones":0,"num_object_copies":88629,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29543,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":39255,"num_write_kb":2201536,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.6","version":"84'39298","reported_seq":39631,"reported_epoch":84,"state":"active+clean","last_fresh":"2026-02-24T07:53:35.857374+0000","last_change":"2026-02-24T07:53:10.804133+0000","last_active":"2026-02-24T07:53:35.857374+0000","last_peered":"2026-02-24T07:53:35.857374+0000","last_clean":"2026-02-24T07:53:35.857374+0000","last_became_active":"2026-02-24T07:51:53.898308+0000","last_became_peered":"2026-02-24T07:51:53.898308+0000","last_unstale":"2026-02-24T07:53:35.857374+0000","last_undegraded":"2026-02-24T07:53:35.857374+0000","last_fullsized":"2026-02-24T07:53:35.857374+0000","mapping_epoch":74,"log_start":"84'39295","ondisk_log_start":"84'39295","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"84'29829","last_scrub_stamp":"2026-02-24T07:53:10.804104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_clean_scrub_stamp":"2026-02-24T07:53:10.804104+0000","objects_scrubbed":29351,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:54:13.621421+0000","scrub_duration":2501,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1943011328,"num_objects":29648,"num_object_clones":0,"num_object_copies":88944,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29648,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":39298,"num_write_kb":2206272,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.5","version":"84'39622","reported_seq":40007,"reported_epoch":84,"state":"active+clean","last_fresh":"2026-02-24T07:53:35.769242+0000","last_change":"2026-02-24T07:53:26.781324+0000","last_active":"2026-02-24T07:53:35.769242+0000","last_peered":"2026-02-24T07:53:35.769242+0000","last_clean":"2026-02-24T07:53:35.769242+0000","last_became_active":"2026-02-24T07:51:53.899749+0000","last_became_peered":"2026-02-24T07:51:53.899749+0000","last_unstale":"2026-02-24T07:53:35.769242+0000","last_undegraded":"2026-02-24T07:53:35.769242+0000","last_fullsized":"2026-02-24T07:53:35.769242+0000","mapping_epoch":74,"log_start":"84'39617","ondisk_log_start":"84'39617","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"84'35538","last_scrub_stamp":"2026-02-24T07:53:26.781276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_clean_scrub_stamp":"2026-02-24T07:53:26.781276+0000","objects_scrubbed":34358,"log_size":5,"log_dups_size":3000,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:54:37.440161+0000","scrub_duration":2982,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1952055296,"num_objects":29786,"num_object_clones":0,"num_object_copies":89358,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29786,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":39622,"num_write_kb":2221056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"84'39672","reported_seq":39985,"reported_epoch":84,"state":"active+clean","last_fresh":"2026-02-24T07:53:35.778266+0000","last_change":"2026-02-24T07:53:05.312364+0000","last_active":"2026-02-24T07:53:35.778266+0000","last_peered":"2026-02-24T07:53:35.778266+0000","last_clean":"2026-02-24T07:53:35.778266+0000","last_became_active":"2026-02-24T07:51:53.900042+0000","last_became_peered":"2026-02-24T07:51:53.900042+0000","last_unstale":"2026-02-24T07:53:35.778266+0000","last_undegraded":"2026-02-24T07:53:35.778266+0000","last_fullsized":"2026-02-24T07:53:35.778266+0000","mapping_epoch":74,"log_start":"84'39669","ondisk_log_start":"84'39669","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"84'27738","last_scrub_stamp":"2026-02-24T07:53:05.312312+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_clean_scrub_stamp":"2026-02-24T07:53:05.312312+0000","objects_scrubbed":27245,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:54:32.166518+0000","scrub_duration":2472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1952710656,"num_objects":29796,"num_object_clones":0,"num_object_copies":89388,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29796,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":39672,"num_write_kb":2222976,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.2","version":"84'40008","reported_seq":40368,"reported_epoch":84,"state":"active+clean","last_fresh":"2026-02-24T07:53:35.758753+0000","last_change":"2026-02-24T07:53:17.780799+0000","last_active":"2026-02-24T07:53:35.758753+0000","last_peered":"2026-02-24T07:53:35.758753+0000","last_clean":"2026-02-24T07:53:35.758753+0000","last_became_active":"2026-02-24T07:51:53.900096+0000","last_became_peered":"2026-02-24T07:51:53.900096+0000","last_unstale":"2026-02-24T07:53:35.758753+0000","last_undegraded":"2026-02-24T07:53:35.758753+0000","last_fullsized":"2026-02-24T07:53:35.758753+0000","mapping_epoch":74,"log_start":"84'40005","ondisk_log_start":"84'40005","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"84'32548","last_scrub_stamp":"2026-02-24T07:53:17.780705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_clean_scrub_stamp":"2026-02-24T07:53:17.780705+0000","objects_scrubbed":32030,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:54:35.344780+0000","scrub_duration":2741,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1973551104,"num_objects":30114,"num_object_clones":0,"num_object_copies":90342,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30114,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":40008,"num_write_kb":2243904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"84'39200","reported_seq":39541,"reported_epoch":84,"state":"active+clean","last_fresh":"2026-02-24T07:53:35.766622+0000","last_change":"2026-02-24T07:53:14.051194+0000","last_active":"2026-02-24T07:53:35.766622+0000","last_peered":"2026-02-24T07:53:35.766622+0000","last_clean":"2026-02-24T07:53:35.766622+0000","last_became_active":"2026-02-24T07:51:53.901027+0000","last_became_peered":"2026-02-24T07:51:53.901027+0000","last_unstale":"2026-02-24T07:53:35.766622+0000","last_undegraded":"2026-02-24T07:53:35.766622+0000","last_fullsized":"2026-02-24T07:53:35.766622+0000","mapping_epoch":74,"log_start":"84'39197","ondisk_log_start":"84'39197","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"84'30565","last_scrub_stamp":"2026-02-24T07:53:14.051154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_clean_scrub_stamp":"2026-02-24T07:53:14.051154+0000","objects_scrubbed":30140,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:54:38.448076+0000","scrub_duration":3147,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1938292736,"num_objects":29576,"num_object_clones":0,"num_object_copies":88728,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29576,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":39200,"num_write_kb":2200832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.0","version":"84'39974","reported_seq":40340,"reported_epoch":84,"state":"active+clean","last_fresh":"2026-02-24T07:53:35.774498+0000","last_change":"2026-02-24T07:53:20.014077+0000","last_active":"2026-02-24T07:53:35.774498+0000","last_peered":"2026-02-24T07:53:35.774498+0000","last_clean":"2026-02-24T07:53:35.774498+0000","last_became_active":"2026-02-24T07:51:53.898551+0000","last_became_peered":"2026-02-24T07:51:53.898551+0000","last_unstale":"2026-02-24T07:53:35.774498+0000","last_undegraded":"2026-02-24T07:53:35.774498+0000","last_fullsized":"2026-02-24T07:53:35.774498+0000","mapping_epoch":74,"log_start":"84'39971","ondisk_log_start":"84'39971","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"84'33261","last_scrub_stamp":"2026-02-24T07:53:20.014009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_clean_scrub_stamp":"2026-02-24T07:53:20.014009+0000","objects_scrubbed":32679,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:54:37.178341+0000","scrub_duration":3163,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1969225728,"num_objects":30048,"num_object_clones":0,"num_object_copies":90144,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30048,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":39974,"num_write_kb":2240704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.4","version":"84'39502","reported_seq":39839,"reported_epoch":84,"state":"active+clean","last_fresh":"2026-02-24T07:53:35.753777+0000","last_change":"2026-02-24T07:53:11.592191+0000","last_active":"2026-02-24T07:53:35.753777+0000","last_peered":"2026-02-24T07:53:35.753777+0000","last_clean":"2026-02-24T07:53:35.753777+0000","last_became_active":"2026-02-24T07:51:53.899675+0000","last_became_peered":"2026-02-24T07:51:53.899675+0000","last_unstale":"2026-02-24T07:53:35.753777+0000","last_undegraded":"2026-02-24T07:53:35.753777+0000","last_fullsized":"2026-02-24T07:53:35.753777+0000","mapping_epoch":74,"log_start":"84'39499","ondisk_log_start":"84'39499","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"84'30127","last_scrub_stamp":"2026-02-24T07:53:11.592155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_clean_scrub_stamp":"2026-02-24T07:53:11.592155+0000","objects_scrubbed":29709,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:54:23.794472+0000","scrub_duration":2432,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1949564928,"num_objects":29748,"num_object_clones":0,"num_object_copies":89244,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29748,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":39502,"num_write_kb":2216000,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"29'99","reported_seq":357,"reported_epoch":84,"state":"active+clean","last_fresh":"2026-02-24T07:52:58.893578+0000","last_change":"2026-02-24T07:52:56.946143+0000","last_active":"2026-02-24T07:52:58.893578+0000","last_peered":"2026-02-24T07:52:58.893578+0000","last_clean":"2026-02-24T07:52:58.893578+0000","last_became_active":"2026-02-24T07:42:54.186719+0000","last_became_peered":"2026-02-24T07:42:54.186719+0000","last_unstale":"2026-02-24T07:52:58.893578+0000","last_undegraded":"2026-02-24T07:52:58.893578+0000","last_fullsized":"2026-02-24T07:52:58.893578+0000","mapping_epoch":27,"log_start":"29'96","ondisk_log_start":"29'96","created":20,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'99","last_scrub_stamp":"2026-02-24T07:52:56.946109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:42:43.640776+0000","last_clean_scrub_stamp":"2026-02-24T07:52:56.946109+0000","objects_scrubbed":2,"log_size":3,"log_dups_size":96,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:54:05.896060+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":240,"num_write":135,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":16,"stat_sum":{"num_bytes":31272665112,"num_objects":477184,"num_object_clones":0,"num_object_copies":1431552,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":477184,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":633956,"num_write_kb":35556417,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":93812502528,"data_stored":93812490312,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":50,"ondisk_log_size":50,"up":48,"acting":48,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":240,"num_write":135,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1682,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":3,"up_from":25,"seq":107374182531,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":28578400,"kb_used_data":26996428,"kb_used_omap":16,"kb_used_meta":1581743,"kb_avail":705424800,"statfs":{"total":751619276800,"available":722354995200,"internally_reserved":0,"allocated":27644342272,"data_stored":27643744667,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16915,"internal_metadata":1619705325},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[7],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":19,"seq":81604378757,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":23225056,"kb_used_data":21257228,"kb_used_omap":15,"kb_used_meta":1967664,"kb_avail":710778144,"statfs":{"total":751619276800,"available":727836819456,"internally_reserved":0,"allocated":21767401472,"data_stored":21766803867,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":15701,"internal_metadata":2014888619},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542278,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":21124120,"kb_used_data":19239364,"kb_used_omap":20,"kb_used_meta":1884715,"kb_avail":712879080,"statfs":{"total":751619276800,"available":729988177920,"internally_reserved":0,"allocated":19701108736,"data_stored":19700518755,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":20777,"internal_metadata":1929948887},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705801,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":25774688,"kb_used_data":24864972,"kb_used_omap":13,"kb_used_meta":909682,"kb_avail":708228512,"statfs":{"total":751619276800,"available":725225996288,"internally_reserved":0,"allocated":25461731328,"data_stored":25461133723,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13632,"internal_metadata":931515072},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1682,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":25403265024,"data_stored":25403260952,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":19517276160,"data_stored":19517276160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":21531398144,"data_stored":21531394072,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":27360563200,"data_stored":27360559128,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-24T07:53:40.857 INFO:tasks.cephadm.ceph_manager.ceph:recovered! 2026-02-24T07:53:40.858 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2026-02-24T07:53:40.858 INFO:tasks.thrashosds.thrasher:Growing pool unique_pool_3 2026-02-24T07:53:40.858 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph pg dump --format=json 2026-02-24T07:53:40.994 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:53:41.065 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:53:40 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/389092402' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 3, "weight": 1.0}]': finished 2026-02-24T07:53:41.065 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:53:40 trial035 ceph-mon[39284]: osdmap e88: 4 total, 4 up, 4 in 2026-02-24T07:53:41.065 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:53:40 trial035 ceph-mon[39284]: from='client.15302 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2026-02-24T07:53:41.065 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:53:40 trial035 ceph-mon[39284]: from='client.15304 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2026-02-24T07:53:41.119 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:53:41.117+0000 7f8abf94a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:53:41.119 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:53:41.117+0000 7f8abf94a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:53:41.251 INFO:teuthology.orchestra.run.trial035.stdout: 2026-02-24T07:53:41.252 INFO:teuthology.orchestra.run.trial035.stderr:dumped all 2026-02-24T07:53:41.294 INFO:teuthology.orchestra.run.trial035.stdout:{"pg_ready":true,"pg_map":{"version":382,"stamp":"2026-02-24T07:53:39.923727+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":31273255480,"num_objects":477186,"num_object_clones":0,"num_object_copies":1431558,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":477186,"num_whiteouts":0,"num_read":137,"num_read_kb":241,"num_write":634091,"num_write_kb":35558567,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":53,"ondisk_log_size":53,"up":51,"acting":51,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":51,"num_osds":4,"num_per_pool_osds":4,"num_per_pool_omap_osds":4,"kb":2936012800,"kb_used":98702264,"kb_used_data":92357992,"kb_used_omap":65,"kb_used_meta":6343806,"kb_avail":2837310536,"statfs":{"total":3006477107200,"available":2905405988864,"internally_reserved":0,"allocated":94574583808,"data_stored":94572201012,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":67025,"internal_metadata":6496057903},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[8],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-2654797824,"num_objects":-40509,"num_object_clones":0,"num_object_copies":-121527,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-40509,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":40509,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.180003"},"pg_stats":[{"pgid":"5.f","version":"84'39489","reported_seq":39784,"reported_epoch":84,"state":"active+clean","last_fresh":"2026-02-24T07:53:35.751195+0000","last_change":"2026-02-24T07:53:01.242353+0000","last_active":"2026-02-24T07:53:35.751195+0000","last_peered":"2026-02-24T07:53:35.751195+0000","last_clean":"2026-02-24T07:53:35.751195+0000","last_became_active":"2026-02-24T07:51:53.900431+0000","last_became_peered":"2026-02-24T07:51:53.900431+0000","last_unstale":"2026-02-24T07:53:35.751195+0000","last_undegraded":"2026-02-24T07:53:35.751195+0000","last_fullsized":"2026-02-24T07:53:35.751195+0000","mapping_epoch":74,"log_start":"84'39486","ondisk_log_start":"84'39486","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"84'26061","last_scrub_stamp":"2026-02-24T07:53:01.242306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_clean_scrub_stamp":"2026-02-24T07:53:01.242306+0000","objects_scrubbed":25674,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:54:22.699112+0000","scrub_duration":2129,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1955790848,"num_objects":29843,"num_object_clones":0,"num_object_copies":89529,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29843,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":39489,"num_write_kb":2218624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.e","version":"84'39369","reported_seq":39651,"reported_epoch":84,"state":"active+clean","last_fresh":"2026-02-24T07:53:35.763308+0000","last_change":"2026-02-24T07:52:56.767335+0000","last_active":"2026-02-24T07:53:35.763308+0000","last_peered":"2026-02-24T07:53:35.763308+0000","last_clean":"2026-02-24T07:53:35.763308+0000","last_became_active":"2026-02-24T07:51:53.899294+0000","last_became_peered":"2026-02-24T07:51:53.899294+0000","last_unstale":"2026-02-24T07:53:35.763308+0000","last_undegraded":"2026-02-24T07:53:35.763308+0000","last_fullsized":"2026-02-24T07:53:35.763308+0000","mapping_epoch":74,"log_start":"84'39366","ondisk_log_start":"84'39366","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"83'24557","last_scrub_stamp":"2026-02-24T07:52:56.767267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_clean_scrub_stamp":"2026-02-24T07:52:56.767267+0000","objects_scrubbed":24118,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:54:24.913755+0000","scrub_duration":2806,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1944125440,"num_objects":29665,"num_object_clones":0,"num_object_copies":88995,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29665,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":39369,"num_write_kb":2209088,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"84'39608","reported_seq":39970,"reported_epoch":84,"state":"active+clean","last_fresh":"2026-02-24T07:53:35.778110+0000","last_change":"2026-02-24T07:53:32.480551+0000","last_active":"2026-02-24T07:53:35.778110+0000","last_peered":"2026-02-24T07:53:35.778110+0000","last_clean":"2026-02-24T07:53:35.778110+0000","last_became_active":"2026-02-24T07:51:53.899772+0000","last_became_peered":"2026-02-24T07:51:53.899772+0000","last_unstale":"2026-02-24T07:53:35.778110+0000","last_undegraded":"2026-02-24T07:53:35.778110+0000","last_fullsized":"2026-02-24T07:53:35.778110+0000","mapping_epoch":74,"log_start":"84'39605","ondisk_log_start":"84'39605","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"84'37904","last_scrub_stamp":"2026-02-24T07:53:32.480499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_clean_scrub_stamp":"2026-02-24T07:53:32.480499+0000","objects_scrubbed":32149,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:54:51.939533+0000","scrub_duration":2742,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1956446232,"num_objects":29854,"num_object_clones":0,"num_object_copies":89562,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29854,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":39608,"num_write_kb":2222721,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,3],"acting":[0,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.c","version":"84'39740","reported_seq":40094,"reported_epoch":84,"state":"active+clean","last_fresh":"2026-02-24T07:53:35.854338+0000","last_change":"2026-02-24T07:53:17.083868+0000","last_active":"2026-02-24T07:53:35.854338+0000","last_peered":"2026-02-24T07:53:35.854338+0000","last_clean":"2026-02-24T07:53:35.854338+0000","last_became_active":"2026-02-24T07:51:53.898285+0000","last_became_peered":"2026-02-24T07:51:53.898285+0000","last_unstale":"2026-02-24T07:53:35.854338+0000","last_undegraded":"2026-02-24T07:53:35.854338+0000","last_fullsized":"2026-02-24T07:53:35.854338+0000","mapping_epoch":74,"log_start":"84'39737","ondisk_log_start":"84'39737","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"84'32067","last_scrub_stamp":"2026-02-24T07:53:17.083821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_clean_scrub_stamp":"2026-02-24T07:53:17.083821+0000","objects_scrubbed":31581,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:54:37.993683+0000","scrub_duration":2856,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1946812416,"num_objects":29706,"num_object_clones":0,"num_object_copies":89118,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29706,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":39740,"num_write_kb":2222272,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.b","version":"84'39625","reported_seq":39996,"reported_epoch":84,"state":"active+clean","last_fresh":"2026-02-24T07:53:35.778024+0000","last_change":"2026-02-24T07:53:30.225127+0000","last_active":"2026-02-24T07:53:35.778024+0000","last_peered":"2026-02-24T07:53:35.778024+0000","last_clean":"2026-02-24T07:53:35.778024+0000","last_became_active":"2026-02-24T07:51:53.899901+0000","last_became_peered":"2026-02-24T07:51:53.899901+0000","last_unstale":"2026-02-24T07:53:35.778024+0000","last_undegraded":"2026-02-24T07:53:35.778024+0000","last_fullsized":"2026-02-24T07:53:35.778024+0000","mapping_epoch":74,"log_start":"84'39622","ondisk_log_start":"84'39622","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"84'36879","last_scrub_stamp":"2026-02-24T07:53:30.225075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_clean_scrub_stamp":"2026-02-24T07:53:30.225075+0000","objects_scrubbed":33279,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:54:35.718496+0000","scrub_duration":3524,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1957756928,"num_objects":29873,"num_object_clones":0,"num_object_copies":89619,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29873,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":39625,"num_write_kb":2223936,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"84'39620","reported_seq":39963,"reported_epoch":84,"state":"active+clean","last_fresh":"2026-02-24T07:53:35.778175+0000","last_change":"2026-02-24T07:53:14.212339+0000","last_active":"2026-02-24T07:53:35.778175+0000","last_peered":"2026-02-24T07:53:35.778175+0000","last_clean":"2026-02-24T07:53:35.778175+0000","last_became_active":"2026-02-24T07:51:53.899549+0000","last_became_peered":"2026-02-24T07:51:53.899549+0000","last_unstale":"2026-02-24T07:53:35.778175+0000","last_undegraded":"2026-02-24T07:53:35.778175+0000","last_fullsized":"2026-02-24T07:53:35.778175+0000","mapping_epoch":74,"log_start":"84'39617","ondisk_log_start":"84'39617","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"84'30986","last_scrub_stamp":"2026-02-24T07:53:14.212265+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_clean_scrub_stamp":"2026-02-24T07:53:14.212265+0000","objects_scrubbed":30490,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:54:35.341423+0000","scrub_duration":3381,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1954545664,"num_objects":29824,"num_object_clones":0,"num_object_copies":89472,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29824,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":39620,"num_write_kb":2222208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,1],"acting":[0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.9","version":"84'40146","reported_seq":40521,"reported_epoch":84,"state":"active+clean","last_fresh":"2026-02-24T07:53:35.765690+0000","last_change":"2026-02-24T07:53:22.113835+0000","last_active":"2026-02-24T07:53:35.765690+0000","last_peered":"2026-02-24T07:53:35.765690+0000","last_clean":"2026-02-24T07:53:35.765690+0000","last_became_active":"2026-02-24T07:51:53.901610+0000","last_became_peered":"2026-02-24T07:51:53.901610+0000","last_unstale":"2026-02-24T07:53:35.765690+0000","last_undegraded":"2026-02-24T07:53:35.765690+0000","last_fullsized":"2026-02-24T07:53:35.765690+0000","mapping_epoch":74,"log_start":"84'40143","ondisk_log_start":"84'40143","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"84'34029","last_scrub_stamp":"2026-02-24T07:53:22.113774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_clean_scrub_stamp":"2026-02-24T07:53:22.113774+0000","objects_scrubbed":33520,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:54:41.903404+0000","scrub_duration":3005,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1976958976,"num_objects":30166,"num_object_clones":0,"num_object_copies":90498,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30166,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":40146,"num_write_kb":2249984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.8","version":"84'39828","reported_seq":40120,"reported_epoch":84,"state":"active+clean","last_fresh":"2026-02-24T07:53:35.778220+0000","last_change":"2026-02-24T07:53:00.549866+0000","last_active":"2026-02-24T07:53:35.778220+0000","last_peered":"2026-02-24T07:53:35.778220+0000","last_clean":"2026-02-24T07:53:35.778220+0000","last_became_active":"2026-02-24T07:51:53.900207+0000","last_became_peered":"2026-02-24T07:51:53.900207+0000","last_unstale":"2026-02-24T07:53:35.778220+0000","last_undegraded":"2026-02-24T07:53:35.778220+0000","last_fullsized":"2026-02-24T07:53:35.778220+0000","mapping_epoch":74,"log_start":"84'39825","ondisk_log_start":"84'39825","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"84'25943","last_scrub_stamp":"2026-02-24T07:53:00.549789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_clean_scrub_stamp":"2026-02-24T07:53:00.549789+0000","objects_scrubbed":25486,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:54:18.896412+0000","scrub_duration":2647,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1965686784,"num_objects":29994,"num_object_clones":0,"num_object_copies":89982,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29994,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":39828,"num_write_kb":2234304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.7","version":"84'39255","reported_seq":39547,"reported_epoch":84,"state":"active+clean","last_fresh":"2026-02-24T07:53:35.764526+0000","last_change":"2026-02-24T07:53:00.188453+0000","last_active":"2026-02-24T07:53:35.764526+0000","last_peered":"2026-02-24T07:53:35.764526+0000","last_clean":"2026-02-24T07:53:35.764526+0000","last_became_active":"2026-02-24T07:51:53.899472+0000","last_became_peered":"2026-02-24T07:51:53.899472+0000","last_unstale":"2026-02-24T07:53:35.764526+0000","last_undegraded":"2026-02-24T07:53:35.764526+0000","last_fullsized":"2026-02-24T07:53:35.764526+0000","mapping_epoch":74,"log_start":"84'39252","ondisk_log_start":"84'39252","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"84'25630","last_scrub_stamp":"2026-02-24T07:53:00.188390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_clean_scrub_stamp":"2026-02-24T07:53:00.188390+0000","objects_scrubbed":25246,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:54:07.242502+0000","scrub_duration":2262,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1936130048,"num_objects":29543,"num_object_clones":0,"num_object_copies":88629,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29543,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":39255,"num_write_kb":2201536,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.6","version":"84'39298","reported_seq":39631,"reported_epoch":84,"state":"active+clean","last_fresh":"2026-02-24T07:53:35.857374+0000","last_change":"2026-02-24T07:53:10.804133+0000","last_active":"2026-02-24T07:53:35.857374+0000","last_peered":"2026-02-24T07:53:35.857374+0000","last_clean":"2026-02-24T07:53:35.857374+0000","last_became_active":"2026-02-24T07:51:53.898308+0000","last_became_peered":"2026-02-24T07:51:53.898308+0000","last_unstale":"2026-02-24T07:53:35.857374+0000","last_undegraded":"2026-02-24T07:53:35.857374+0000","last_fullsized":"2026-02-24T07:53:35.857374+0000","mapping_epoch":74,"log_start":"84'39295","ondisk_log_start":"84'39295","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"84'29829","last_scrub_stamp":"2026-02-24T07:53:10.804104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_clean_scrub_stamp":"2026-02-24T07:53:10.804104+0000","objects_scrubbed":29351,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:54:13.621421+0000","scrub_duration":2501,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1943011328,"num_objects":29648,"num_object_clones":0,"num_object_copies":88944,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29648,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":39298,"num_write_kb":2206272,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.5","version":"84'39622","reported_seq":40007,"reported_epoch":84,"state":"active+clean","last_fresh":"2026-02-24T07:53:35.769242+0000","last_change":"2026-02-24T07:53:26.781324+0000","last_active":"2026-02-24T07:53:35.769242+0000","last_peered":"2026-02-24T07:53:35.769242+0000","last_clean":"2026-02-24T07:53:35.769242+0000","last_became_active":"2026-02-24T07:51:53.899749+0000","last_became_peered":"2026-02-24T07:51:53.899749+0000","last_unstale":"2026-02-24T07:53:35.769242+0000","last_undegraded":"2026-02-24T07:53:35.769242+0000","last_fullsized":"2026-02-24T07:53:35.769242+0000","mapping_epoch":74,"log_start":"84'39617","ondisk_log_start":"84'39617","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"84'35538","last_scrub_stamp":"2026-02-24T07:53:26.781276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_clean_scrub_stamp":"2026-02-24T07:53:26.781276+0000","objects_scrubbed":34358,"log_size":5,"log_dups_size":3000,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:54:37.440161+0000","scrub_duration":2982,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1952055296,"num_objects":29786,"num_object_clones":0,"num_object_copies":89358,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29786,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":39622,"num_write_kb":2221056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"84'39672","reported_seq":39985,"reported_epoch":84,"state":"active+clean","last_fresh":"2026-02-24T07:53:35.778266+0000","last_change":"2026-02-24T07:53:05.312364+0000","last_active":"2026-02-24T07:53:35.778266+0000","last_peered":"2026-02-24T07:53:35.778266+0000","last_clean":"2026-02-24T07:53:35.778266+0000","last_became_active":"2026-02-24T07:51:53.900042+0000","last_became_peered":"2026-02-24T07:51:53.900042+0000","last_unstale":"2026-02-24T07:53:35.778266+0000","last_undegraded":"2026-02-24T07:53:35.778266+0000","last_fullsized":"2026-02-24T07:53:35.778266+0000","mapping_epoch":74,"log_start":"84'39669","ondisk_log_start":"84'39669","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"84'27738","last_scrub_stamp":"2026-02-24T07:53:05.312312+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_clean_scrub_stamp":"2026-02-24T07:53:05.312312+0000","objects_scrubbed":27245,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:54:32.166518+0000","scrub_duration":2472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1952710656,"num_objects":29796,"num_object_clones":0,"num_object_copies":89388,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29796,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":39672,"num_write_kb":2222976,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.2","version":"84'40008","reported_seq":40368,"reported_epoch":84,"state":"active+clean","last_fresh":"2026-02-24T07:53:35.758753+0000","last_change":"2026-02-24T07:53:17.780799+0000","last_active":"2026-02-24T07:53:35.758753+0000","last_peered":"2026-02-24T07:53:35.758753+0000","last_clean":"2026-02-24T07:53:35.758753+0000","last_became_active":"2026-02-24T07:51:53.900096+0000","last_became_peered":"2026-02-24T07:51:53.900096+0000","last_unstale":"2026-02-24T07:53:35.758753+0000","last_undegraded":"2026-02-24T07:53:35.758753+0000","last_fullsized":"2026-02-24T07:53:35.758753+0000","mapping_epoch":74,"log_start":"84'40005","ondisk_log_start":"84'40005","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"84'32548","last_scrub_stamp":"2026-02-24T07:53:17.780705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_clean_scrub_stamp":"2026-02-24T07:53:17.780705+0000","objects_scrubbed":32030,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:54:35.344780+0000","scrub_duration":2741,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1973551104,"num_objects":30114,"num_object_clones":0,"num_object_copies":90342,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30114,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":40008,"num_write_kb":2243904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"84'39200","reported_seq":39541,"reported_epoch":84,"state":"active+clean","last_fresh":"2026-02-24T07:53:35.766622+0000","last_change":"2026-02-24T07:53:14.051194+0000","last_active":"2026-02-24T07:53:35.766622+0000","last_peered":"2026-02-24T07:53:35.766622+0000","last_clean":"2026-02-24T07:53:35.766622+0000","last_became_active":"2026-02-24T07:51:53.901027+0000","last_became_peered":"2026-02-24T07:51:53.901027+0000","last_unstale":"2026-02-24T07:53:35.766622+0000","last_undegraded":"2026-02-24T07:53:35.766622+0000","last_fullsized":"2026-02-24T07:53:35.766622+0000","mapping_epoch":74,"log_start":"84'39197","ondisk_log_start":"84'39197","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"84'30565","last_scrub_stamp":"2026-02-24T07:53:14.051154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_clean_scrub_stamp":"2026-02-24T07:53:14.051154+0000","objects_scrubbed":30140,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:54:38.448076+0000","scrub_duration":3147,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1938292736,"num_objects":29576,"num_object_clones":0,"num_object_copies":88728,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29576,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":39200,"num_write_kb":2200832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.0","version":"84'39974","reported_seq":40340,"reported_epoch":84,"state":"active+clean","last_fresh":"2026-02-24T07:53:35.774498+0000","last_change":"2026-02-24T07:53:20.014077+0000","last_active":"2026-02-24T07:53:35.774498+0000","last_peered":"2026-02-24T07:53:35.774498+0000","last_clean":"2026-02-24T07:53:35.774498+0000","last_became_active":"2026-02-24T07:51:53.898551+0000","last_became_peered":"2026-02-24T07:51:53.898551+0000","last_unstale":"2026-02-24T07:53:35.774498+0000","last_undegraded":"2026-02-24T07:53:35.774498+0000","last_fullsized":"2026-02-24T07:53:35.774498+0000","mapping_epoch":74,"log_start":"84'39971","ondisk_log_start":"84'39971","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"84'33261","last_scrub_stamp":"2026-02-24T07:53:20.014009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_clean_scrub_stamp":"2026-02-24T07:53:20.014009+0000","objects_scrubbed":32679,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:54:37.178341+0000","scrub_duration":3163,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1969225728,"num_objects":30048,"num_object_clones":0,"num_object_copies":90144,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30048,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":39974,"num_write_kb":2240704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.4","version":"84'39502","reported_seq":39839,"reported_epoch":84,"state":"active+clean","last_fresh":"2026-02-24T07:53:35.753777+0000","last_change":"2026-02-24T07:53:11.592191+0000","last_active":"2026-02-24T07:53:35.753777+0000","last_peered":"2026-02-24T07:53:35.753777+0000","last_clean":"2026-02-24T07:53:35.753777+0000","last_became_active":"2026-02-24T07:51:53.899675+0000","last_became_peered":"2026-02-24T07:51:53.899675+0000","last_unstale":"2026-02-24T07:53:35.753777+0000","last_undegraded":"2026-02-24T07:53:35.753777+0000","last_fullsized":"2026-02-24T07:53:35.753777+0000","mapping_epoch":74,"log_start":"84'39499","ondisk_log_start":"84'39499","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"84'30127","last_scrub_stamp":"2026-02-24T07:53:11.592155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_clean_scrub_stamp":"2026-02-24T07:53:11.592155+0000","objects_scrubbed":29709,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:54:23.794472+0000","scrub_duration":2432,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1949564928,"num_objects":29748,"num_object_clones":0,"num_object_copies":89244,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29748,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":39502,"num_write_kb":2216000,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"29'99","reported_seq":357,"reported_epoch":84,"state":"active+clean","last_fresh":"2026-02-24T07:52:58.893578+0000","last_change":"2026-02-24T07:52:56.946143+0000","last_active":"2026-02-24T07:52:58.893578+0000","last_peered":"2026-02-24T07:52:58.893578+0000","last_clean":"2026-02-24T07:52:58.893578+0000","last_became_active":"2026-02-24T07:42:54.186719+0000","last_became_peered":"2026-02-24T07:42:54.186719+0000","last_unstale":"2026-02-24T07:52:58.893578+0000","last_undegraded":"2026-02-24T07:52:58.893578+0000","last_fullsized":"2026-02-24T07:52:58.893578+0000","mapping_epoch":27,"log_start":"29'96","ondisk_log_start":"29'96","created":20,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'99","last_scrub_stamp":"2026-02-24T07:52:56.946109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:42:43.640776+0000","last_clean_scrub_stamp":"2026-02-24T07:52:56.946109+0000","objects_scrubbed":2,"log_size":3,"log_dups_size":96,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:54:05.896060+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":240,"num_write":135,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":16,"stat_sum":{"num_bytes":31272665112,"num_objects":477184,"num_object_clones":0,"num_object_copies":1431552,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":477184,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":633956,"num_write_kb":35556417,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":93812502528,"data_stored":93812490312,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":50,"ondisk_log_size":50,"up":48,"acting":48,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":240,"num_write":135,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1682,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":3,"up_from":25,"seq":107374182531,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":28578400,"kb_used_data":26996428,"kb_used_omap":16,"kb_used_meta":1581743,"kb_avail":705424800,"statfs":{"total":751619276800,"available":722354995200,"internally_reserved":0,"allocated":27644342272,"data_stored":27643744667,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16915,"internal_metadata":1619705325},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[7],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":19,"seq":81604378757,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":23225056,"kb_used_data":21257228,"kb_used_omap":15,"kb_used_meta":1967664,"kb_avail":710778144,"statfs":{"total":751619276800,"available":727836819456,"internally_reserved":0,"allocated":21767401472,"data_stored":21766803867,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":15701,"internal_metadata":2014888619},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542278,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":21124120,"kb_used_data":19239364,"kb_used_omap":20,"kb_used_meta":1884715,"kb_avail":712879080,"statfs":{"total":751619276800,"available":729988177920,"internally_reserved":0,"allocated":19701108736,"data_stored":19700518755,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":20777,"internal_metadata":1929948887},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705801,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":25774688,"kb_used_data":24864972,"kb_used_omap":13,"kb_used_meta":909682,"kb_avail":708228512,"statfs":{"total":751619276800,"available":725225996288,"internally_reserved":0,"allocated":25461731328,"data_stored":25461133723,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13632,"internal_metadata":931515072},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1682,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":25403265024,"data_stored":25403260952,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":19517276160,"data_stored":19517276160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":21531398144,"data_stored":21531394072,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":27360563200,"data_stored":27360559128,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-24T07:53:41.298 INFO:tasks.cephadm.ceph_manager.ceph:increase pool size by 10 2026-02-24T07:53:41.299 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph osd pool set unique_pool_3 pg_num 46 2026-02-24T07:53:41.440 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:53:41.566 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:53:41.564+0000 7ff323b81640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:53:41.566 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:53:41.564+0000 7ff323b81640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:53:41.704 INFO:teuthology.orchestra.run.trial035.stderr:Error EPERM: crimson_allow_pg_split is false; pg_num increase denied for crimson pool 2026-02-24T07:53:41.744 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-24T07:53:41.786 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2026-02-24T07:53:41.927 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:53:42.051 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:53:42.048+0000 7f978075a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:53:42.051 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:53:42.048+0000 7f978075a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:53:42.065 INFO:teuthology.orchestra.run.trial035.stdout:osd.0: {} 2026-02-24T07:53:42.065 INFO:teuthology.orchestra.run.trial035.stderr:osd.0: osd_enable_op_tracker = '' 2026-02-24T07:53:42.076 INFO:teuthology.orchestra.run.trial035.stdout:osd.1: {} 2026-02-24T07:53:42.076 INFO:teuthology.orchestra.run.trial035.stderr:osd.1: osd_enable_op_tracker = '' 2026-02-24T07:53:42.088 INFO:teuthology.orchestra.run.trial035.stdout:osd.2: {} 2026-02-24T07:53:42.088 INFO:teuthology.orchestra.run.trial035.stderr:osd.2: osd_enable_op_tracker = '' 2026-02-24T07:53:42.099 INFO:teuthology.orchestra.run.trial035.stdout:osd.3: {} 2026-02-24T07:53:42.100 INFO:teuthology.orchestra.run.trial035.stderr:osd.3: osd_enable_op_tracker = '' 2026-02-24T07:53:42.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:53:41 trial035 ceph-mon[39284]: from='client.15306 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2026-02-24T07:53:42.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:53:41 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/1615842899' entity='client.admin' cmd={"prefix": "osd pool set", "pool": "unique_pool_3", "var": "pg_num", "val": "46"} : dispatch 2026-02-24T07:53:42.205 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:53:41 trial035 ceph-mon[39284]: pgmap v383: 17 pgs: 17 active+clean; 27 GiB data, 87 GiB used, 2.6 TiB / 2.7 TiB avail; 0 B/s wr, 8.24k op/s 2026-02-24T07:53:43.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:53:42 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/3803401890' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2026-02-24T07:53:44.141 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2026-02-24T07:53:44.282 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:53:44.398 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:53:44.395+0000 7f3539156640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:53:44.398 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:53:44.396+0000 7f3539156640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:53:44.413 INFO:teuthology.orchestra.run.trial035.stdout:osd.0: {} 2026-02-24T07:53:44.413 INFO:teuthology.orchestra.run.trial035.stderr:osd.0: osd_enable_op_tracker = '' 2026-02-24T07:53:44.426 INFO:teuthology.orchestra.run.trial035.stdout:osd.1: {} 2026-02-24T07:53:44.427 INFO:teuthology.orchestra.run.trial035.stderr:osd.1: osd_enable_op_tracker = '' 2026-02-24T07:53:44.442 INFO:teuthology.orchestra.run.trial035.stdout:osd.2: {} 2026-02-24T07:53:44.442 INFO:teuthology.orchestra.run.trial035.stderr:osd.2: osd_enable_op_tracker = '' 2026-02-24T07:53:44.456 INFO:teuthology.orchestra.run.trial035.stdout:osd.3: {} 2026-02-24T07:53:44.456 INFO:teuthology.orchestra.run.trial035.stderr:osd.3: osd_enable_op_tracker = '' 2026-02-24T07:53:45.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:53:44 trial035 ceph-mon[39284]: pgmap v384: 17 pgs: 17 active+clean; 27 GiB data, 87 GiB used, 2.6 TiB / 2.7 TiB avail; 0 B/s wr, 6.63k op/s 2026-02-24T07:53:45.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:53:44 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/3306022278' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2026-02-24T07:53:45.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:53:44 trial035 ceph-mon[39284]: from='mgr.14284 10.20.193.35:0/3681373222' entity='mgr.x' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2026-02-24T07:53:46.498 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2026-02-24T07:53:46.646 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:53:46.746 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2026-02-24T07:53:46.746 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2026-02-24T07:53:46.746 INFO:tasks.thrashosds.thrasher:Growing pool .mgr 2026-02-24T07:53:46.746 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph pg dump --format=json 2026-02-24T07:53:46.873 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:53:47.049 INFO:teuthology.orchestra.run.trial035.stdout:osd.0: {} 2026-02-24T07:53:47.049 INFO:teuthology.orchestra.run.trial035.stdout:osd.1: {} 2026-02-24T07:53:47.049 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:53:47.009+0000 7f28c6f07640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:53:47.049 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:53:47.010+0000 7f28c6f07640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:53:47.049 INFO:teuthology.orchestra.run.trial035.stderr:osd.0: osd_enable_op_tracker = '' 2026-02-24T07:53:47.049 INFO:teuthology.orchestra.run.trial035.stderr:osd.1: osd_enable_op_tracker = '' 2026-02-24T07:53:47.052 INFO:teuthology.orchestra.run.trial035.stdout:osd.2: {} 2026-02-24T07:53:47.053 INFO:teuthology.orchestra.run.trial035.stderr:osd.2: osd_enable_op_tracker = '' 2026-02-24T07:53:47.066 INFO:teuthology.orchestra.run.trial035.stdout:osd.3: {} 2026-02-24T07:53:47.067 INFO:teuthology.orchestra.run.trial035.stderr:osd.3: osd_enable_op_tracker = '' 2026-02-24T07:53:47.067 INFO:journalctl@ceph.mgr.x.trial035.stdout:Feb 24 07:53:46 trial035 ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-mgr-x[39621]: ::ffff:10.20.193.91 - - [24/Feb/2026:07:53:46] "GET /metrics HTTP/1.1" 200 4230 "" "Prometheus/3.6.0" 2026-02-24T07:53:47.067 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:53:46 trial035 ceph-mon[39284]: pgmap v385: 17 pgs: 17 active+clean; 27 GiB data, 87 GiB used, 2.6 TiB / 2.7 TiB avail; 0 B/s wr, 5.09k op/s 2026-02-24T07:53:47.264 INFO:journalctl@ceph.prometheus.a.trial091.stdout:Feb 24 07:53:46 trial091 ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-prometheus-a[66007]: time=2026-02-24T07:53:46.973Z level=ERROR source=scrape.go:1630 msg="Failed to determine correct type of scrape target." component="scrape manager" scrape_pool=ceph target=http://trial035:9283/metrics content_type="text/html;charset=utf-8" fallback_media_type="" err="received unsupported Content-Type \"text/html;charset=utf-8\" and no fallback_scrape_protocol specified for target" 2026-02-24T07:53:47.824 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:53:47.814+0000 7fbfed97b640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:53:47.825 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:53:47.814+0000 7fbfed97b640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:53:47.961 INFO:teuthology.orchestra.run.trial035.stdout: 2026-02-24T07:53:47.961 INFO:teuthology.orchestra.run.trial035.stderr:dumped all 2026-02-24T07:53:48.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:53:47 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/3994685685' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2026-02-24T07:53:48.565 INFO:teuthology.orchestra.run.trial035.stdout:{"pg_ready":true,"pg_map":{"version":386,"stamp":"2026-02-24T07:53:47.745276+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":25949700664,"num_objects":395955,"num_object_clones":0,"num_object_copies":1187865,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":395955,"num_whiteouts":0,"num_read":137,"num_read_kb":241,"num_write":715322,"num_write_kb":35558567,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":51,"ondisk_log_size":51,"up":51,"acting":51,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":51,"num_osds":4,"num_per_pool_osds":4,"num_per_pool_omap_osds":4,"kb":2936012800,"kb_used":83026520,"kb_used_data":76989512,"kb_used_omap":84,"kb_used_meta":6036715,"kb_avail":2852986280,"statfs":{"total":3006477107200,"available":2921457950720,"internally_reserved":0,"allocated":78837260288,"data_stored":78835489056,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":86105,"internal_metadata":6181597095},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[13,1],"upper_bound":4},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-5323554816,"num_objects":-81231,"num_object_clones":0,"num_object_copies":-243693,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-81231,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":81231,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.823848"},"pg_stats":[{"pgid":"5.f","version":"88'44514","reported_seq":44817,"reported_epoch":88,"state":"active+clean","last_fresh":"2026-02-24T07:53:45.746509+0000","last_change":"2026-02-24T07:53:01.242353+0000","last_active":"2026-02-24T07:53:45.746509+0000","last_peered":"2026-02-24T07:53:45.746509+0000","last_clean":"2026-02-24T07:53:45.746509+0000","last_became_active":"2026-02-24T07:51:53.900431+0000","last_became_peered":"2026-02-24T07:51:53.900431+0000","last_unstale":"2026-02-24T07:53:45.746509+0000","last_undegraded":"2026-02-24T07:53:45.746509+0000","last_fullsized":"2026-02-24T07:53:45.746509+0000","mapping_epoch":74,"log_start":"88'44511","ondisk_log_start":"88'44511","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"84'26061","last_scrub_stamp":"2026-02-24T07:53:01.242306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_clean_scrub_stamp":"2026-02-24T07:53:01.242306+0000","objects_scrubbed":25674,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:54:22.699112+0000","scrub_duration":2129,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1626472448,"num_objects":24818,"num_object_clones":0,"num_object_copies":74454,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24818,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44514,"num_write_kb":2218624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.e","version":"88'44459","reported_seq":44749,"reported_epoch":88,"state":"active+clean","last_fresh":"2026-02-24T07:53:45.768664+0000","last_change":"2026-02-24T07:52:56.767335+0000","last_active":"2026-02-24T07:53:45.768664+0000","last_peered":"2026-02-24T07:53:45.768664+0000","last_clean":"2026-02-24T07:53:45.768664+0000","last_became_active":"2026-02-24T07:51:53.899294+0000","last_became_peered":"2026-02-24T07:51:53.899294+0000","last_unstale":"2026-02-24T07:53:45.768664+0000","last_undegraded":"2026-02-24T07:53:45.768664+0000","last_fullsized":"2026-02-24T07:53:45.768664+0000","mapping_epoch":74,"log_start":"88'44456","ondisk_log_start":"88'44456","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"83'24557","last_scrub_stamp":"2026-02-24T07:52:56.767267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_clean_scrub_stamp":"2026-02-24T07:52:56.767267+0000","objects_scrubbed":24118,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:54:24.913755+0000","scrub_duration":2806,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1610547200,"num_objects":24575,"num_object_clones":0,"num_object_copies":73725,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24575,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44459,"num_write_kb":2209088,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"88'44680","reported_seq":45050,"reported_epoch":88,"state":"active+clean","last_fresh":"2026-02-24T07:53:45.781150+0000","last_change":"2026-02-24T07:53:32.480551+0000","last_active":"2026-02-24T07:53:45.781150+0000","last_peered":"2026-02-24T07:53:45.781150+0000","last_clean":"2026-02-24T07:53:45.781150+0000","last_became_active":"2026-02-24T07:51:53.899772+0000","last_became_peered":"2026-02-24T07:51:53.899772+0000","last_unstale":"2026-02-24T07:53:45.781150+0000","last_undegraded":"2026-02-24T07:53:45.781150+0000","last_fullsized":"2026-02-24T07:53:45.781150+0000","mapping_epoch":74,"log_start":"88'44677","ondisk_log_start":"88'44677","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"84'37904","last_scrub_stamp":"2026-02-24T07:53:32.480499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_clean_scrub_stamp":"2026-02-24T07:53:32.480499+0000","objects_scrubbed":32149,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:54:51.939533+0000","scrub_duration":2742,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1624047640,"num_objects":24782,"num_object_clones":0,"num_object_copies":74346,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24782,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":44680,"num_write_kb":2222721,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,3],"acting":[0,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.c","version":"88'44721","reported_seq":45084,"reported_epoch":88,"state":"active+clean","last_fresh":"2026-02-24T07:53:45.854577+0000","last_change":"2026-02-24T07:53:17.083868+0000","last_active":"2026-02-24T07:53:45.854577+0000","last_peered":"2026-02-24T07:53:45.854577+0000","last_clean":"2026-02-24T07:53:45.854577+0000","last_became_active":"2026-02-24T07:51:53.898285+0000","last_became_peered":"2026-02-24T07:51:53.898285+0000","last_unstale":"2026-02-24T07:53:45.854577+0000","last_undegraded":"2026-02-24T07:53:45.854577+0000","last_fullsized":"2026-02-24T07:53:45.854577+0000","mapping_epoch":74,"log_start":"88'44718","ondisk_log_start":"88'44718","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"84'32067","last_scrub_stamp":"2026-02-24T07:53:17.083821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_clean_scrub_stamp":"2026-02-24T07:53:17.083821+0000","objects_scrubbed":31581,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:54:37.993683+0000","scrub_duration":2856,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1620377600,"num_objects":24725,"num_object_clones":0,"num_object_copies":74175,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24725,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44721,"num_write_kb":2222272,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.b","version":"88'44778","reported_seq":45157,"reported_epoch":88,"state":"active+clean","last_fresh":"2026-02-24T07:53:45.779400+0000","last_change":"2026-02-24T07:53:30.225127+0000","last_active":"2026-02-24T07:53:45.779400+0000","last_peered":"2026-02-24T07:53:45.779400+0000","last_clean":"2026-02-24T07:53:45.779400+0000","last_became_active":"2026-02-24T07:51:53.899901+0000","last_became_peered":"2026-02-24T07:51:53.899901+0000","last_unstale":"2026-02-24T07:53:45.779400+0000","last_undegraded":"2026-02-24T07:53:45.779400+0000","last_fullsized":"2026-02-24T07:53:45.779400+0000","mapping_epoch":74,"log_start":"88'44775","ondisk_log_start":"88'44775","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"84'36879","last_scrub_stamp":"2026-02-24T07:53:30.225075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_clean_scrub_stamp":"2026-02-24T07:53:30.225075+0000","objects_scrubbed":33279,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:54:35.718496+0000","scrub_duration":3524,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1620049920,"num_objects":24720,"num_object_clones":0,"num_object_copies":74160,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24720,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44778,"num_write_kb":2223936,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"88'44656","reported_seq":45007,"reported_epoch":88,"state":"active+clean","last_fresh":"2026-02-24T07:53:45.777598+0000","last_change":"2026-02-24T07:53:14.212339+0000","last_active":"2026-02-24T07:53:45.777598+0000","last_peered":"2026-02-24T07:53:45.777598+0000","last_clean":"2026-02-24T07:53:45.777598+0000","last_became_active":"2026-02-24T07:51:53.899549+0000","last_became_peered":"2026-02-24T07:51:53.899549+0000","last_unstale":"2026-02-24T07:53:45.777598+0000","last_undegraded":"2026-02-24T07:53:45.777598+0000","last_fullsized":"2026-02-24T07:53:45.777598+0000","mapping_epoch":74,"log_start":"88'44653","ondisk_log_start":"88'44653","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"84'30986","last_scrub_stamp":"2026-02-24T07:53:14.212265+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_clean_scrub_stamp":"2026-02-24T07:53:14.212265+0000","objects_scrubbed":30490,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:54:35.341423+0000","scrub_duration":3381,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1624506368,"num_objects":24788,"num_object_clones":0,"num_object_copies":74364,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24788,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44656,"num_write_kb":2222208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,1],"acting":[0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.9","version":"88'45304","reported_seq":45687,"reported_epoch":88,"state":"active+clean","last_fresh":"2026-02-24T07:53:45.768149+0000","last_change":"2026-02-24T07:53:22.113835+0000","last_active":"2026-02-24T07:53:45.768149+0000","last_peered":"2026-02-24T07:53:45.768149+0000","last_clean":"2026-02-24T07:53:45.768149+0000","last_became_active":"2026-02-24T07:51:53.901610+0000","last_became_peered":"2026-02-24T07:51:53.901610+0000","last_unstale":"2026-02-24T07:53:45.768149+0000","last_undegraded":"2026-02-24T07:53:45.768149+0000","last_fullsized":"2026-02-24T07:53:45.768149+0000","mapping_epoch":74,"log_start":"88'45301","ondisk_log_start":"88'45301","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"84'34029","last_scrub_stamp":"2026-02-24T07:53:22.113774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_clean_scrub_stamp":"2026-02-24T07:53:22.113774+0000","objects_scrubbed":33520,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:54:41.903404+0000","scrub_duration":3005,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1638924288,"num_objects":25008,"num_object_clones":0,"num_object_copies":75024,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25008,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45304,"num_write_kb":2249984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.8","version":"88'44820","reported_seq":45121,"reported_epoch":88,"state":"active+clean","last_fresh":"2026-02-24T07:53:45.778596+0000","last_change":"2026-02-24T07:53:00.549866+0000","last_active":"2026-02-24T07:53:45.778596+0000","last_peered":"2026-02-24T07:53:45.778596+0000","last_clean":"2026-02-24T07:53:45.778596+0000","last_became_active":"2026-02-24T07:51:53.900207+0000","last_became_peered":"2026-02-24T07:51:53.900207+0000","last_unstale":"2026-02-24T07:53:45.778596+0000","last_undegraded":"2026-02-24T07:53:45.778596+0000","last_fullsized":"2026-02-24T07:53:45.778596+0000","mapping_epoch":74,"log_start":"88'44817","ondisk_log_start":"88'44817","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"84'25943","last_scrub_stamp":"2026-02-24T07:53:00.549789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_clean_scrub_stamp":"2026-02-24T07:53:00.549789+0000","objects_scrubbed":25486,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:54:18.896412+0000","scrub_duration":2647,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1638531072,"num_objects":25002,"num_object_clones":0,"num_object_copies":75006,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25002,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44820,"num_write_kb":2234304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.7","version":"88'44305","reported_seq":44605,"reported_epoch":88,"state":"active+clean","last_fresh":"2026-02-24T07:53:45.760305+0000","last_change":"2026-02-24T07:53:00.188453+0000","last_active":"2026-02-24T07:53:45.760305+0000","last_peered":"2026-02-24T07:53:45.760305+0000","last_clean":"2026-02-24T07:53:45.760305+0000","last_became_active":"2026-02-24T07:51:53.899472+0000","last_became_peered":"2026-02-24T07:51:53.899472+0000","last_unstale":"2026-02-24T07:53:45.760305+0000","last_undegraded":"2026-02-24T07:53:45.760305+0000","last_fullsized":"2026-02-24T07:53:45.760305+0000","mapping_epoch":74,"log_start":"88'44302","ondisk_log_start":"88'44302","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"84'25630","last_scrub_stamp":"2026-02-24T07:53:00.188390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_clean_scrub_stamp":"2026-02-24T07:53:00.188390+0000","objects_scrubbed":25246,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:54:07.242502+0000","scrub_duration":2262,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1605173248,"num_objects":24493,"num_object_clones":0,"num_object_copies":73479,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24493,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44305,"num_write_kb":2201536,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.6","version":"88'44309","reported_seq":44649,"reported_epoch":88,"state":"active+clean","last_fresh":"2026-02-24T07:53:45.860130+0000","last_change":"2026-02-24T07:53:10.804133+0000","last_active":"2026-02-24T07:53:45.860130+0000","last_peered":"2026-02-24T07:53:45.860130+0000","last_clean":"2026-02-24T07:53:45.860130+0000","last_became_active":"2026-02-24T07:51:53.898308+0000","last_became_peered":"2026-02-24T07:51:53.898308+0000","last_unstale":"2026-02-24T07:53:45.860130+0000","last_undegraded":"2026-02-24T07:53:45.860130+0000","last_fullsized":"2026-02-24T07:53:45.860130+0000","mapping_epoch":74,"log_start":"88'44306","ondisk_log_start":"88'44306","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"84'29829","last_scrub_stamp":"2026-02-24T07:53:10.804104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_clean_scrub_stamp":"2026-02-24T07:53:10.804104+0000","objects_scrubbed":29351,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:54:13.621421+0000","scrub_duration":2501,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1614610432,"num_objects":24637,"num_object_clones":0,"num_object_copies":73911,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24637,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44309,"num_write_kb":2206272,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.5","version":"88'44726","reported_seq":45119,"reported_epoch":88,"state":"active+clean","last_fresh":"2026-02-24T07:53:45.780077+0000","last_change":"2026-02-24T07:53:26.781324+0000","last_active":"2026-02-24T07:53:45.780077+0000","last_peered":"2026-02-24T07:53:45.780077+0000","last_clean":"2026-02-24T07:53:45.780077+0000","last_became_active":"2026-02-24T07:51:53.899749+0000","last_became_peered":"2026-02-24T07:51:53.899749+0000","last_unstale":"2026-02-24T07:53:45.780077+0000","last_undegraded":"2026-02-24T07:53:45.780077+0000","last_fullsized":"2026-02-24T07:53:45.780077+0000","mapping_epoch":74,"log_start":"88'44723","ondisk_log_start":"88'44723","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"84'35538","last_scrub_stamp":"2026-02-24T07:53:26.781276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_clean_scrub_stamp":"2026-02-24T07:53:26.781276+0000","objects_scrubbed":34358,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:54:37.440161+0000","scrub_duration":2982,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1617559552,"num_objects":24682,"num_object_clones":0,"num_object_copies":74046,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24682,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44726,"num_write_kb":2221056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"88'44656","reported_seq":44977,"reported_epoch":88,"state":"active+clean","last_fresh":"2026-02-24T07:53:45.777824+0000","last_change":"2026-02-24T07:53:05.312364+0000","last_active":"2026-02-24T07:53:45.777824+0000","last_peered":"2026-02-24T07:53:45.777824+0000","last_clean":"2026-02-24T07:53:45.777824+0000","last_became_active":"2026-02-24T07:51:53.900042+0000","last_became_peered":"2026-02-24T07:51:53.900042+0000","last_unstale":"2026-02-24T07:53:45.777824+0000","last_undegraded":"2026-02-24T07:53:45.777824+0000","last_fullsized":"2026-02-24T07:53:45.777824+0000","mapping_epoch":74,"log_start":"88'44653","ondisk_log_start":"88'44653","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"84'27738","last_scrub_stamp":"2026-02-24T07:53:05.312312+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_clean_scrub_stamp":"2026-02-24T07:53:05.312312+0000","objects_scrubbed":27245,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:54:32.166518+0000","scrub_duration":2472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1626079232,"num_objects":24812,"num_object_clones":0,"num_object_copies":74436,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24812,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44656,"num_write_kb":2222976,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.2","version":"88'45268","reported_seq":45636,"reported_epoch":88,"state":"active+clean","last_fresh":"2026-02-24T07:53:45.768611+0000","last_change":"2026-02-24T07:53:17.780799+0000","last_active":"2026-02-24T07:53:45.768611+0000","last_peered":"2026-02-24T07:53:45.768611+0000","last_clean":"2026-02-24T07:53:45.768611+0000","last_became_active":"2026-02-24T07:51:53.900096+0000","last_became_peered":"2026-02-24T07:51:53.900096+0000","last_unstale":"2026-02-24T07:53:45.768611+0000","last_undegraded":"2026-02-24T07:53:45.768611+0000","last_fullsized":"2026-02-24T07:53:45.768611+0000","mapping_epoch":74,"log_start":"88'45265","ondisk_log_start":"88'45265","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"84'32548","last_scrub_stamp":"2026-02-24T07:53:17.780705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_clean_scrub_stamp":"2026-02-24T07:53:17.780705+0000","objects_scrubbed":32030,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:54:35.344780+0000","scrub_duration":2741,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1628831744,"num_objects":24854,"num_object_clones":0,"num_object_copies":74562,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24854,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45268,"num_write_kb":2243904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"88'44208","reported_seq":44557,"reported_epoch":88,"state":"active+clean","last_fresh":"2026-02-24T07:53:45.769274+0000","last_change":"2026-02-24T07:53:14.051194+0000","last_active":"2026-02-24T07:53:45.769274+0000","last_peered":"2026-02-24T07:53:45.769274+0000","last_clean":"2026-02-24T07:53:45.769274+0000","last_became_active":"2026-02-24T07:51:53.901027+0000","last_became_peered":"2026-02-24T07:51:53.901027+0000","last_unstale":"2026-02-24T07:53:45.769274+0000","last_undegraded":"2026-02-24T07:53:45.769274+0000","last_fullsized":"2026-02-24T07:53:45.769274+0000","mapping_epoch":74,"log_start":"88'44205","ondisk_log_start":"88'44205","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"84'30565","last_scrub_stamp":"2026-02-24T07:53:14.051154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_clean_scrub_stamp":"2026-02-24T07:53:14.051154+0000","objects_scrubbed":30140,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:54:38.448076+0000","scrub_duration":3147,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1610088448,"num_objects":24568,"num_object_clones":0,"num_object_copies":73704,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24568,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44208,"num_write_kb":2200832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.0","version":"88'45207","reported_seq":45581,"reported_epoch":88,"state":"active+clean","last_fresh":"2026-02-24T07:53:45.772645+0000","last_change":"2026-02-24T07:53:20.014077+0000","last_active":"2026-02-24T07:53:45.772645+0000","last_peered":"2026-02-24T07:53:45.772645+0000","last_clean":"2026-02-24T07:53:45.772645+0000","last_became_active":"2026-02-24T07:51:53.898551+0000","last_became_peered":"2026-02-24T07:51:53.898551+0000","last_unstale":"2026-02-24T07:53:45.772645+0000","last_undegraded":"2026-02-24T07:53:45.772645+0000","last_fullsized":"2026-02-24T07:53:45.772645+0000","mapping_epoch":74,"log_start":"88'45204","ondisk_log_start":"88'45204","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"84'33261","last_scrub_stamp":"2026-02-24T07:53:20.014009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_clean_scrub_stamp":"2026-02-24T07:53:20.014009+0000","objects_scrubbed":32679,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:54:37.178341+0000","scrub_duration":3163,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1626275840,"num_objects":24815,"num_object_clones":0,"num_object_copies":74445,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24815,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45207,"num_write_kb":2240704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.4","version":"88'44576","reported_seq":44921,"reported_epoch":88,"state":"active+clean","last_fresh":"2026-02-24T07:53:45.750250+0000","last_change":"2026-02-24T07:53:11.592191+0000","last_active":"2026-02-24T07:53:45.750250+0000","last_peered":"2026-02-24T07:53:45.750250+0000","last_clean":"2026-02-24T07:53:45.750250+0000","last_became_active":"2026-02-24T07:51:53.899675+0000","last_became_peered":"2026-02-24T07:51:53.899675+0000","last_unstale":"2026-02-24T07:53:45.750250+0000","last_undegraded":"2026-02-24T07:53:45.750250+0000","last_fullsized":"2026-02-24T07:53:45.750250+0000","mapping_epoch":74,"log_start":"88'44573","ondisk_log_start":"88'44573","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"84'30127","last_scrub_stamp":"2026-02-24T07:53:11.592155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_clean_scrub_stamp":"2026-02-24T07:53:11.592155+0000","objects_scrubbed":29709,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:54:23.794472+0000","scrub_duration":2432,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1617035264,"num_objects":24674,"num_object_clones":0,"num_object_copies":74022,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24674,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44576,"num_write_kb":2216000,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"29'99","reported_seq":365,"reported_epoch":88,"state":"active+clean","last_fresh":"2026-02-24T07:53:39.925612+0000","last_change":"2026-02-24T07:52:56.946143+0000","last_active":"2026-02-24T07:53:39.925612+0000","last_peered":"2026-02-24T07:53:39.925612+0000","last_clean":"2026-02-24T07:53:39.925612+0000","last_became_active":"2026-02-24T07:42:54.186719+0000","last_became_peered":"2026-02-24T07:42:54.186719+0000","last_unstale":"2026-02-24T07:53:39.925612+0000","last_undegraded":"2026-02-24T07:53:39.925612+0000","last_fullsized":"2026-02-24T07:53:39.925612+0000","mapping_epoch":27,"log_start":"29'96","ondisk_log_start":"29'96","created":20,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'99","last_scrub_stamp":"2026-02-24T07:52:56.946109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:42:43.640776+0000","last_clean_scrub_stamp":"2026-02-24T07:52:56.946109+0000","objects_scrubbed":2,"log_size":3,"log_dups_size":96,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:54:05.896060+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":240,"num_write":135,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":16,"stat_sum":{"num_bytes":25949110296,"num_objects":395953,"num_object_clones":0,"num_object_copies":1187859,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":395953,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":715187,"num_write_kb":35556417,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":77836595200,"data_stored":77836582984,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":48,"ondisk_log_size":48,"up":48,"acting":48,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":240,"num_write":135,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1682,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":3,"up_from":25,"seq":107374182533,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":24142456,"kb_used_data":22434340,"kb_used_omap":20,"kb_used_meta":1708075,"kb_avail":709860744,"statfs":{"total":751619276800,"available":726897401856,"internally_reserved":0,"allocated":22972764160,"data_stored":22972319446,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":21479,"internal_metadata":1749068825},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":19,"seq":81604378759,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":19273720,"kb_used_data":17757412,"kb_used_omap":20,"kb_used_meta":1516139,"kb_avail":714729480,"statfs":{"total":751619276800,"available":731882987520,"internally_reserved":0,"allocated":18183589888,"data_stored":18183145174,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":21284,"internal_metadata":1552526556},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542280,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":17879856,"kb_used_data":16085276,"kb_used_omap":23,"kb_used_meta":1794536,"kb_avail":716123344,"statfs":{"total":751619276800,"available":733310304256,"internally_reserved":0,"allocated":16471322624,"data_stored":16470885534,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":24494,"internal_metadata":1837604946},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705803,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":21730488,"kb_used_data":20712484,"kb_used_omap":18,"kb_used_meta":1017965,"kb_avail":712272712,"statfs":{"total":751619276800,"available":729367257088,"internally_reserved":0,"allocated":21209583616,"data_stored":21209138902,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":18848,"internal_metadata":1042396768},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[7,1],"upper_bound":4},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1682,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":21077430272,"data_stored":21077426200,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16189030400,"data_stored":16189030400,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":17863806976,"data_stored":17863802904,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":22706327552,"data_stored":22706323480,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-24T07:53:48.567 INFO:tasks.cephadm.ceph_manager.ceph:increase pool size by 10 2026-02-24T07:53:48.567 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph osd pool set .mgr pg_num 361 2026-02-24T07:53:48.707 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:53:49.085 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:53:48 trial035 ceph-mon[39284]: pgmap v386: 17 pgs: 17 active+clean; 24 GiB data, 79 GiB used, 2.7 TiB / 2.7 TiB avail; 0 B/s wr, 9.21k op/s 2026-02-24T07:53:49.123 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:53:49.083+0000 7f6e43df0640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:53:49.123 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:53:49.083+0000 7f6e43df0640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:53:49.226 INFO:teuthology.orchestra.run.trial035.stderr:Error EPERM: crimson_allow_pg_split is false; pg_num increase denied for crimson pool 2026-02-24T07:53:49.574 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2026-02-24T07:53:49.931 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-24T07:53:49.934 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:53:50.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:53:49 trial035 ceph-mon[39284]: from='client.15316 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2026-02-24T07:53:50.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:53:49 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/2374155925' entity='client.admin' cmd={"prefix": "osd pool set", "pool": ".mgr", "var": "pg_num", "val": "361"} : dispatch 2026-02-24T07:53:50.343 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:53:50.340+0000 7f93a1d91640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:53:50.343 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:53:50.340+0000 7f93a1d91640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:53:50.355 INFO:teuthology.orchestra.run.trial035.stdout:osd.0: {} 2026-02-24T07:53:50.356 INFO:teuthology.orchestra.run.trial035.stderr:osd.0: osd_enable_op_tracker = '' 2026-02-24T07:53:50.370 INFO:teuthology.orchestra.run.trial035.stdout:osd.1: {} 2026-02-24T07:53:50.370 INFO:teuthology.orchestra.run.trial035.stderr:osd.1: osd_enable_op_tracker = '' 2026-02-24T07:53:50.382 INFO:teuthology.orchestra.run.trial035.stdout:osd.2: {} 2026-02-24T07:53:50.382 INFO:teuthology.orchestra.run.trial035.stderr:osd.2: osd_enable_op_tracker = '' 2026-02-24T07:53:50.394 INFO:teuthology.orchestra.run.trial035.stdout:osd.3: {} 2026-02-24T07:53:50.395 INFO:teuthology.orchestra.run.trial035.stderr:osd.3: osd_enable_op_tracker = '' 2026-02-24T07:53:51.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:53:50 trial035 ceph-mon[39284]: pgmap v387: 17 pgs: 17 active+clean; 24 GiB data, 79 GiB used, 2.7 TiB / 2.7 TiB avail; 0 B/s wr, 8.12k op/s 2026-02-24T07:53:51.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:53:50 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/2834368803' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2026-02-24T07:53:52.494 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2026-02-24T07:53:52.635 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:53:52.757 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:53:52.755+0000 7f1e47c67640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:53:52.758 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:53:52.755+0000 7f1e47c67640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:53:52.771 INFO:teuthology.orchestra.run.trial035.stdout:osd.0: {} 2026-02-24T07:53:52.771 INFO:teuthology.orchestra.run.trial035.stderr:osd.0: osd_enable_op_tracker = '' 2026-02-24T07:53:52.787 INFO:teuthology.orchestra.run.trial035.stdout:osd.1: {} 2026-02-24T07:53:52.788 INFO:teuthology.orchestra.run.trial035.stderr:osd.1: osd_enable_op_tracker = '' 2026-02-24T07:53:52.800 INFO:teuthology.orchestra.run.trial035.stdout:osd.2: {} 2026-02-24T07:53:52.800 INFO:teuthology.orchestra.run.trial035.stderr:osd.2: osd_enable_op_tracker = '' 2026-02-24T07:53:52.813 INFO:teuthology.orchestra.run.trial035.stdout:osd.3: {} 2026-02-24T07:53:52.813 INFO:teuthology.orchestra.run.trial035.stderr:osd.3: osd_enable_op_tracker = '' 2026-02-24T07:53:53.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:53:52 trial035 ceph-mon[39284]: pgmap v388: 17 pgs: 17 active+clean; 22 GiB data, 73 GiB used, 2.7 TiB / 2.7 TiB avail; 0 B/s wr, 9.60k op/s 2026-02-24T07:53:53.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:53:52 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/1175778940' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2026-02-24T07:53:54.856 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2026-02-24T07:53:54.933 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2026-02-24T07:53:54.933 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2026-02-24T07:53:54.933 INFO:tasks.thrashosds.thrasher:Growing pool unique_pool_3 2026-02-24T07:53:54.934 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph pg dump --format=json 2026-02-24T07:53:54.999 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:53:55.070 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:53:55.116 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:53:55.113+0000 7ff0b54f8640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:53:55.116 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:53:55.114+0000 7ff0b54f8640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:53:55.132 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:53:54 trial035 ceph-mon[39284]: pgmap v389: 17 pgs: 17 active+clean; 22 GiB data, 73 GiB used, 2.7 TiB / 2.7 TiB avail; 0 B/s wr, 6.14k op/s 2026-02-24T07:53:55.133 INFO:teuthology.orchestra.run.trial035.stdout:osd.0: {} 2026-02-24T07:53:55.133 INFO:teuthology.orchestra.run.trial035.stderr:osd.0: osd_enable_op_tracker = '' 2026-02-24T07:53:55.147 INFO:teuthology.orchestra.run.trial035.stdout:osd.1: {} 2026-02-24T07:53:55.147 INFO:teuthology.orchestra.run.trial035.stderr:osd.1: osd_enable_op_tracker = '' 2026-02-24T07:53:55.160 INFO:teuthology.orchestra.run.trial035.stdout:osd.2: {} 2026-02-24T07:53:55.160 INFO:teuthology.orchestra.run.trial035.stderr:osd.2: osd_enable_op_tracker = '' 2026-02-24T07:53:55.172 INFO:teuthology.orchestra.run.trial035.stdout:osd.3: {} 2026-02-24T07:53:55.172 INFO:teuthology.orchestra.run.trial035.stderr:osd.3: osd_enable_op_tracker = '' 2026-02-24T07:53:55.185 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:53:55.184+0000 7f186a79e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:53:55.186 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:53:55.184+0000 7f186a79e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:53:55.320 INFO:teuthology.orchestra.run.trial035.stdout: 2026-02-24T07:53:55.320 INFO:teuthology.orchestra.run.trial035.stderr:dumped all 2026-02-24T07:53:55.361 INFO:teuthology.orchestra.run.trial035.stdout:{"pg_ready":true,"pg_map":{"version":389,"stamp":"2026-02-24T07:53:53.746081+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":23835312696,"num_objects":363692,"num_object_clones":0,"num_object_copies":1091076,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":363692,"num_whiteouts":0,"num_read":137,"num_read_kb":241,"num_write":747585,"num_write_kb":35558567,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":53,"ondisk_log_size":53,"up":51,"acting":51,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":51,"num_osds":4,"num_per_pool_osds":4,"num_per_pool_omap_osds":4,"kb":2936012800,"kb_used":76711192,"kb_used_data":70493640,"kb_used_omap":84,"kb_used_meta":6217195,"kb_avail":2859301608,"statfs":{"total":3006477107200,"available":2927924846592,"internally_reserved":0,"allocated":72185487360,"data_stored":72183716128,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":86105,"internal_metadata":6366408615},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-4829741056,"num_objects":-73696,"num_object_clones":0,"num_object_copies":-221088,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-73696,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":73696,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001576"},"pg_stats":[{"pgid":"5.f","version":"88'46497","reported_seq":46799,"reported_epoch":88,"state":"active+clean","last_fresh":"2026-02-24T07:53:50.751051+0000","last_change":"2026-02-24T07:53:01.242353+0000","last_active":"2026-02-24T07:53:50.751051+0000","last_peered":"2026-02-24T07:53:50.751051+0000","last_clean":"2026-02-24T07:53:50.751051+0000","last_became_active":"2026-02-24T07:51:53.900431+0000","last_became_peered":"2026-02-24T07:51:53.900431+0000","last_unstale":"2026-02-24T07:53:50.751051+0000","last_undegraded":"2026-02-24T07:53:50.751051+0000","last_fullsized":"2026-02-24T07:53:50.751051+0000","mapping_epoch":74,"log_start":"88'46494","ondisk_log_start":"88'46494","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"84'26061","last_scrub_stamp":"2026-02-24T07:53:01.242306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_clean_scrub_stamp":"2026-02-24T07:53:01.242306+0000","objects_scrubbed":25674,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:54:22.699112+0000","scrub_duration":2129,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1496514560,"num_objects":22835,"num_object_clones":0,"num_object_copies":68505,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22835,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":46497,"num_write_kb":2218624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.e","version":"88'46473","reported_seq":46762,"reported_epoch":88,"state":"active+clean","last_fresh":"2026-02-24T07:53:50.766300+0000","last_change":"2026-02-24T07:52:56.767335+0000","last_active":"2026-02-24T07:53:50.766300+0000","last_peered":"2026-02-24T07:53:50.766300+0000","last_clean":"2026-02-24T07:53:50.766300+0000","last_became_active":"2026-02-24T07:51:53.899294+0000","last_became_peered":"2026-02-24T07:51:53.899294+0000","last_unstale":"2026-02-24T07:53:50.766300+0000","last_undegraded":"2026-02-24T07:53:50.766300+0000","last_fullsized":"2026-02-24T07:53:50.766300+0000","mapping_epoch":74,"log_start":"88'46470","ondisk_log_start":"88'46470","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"83'24557","last_scrub_stamp":"2026-02-24T07:52:56.767267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_clean_scrub_stamp":"2026-02-24T07:52:56.767267+0000","objects_scrubbed":24118,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:54:24.913755+0000","scrub_duration":2806,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1478557696,"num_objects":22561,"num_object_clones":0,"num_object_copies":67683,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22561,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":46473,"num_write_kb":2209088,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"88'46685","reported_seq":47055,"reported_epoch":88,"state":"active+clean","last_fresh":"2026-02-24T07:53:50.781355+0000","last_change":"2026-02-24T07:53:32.480551+0000","last_active":"2026-02-24T07:53:50.781355+0000","last_peered":"2026-02-24T07:53:50.781355+0000","last_clean":"2026-02-24T07:53:50.781355+0000","last_became_active":"2026-02-24T07:51:53.899772+0000","last_became_peered":"2026-02-24T07:51:53.899772+0000","last_unstale":"2026-02-24T07:53:50.781355+0000","last_undegraded":"2026-02-24T07:53:50.781355+0000","last_fullsized":"2026-02-24T07:53:50.781355+0000","mapping_epoch":74,"log_start":"88'46682","ondisk_log_start":"88'46682","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"84'37904","last_scrub_stamp":"2026-02-24T07:53:32.480499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_clean_scrub_stamp":"2026-02-24T07:53:32.480499+0000","objects_scrubbed":32149,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:54:51.939533+0000","scrub_duration":2742,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1492647960,"num_objects":22777,"num_object_clones":0,"num_object_copies":68331,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22777,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":46685,"num_write_kb":2222721,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,3],"acting":[0,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.c","version":"88'46726","reported_seq":47089,"reported_epoch":88,"state":"active+clean","last_fresh":"2026-02-24T07:53:50.859186+0000","last_change":"2026-02-24T07:53:17.083868+0000","last_active":"2026-02-24T07:53:50.859186+0000","last_peered":"2026-02-24T07:53:50.859186+0000","last_clean":"2026-02-24T07:53:50.859186+0000","last_became_active":"2026-02-24T07:51:53.898285+0000","last_became_peered":"2026-02-24T07:51:53.898285+0000","last_unstale":"2026-02-24T07:53:50.859186+0000","last_undegraded":"2026-02-24T07:53:50.859186+0000","last_fullsized":"2026-02-24T07:53:50.859186+0000","mapping_epoch":74,"log_start":"88'46722","ondisk_log_start":"88'46722","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"84'32067","last_scrub_stamp":"2026-02-24T07:53:17.083821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_clean_scrub_stamp":"2026-02-24T07:53:17.083821+0000","objects_scrubbed":31581,"log_size":4,"log_dups_size":3000,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:54:37.993683+0000","scrub_duration":2856,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1488977920,"num_objects":22720,"num_object_clones":0,"num_object_copies":68160,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22720,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":46726,"num_write_kb":2222272,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.b","version":"88'46824","reported_seq":47203,"reported_epoch":88,"state":"active+clean","last_fresh":"2026-02-24T07:53:50.779224+0000","last_change":"2026-02-24T07:53:30.225127+0000","last_active":"2026-02-24T07:53:50.779224+0000","last_peered":"2026-02-24T07:53:50.779224+0000","last_clean":"2026-02-24T07:53:50.779224+0000","last_became_active":"2026-02-24T07:51:53.899901+0000","last_became_peered":"2026-02-24T07:51:53.899901+0000","last_unstale":"2026-02-24T07:53:50.779224+0000","last_undegraded":"2026-02-24T07:53:50.779224+0000","last_fullsized":"2026-02-24T07:53:50.779224+0000","mapping_epoch":74,"log_start":"88'46821","ondisk_log_start":"88'46821","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"84'36879","last_scrub_stamp":"2026-02-24T07:53:30.225075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_clean_scrub_stamp":"2026-02-24T07:53:30.225075+0000","objects_scrubbed":33279,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:54:35.718496+0000","scrub_duration":3524,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1485963264,"num_objects":22674,"num_object_clones":0,"num_object_copies":68022,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22674,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":46824,"num_write_kb":2223936,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"88'46639","reported_seq":46990,"reported_epoch":88,"state":"active+clean","last_fresh":"2026-02-24T07:53:50.781732+0000","last_change":"2026-02-24T07:53:14.212339+0000","last_active":"2026-02-24T07:53:50.781732+0000","last_peered":"2026-02-24T07:53:50.781732+0000","last_clean":"2026-02-24T07:53:50.781732+0000","last_became_active":"2026-02-24T07:51:53.899549+0000","last_became_peered":"2026-02-24T07:51:53.899549+0000","last_unstale":"2026-02-24T07:53:50.781732+0000","last_undegraded":"2026-02-24T07:53:50.781732+0000","last_fullsized":"2026-02-24T07:53:50.781732+0000","mapping_epoch":74,"log_start":"88'46636","ondisk_log_start":"88'46636","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"84'30986","last_scrub_stamp":"2026-02-24T07:53:14.212265+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_clean_scrub_stamp":"2026-02-24T07:53:14.212265+0000","objects_scrubbed":30490,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:54:35.341423+0000","scrub_duration":3381,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1494548480,"num_objects":22805,"num_object_clones":0,"num_object_copies":68415,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22805,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":46639,"num_write_kb":2222208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,1],"acting":[0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.9","version":"88'47321","reported_seq":47704,"reported_epoch":88,"state":"active+clean","last_fresh":"2026-02-24T07:53:50.765976+0000","last_change":"2026-02-24T07:53:22.113835+0000","last_active":"2026-02-24T07:53:50.765976+0000","last_peered":"2026-02-24T07:53:50.765976+0000","last_clean":"2026-02-24T07:53:50.765976+0000","last_became_active":"2026-02-24T07:51:53.901610+0000","last_became_peered":"2026-02-24T07:51:53.901610+0000","last_unstale":"2026-02-24T07:53:50.765976+0000","last_undegraded":"2026-02-24T07:53:50.765976+0000","last_fullsized":"2026-02-24T07:53:50.765976+0000","mapping_epoch":74,"log_start":"88'47318","ondisk_log_start":"88'47318","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"84'34029","last_scrub_stamp":"2026-02-24T07:53:22.113774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_clean_scrub_stamp":"2026-02-24T07:53:22.113774+0000","objects_scrubbed":33520,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:54:41.903404+0000","scrub_duration":3005,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1506738176,"num_objects":22991,"num_object_clones":0,"num_object_copies":68973,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22991,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":47321,"num_write_kb":2249984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.8","version":"88'46885","reported_seq":47186,"reported_epoch":88,"state":"active+clean","last_fresh":"2026-02-24T07:53:50.779659+0000","last_change":"2026-02-24T07:53:00.549866+0000","last_active":"2026-02-24T07:53:50.779659+0000","last_peered":"2026-02-24T07:53:50.779659+0000","last_clean":"2026-02-24T07:53:50.779659+0000","last_became_active":"2026-02-24T07:51:53.900207+0000","last_became_peered":"2026-02-24T07:51:53.900207+0000","last_unstale":"2026-02-24T07:53:50.779659+0000","last_undegraded":"2026-02-24T07:53:50.779659+0000","last_fullsized":"2026-02-24T07:53:50.779659+0000","mapping_epoch":74,"log_start":"88'46882","ondisk_log_start":"88'46882","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"84'25943","last_scrub_stamp":"2026-02-24T07:53:00.549789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_clean_scrub_stamp":"2026-02-24T07:53:00.549789+0000","objects_scrubbed":25486,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:54:18.896412+0000","scrub_duration":2647,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1503199232,"num_objects":22937,"num_object_clones":0,"num_object_copies":68811,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22937,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":46885,"num_write_kb":2234304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.7","version":"88'46318","reported_seq":46618,"reported_epoch":88,"state":"active+clean","last_fresh":"2026-02-24T07:53:50.768869+0000","last_change":"2026-02-24T07:53:00.188453+0000","last_active":"2026-02-24T07:53:50.768869+0000","last_peered":"2026-02-24T07:53:50.768869+0000","last_clean":"2026-02-24T07:53:50.768869+0000","last_became_active":"2026-02-24T07:51:53.899472+0000","last_became_peered":"2026-02-24T07:51:53.899472+0000","last_unstale":"2026-02-24T07:53:50.768869+0000","last_undegraded":"2026-02-24T07:53:50.768869+0000","last_fullsized":"2026-02-24T07:53:50.768869+0000","mapping_epoch":74,"log_start":"88'46315","ondisk_log_start":"88'46315","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"84'25630","last_scrub_stamp":"2026-02-24T07:53:00.188390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_clean_scrub_stamp":"2026-02-24T07:53:00.188390+0000","objects_scrubbed":25246,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:54:07.242502+0000","scrub_duration":2262,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1473249280,"num_objects":22480,"num_object_clones":0,"num_object_copies":67440,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22480,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":46318,"num_write_kb":2201536,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.6","version":"88'46270","reported_seq":46611,"reported_epoch":88,"state":"active+clean","last_fresh":"2026-02-24T07:53:50.856408+0000","last_change":"2026-02-24T07:53:10.804133+0000","last_active":"2026-02-24T07:53:50.856408+0000","last_peered":"2026-02-24T07:53:50.856408+0000","last_clean":"2026-02-24T07:53:50.856408+0000","last_became_active":"2026-02-24T07:51:53.898308+0000","last_became_peered":"2026-02-24T07:51:53.898308+0000","last_unstale":"2026-02-24T07:53:50.856408+0000","last_undegraded":"2026-02-24T07:53:50.856408+0000","last_fullsized":"2026-02-24T07:53:50.856408+0000","mapping_epoch":74,"log_start":"88'46267","ondisk_log_start":"88'46267","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"84'29829","last_scrub_stamp":"2026-02-24T07:53:10.804104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_clean_scrub_stamp":"2026-02-24T07:53:10.804104+0000","objects_scrubbed":29351,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:54:13.621421+0000","scrub_duration":2501,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1486094336,"num_objects":22676,"num_object_clones":0,"num_object_copies":68028,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22676,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":46270,"num_write_kb":2206272,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.5","version":"88'46813","reported_seq":47206,"reported_epoch":88,"state":"active+clean","last_fresh":"2026-02-24T07:53:50.782440+0000","last_change":"2026-02-24T07:53:26.781324+0000","last_active":"2026-02-24T07:53:50.782440+0000","last_peered":"2026-02-24T07:53:50.782440+0000","last_clean":"2026-02-24T07:53:50.782440+0000","last_became_active":"2026-02-24T07:51:53.899749+0000","last_became_peered":"2026-02-24T07:51:53.899749+0000","last_unstale":"2026-02-24T07:53:50.782440+0000","last_undegraded":"2026-02-24T07:53:50.782440+0000","last_fullsized":"2026-02-24T07:53:50.782440+0000","mapping_epoch":74,"log_start":"88'46810","ondisk_log_start":"88'46810","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"84'35538","last_scrub_stamp":"2026-02-24T07:53:26.781276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_clean_scrub_stamp":"2026-02-24T07:53:26.781276+0000","objects_scrubbed":34358,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:54:37.440161+0000","scrub_duration":2982,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1480785920,"num_objects":22595,"num_object_clones":0,"num_object_copies":67785,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22595,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":46813,"num_write_kb":2221056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"88'46681","reported_seq":47002,"reported_epoch":88,"state":"active+clean","last_fresh":"2026-02-24T07:53:50.779080+0000","last_change":"2026-02-24T07:53:05.312364+0000","last_active":"2026-02-24T07:53:50.779080+0000","last_peered":"2026-02-24T07:53:50.779080+0000","last_clean":"2026-02-24T07:53:50.779080+0000","last_became_active":"2026-02-24T07:51:53.900042+0000","last_became_peered":"2026-02-24T07:51:53.900042+0000","last_unstale":"2026-02-24T07:53:50.779080+0000","last_undegraded":"2026-02-24T07:53:50.779080+0000","last_fullsized":"2026-02-24T07:53:50.779080+0000","mapping_epoch":74,"log_start":"88'46678","ondisk_log_start":"88'46678","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"84'27738","last_scrub_stamp":"2026-02-24T07:53:05.312312+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_clean_scrub_stamp":"2026-02-24T07:53:05.312312+0000","objects_scrubbed":27245,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:54:32.166518+0000","scrub_duration":2472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1493368832,"num_objects":22787,"num_object_clones":0,"num_object_copies":68361,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22787,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":46681,"num_write_kb":2222976,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.2","version":"88'47294","reported_seq":47662,"reported_epoch":88,"state":"active+clean","last_fresh":"2026-02-24T07:53:50.768770+0000","last_change":"2026-02-24T07:53:17.780799+0000","last_active":"2026-02-24T07:53:50.768770+0000","last_peered":"2026-02-24T07:53:50.768770+0000","last_clean":"2026-02-24T07:53:50.768770+0000","last_became_active":"2026-02-24T07:51:53.900096+0000","last_became_peered":"2026-02-24T07:51:53.900096+0000","last_unstale":"2026-02-24T07:53:50.768770+0000","last_undegraded":"2026-02-24T07:53:50.768770+0000","last_fullsized":"2026-02-24T07:53:50.768770+0000","mapping_epoch":74,"log_start":"88'47291","ondisk_log_start":"88'47291","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"84'32548","last_scrub_stamp":"2026-02-24T07:53:17.780705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_clean_scrub_stamp":"2026-02-24T07:53:17.780705+0000","objects_scrubbed":32030,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:54:35.344780+0000","scrub_duration":2741,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1496055808,"num_objects":22828,"num_object_clones":0,"num_object_copies":68484,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22828,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":47294,"num_write_kb":2243904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"88'46200","reported_seq":46549,"reported_epoch":88,"state":"active+clean","last_fresh":"2026-02-24T07:53:50.764546+0000","last_change":"2026-02-24T07:53:14.051194+0000","last_active":"2026-02-24T07:53:50.764546+0000","last_peered":"2026-02-24T07:53:50.764546+0000","last_clean":"2026-02-24T07:53:50.764546+0000","last_became_active":"2026-02-24T07:51:53.901027+0000","last_became_peered":"2026-02-24T07:51:53.901027+0000","last_unstale":"2026-02-24T07:53:50.764546+0000","last_undegraded":"2026-02-24T07:53:50.764546+0000","last_fullsized":"2026-02-24T07:53:50.764546+0000","mapping_epoch":74,"log_start":"88'46197","ondisk_log_start":"88'46197","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"84'30565","last_scrub_stamp":"2026-02-24T07:53:14.051154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_clean_scrub_stamp":"2026-02-24T07:53:14.051154+0000","objects_scrubbed":30140,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:54:38.448076+0000","scrub_duration":3147,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1479540736,"num_objects":22576,"num_object_clones":0,"num_object_copies":67728,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22576,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":46200,"num_write_kb":2200832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.0","version":"88'47262","reported_seq":47635,"reported_epoch":88,"state":"active+clean","last_fresh":"2026-02-24T07:53:50.784548+0000","last_change":"2026-02-24T07:53:20.014077+0000","last_active":"2026-02-24T07:53:50.784548+0000","last_peered":"2026-02-24T07:53:50.784548+0000","last_clean":"2026-02-24T07:53:50.784548+0000","last_became_active":"2026-02-24T07:51:53.898551+0000","last_became_peered":"2026-02-24T07:51:53.898551+0000","last_unstale":"2026-02-24T07:53:50.784548+0000","last_undegraded":"2026-02-24T07:53:50.784548+0000","last_fullsized":"2026-02-24T07:53:50.784548+0000","mapping_epoch":74,"log_start":"88'47258","ondisk_log_start":"88'47258","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"84'33261","last_scrub_stamp":"2026-02-24T07:53:20.014009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_clean_scrub_stamp":"2026-02-24T07:53:20.014009+0000","objects_scrubbed":32679,"log_size":4,"log_dups_size":3000,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:54:37.178341+0000","scrub_duration":3163,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1491599360,"num_objects":22760,"num_object_clones":0,"num_object_copies":68280,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22760,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":47262,"num_write_kb":2240704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.4","version":"88'46562","reported_seq":46907,"reported_epoch":88,"state":"active+clean","last_fresh":"2026-02-24T07:53:50.754121+0000","last_change":"2026-02-24T07:53:11.592191+0000","last_active":"2026-02-24T07:53:50.754121+0000","last_peered":"2026-02-24T07:53:50.754121+0000","last_clean":"2026-02-24T07:53:50.754121+0000","last_became_active":"2026-02-24T07:51:53.899675+0000","last_became_peered":"2026-02-24T07:51:53.899675+0000","last_unstale":"2026-02-24T07:53:50.754121+0000","last_undegraded":"2026-02-24T07:53:50.754121+0000","last_fullsized":"2026-02-24T07:53:50.754121+0000","mapping_epoch":74,"log_start":"88'46559","ondisk_log_start":"88'46559","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"84'30127","last_scrub_stamp":"2026-02-24T07:53:11.592155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_clean_scrub_stamp":"2026-02-24T07:53:11.592155+0000","objects_scrubbed":29709,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:54:23.794472+0000","scrub_duration":2432,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1486880768,"num_objects":22688,"num_object_clones":0,"num_object_copies":68064,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22688,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":46562,"num_write_kb":2216000,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"29'99","reported_seq":365,"reported_epoch":88,"state":"active+clean","last_fresh":"2026-02-24T07:53:39.925612+0000","last_change":"2026-02-24T07:52:56.946143+0000","last_active":"2026-02-24T07:53:39.925612+0000","last_peered":"2026-02-24T07:53:39.925612+0000","last_clean":"2026-02-24T07:53:39.925612+0000","last_became_active":"2026-02-24T07:42:54.186719+0000","last_became_peered":"2026-02-24T07:42:54.186719+0000","last_unstale":"2026-02-24T07:53:39.925612+0000","last_undegraded":"2026-02-24T07:53:39.925612+0000","last_fullsized":"2026-02-24T07:53:39.925612+0000","mapping_epoch":27,"log_start":"29'96","ondisk_log_start":"29'96","created":20,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'99","last_scrub_stamp":"2026-02-24T07:52:56.946109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:42:43.640776+0000","last_clean_scrub_stamp":"2026-02-24T07:52:56.946109+0000","objects_scrubbed":2,"log_size":3,"log_dups_size":96,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:54:05.896060+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":240,"num_write":135,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":16,"stat_sum":{"num_bytes":23834722328,"num_objects":363690,"num_object_clones":0,"num_object_copies":1091070,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":363690,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":747450,"num_write_kb":35556417,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":71501099008,"data_stored":71501086792,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":50,"ondisk_log_size":50,"up":48,"acting":48,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":240,"num_write":135,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1682,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":3,"up_from":25,"seq":107374182534,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":22296696,"kb_used_data":20534436,"kb_used_omap":20,"kb_used_meta":1762219,"kb_avail":711706504,"statfs":{"total":751619276800,"available":728787460096,"internally_reserved":0,"allocated":21027262464,"data_stored":21026817750,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":21479,"internal_metadata":1804512281},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":19,"seq":81604378760,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":17800504,"kb_used_data":16248036,"kb_used_omap":20,"kb_used_meta":1552235,"kb_avail":716202696,"statfs":{"total":751619276800,"available":733391560704,"internally_reserved":0,"allocated":16637988864,"data_stored":16637544150,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":21284,"internal_metadata":1589488860},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542281,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":16549552,"kb_used_data":14718876,"kb_used_omap":23,"kb_used_meta":1830632,"kb_avail":717453648,"statfs":{"total":751619276800,"available":734672535552,"internally_reserved":0,"allocated":15072129024,"data_stored":15071691934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":24494,"internal_metadata":1874567250},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705804,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":20064440,"kb_used_data":18992292,"kb_used_omap":18,"kb_used_meta":1072109,"kb_avail":713938760,"statfs":{"total":751619276800,"available":731073290240,"internally_reserved":0,"allocated":19448107008,"data_stored":19447662294,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":18848,"internal_metadata":1097840224},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1682,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":19355078656,"data_stored":19355074584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":14870708224,"data_stored":14870708224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16416444416,"data_stored":16416440344,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":20858867712,"data_stored":20858863640,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-24T07:53:55.363 INFO:tasks.cephadm.ceph_manager.ceph:increase pool size by 10 2026-02-24T07:53:55.364 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph osd pool set unique_pool_3 pg_num 56 2026-02-24T07:53:55.494 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:53:55.901 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:53:55.878+0000 7f97e70d7640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:53:55.902 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:53:55.879+0000 7f97e70d7640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:53:56.022 INFO:teuthology.orchestra.run.trial035.stderr:Error EPERM: crimson_allow_pg_split is false; pg_num increase denied for crimson pool 2026-02-24T07:53:56.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:53:55 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/75241074' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2026-02-24T07:53:56.630 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-24T07:53:57.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:53:56 trial035 ceph-mon[39284]: from='client.15326 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2026-02-24T07:53:57.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:53:56 trial035 ceph-mon[39284]: pgmap v390: 17 pgs: 17 active+clean; 22 GiB data, 73 GiB used, 2.7 TiB / 2.7 TiB avail; 0 B/s wr, 6.14k op/s 2026-02-24T07:53:57.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:53:56 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/1464369409' entity='client.admin' cmd={"prefix": "osd pool set", "pool": "unique_pool_3", "var": "pg_num", "val": "56"} : dispatch 2026-02-24T07:53:57.204 INFO:journalctl@ceph.mgr.x.trial035.stdout:Feb 24 07:53:56 trial035 ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-mgr-x[39621]: ::ffff:10.20.193.91 - - [24/Feb/2026:07:53:56] "GET /metrics HTTP/1.1" 200 4230 "" "Prometheus/3.6.0" 2026-02-24T07:53:57.216 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2026-02-24T07:53:57.354 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:53:57.356 INFO:journalctl@ceph.prometheus.a.trial091.stdout:Feb 24 07:53:56 trial091 ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-prometheus-a[66007]: time=2026-02-24T07:53:56.973Z level=ERROR source=scrape.go:1630 msg="Failed to determine correct type of scrape target." component="scrape manager" scrape_pool=ceph target=http://trial035:9283/metrics content_type="text/html;charset=utf-8" fallback_media_type="" err="received unsupported Content-Type \"text/html;charset=utf-8\" and no fallback_scrape_protocol specified for target" 2026-02-24T07:53:57.796 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:53:57.751+0000 7fc592e76640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:53:57.796 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:53:57.751+0000 7fc592e76640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:53:57.796 INFO:teuthology.orchestra.run.trial035.stderr:osd.0: osd_enable_op_tracker = '' 2026-02-24T07:53:57.796 INFO:teuthology.orchestra.run.trial035.stderr:osd.1: osd_enable_op_tracker = '' 2026-02-24T07:53:57.796 INFO:teuthology.orchestra.run.trial035.stdout:osd.0: {} 2026-02-24T07:53:57.797 INFO:teuthology.orchestra.run.trial035.stdout:osd.1: {} 2026-02-24T07:53:57.806 INFO:teuthology.orchestra.run.trial035.stdout:osd.2: {} 2026-02-24T07:53:57.806 INFO:teuthology.orchestra.run.trial035.stderr:osd.2: osd_enable_op_tracker = '' 2026-02-24T07:53:57.818 INFO:teuthology.orchestra.run.trial035.stdout:osd.3: {} 2026-02-24T07:53:57.819 INFO:teuthology.orchestra.run.trial035.stderr:osd.3: osd_enable_op_tracker = '' 2026-02-24T07:53:58.085 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:53:57 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/2716633767' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2026-02-24T07:53:59.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:53:58 trial035 ceph-mon[39284]: pgmap v391: 17 pgs: 17 active+clean; 20 GiB data, 66 GiB used, 2.7 TiB / 2.7 TiB avail; 0 B/s wr, 9.47k op/s 2026-02-24T07:54:00.133 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2026-02-24T07:54:00.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:53:59 trial035 ceph-mon[39284]: from='mgr.14284 10.20.193.35:0/3681373222' entity='mgr.x' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2026-02-24T07:54:00.270 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:54:00.388 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:54:00.386+0000 7fbcaecc1640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:54:00.388 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:54:00.387+0000 7fbcaecc1640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:54:00.402 INFO:teuthology.orchestra.run.trial035.stdout:osd.0: {} 2026-02-24T07:54:00.403 INFO:teuthology.orchestra.run.trial035.stderr:osd.0: osd_enable_op_tracker = '' 2026-02-24T07:54:00.415 INFO:teuthology.orchestra.run.trial035.stdout:osd.1: {} 2026-02-24T07:54:00.416 INFO:teuthology.orchestra.run.trial035.stderr:osd.1: osd_enable_op_tracker = '' 2026-02-24T07:54:00.429 INFO:teuthology.orchestra.run.trial035.stdout:osd.2: {} 2026-02-24T07:54:00.429 INFO:teuthology.orchestra.run.trial035.stderr:osd.2: osd_enable_op_tracker = '' 2026-02-24T07:54:00.444 INFO:teuthology.orchestra.run.trial035.stdout:osd.3: {} 2026-02-24T07:54:00.444 INFO:teuthology.orchestra.run.trial035.stderr:osd.3: osd_enable_op_tracker = '' 2026-02-24T07:54:01.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:54:00 trial035 ceph-mon[39284]: pgmap v392: 17 pgs: 17 active+clean; 20 GiB data, 66 GiB used, 2.7 TiB / 2.7 TiB avail; 0 B/s wr, 6.02k op/s 2026-02-24T07:54:01.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:54:00 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/1323837297' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2026-02-24T07:54:01.631 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2026-02-24T07:54:01.632 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2026-02-24T07:54:01.632 INFO:tasks.thrashosds.thrasher:trim_stale_osdmaps 2026-02-24T07:54:01.632 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph tell osd.0 'trim stale osdmaps' 2026-02-24T07:54:01.768 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:54:01.894 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:54:01.893+0000 7f122ef04640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:54:01.895 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:54:01.893+0000 7f122ef04640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:54:01.915 INFO:teuthology.orchestra.run.trial035.stderr:no valid command found; 10 closest matches: 2026-02-24T07:54:01.916 INFO:teuthology.orchestra.run.trial035.stderr:0 2026-02-24T07:54:01.916 INFO:teuthology.orchestra.run.trial035.stderr:1 2026-02-24T07:54:01.916 INFO:teuthology.orchestra.run.trial035.stderr:2 2026-02-24T07:54:01.916 INFO:teuthology.orchestra.run.trial035.stderr:abort 2026-02-24T07:54:01.916 INFO:teuthology.orchestra.run.trial035.stderr:assert 2026-02-24T07:54:01.916 INFO:teuthology.orchestra.run.trial035.stderr:bench [] [] [] [] 2026-02-24T07:54:01.916 INFO:teuthology.orchestra.run.trial035.stderr:bluefs debug_inject_read_zeros 2026-02-24T07:54:01.916 INFO:teuthology.orchestra.run.trial035.stderr:bluefs files list 2026-02-24T07:54:01.916 INFO:teuthology.orchestra.run.trial035.stderr:bluefs stats 2026-02-24T07:54:01.917 INFO:teuthology.orchestra.run.trial035.stderr:bluestore allocator dump block 2026-02-24T07:54:01.917 INFO:teuthology.orchestra.run.trial035.stderr:Error EINVAL: invalid command 2026-02-24T07:54:02.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:54:01 trial035 ceph-mon[39284]: pgmap v393: 17 pgs: 17 active+clean; 18 GiB data, 60 GiB used, 2.7 TiB / 2.7 TiB avail; 0 B/s wr, 9.06k op/s 2026-02-24T07:54:02.486 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2026-02-24T07:54:02.572 DEBUG:teuthology.orchestra.run:got remote process result: 22 2026-02-24T07:54:02.573 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2026-02-24T07:54:02.573 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph tell osd.1 'trim stale osdmaps' 2026-02-24T07:54:02.617 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:54:02.815 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:54:03.083 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:54:03.001+0000 7fcd76dbd640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:54:03.083 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:54:03.001+0000 7fcd76dbd640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:54:03.083 INFO:teuthology.orchestra.run.trial035.stderr:osd.0: osd_enable_op_tracker = '' 2026-02-24T07:54:03.083 INFO:teuthology.orchestra.run.trial035.stderr:osd.1: osd_enable_op_tracker = '' 2026-02-24T07:54:03.084 INFO:teuthology.orchestra.run.trial035.stdout:osd.0: {} 2026-02-24T07:54:03.084 INFO:teuthology.orchestra.run.trial035.stdout:osd.1: {} 2026-02-24T07:54:03.084 INFO:teuthology.orchestra.run.trial035.stdout:osd.2: {} 2026-02-24T07:54:03.084 INFO:teuthology.orchestra.run.trial035.stdout:osd.3: {} 2026-02-24T07:54:03.084 INFO:teuthology.orchestra.run.trial035.stderr:osd.2: osd_enable_op_tracker = '' 2026-02-24T07:54:03.084 INFO:teuthology.orchestra.run.trial035.stderr:osd.3: osd_enable_op_tracker = '' 2026-02-24T07:54:03.084 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:54:03 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/3560376955' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2026-02-24T07:54:03.811 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:54:03.810+0000 7f4f5ea7d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:54:03.811 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:54:03.810+0000 7f4f5ea7d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:54:03.832 INFO:teuthology.orchestra.run.trial035.stderr:no valid command found; 10 closest matches: 2026-02-24T07:54:03.832 INFO:teuthology.orchestra.run.trial035.stderr:0 2026-02-24T07:54:03.832 INFO:teuthology.orchestra.run.trial035.stderr:1 2026-02-24T07:54:03.833 INFO:teuthology.orchestra.run.trial035.stderr:2 2026-02-24T07:54:03.833 INFO:teuthology.orchestra.run.trial035.stderr:abort 2026-02-24T07:54:03.833 INFO:teuthology.orchestra.run.trial035.stderr:assert 2026-02-24T07:54:03.833 INFO:teuthology.orchestra.run.trial035.stderr:bench [] [] [] [] 2026-02-24T07:54:03.833 INFO:teuthology.orchestra.run.trial035.stderr:bluefs debug_inject_read_zeros 2026-02-24T07:54:03.833 INFO:teuthology.orchestra.run.trial035.stderr:bluefs files list 2026-02-24T07:54:03.833 INFO:teuthology.orchestra.run.trial035.stderr:bluefs stats 2026-02-24T07:54:03.833 INFO:teuthology.orchestra.run.trial035.stderr:bluestore allocator dump block 2026-02-24T07:54:03.833 INFO:teuthology.orchestra.run.trial035.stderr:Error EINVAL: invalid command 2026-02-24T07:54:03.865 DEBUG:teuthology.orchestra.run:got remote process result: 22 2026-02-24T07:54:03.866 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2026-02-24T07:54:03.866 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph tell osd.2 'trim stale osdmaps' 2026-02-24T07:54:04.004 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:54:04.062 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:54:04 trial035 ceph-mon[39284]: pgmap v394: 17 pgs: 17 active+clean; 18 GiB data, 60 GiB used, 2.7 TiB / 2.7 TiB avail; 0 B/s wr, 6.37k op/s 2026-02-24T07:54:04.119 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:54:04.118+0000 7f9a3a2f2640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:54:04.127 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:54:04.118+0000 7f9a3a2f2640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:54:04.139 INFO:teuthology.orchestra.run.trial035.stderr:no valid command found; 10 closest matches: 2026-02-24T07:54:04.139 INFO:teuthology.orchestra.run.trial035.stderr:0 2026-02-24T07:54:04.139 INFO:teuthology.orchestra.run.trial035.stderr:1 2026-02-24T07:54:04.140 INFO:teuthology.orchestra.run.trial035.stderr:2 2026-02-24T07:54:04.140 INFO:teuthology.orchestra.run.trial035.stderr:abort 2026-02-24T07:54:04.140 INFO:teuthology.orchestra.run.trial035.stderr:assert 2026-02-24T07:54:04.140 INFO:teuthology.orchestra.run.trial035.stderr:bench [] [] [] [] 2026-02-24T07:54:04.140 INFO:teuthology.orchestra.run.trial035.stderr:bluefs debug_inject_read_zeros 2026-02-24T07:54:04.140 INFO:teuthology.orchestra.run.trial035.stderr:bluefs files list 2026-02-24T07:54:04.140 INFO:teuthology.orchestra.run.trial035.stderr:bluefs stats 2026-02-24T07:54:04.140 INFO:teuthology.orchestra.run.trial035.stderr:bluestore allocator dump block 2026-02-24T07:54:04.140 INFO:teuthology.orchestra.run.trial035.stderr:Error EINVAL: invalid command 2026-02-24T07:54:04.171 DEBUG:teuthology.orchestra.run:got remote process result: 22 2026-02-24T07:54:04.172 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2026-02-24T07:54:04.172 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph tell osd.3 'trim stale osdmaps' 2026-02-24T07:54:04.306 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:54:04.425 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:54:04.424+0000 7f4fdafee640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:54:04.425 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:54:04.424+0000 7f4fdafee640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:54:04.444 INFO:teuthology.orchestra.run.trial035.stderr:no valid command found; 10 closest matches: 2026-02-24T07:54:04.445 INFO:teuthology.orchestra.run.trial035.stderr:0 2026-02-24T07:54:04.445 INFO:teuthology.orchestra.run.trial035.stderr:1 2026-02-24T07:54:04.445 INFO:teuthology.orchestra.run.trial035.stderr:2 2026-02-24T07:54:04.445 INFO:teuthology.orchestra.run.trial035.stderr:abort 2026-02-24T07:54:04.445 INFO:teuthology.orchestra.run.trial035.stderr:assert 2026-02-24T07:54:04.445 INFO:teuthology.orchestra.run.trial035.stderr:bench [] [] [] [] 2026-02-24T07:54:04.445 INFO:teuthology.orchestra.run.trial035.stderr:bluefs debug_inject_read_zeros 2026-02-24T07:54:04.445 INFO:teuthology.orchestra.run.trial035.stderr:bluefs files list 2026-02-24T07:54:04.445 INFO:teuthology.orchestra.run.trial035.stderr:bluefs stats 2026-02-24T07:54:04.445 INFO:teuthology.orchestra.run.trial035.stderr:bluestore allocator dump block 2026-02-24T07:54:04.446 INFO:teuthology.orchestra.run.trial035.stderr:Error EINVAL: invalid command 2026-02-24T07:54:04.475 DEBUG:teuthology.orchestra.run:got remote process result: 22 2026-02-24T07:54:04.476 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2026-02-24T07:54:05.691 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2026-02-24T07:54:05.828 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:54:05.947 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:54:05.946+0000 7f49971ee640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:54:05.947 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:54:05.946+0000 7f49971ee640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:54:05.961 INFO:teuthology.orchestra.run.trial035.stdout:osd.0: {} 2026-02-24T07:54:05.962 INFO:teuthology.orchestra.run.trial035.stderr:osd.0: osd_enable_op_tracker = '' 2026-02-24T07:54:05.976 INFO:teuthology.orchestra.run.trial035.stdout:osd.1: {} 2026-02-24T07:54:05.976 INFO:teuthology.orchestra.run.trial035.stderr:osd.1: osd_enable_op_tracker = '' 2026-02-24T07:54:05.991 INFO:teuthology.orchestra.run.trial035.stdout:osd.2: {} 2026-02-24T07:54:05.991 INFO:teuthology.orchestra.run.trial035.stderr:osd.2: osd_enable_op_tracker = '' 2026-02-24T07:54:06.003 INFO:teuthology.orchestra.run.trial035.stdout:osd.3: {} 2026-02-24T07:54:06.003 INFO:teuthology.orchestra.run.trial035.stderr:osd.3: osd_enable_op_tracker = '' 2026-02-24T07:54:07.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:54:06 trial035 ceph-mon[39284]: pgmap v395: 17 pgs: 17 active+clean; 18 GiB data, 60 GiB used, 2.7 TiB / 2.7 TiB avail; 0 B/s wr, 6.37k op/s 2026-02-24T07:54:07.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:54:06 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/691717649' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2026-02-24T07:54:07.205 INFO:journalctl@ceph.mgr.x.trial035.stdout:Feb 24 07:54:06 trial035 ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-mgr-x[39621]: ::ffff:10.20.193.91 - - [24/Feb/2026:07:54:06] "GET /metrics HTTP/1.1" 200 4237 "" "Prometheus/3.6.0" 2026-02-24T07:54:07.356 INFO:journalctl@ceph.prometheus.a.trial091.stdout:Feb 24 07:54:06 trial091 ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-prometheus-a[66007]: time=2026-02-24T07:54:06.973Z level=ERROR source=scrape.go:1630 msg="Failed to determine correct type of scrape target." component="scrape manager" scrape_pool=ceph target=http://trial035:9283/metrics content_type="text/html;charset=utf-8" fallback_media_type="" err="received unsupported Content-Type \"text/html;charset=utf-8\" and no fallback_scrape_protocol specified for target" 2026-02-24T07:54:08.046 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2026-02-24T07:54:08.190 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:54:08.307 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:54:08.304+0000 7f19cbd3b640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:54:08.307 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:54:08.305+0000 7f19cbd3b640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:54:08.321 INFO:teuthology.orchestra.run.trial035.stdout:osd.0: {} 2026-02-24T07:54:08.322 INFO:teuthology.orchestra.run.trial035.stderr:osd.0: osd_enable_op_tracker = '' 2026-02-24T07:54:08.335 INFO:teuthology.orchestra.run.trial035.stdout:osd.1: {} 2026-02-24T07:54:08.335 INFO:teuthology.orchestra.run.trial035.stderr:osd.1: osd_enable_op_tracker = '' 2026-02-24T07:54:08.347 INFO:teuthology.orchestra.run.trial035.stdout:osd.2: {} 2026-02-24T07:54:08.347 INFO:teuthology.orchestra.run.trial035.stderr:osd.2: osd_enable_op_tracker = '' 2026-02-24T07:54:08.359 INFO:teuthology.orchestra.run.trial035.stdout:osd.3: {} 2026-02-24T07:54:08.360 INFO:teuthology.orchestra.run.trial035.stderr:osd.3: osd_enable_op_tracker = '' 2026-02-24T07:54:08.945 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:54:08 trial035 ceph-mon[39284]: pgmap v396: 17 pgs: 17 active+clean; 15 GiB data, 52 GiB used, 2.7 TiB / 2.7 TiB avail; 0 B/s wr, 9.36k op/s 2026-02-24T07:54:08.945 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:54:08 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/2819827717' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2026-02-24T07:54:09.477 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2026-02-24T07:54:09.477 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2026-02-24T07:54:09.477 INFO:tasks.thrashosds.thrasher:fixing pg num pool unique_pool_3 2026-02-24T07:54:09.478 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph pg dump --format=json 2026-02-24T07:54:09.605 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:54:09.934 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:54:09.931+0000 7fd856cba640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:54:09.934 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:54:09.932+0000 7fd856cba640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:54:10.073 INFO:teuthology.orchestra.run.trial035.stdout: 2026-02-24T07:54:10.073 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:54:09 trial035 ceph-mon[39284]: 1.0 scrub starts 2026-02-24T07:54:10.073 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:54:09 trial035 ceph-mon[39284]: 1.0 scrub ok 2026-02-24T07:54:10.074 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:54:09 trial035 ceph-mon[39284]: purged_snaps scrub starts 2026-02-24T07:54:10.074 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:54:09 trial035 ceph-mon[39284]: purged_snaps scrub ok 2026-02-24T07:54:10.074 INFO:teuthology.orchestra.run.trial035.stderr:dumped all 2026-02-24T07:54:10.443 INFO:teuthology.orchestra.run.trial035.stdout:{"pg_ready":true,"pg_map":{"version":397,"stamp":"2026-02-24T07:54:09.748195+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":16476209720,"num_objects":251401,"num_object_clones":0,"num_object_copies":754203,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":251401,"num_whiteouts":0,"num_read":137,"num_read_kb":241,"num_write":859876,"num_write_kb":35558567,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":51,"ondisk_log_size":51,"up":51,"acting":51,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":51,"num_osds":4,"num_per_pool_osds":4,"num_per_pool_omap_osds":4,"kb":2936012800,"kb_used":54556504,"kb_used_data":48958728,"kb_used_omap":85,"kb_used_meta":5597482,"kb_avail":2881456296,"statfs":{"total":3006477107200,"available":2950611247104,"internally_reserved":0,"allocated":50133737472,"data_stored":50131966240,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":87086,"internal_metadata":5731822546},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[4],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-4736221184,"num_objects":-72269,"num_object_clones":0,"num_object_copies":-216807,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-72269,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":72269,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001614"},"pg_stats":[{"pgid":"5.f","version":"88'53479","reported_seq":53782,"reported_epoch":88,"state":"active+clean","last_fresh":"2026-02-24T07:54:05.752537+0000","last_change":"2026-02-24T07:53:01.242353+0000","last_active":"2026-02-24T07:54:05.752537+0000","last_peered":"2026-02-24T07:54:05.752537+0000","last_clean":"2026-02-24T07:54:05.752537+0000","last_became_active":"2026-02-24T07:51:53.900431+0000","last_became_peered":"2026-02-24T07:51:53.900431+0000","last_unstale":"2026-02-24T07:54:05.752537+0000","last_undegraded":"2026-02-24T07:54:05.752537+0000","last_fullsized":"2026-02-24T07:54:05.752537+0000","mapping_epoch":74,"log_start":"88'53476","ondisk_log_start":"88'53476","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"84'26061","last_scrub_stamp":"2026-02-24T07:53:01.242306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_clean_scrub_stamp":"2026-02-24T07:53:01.242306+0000","objects_scrubbed":25674,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:54:22.699112+0000","scrub_duration":2129,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1038942208,"num_objects":15853,"num_object_clones":0,"num_object_copies":47559,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15853,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":53479,"num_write_kb":2218624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.e","version":"88'53424","reported_seq":53714,"reported_epoch":88,"state":"active+clean","last_fresh":"2026-02-24T07:54:05.763769+0000","last_change":"2026-02-24T07:52:56.767335+0000","last_active":"2026-02-24T07:54:05.763769+0000","last_peered":"2026-02-24T07:54:05.763769+0000","last_clean":"2026-02-24T07:54:05.763769+0000","last_became_active":"2026-02-24T07:51:53.899294+0000","last_became_peered":"2026-02-24T07:51:53.899294+0000","last_unstale":"2026-02-24T07:54:05.763769+0000","last_undegraded":"2026-02-24T07:54:05.763769+0000","last_fullsized":"2026-02-24T07:54:05.763769+0000","mapping_epoch":74,"log_start":"88'53421","ondisk_log_start":"88'53421","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"83'24557","last_scrub_stamp":"2026-02-24T07:52:56.767267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_clean_scrub_stamp":"2026-02-24T07:52:56.767267+0000","objects_scrubbed":24118,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:54:24.913755+0000","scrub_duration":2806,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1023016960,"num_objects":15610,"num_object_clones":0,"num_object_copies":46830,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15610,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":53424,"num_write_kb":2209088,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"88'53743","reported_seq":54113,"reported_epoch":88,"state":"active+clean","last_fresh":"2026-02-24T07:54:05.783574+0000","last_change":"2026-02-24T07:53:32.480551+0000","last_active":"2026-02-24T07:54:05.783574+0000","last_peered":"2026-02-24T07:54:05.783574+0000","last_clean":"2026-02-24T07:54:05.783574+0000","last_became_active":"2026-02-24T07:51:53.899772+0000","last_became_peered":"2026-02-24T07:51:53.899772+0000","last_unstale":"2026-02-24T07:54:05.783574+0000","last_undegraded":"2026-02-24T07:54:05.783574+0000","last_fullsized":"2026-02-24T07:54:05.783574+0000","mapping_epoch":74,"log_start":"88'53740","ondisk_log_start":"88'53740","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"84'37904","last_scrub_stamp":"2026-02-24T07:53:32.480499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_clean_scrub_stamp":"2026-02-24T07:53:32.480499+0000","objects_scrubbed":32149,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:54:51.939533+0000","scrub_duration":2742,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1030094872,"num_objects":15719,"num_object_clones":0,"num_object_copies":47157,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15719,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":53743,"num_write_kb":2222721,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,3],"acting":[0,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.c","version":"88'53681","reported_seq":54044,"reported_epoch":88,"state":"active+clean","last_fresh":"2026-02-24T07:54:05.857709+0000","last_change":"2026-02-24T07:53:17.083868+0000","last_active":"2026-02-24T07:54:05.857709+0000","last_peered":"2026-02-24T07:54:05.857709+0000","last_clean":"2026-02-24T07:54:05.857709+0000","last_became_active":"2026-02-24T07:51:53.898285+0000","last_became_peered":"2026-02-24T07:51:53.898285+0000","last_unstale":"2026-02-24T07:54:05.857709+0000","last_undegraded":"2026-02-24T07:54:05.857709+0000","last_fullsized":"2026-02-24T07:54:05.857709+0000","mapping_epoch":74,"log_start":"88'53678","ondisk_log_start":"88'53678","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"84'32067","last_scrub_stamp":"2026-02-24T07:53:17.083821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_clean_scrub_stamp":"2026-02-24T07:53:17.083821+0000","objects_scrubbed":31581,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:54:37.993683+0000","scrub_duration":2856,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1033175040,"num_objects":15765,"num_object_clones":0,"num_object_copies":47295,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15765,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":53681,"num_write_kb":2222272,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.b","version":"88'53955","reported_seq":54334,"reported_epoch":88,"state":"active+clean","last_fresh":"2026-02-24T07:54:05.777680+0000","last_change":"2026-02-24T07:53:30.225127+0000","last_active":"2026-02-24T07:54:05.777680+0000","last_peered":"2026-02-24T07:54:05.777680+0000","last_clean":"2026-02-24T07:54:05.777680+0000","last_became_active":"2026-02-24T07:51:53.899901+0000","last_became_peered":"2026-02-24T07:51:53.899901+0000","last_unstale":"2026-02-24T07:54:05.777680+0000","last_undegraded":"2026-02-24T07:54:05.777680+0000","last_fullsized":"2026-02-24T07:54:05.777680+0000","mapping_epoch":74,"log_start":"88'53952","ondisk_log_start":"88'53952","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"84'36879","last_scrub_stamp":"2026-02-24T07:53:30.225075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_clean_scrub_stamp":"2026-02-24T07:53:30.225075+0000","objects_scrubbed":33279,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:54:35.718496+0000","scrub_duration":3524,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1018626048,"num_objects":15543,"num_object_clones":0,"num_object_copies":46629,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15543,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":53955,"num_write_kb":2223936,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"88'53710","reported_seq":54060,"reported_epoch":88,"state":"active+clean","last_fresh":"2026-02-24T07:54:05.784789+0000","last_change":"2026-02-24T07:53:14.212339+0000","last_active":"2026-02-24T07:54:05.784789+0000","last_peered":"2026-02-24T07:54:05.784789+0000","last_clean":"2026-02-24T07:54:05.784789+0000","last_became_active":"2026-02-24T07:51:53.899549+0000","last_became_peered":"2026-02-24T07:51:53.899549+0000","last_unstale":"2026-02-24T07:54:05.784789+0000","last_undegraded":"2026-02-24T07:54:05.784789+0000","last_fullsized":"2026-02-24T07:54:05.784789+0000","mapping_epoch":74,"log_start":"88'53707","ondisk_log_start":"88'53707","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"84'30986","last_scrub_stamp":"2026-02-24T07:53:14.212265+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_clean_scrub_stamp":"2026-02-24T07:53:14.212265+0000","objects_scrubbed":30490,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:54:35.341423+0000","scrub_duration":3381,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1031143424,"num_objects":15734,"num_object_clones":0,"num_object_copies":47202,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15734,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":53710,"num_write_kb":2222208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,1],"acting":[0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.9","version":"88'54416","reported_seq":54799,"reported_epoch":88,"state":"active+clean","last_fresh":"2026-02-24T07:54:05.767969+0000","last_change":"2026-02-24T07:53:22.113835+0000","last_active":"2026-02-24T07:54:05.767969+0000","last_peered":"2026-02-24T07:54:05.767969+0000","last_clean":"2026-02-24T07:54:05.767969+0000","last_became_active":"2026-02-24T07:51:53.901610+0000","last_became_peered":"2026-02-24T07:51:53.901610+0000","last_unstale":"2026-02-24T07:54:05.767969+0000","last_undegraded":"2026-02-24T07:54:05.767969+0000","last_fullsized":"2026-02-24T07:54:05.767969+0000","mapping_epoch":74,"log_start":"88'54413","ondisk_log_start":"88'54413","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"84'34029","last_scrub_stamp":"2026-02-24T07:53:22.113774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_clean_scrub_stamp":"2026-02-24T07:53:22.113774+0000","objects_scrubbed":33520,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:54:41.903404+0000","scrub_duration":3005,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1041760256,"num_objects":15896,"num_object_clones":0,"num_object_copies":47688,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15896,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":54416,"num_write_kb":2249984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.8","version":"88'53863","reported_seq":54165,"reported_epoch":88,"state":"active+clean","last_fresh":"2026-02-24T07:54:05.785335+0000","last_change":"2026-02-24T07:53:00.549866+0000","last_active":"2026-02-24T07:54:05.785335+0000","last_peered":"2026-02-24T07:54:05.785335+0000","last_clean":"2026-02-24T07:54:05.785335+0000","last_became_active":"2026-02-24T07:51:53.900207+0000","last_became_peered":"2026-02-24T07:51:53.900207+0000","last_unstale":"2026-02-24T07:54:05.785335+0000","last_undegraded":"2026-02-24T07:54:05.785335+0000","last_fullsized":"2026-02-24T07:54:05.785335+0000","mapping_epoch":74,"log_start":"88'53860","ondisk_log_start":"88'53860","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"84'25943","last_scrub_stamp":"2026-02-24T07:53:00.549789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_clean_scrub_stamp":"2026-02-24T07:53:00.549789+0000","objects_scrubbed":25486,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:54:18.896412+0000","scrub_duration":2647,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1045889024,"num_objects":15959,"num_object_clones":0,"num_object_copies":47877,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15959,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":53863,"num_write_kb":2234304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.7","version":"88'53334","reported_seq":53635,"reported_epoch":88,"state":"active+clean","last_fresh":"2026-02-24T07:54:05.771087+0000","last_change":"2026-02-24T07:53:00.188453+0000","last_active":"2026-02-24T07:54:05.771087+0000","last_peered":"2026-02-24T07:54:05.771087+0000","last_clean":"2026-02-24T07:54:05.771087+0000","last_became_active":"2026-02-24T07:51:53.899472+0000","last_became_peered":"2026-02-24T07:51:53.899472+0000","last_unstale":"2026-02-24T07:54:05.771087+0000","last_undegraded":"2026-02-24T07:54:05.771087+0000","last_fullsized":"2026-02-24T07:54:05.771087+0000","mapping_epoch":74,"log_start":"88'53331","ondisk_log_start":"88'53331","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"84'25630","last_scrub_stamp":"2026-02-24T07:53:00.188390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_clean_scrub_stamp":"2026-02-24T07:53:00.188390+0000","objects_scrubbed":25246,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:54:07.242502+0000","scrub_duration":2262,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1013448704,"num_objects":15464,"num_object_clones":0,"num_object_copies":46392,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15464,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":53334,"num_write_kb":2201536,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.6","version":"88'53394","reported_seq":53735,"reported_epoch":88,"state":"active+clean","last_fresh":"2026-02-24T07:54:05.856725+0000","last_change":"2026-02-24T07:53:10.804133+0000","last_active":"2026-02-24T07:54:05.856725+0000","last_peered":"2026-02-24T07:54:05.856725+0000","last_clean":"2026-02-24T07:54:05.856725+0000","last_became_active":"2026-02-24T07:51:53.898308+0000","last_became_peered":"2026-02-24T07:51:53.898308+0000","last_unstale":"2026-02-24T07:54:05.856725+0000","last_undegraded":"2026-02-24T07:54:05.856725+0000","last_fullsized":"2026-02-24T07:54:05.856725+0000","mapping_epoch":74,"log_start":"88'53391","ondisk_log_start":"88'53391","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"84'29829","last_scrub_stamp":"2026-02-24T07:53:10.804104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_clean_scrub_stamp":"2026-02-24T07:53:10.804104+0000","objects_scrubbed":29351,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:54:13.621421+0000","scrub_duration":2501,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1019215872,"num_objects":15552,"num_object_clones":0,"num_object_copies":46656,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15552,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":53394,"num_write_kb":2206272,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.5","version":"88'53780","reported_seq":54173,"reported_epoch":88,"state":"active+clean","last_fresh":"2026-02-24T07:54:05.784513+0000","last_change":"2026-02-24T07:53:26.781324+0000","last_active":"2026-02-24T07:54:05.784513+0000","last_peered":"2026-02-24T07:54:05.784513+0000","last_clean":"2026-02-24T07:54:05.784513+0000","last_became_active":"2026-02-24T07:51:53.899749+0000","last_became_peered":"2026-02-24T07:51:53.899749+0000","last_unstale":"2026-02-24T07:54:05.784513+0000","last_undegraded":"2026-02-24T07:54:05.784513+0000","last_fullsized":"2026-02-24T07:54:05.784513+0000","mapping_epoch":74,"log_start":"88'53777","ondisk_log_start":"88'53777","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"84'35538","last_scrub_stamp":"2026-02-24T07:53:26.781276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_clean_scrub_stamp":"2026-02-24T07:53:26.781276+0000","objects_scrubbed":34358,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:54:37.440161+0000","scrub_duration":2982,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1024196608,"num_objects":15628,"num_object_clones":0,"num_object_copies":46884,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15628,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":53780,"num_write_kb":2221056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"88'53679","reported_seq":54001,"reported_epoch":88,"state":"active+clean","last_fresh":"2026-02-24T07:54:05.783212+0000","last_change":"2026-02-24T07:53:05.312364+0000","last_active":"2026-02-24T07:54:05.783212+0000","last_peered":"2026-02-24T07:54:05.783212+0000","last_clean":"2026-02-24T07:54:05.783212+0000","last_became_active":"2026-02-24T07:51:53.900042+0000","last_became_peered":"2026-02-24T07:51:53.900042+0000","last_unstale":"2026-02-24T07:54:05.783212+0000","last_undegraded":"2026-02-24T07:54:05.783212+0000","last_fullsized":"2026-02-24T07:54:05.783212+0000","mapping_epoch":74,"log_start":"88'53676","ondisk_log_start":"88'53676","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"84'27738","last_scrub_stamp":"2026-02-24T07:53:05.312312+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_clean_scrub_stamp":"2026-02-24T07:53:05.312312+0000","objects_scrubbed":27245,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:54:32.166518+0000","scrub_duration":2472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1034747904,"num_objects":15789,"num_object_clones":0,"num_object_copies":47367,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15789,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":53679,"num_write_kb":2222976,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.2","version":"88'54241","reported_seq":54609,"reported_epoch":88,"state":"active+clean","last_fresh":"2026-02-24T07:54:05.770222+0000","last_change":"2026-02-24T07:53:17.780799+0000","last_active":"2026-02-24T07:54:05.770222+0000","last_peered":"2026-02-24T07:54:05.770222+0000","last_clean":"2026-02-24T07:54:05.770222+0000","last_became_active":"2026-02-24T07:51:53.900096+0000","last_became_peered":"2026-02-24T07:51:53.900096+0000","last_unstale":"2026-02-24T07:54:05.770222+0000","last_undegraded":"2026-02-24T07:54:05.770222+0000","last_fullsized":"2026-02-24T07:54:05.770222+0000","mapping_epoch":74,"log_start":"88'54238","ondisk_log_start":"88'54238","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"84'32548","last_scrub_stamp":"2026-02-24T07:53:17.780705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_clean_scrub_stamp":"2026-02-24T07:53:17.780705+0000","objects_scrubbed":32030,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:54:35.344780+0000","scrub_duration":2741,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1040777216,"num_objects":15881,"num_object_clones":0,"num_object_copies":47643,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15881,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":54241,"num_write_kb":2243904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"88'53116","reported_seq":53466,"reported_epoch":88,"state":"active+clean","last_fresh":"2026-02-24T07:54:05.769376+0000","last_change":"2026-02-24T07:53:14.051194+0000","last_active":"2026-02-24T07:54:05.769376+0000","last_peered":"2026-02-24T07:54:05.769376+0000","last_clean":"2026-02-24T07:54:05.769376+0000","last_became_active":"2026-02-24T07:51:53.901027+0000","last_became_peered":"2026-02-24T07:51:53.901027+0000","last_unstale":"2026-02-24T07:54:05.769376+0000","last_undegraded":"2026-02-24T07:54:05.769376+0000","last_fullsized":"2026-02-24T07:54:05.769376+0000","mapping_epoch":74,"log_start":"88'53113","ondisk_log_start":"88'53113","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"84'30565","last_scrub_stamp":"2026-02-24T07:53:14.051154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_clean_scrub_stamp":"2026-02-24T07:53:14.051154+0000","objects_scrubbed":30140,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:54:38.448076+0000","scrub_duration":3147,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1026293760,"num_objects":15660,"num_object_clones":0,"num_object_copies":46980,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15660,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":53116,"num_write_kb":2200832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.0","version":"88'54272","reported_seq":54647,"reported_epoch":88,"state":"active+clean","last_fresh":"2026-02-24T07:54:05.780920+0000","last_change":"2026-02-24T07:53:20.014077+0000","last_active":"2026-02-24T07:54:05.780920+0000","last_peered":"2026-02-24T07:54:05.780920+0000","last_clean":"2026-02-24T07:54:05.780920+0000","last_became_active":"2026-02-24T07:51:53.898551+0000","last_became_peered":"2026-02-24T07:51:53.898551+0000","last_unstale":"2026-02-24T07:54:05.780920+0000","last_undegraded":"2026-02-24T07:54:05.780920+0000","last_fullsized":"2026-02-24T07:54:05.780920+0000","mapping_epoch":74,"log_start":"88'54269","ondisk_log_start":"88'54269","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"84'33261","last_scrub_stamp":"2026-02-24T07:53:20.014009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_clean_scrub_stamp":"2026-02-24T07:53:20.014009+0000","objects_scrubbed":32679,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:54:37.178341+0000","scrub_duration":3163,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1032192000,"num_objects":15750,"num_object_clones":0,"num_object_copies":47250,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15750,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":54272,"num_write_kb":2240704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.4","version":"88'53654","reported_seq":53999,"reported_epoch":88,"state":"active+clean","last_fresh":"2026-02-24T07:54:05.750668+0000","last_change":"2026-02-24T07:53:11.592191+0000","last_active":"2026-02-24T07:54:05.750668+0000","last_peered":"2026-02-24T07:54:05.750668+0000","last_clean":"2026-02-24T07:54:05.750668+0000","last_became_active":"2026-02-24T07:51:53.899675+0000","last_became_peered":"2026-02-24T07:51:53.899675+0000","last_unstale":"2026-02-24T07:54:05.750668+0000","last_undegraded":"2026-02-24T07:54:05.750668+0000","last_fullsized":"2026-02-24T07:54:05.750668+0000","mapping_epoch":74,"log_start":"88'53651","ondisk_log_start":"88'53651","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"84'30127","last_scrub_stamp":"2026-02-24T07:53:11.592155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_clean_scrub_stamp":"2026-02-24T07:53:11.592155+0000","objects_scrubbed":29709,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:54:23.794472+0000","scrub_duration":2432,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1022099456,"num_objects":15596,"num_object_clones":0,"num_object_copies":46788,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15596,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":53654,"num_write_kb":2216000,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"29'99","reported_seq":365,"reported_epoch":88,"state":"active+clean","last_fresh":"2026-02-24T07:53:39.925612+0000","last_change":"2026-02-24T07:52:56.946143+0000","last_active":"2026-02-24T07:53:39.925612+0000","last_peered":"2026-02-24T07:53:39.925612+0000","last_clean":"2026-02-24T07:53:39.925612+0000","last_became_active":"2026-02-24T07:42:54.186719+0000","last_became_peered":"2026-02-24T07:42:54.186719+0000","last_unstale":"2026-02-24T07:53:39.925612+0000","last_undegraded":"2026-02-24T07:53:39.925612+0000","last_fullsized":"2026-02-24T07:53:39.925612+0000","mapping_epoch":27,"log_start":"29'96","ondisk_log_start":"29'96","created":20,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'99","last_scrub_stamp":"2026-02-24T07:52:56.946109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:42:43.640776+0000","last_clean_scrub_stamp":"2026-02-24T07:52:56.946109+0000","objects_scrubbed":2,"log_size":3,"log_dups_size":96,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:54:05.896060+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":240,"num_write":135,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":16,"stat_sum":{"num_bytes":16475619352,"num_objects":251399,"num_object_clones":0,"num_object_copies":754197,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":251399,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":859741,"num_write_kb":35556417,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":49421692928,"data_stored":49421680712,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":48,"ondisk_log_size":48,"up":48,"acting":48,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":240,"num_write":135,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1682,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":3,"up_from":25,"seq":107374182537,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":14950008,"kb_used_data":14204132,"kb_used_omap":20,"kb_used_meta":745771,"kb_avail":719053192,"statfs":{"total":751619276800,"available":736310468608,"internally_reserved":0,"allocated":14545031168,"data_stored":14544586454,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":21479,"internal_metadata":763669529},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":19,"seq":81604378763,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":13098872,"kb_used_data":11402596,"kb_used_omap":20,"kb_used_meta":1696235,"kb_avail":720904328,"statfs":{"total":751619276800,"available":738206031872,"internally_reserved":0,"allocated":11676258304,"data_stored":11675813590,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":21284,"internal_metadata":1736944860},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[4],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542284,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":12046512,"kb_used_data":10179612,"kb_used_omap":24,"kb_used_meta":1866791,"kb_avail":721956688,"statfs":{"total":751619276800,"available":739283648512,"internally_reserved":0,"allocated":10423922688,"data_stored":10423485598,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":25475,"internal_metadata":1911594109},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705807,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":14461112,"kb_used_data":13172388,"kb_used_omap":18,"kb_used_meta":1288685,"kb_avail":719542088,"statfs":{"total":751619276800,"available":736811098112,"internally_reserved":0,"allocated":13488525312,"data_stored":13488080598,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":18848,"internal_metadata":1319614048},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1682,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":13392023552,"data_stored":13392019480,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":10264510464,"data_stored":10264510464,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":11349790720,"data_stored":11349786648,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":14415368192,"data_stored":14415364120,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-24T07:54:10.444 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph osd pool set unique_pool_3 pgp_num 56 2026-02-24T07:54:10.585 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:54:10.705 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:54:10.703+0000 7f3a7e9db640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:54:10.706 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:54:10.703+0000 7f3a7e9db640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:54:10.839 INFO:teuthology.orchestra.run.trial035.stderr:Error EPERM: crimson_allow_pg_split is false; pgp_num increase denied 2026-02-24T07:54:10.878 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-24T07:54:10.954 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:54:10 trial035 ceph-mon[39284]: 5.7 scrub starts 2026-02-24T07:54:10.954 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:54:10 trial035 ceph-mon[39284]: pgmap v397: 17 pgs: 17 active+clean; 15 GiB data, 52 GiB used, 2.7 TiB / 2.7 TiB avail; 0 B/s wr, 6.02k op/s 2026-02-24T07:54:10.954 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:54:10 trial035 ceph-mon[39284]: from='mgr.14284 10.20.193.35:0/3681373222' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2026-02-24T07:54:10.955 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:54:10 trial035 ceph-mon[39284]: from='mgr.14284 10.20.193.35:0/3681373222' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-24T07:54:10.955 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:54:10 trial035 ceph-mon[39284]: from='mgr.14284 10.20.193.35:0/3681373222' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2026-02-24T07:54:10.955 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:54:10 trial035 ceph-mon[39284]: from='mgr.14284 10.20.193.35:0/3681373222' entity='mgr.x' 2026-02-24T07:54:10.955 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:54:10 trial035 ceph-mon[39284]: from='mgr.14284 10.20.193.35:0/3681373222' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2026-02-24T07:54:10.955 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:54:10 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/3571033703' entity='client.admin' cmd={"prefix": "osd pool set", "pool": "unique_pool_3", "var": "pgp_num", "val": "56"} : dispatch 2026-02-24T07:54:10.967 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2026-02-24T07:54:11.104 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:54:11.221 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:54:11.219+0000 7f92eaae2640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:54:11.222 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:54:11.219+0000 7f92eaae2640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:54:11.235 INFO:teuthology.orchestra.run.trial035.stdout:osd.0: {} 2026-02-24T07:54:11.236 INFO:teuthology.orchestra.run.trial035.stderr:osd.0: osd_enable_op_tracker = '' 2026-02-24T07:54:11.247 INFO:teuthology.orchestra.run.trial035.stdout:osd.1: {} 2026-02-24T07:54:11.248 INFO:teuthology.orchestra.run.trial035.stderr:osd.1: osd_enable_op_tracker = '' 2026-02-24T07:54:11.260 INFO:teuthology.orchestra.run.trial035.stdout:osd.2: {} 2026-02-24T07:54:11.260 INFO:teuthology.orchestra.run.trial035.stderr:osd.2: osd_enable_op_tracker = '' 2026-02-24T07:54:11.274 INFO:teuthology.orchestra.run.trial035.stdout:osd.3: {} 2026-02-24T07:54:11.275 INFO:teuthology.orchestra.run.trial035.stderr:osd.3: osd_enable_op_tracker = '' 2026-02-24T07:54:12.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:54:11 trial035 ceph-mon[39284]: from='client.15348 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2026-02-24T07:54:12.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:54:11 trial035 ceph-mon[39284]: 5.7 scrub ok 2026-02-24T07:54:12.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:54:11 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/2859544703' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2026-02-24T07:54:13.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:54:12 trial035 ceph-mon[39284]: pgmap v398: 17 pgs: 1 active+clean+scrubbing, 16 active+clean; 13 GiB data, 45 GiB used, 2.7 TiB / 2.7 TiB avail; 0 B/s wr, 9.10k op/s 2026-02-24T07:54:13.340 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2026-02-24T07:54:13.478 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:54:13.601 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:54:13.598+0000 7fa3e5c4d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:54:13.601 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:54:13.599+0000 7fa3e5c4d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:54:13.615 INFO:teuthology.orchestra.run.trial035.stdout:osd.0: {} 2026-02-24T07:54:13.616 INFO:teuthology.orchestra.run.trial035.stderr:osd.0: osd_enable_op_tracker = '' 2026-02-24T07:54:13.627 INFO:teuthology.orchestra.run.trial035.stdout:osd.1: {} 2026-02-24T07:54:13.627 INFO:teuthology.orchestra.run.trial035.stderr:osd.1: osd_enable_op_tracker = '' 2026-02-24T07:54:13.639 INFO:teuthology.orchestra.run.trial035.stdout:osd.2: {} 2026-02-24T07:54:13.639 INFO:teuthology.orchestra.run.trial035.stderr:osd.2: osd_enable_op_tracker = '' 2026-02-24T07:54:13.651 INFO:teuthology.orchestra.run.trial035.stdout:osd.3: {} 2026-02-24T07:54:13.652 INFO:teuthology.orchestra.run.trial035.stderr:osd.3: osd_enable_op_tracker = '' 2026-02-24T07:54:14.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:54:13 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/2027426338' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2026-02-24T07:54:15.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:54:14 trial035 ceph-mon[39284]: pgmap v399: 17 pgs: 1 active+clean+scrubbing, 16 active+clean; 13 GiB data, 45 GiB used, 2.7 TiB / 2.7 TiB avail; 0 B/s wr, 6.07k op/s 2026-02-24T07:54:15.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:54:14 trial035 ceph-mon[39284]: from='mgr.14284 10.20.193.35:0/3681373222' entity='mgr.x' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2026-02-24T07:54:15.693 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2026-02-24T07:54:15.834 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:54:15.879 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2026-02-24T07:54:15.879 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2026-02-24T07:54:15.879 INFO:tasks.thrashosds.thrasher:trim_stale_osdmaps 2026-02-24T07:54:15.880 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph tell osd.0 'trim stale osdmaps' 2026-02-24T07:54:16.031 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:54:16.046 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:54:15 trial035 ceph-mon[39284]: 5.6 scrub starts 2026-02-24T07:54:16.173 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:54:16.166+0000 7efd94b6c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:54:16.173 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:54:16.167+0000 7efd94b6c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:54:16.185 INFO:teuthology.orchestra.run.trial035.stdout:osd.0: {} 2026-02-24T07:54:16.185 INFO:teuthology.orchestra.run.trial035.stderr:osd.0: osd_enable_op_tracker = '' 2026-02-24T07:54:16.197 INFO:teuthology.orchestra.run.trial035.stdout:osd.1: {} 2026-02-24T07:54:16.197 INFO:teuthology.orchestra.run.trial035.stderr:osd.1: osd_enable_op_tracker = '' 2026-02-24T07:54:16.209 INFO:teuthology.orchestra.run.trial035.stdout:osd.2: {} 2026-02-24T07:54:16.209 INFO:teuthology.orchestra.run.trial035.stderr:osd.2: osd_enable_op_tracker = '' 2026-02-24T07:54:16.222 INFO:teuthology.orchestra.run.trial035.stdout:osd.3: {} 2026-02-24T07:54:16.222 INFO:teuthology.orchestra.run.trial035.stderr:osd.3: osd_enable_op_tracker = '' 2026-02-24T07:54:16.874 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:54:16.872+0000 7f9ed6c24640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:54:16.874 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:54:16.873+0000 7f9ed6c24640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:54:16.895 INFO:teuthology.orchestra.run.trial035.stderr:no valid command found; 10 closest matches: 2026-02-24T07:54:16.895 INFO:teuthology.orchestra.run.trial035.stderr:0 2026-02-24T07:54:16.895 INFO:teuthology.orchestra.run.trial035.stderr:1 2026-02-24T07:54:16.895 INFO:teuthology.orchestra.run.trial035.stderr:2 2026-02-24T07:54:16.895 INFO:teuthology.orchestra.run.trial035.stderr:abort 2026-02-24T07:54:16.895 INFO:teuthology.orchestra.run.trial035.stderr:assert 2026-02-24T07:54:16.895 INFO:teuthology.orchestra.run.trial035.stderr:bench [] [] [] [] 2026-02-24T07:54:16.896 INFO:teuthology.orchestra.run.trial035.stderr:bluefs debug_inject_read_zeros 2026-02-24T07:54:16.896 INFO:teuthology.orchestra.run.trial035.stderr:bluefs files list 2026-02-24T07:54:16.896 INFO:teuthology.orchestra.run.trial035.stderr:bluefs stats 2026-02-24T07:54:16.896 INFO:teuthology.orchestra.run.trial035.stderr:bluestore allocator dump block 2026-02-24T07:54:16.896 INFO:teuthology.orchestra.run.trial035.stderr:Error EINVAL: invalid command 2026-02-24T07:54:16.927 DEBUG:teuthology.orchestra.run:got remote process result: 22 2026-02-24T07:54:16.927 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2026-02-24T07:54:16.927 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph tell osd.1 'trim stale osdmaps' 2026-02-24T07:54:17.069 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:54:17.093 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:54:16 trial035 ceph-mon[39284]: 5.6 scrub ok 2026-02-24T07:54:17.093 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:54:16 trial035 ceph-mon[39284]: pgmap v400: 17 pgs: 1 active+clean+scrubbing, 16 active+clean; 13 GiB data, 45 GiB used, 2.7 TiB / 2.7 TiB avail; 0 B/s wr, 6.07k op/s 2026-02-24T07:54:17.093 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:54:16 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/1892158174' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2026-02-24T07:54:17.093 INFO:journalctl@ceph.mgr.x.trial035.stdout:Feb 24 07:54:16 trial035 ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-mgr-x[39621]: ::ffff:10.20.193.91 - - [24/Feb/2026:07:54:16] "GET /metrics HTTP/1.1" 200 4235 "" "Prometheus/3.6.0" 2026-02-24T07:54:17.193 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:54:17.191+0000 7fb86404a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:54:17.194 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:54:17.191+0000 7fb86404a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:54:17.214 INFO:teuthology.orchestra.run.trial035.stderr:no valid command found; 10 closest matches: 2026-02-24T07:54:17.214 INFO:teuthology.orchestra.run.trial035.stderr:0 2026-02-24T07:54:17.214 INFO:teuthology.orchestra.run.trial035.stderr:1 2026-02-24T07:54:17.214 INFO:teuthology.orchestra.run.trial035.stderr:2 2026-02-24T07:54:17.214 INFO:teuthology.orchestra.run.trial035.stderr:abort 2026-02-24T07:54:17.215 INFO:teuthology.orchestra.run.trial035.stderr:assert 2026-02-24T07:54:17.215 INFO:teuthology.orchestra.run.trial035.stderr:bench [] [] [] [] 2026-02-24T07:54:17.215 INFO:teuthology.orchestra.run.trial035.stderr:bluefs debug_inject_read_zeros 2026-02-24T07:54:17.215 INFO:teuthology.orchestra.run.trial035.stderr:bluefs files list 2026-02-24T07:54:17.215 INFO:teuthology.orchestra.run.trial035.stderr:bluefs stats 2026-02-24T07:54:17.215 INFO:teuthology.orchestra.run.trial035.stderr:bluestore allocator dump block 2026-02-24T07:54:17.215 INFO:teuthology.orchestra.run.trial035.stderr:Error EINVAL: invalid command 2026-02-24T07:54:17.245 DEBUG:teuthology.orchestra.run:got remote process result: 22 2026-02-24T07:54:17.246 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2026-02-24T07:54:17.246 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph tell osd.2 'trim stale osdmaps' 2026-02-24T07:54:17.262 INFO:journalctl@ceph.prometheus.a.trial091.stdout:Feb 24 07:54:16 trial091 ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-prometheus-a[66007]: time=2026-02-24T07:54:16.973Z level=ERROR source=scrape.go:1630 msg="Failed to determine correct type of scrape target." component="scrape manager" scrape_pool=ceph target=http://trial035:9283/metrics content_type="text/html;charset=utf-8" fallback_media_type="" err="received unsupported Content-Type \"text/html;charset=utf-8\" and no fallback_scrape_protocol specified for target" 2026-02-24T07:54:17.384 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:54:17.498 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:54:17.495+0000 7f4b9303d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:54:17.498 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:54:17.496+0000 7f4b9303d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:54:17.518 INFO:teuthology.orchestra.run.trial035.stderr:no valid command found; 10 closest matches: 2026-02-24T07:54:17.518 INFO:teuthology.orchestra.run.trial035.stderr:0 2026-02-24T07:54:17.518 INFO:teuthology.orchestra.run.trial035.stderr:1 2026-02-24T07:54:17.519 INFO:teuthology.orchestra.run.trial035.stderr:2 2026-02-24T07:54:17.519 INFO:teuthology.orchestra.run.trial035.stderr:abort 2026-02-24T07:54:17.519 INFO:teuthology.orchestra.run.trial035.stderr:assert 2026-02-24T07:54:17.519 INFO:teuthology.orchestra.run.trial035.stderr:bench [] [] [] [] 2026-02-24T07:54:17.519 INFO:teuthology.orchestra.run.trial035.stderr:bluefs debug_inject_read_zeros 2026-02-24T07:54:17.519 INFO:teuthology.orchestra.run.trial035.stderr:bluefs files list 2026-02-24T07:54:17.519 INFO:teuthology.orchestra.run.trial035.stderr:bluefs stats 2026-02-24T07:54:17.519 INFO:teuthology.orchestra.run.trial035.stderr:bluestore allocator dump block 2026-02-24T07:54:17.519 INFO:teuthology.orchestra.run.trial035.stderr:Error EINVAL: invalid command 2026-02-24T07:54:17.550 DEBUG:teuthology.orchestra.run:got remote process result: 22 2026-02-24T07:54:17.551 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2026-02-24T07:54:17.551 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph tell osd.3 'trim stale osdmaps' 2026-02-24T07:54:17.693 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:54:17.813 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:54:17.810+0000 7fc6a55d7640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:54:17.813 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:54:17.811+0000 7fc6a55d7640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:54:17.832 INFO:teuthology.orchestra.run.trial035.stderr:no valid command found; 10 closest matches: 2026-02-24T07:54:17.833 INFO:teuthology.orchestra.run.trial035.stderr:0 2026-02-24T07:54:17.833 INFO:teuthology.orchestra.run.trial035.stderr:1 2026-02-24T07:54:17.833 INFO:teuthology.orchestra.run.trial035.stderr:2 2026-02-24T07:54:17.833 INFO:teuthology.orchestra.run.trial035.stderr:abort 2026-02-24T07:54:17.833 INFO:teuthology.orchestra.run.trial035.stderr:assert 2026-02-24T07:54:17.833 INFO:teuthology.orchestra.run.trial035.stderr:bench [] [] [] [] 2026-02-24T07:54:17.833 INFO:teuthology.orchestra.run.trial035.stderr:bluefs debug_inject_read_zeros 2026-02-24T07:54:17.833 INFO:teuthology.orchestra.run.trial035.stderr:bluefs files list 2026-02-24T07:54:17.833 INFO:teuthology.orchestra.run.trial035.stderr:bluefs stats 2026-02-24T07:54:17.833 INFO:teuthology.orchestra.run.trial035.stderr:bluestore allocator dump block 2026-02-24T07:54:17.833 INFO:teuthology.orchestra.run.trial035.stderr:Error EINVAL: invalid command 2026-02-24T07:54:17.865 DEBUG:teuthology.orchestra.run:got remote process result: 22 2026-02-24T07:54:17.866 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2026-02-24T07:54:18.752 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2026-02-24T07:54:18.889 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:54:19.006 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:54:19.004+0000 7fa5199c6640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:54:19.006 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:54:19.004+0000 7fa5199c6640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:54:19.021 INFO:teuthology.orchestra.run.trial035.stdout:osd.0: {} 2026-02-24T07:54:19.022 INFO:teuthology.orchestra.run.trial035.stderr:osd.0: osd_enable_op_tracker = '' 2026-02-24T07:54:19.034 INFO:teuthology.orchestra.run.trial035.stdout:osd.1: {} 2026-02-24T07:54:19.034 INFO:teuthology.orchestra.run.trial035.stderr:osd.1: osd_enable_op_tracker = '' 2026-02-24T07:54:19.035 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:54:18 trial035 ceph-mon[39284]: pgmap v401: 17 pgs: 17 active+clean; 11 GiB data, 37 GiB used, 2.7 TiB / 2.7 TiB avail; 0 B/s wr, 9.21k op/s 2026-02-24T07:54:19.047 INFO:teuthology.orchestra.run.trial035.stdout:osd.2: {} 2026-02-24T07:54:19.047 INFO:teuthology.orchestra.run.trial035.stderr:osd.2: osd_enable_op_tracker = '' 2026-02-24T07:54:19.058 INFO:teuthology.orchestra.run.trial035.stdout:osd.3: {} 2026-02-24T07:54:19.059 INFO:teuthology.orchestra.run.trial035.stderr:osd.3: osd_enable_op_tracker = '' 2026-02-24T07:54:20.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:54:19 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/1114897179' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2026-02-24T07:54:21.100 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2026-02-24T07:54:21.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:54:20 trial035 ceph-mon[39284]: 5.8 scrub starts 2026-02-24T07:54:21.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:54:20 trial035 ceph-mon[39284]: pgmap v402: 17 pgs: 17 active+clean; 11 GiB data, 37 GiB used, 2.7 TiB / 2.7 TiB avail; 0 B/s wr, 6.22k op/s 2026-02-24T07:54:21.246 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:54:21.380 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:54:21.377+0000 7fe67de1a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:54:21.380 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:54:21.378+0000 7fe67de1a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:54:21.394 INFO:teuthology.orchestra.run.trial035.stdout:osd.0: {} 2026-02-24T07:54:21.395 INFO:teuthology.orchestra.run.trial035.stderr:osd.0: osd_enable_op_tracker = '' 2026-02-24T07:54:21.410 INFO:teuthology.orchestra.run.trial035.stdout:osd.1: {} 2026-02-24T07:54:21.410 INFO:teuthology.orchestra.run.trial035.stderr:osd.1: osd_enable_op_tracker = '' 2026-02-24T07:54:21.423 INFO:teuthology.orchestra.run.trial035.stdout:osd.2: {} 2026-02-24T07:54:21.424 INFO:teuthology.orchestra.run.trial035.stderr:osd.2: osd_enable_op_tracker = '' 2026-02-24T07:54:21.435 INFO:teuthology.orchestra.run.trial035.stdout:osd.3: {} 2026-02-24T07:54:21.435 INFO:teuthology.orchestra.run.trial035.stderr:osd.3: osd_enable_op_tracker = '' 2026-02-24T07:54:22.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:54:21 trial035 ceph-mon[39284]: 5.8 scrub ok 2026-02-24T07:54:22.205 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:54:21 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/4047186045' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2026-02-24T07:54:22.866 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2026-02-24T07:54:22.867 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2026-02-24T07:54:22.867 INFO:tasks.thrashosds.thrasher:Growing pool unique_pool_3 2026-02-24T07:54:22.867 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph pg dump --format=json 2026-02-24T07:54:23.005 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:54:23.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:54:22 trial035 ceph-mon[39284]: pgmap v403: 17 pgs: 17 active+clean; 8.6 GiB data, 31 GiB used, 2.7 TiB / 2.7 TiB avail; 0 B/s wr, 9.20k op/s 2026-02-24T07:54:23.476 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2026-02-24T07:54:23.493 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:54:23.440+0000 7f4f1b3e4640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:54:23.494 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:54:23.440+0000 7f4f1b3e4640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:54:23.570 INFO:teuthology.orchestra.run.trial035.stdout: 2026-02-24T07:54:23.571 INFO:teuthology.orchestra.run.trial035.stderr:dumped all 2026-02-24T07:54:23.730 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:54:23.754 INFO:teuthology.orchestra.run.trial035.stdout:{"pg_ready":true,"pg_map":{"version":403,"stamp":"2026-02-24T07:54:21.749842+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":9241821752,"num_objects":141013,"num_object_clones":0,"num_object_copies":423039,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":141013,"num_whiteouts":0,"num_read":137,"num_read_kb":241,"num_write":970264,"num_write_kb":35558567,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":53,"ondisk_log_size":53,"up":51,"acting":51,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":51,"num_osds":4,"num_per_pool_osds":4,"num_per_pool_omap_osds":4,"kb":2936012800,"kb_used":32044888,"kb_used_data":27282952,"kb_used_omap":78,"kb_used_meta":4761585,"kb_avail":2903967912,"statfs":{"total":3006477107200,"available":2973663141888,"internally_reserved":0,"allocated":27937742848,"data_stored":27935971616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":80630,"internal_metadata":4875863306},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-7234387968,"num_objects":-110388,"num_object_clones":0,"num_object_copies":-331164,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-110388,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":110388,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001646"},"pg_stats":[{"pgid":"5.f","version":"88'60445","reported_seq":60747,"reported_epoch":88,"state":"active+clean","last_fresh":"2026-02-24T07:54:20.758696+0000","last_change":"2026-02-24T07:53:01.242353+0000","last_active":"2026-02-24T07:54:20.758696+0000","last_peered":"2026-02-24T07:54:20.758696+0000","last_clean":"2026-02-24T07:54:20.758696+0000","last_became_active":"2026-02-24T07:51:53.900431+0000","last_became_peered":"2026-02-24T07:51:53.900431+0000","last_unstale":"2026-02-24T07:54:20.758696+0000","last_undegraded":"2026-02-24T07:54:20.758696+0000","last_fullsized":"2026-02-24T07:54:20.758696+0000","mapping_epoch":74,"log_start":"88'60440","ondisk_log_start":"88'60440","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"84'26061","last_scrub_stamp":"2026-02-24T07:53:01.242306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_clean_scrub_stamp":"2026-02-24T07:53:01.242306+0000","objects_scrubbed":25674,"log_size":5,"log_dups_size":3000,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:54:22.699112+0000","scrub_duration":2129,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582418432,"num_objects":8887,"num_object_clones":0,"num_object_copies":26661,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8887,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":60445,"num_write_kb":2218624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.e","version":"88'60385","reported_seq":60675,"reported_epoch":88,"state":"active+clean","last_fresh":"2026-02-24T07:54:20.771109+0000","last_change":"2026-02-24T07:52:56.767335+0000","last_active":"2026-02-24T07:54:20.771109+0000","last_peered":"2026-02-24T07:54:20.771109+0000","last_clean":"2026-02-24T07:54:20.771109+0000","last_became_active":"2026-02-24T07:51:53.899294+0000","last_became_peered":"2026-02-24T07:51:53.899294+0000","last_unstale":"2026-02-24T07:54:20.771109+0000","last_undegraded":"2026-02-24T07:54:20.771109+0000","last_fullsized":"2026-02-24T07:54:20.771109+0000","mapping_epoch":74,"log_start":"88'60382","ondisk_log_start":"88'60382","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"83'24557","last_scrub_stamp":"2026-02-24T07:52:56.767267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_clean_scrub_stamp":"2026-02-24T07:52:56.767267+0000","objects_scrubbed":24118,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:54:24.913755+0000","scrub_duration":2806,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":566820864,"num_objects":8649,"num_object_clones":0,"num_object_copies":25947,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8649,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":60385,"num_write_kb":2209088,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"88'60677","reported_seq":61047,"reported_epoch":88,"state":"active+clean","last_fresh":"2026-02-24T07:54:20.786199+0000","last_change":"2026-02-24T07:53:32.480551+0000","last_active":"2026-02-24T07:54:20.786199+0000","last_peered":"2026-02-24T07:54:20.786199+0000","last_clean":"2026-02-24T07:54:20.786199+0000","last_became_active":"2026-02-24T07:51:53.899772+0000","last_became_peered":"2026-02-24T07:51:53.899772+0000","last_unstale":"2026-02-24T07:54:20.786199+0000","last_undegraded":"2026-02-24T07:54:20.786199+0000","last_fullsized":"2026-02-24T07:54:20.786199+0000","mapping_epoch":74,"log_start":"88'60674","ondisk_log_start":"88'60674","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"84'37904","last_scrub_stamp":"2026-02-24T07:53:32.480499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_clean_scrub_stamp":"2026-02-24T07:53:32.480499+0000","objects_scrubbed":32149,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:54:51.939533+0000","scrub_duration":2742,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":575668248,"num_objects":8785,"num_object_clones":0,"num_object_copies":26355,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8785,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":60677,"num_write_kb":2222721,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,3],"acting":[0,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.c","version":"88'60704","reported_seq":61067,"reported_epoch":88,"state":"active+clean","last_fresh":"2026-02-24T07:54:20.860452+0000","last_change":"2026-02-24T07:53:17.083868+0000","last_active":"2026-02-24T07:54:20.860452+0000","last_peered":"2026-02-24T07:54:20.860452+0000","last_clean":"2026-02-24T07:54:20.860452+0000","last_became_active":"2026-02-24T07:51:53.898285+0000","last_became_peered":"2026-02-24T07:51:53.898285+0000","last_unstale":"2026-02-24T07:54:20.860452+0000","last_undegraded":"2026-02-24T07:54:20.860452+0000","last_fullsized":"2026-02-24T07:54:20.860452+0000","mapping_epoch":74,"log_start":"88'60701","ondisk_log_start":"88'60701","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"84'32067","last_scrub_stamp":"2026-02-24T07:53:17.083821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_clean_scrub_stamp":"2026-02-24T07:53:17.083821+0000","objects_scrubbed":31581,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:54:37.993683+0000","scrub_duration":2856,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":572915712,"num_objects":8742,"num_object_clones":0,"num_object_copies":26226,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8742,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":60704,"num_write_kb":2222272,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.b","version":"88'60666","reported_seq":61046,"reported_epoch":88,"state":"active+clean","last_fresh":"2026-02-24T07:54:20.787706+0000","last_change":"2026-02-24T07:53:30.225127+0000","last_active":"2026-02-24T07:54:20.787706+0000","last_peered":"2026-02-24T07:54:20.787706+0000","last_clean":"2026-02-24T07:54:20.787706+0000","last_became_active":"2026-02-24T07:51:53.899901+0000","last_became_peered":"2026-02-24T07:51:53.899901+0000","last_unstale":"2026-02-24T07:54:20.787706+0000","last_undegraded":"2026-02-24T07:54:20.787706+0000","last_fullsized":"2026-02-24T07:54:20.787706+0000","mapping_epoch":74,"log_start":"88'60663","ondisk_log_start":"88'60663","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"84'36879","last_scrub_stamp":"2026-02-24T07:53:30.225075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_clean_scrub_stamp":"2026-02-24T07:53:30.225075+0000","objects_scrubbed":33279,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:54:35.718496+0000","scrub_duration":3524,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":578813952,"num_objects":8832,"num_object_clones":0,"num_object_copies":26496,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8832,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":60666,"num_write_kb":2223936,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"88'60561","reported_seq":60913,"reported_epoch":88,"state":"active+clean","last_fresh":"2026-02-24T07:54:20.783159+0000","last_change":"2026-02-24T07:53:14.212339+0000","last_active":"2026-02-24T07:54:20.783159+0000","last_peered":"2026-02-24T07:54:20.783159+0000","last_clean":"2026-02-24T07:54:20.783159+0000","last_became_active":"2026-02-24T07:51:53.899549+0000","last_became_peered":"2026-02-24T07:51:53.899549+0000","last_unstale":"2026-02-24T07:54:20.783159+0000","last_undegraded":"2026-02-24T07:54:20.783159+0000","last_fullsized":"2026-02-24T07:54:20.783159+0000","mapping_epoch":74,"log_start":"88'60558","ondisk_log_start":"88'60558","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"84'30986","last_scrub_stamp":"2026-02-24T07:53:14.212265+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_clean_scrub_stamp":"2026-02-24T07:53:14.212265+0000","objects_scrubbed":30490,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:54:35.341423+0000","scrub_duration":3381,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582156288,"num_objects":8883,"num_object_clones":0,"num_object_copies":26649,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8883,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":60561,"num_write_kb":2222208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,1],"acting":[0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.9","version":"88'61324","reported_seq":61707,"reported_epoch":88,"state":"active+clean","last_fresh":"2026-02-24T07:54:20.772739+0000","last_change":"2026-02-24T07:53:22.113835+0000","last_active":"2026-02-24T07:54:20.772739+0000","last_peered":"2026-02-24T07:54:20.772739+0000","last_clean":"2026-02-24T07:54:20.772739+0000","last_became_active":"2026-02-24T07:51:53.901610+0000","last_became_peered":"2026-02-24T07:51:53.901610+0000","last_unstale":"2026-02-24T07:54:20.772739+0000","last_undegraded":"2026-02-24T07:54:20.772739+0000","last_fullsized":"2026-02-24T07:54:20.772739+0000","mapping_epoch":74,"log_start":"88'61321","ondisk_log_start":"88'61321","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"84'34029","last_scrub_stamp":"2026-02-24T07:53:22.113774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_clean_scrub_stamp":"2026-02-24T07:53:22.113774+0000","objects_scrubbed":33520,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:54:41.903404+0000","scrub_duration":3005,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":589037568,"num_objects":8988,"num_object_clones":0,"num_object_copies":26964,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8988,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":61324,"num_write_kb":2249984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.8","version":"88'60862","reported_seq":61264,"reported_epoch":88,"state":"active+clean","last_fresh":"2026-02-24T07:54:20.786229+0000","last_change":"2026-02-24T07:54:20.727673+0000","last_active":"2026-02-24T07:54:20.786229+0000","last_peered":"2026-02-24T07:54:20.786229+0000","last_clean":"2026-02-24T07:54:20.786229+0000","last_became_active":"2026-02-24T07:51:53.900207+0000","last_became_peered":"2026-02-24T07:51:53.900207+0000","last_unstale":"2026-02-24T07:54:20.786229+0000","last_undegraded":"2026-02-24T07:54:20.786229+0000","last_fullsized":"2026-02-24T07:54:20.786229+0000","mapping_epoch":74,"log_start":"88'60859","ondisk_log_start":"88'60859","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"88'60829","last_scrub_stamp":"2026-02-24T07:54:20.727592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_clean_scrub_stamp":"2026-02-24T07:54:20.727592+0000","objects_scrubbed":9200,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:55:41.278790+0000","scrub_duration":1000,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":587202560,"num_objects":8960,"num_object_clones":0,"num_object_copies":26880,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8960,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":60862,"num_write_kb":2234304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.7","version":"88'60139","reported_seq":60582,"reported_epoch":88,"state":"active+clean","last_fresh":"2026-02-24T07:54:20.773817+0000","last_change":"2026-02-24T07:54:10.924892+0000","last_active":"2026-02-24T07:54:20.773817+0000","last_peered":"2026-02-24T07:54:20.773817+0000","last_clean":"2026-02-24T07:54:20.773817+0000","last_became_active":"2026-02-24T07:51:53.899472+0000","last_became_peered":"2026-02-24T07:51:53.899472+0000","last_unstale":"2026-02-24T07:54:20.773817+0000","last_undegraded":"2026-02-24T07:54:20.773817+0000","last_fullsized":"2026-02-24T07:54:20.773817+0000","mapping_epoch":74,"log_start":"88'60136","ondisk_log_start":"88'60136","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"88'55711","last_scrub_stamp":"2026-02-24T07:54:10.924866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_clean_scrub_stamp":"2026-02-24T07:54:10.924866+0000","objects_scrubbed":13335,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:55:37.296783+0000","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":567476224,"num_objects":8659,"num_object_clones":0,"num_object_copies":25977,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8659,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":60139,"num_write_kb":2201536,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.6","version":"88'60242","reported_seq":60705,"reported_epoch":88,"state":"active+clean","last_fresh":"2026-02-24T07:54:20.861225+0000","last_change":"2026-02-24T07:54:15.288914+0000","last_active":"2026-02-24T07:54:20.861225+0000","last_peered":"2026-02-24T07:54:20.861225+0000","last_clean":"2026-02-24T07:54:20.861225+0000","last_became_active":"2026-02-24T07:51:53.898308+0000","last_became_peered":"2026-02-24T07:51:53.898308+0000","last_unstale":"2026-02-24T07:54:20.861225+0000","last_undegraded":"2026-02-24T07:54:20.861225+0000","last_fullsized":"2026-02-24T07:54:20.861225+0000","mapping_epoch":74,"log_start":"88'60239","ondisk_log_start":"88'60239","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"88'57815","last_scrub_stamp":"2026-02-24T07:54:15.288870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_clean_scrub_stamp":"2026-02-24T07:54:15.288870+0000","objects_scrubbed":11321,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:55:31.761926+0000","scrub_duration":822,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":570425344,"num_objects":8704,"num_object_clones":0,"num_object_copies":26112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8704,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":60242,"num_write_kb":2206272,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.5","version":"88'60580","reported_seq":60973,"reported_epoch":88,"state":"active+clean","last_fresh":"2026-02-24T07:54:20.781715+0000","last_change":"2026-02-24T07:53:26.781324+0000","last_active":"2026-02-24T07:54:20.781715+0000","last_peered":"2026-02-24T07:54:20.781715+0000","last_clean":"2026-02-24T07:54:20.781715+0000","last_became_active":"2026-02-24T07:51:53.899749+0000","last_became_peered":"2026-02-24T07:51:53.899749+0000","last_unstale":"2026-02-24T07:54:20.781715+0000","last_undegraded":"2026-02-24T07:54:20.781715+0000","last_fullsized":"2026-02-24T07:54:20.781715+0000","mapping_epoch":74,"log_start":"88'60577","ondisk_log_start":"88'60577","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"84'35538","last_scrub_stamp":"2026-02-24T07:53:26.781276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_clean_scrub_stamp":"2026-02-24T07:53:26.781276+0000","objects_scrubbed":34358,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:54:37.440161+0000","scrub_duration":2982,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":578551808,"num_objects":8828,"num_object_clones":0,"num_object_copies":26484,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8828,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":60580,"num_write_kb":2221056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"88'60594","reported_seq":60916,"reported_epoch":88,"state":"active+clean","last_fresh":"2026-02-24T07:54:20.787601+0000","last_change":"2026-02-24T07:53:05.312364+0000","last_active":"2026-02-24T07:54:20.787601+0000","last_peered":"2026-02-24T07:54:20.787601+0000","last_clean":"2026-02-24T07:54:20.787601+0000","last_became_active":"2026-02-24T07:51:53.900042+0000","last_became_peered":"2026-02-24T07:51:53.900042+0000","last_unstale":"2026-02-24T07:54:20.787601+0000","last_undegraded":"2026-02-24T07:54:20.787601+0000","last_fullsized":"2026-02-24T07:54:20.787601+0000","mapping_epoch":74,"log_start":"88'60591","ondisk_log_start":"88'60591","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"84'27738","last_scrub_stamp":"2026-02-24T07:53:05.312312+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_clean_scrub_stamp":"2026-02-24T07:53:05.312312+0000","objects_scrubbed":27245,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:54:32.166518+0000","scrub_duration":2472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":581566464,"num_objects":8874,"num_object_clones":0,"num_object_copies":26622,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8874,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":60594,"num_write_kb":2222976,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.2","version":"88'61246","reported_seq":61614,"reported_epoch":88,"state":"active+clean","last_fresh":"2026-02-24T07:54:20.771161+0000","last_change":"2026-02-24T07:53:17.780799+0000","last_active":"2026-02-24T07:54:20.771161+0000","last_peered":"2026-02-24T07:54:20.771161+0000","last_clean":"2026-02-24T07:54:20.771161+0000","last_became_active":"2026-02-24T07:51:53.900096+0000","last_became_peered":"2026-02-24T07:51:53.900096+0000","last_unstale":"2026-02-24T07:54:20.771161+0000","last_undegraded":"2026-02-24T07:54:20.771161+0000","last_fullsized":"2026-02-24T07:54:20.771161+0000","mapping_epoch":74,"log_start":"88'61243","ondisk_log_start":"88'61243","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"84'32548","last_scrub_stamp":"2026-02-24T07:53:17.780705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_clean_scrub_stamp":"2026-02-24T07:53:17.780705+0000","objects_scrubbed":32030,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:54:35.344780+0000","scrub_duration":2741,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":581697536,"num_objects":8876,"num_object_clones":0,"num_object_copies":26628,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8876,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":61246,"num_write_kb":2243904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"88'59962","reported_seq":60312,"reported_epoch":88,"state":"active+clean","last_fresh":"2026-02-24T07:54:20.764841+0000","last_change":"2026-02-24T07:53:14.051194+0000","last_active":"2026-02-24T07:54:20.764841+0000","last_peered":"2026-02-24T07:54:20.764841+0000","last_clean":"2026-02-24T07:54:20.764841+0000","last_became_active":"2026-02-24T07:51:53.901027+0000","last_became_peered":"2026-02-24T07:51:53.901027+0000","last_unstale":"2026-02-24T07:54:20.764841+0000","last_undegraded":"2026-02-24T07:54:20.764841+0000","last_fullsized":"2026-02-24T07:54:20.764841+0000","mapping_epoch":74,"log_start":"88'59959","ondisk_log_start":"88'59959","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"84'30565","last_scrub_stamp":"2026-02-24T07:53:14.051154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_clean_scrub_stamp":"2026-02-24T07:53:14.051154+0000","objects_scrubbed":30140,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:54:38.448076+0000","scrub_duration":3147,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":577634304,"num_objects":8814,"num_object_clones":0,"num_object_copies":26442,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8814,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":59962,"num_write_kb":2200832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.0","version":"88'61294","reported_seq":61668,"reported_epoch":88,"state":"active+clean","last_fresh":"2026-02-24T07:54:20.787137+0000","last_change":"2026-02-24T07:53:20.014077+0000","last_active":"2026-02-24T07:54:20.787137+0000","last_peered":"2026-02-24T07:54:20.787137+0000","last_clean":"2026-02-24T07:54:20.787137+0000","last_became_active":"2026-02-24T07:51:53.898551+0000","last_became_peered":"2026-02-24T07:51:53.898551+0000","last_unstale":"2026-02-24T07:54:20.787137+0000","last_undegraded":"2026-02-24T07:54:20.787137+0000","last_fullsized":"2026-02-24T07:54:20.787137+0000","mapping_epoch":74,"log_start":"88'61291","ondisk_log_start":"88'61291","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"84'33261","last_scrub_stamp":"2026-02-24T07:53:20.014009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_clean_scrub_stamp":"2026-02-24T07:53:20.014009+0000","objects_scrubbed":32679,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:54:37.178341+0000","scrub_duration":3163,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":571998208,"num_objects":8728,"num_object_clones":0,"num_object_copies":26184,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8728,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":61294,"num_write_kb":2240704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.4","version":"88'60448","reported_seq":60793,"reported_epoch":88,"state":"active+clean","last_fresh":"2026-02-24T07:54:20.755687+0000","last_change":"2026-02-24T07:53:11.592191+0000","last_active":"2026-02-24T07:54:20.755687+0000","last_peered":"2026-02-24T07:54:20.755687+0000","last_clean":"2026-02-24T07:54:20.755687+0000","last_became_active":"2026-02-24T07:51:53.899675+0000","last_became_peered":"2026-02-24T07:51:53.899675+0000","last_unstale":"2026-02-24T07:54:20.755687+0000","last_undegraded":"2026-02-24T07:54:20.755687+0000","last_fullsized":"2026-02-24T07:54:20.755687+0000","mapping_epoch":74,"log_start":"88'60445","ondisk_log_start":"88'60445","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"84'30127","last_scrub_stamp":"2026-02-24T07:53:11.592155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_clean_scrub_stamp":"2026-02-24T07:53:11.592155+0000","objects_scrubbed":29709,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:54:23.794472+0000","scrub_duration":2432,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":576847872,"num_objects":8802,"num_object_clones":0,"num_object_copies":26406,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8802,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":60448,"num_write_kb":2216000,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"29'99","reported_seq":373,"reported_epoch":88,"state":"active+clean","last_fresh":"2026-02-24T07:54:08.521371+0000","last_change":"2026-02-24T07:54:08.521371+0000","last_active":"2026-02-24T07:54:08.521371+0000","last_peered":"2026-02-24T07:54:08.521371+0000","last_clean":"2026-02-24T07:54:08.521371+0000","last_became_active":"2026-02-24T07:42:54.186719+0000","last_became_peered":"2026-02-24T07:42:54.186719+0000","last_unstale":"2026-02-24T07:54:08.521371+0000","last_undegraded":"2026-02-24T07:54:08.521371+0000","last_fullsized":"2026-02-24T07:54:08.521371+0000","mapping_epoch":27,"log_start":"29'96","ondisk_log_start":"29'96","created":20,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'99","last_scrub_stamp":"2026-02-24T07:54:08.521342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:42:43.640776+0000","last_clean_scrub_stamp":"2026-02-24T07:54:08.521342+0000","objects_scrubbed":2,"log_size":3,"log_dups_size":96,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:55:15.605551+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":240,"num_write":135,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":16,"stat_sum":{"num_bytes":9241231384,"num_objects":141011,"num_object_clones":0,"num_object_copies":423033,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":141011,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":970129,"num_write_kb":35556417,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":27721674752,"data_stored":27721662536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":5555,"internal_metadata":0},"log_size":50,"ondisk_log_size":50,"up":48,"acting":48,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":240,"num_write":135,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":2508,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":3,"up_from":25,"seq":107374182540,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":8862840,"kb_used_data":7954596,"kb_used_omap":21,"kb_used_meta":908202,"kb_avail":725140360,"statfs":{"total":751619276800,"available":742543728640,"internally_reserved":0,"allocated":8145506304,"data_stored":8145061590,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":21632,"internal_metadata":929999744},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":19,"seq":81604378766,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":8103864,"kb_used_data":6263140,"kb_used_omap":20,"kb_used_meta":1840619,"kb_avail":725899336,"statfs":{"total":751619276800,"available":743320920064,"internally_reserved":0,"allocated":6413455360,"data_stored":6413010646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":21284,"internal_metadata":1884794076},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542287,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":6291056,"kb_used_data":5711260,"kb_used_omap":18,"kb_used_meta":579693,"kb_avail":727712144,"statfs":{"total":751619276800,"available":745177235456,"internally_reserved":0,"allocated":5848330240,"data_stored":5847893150,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":18773,"internal_metadata":593606315},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705810,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":8787128,"kb_used_data":7353956,"kb_used_omap":18,"kb_used_meta":1433069,"kb_avail":725216072,"statfs":{"total":751619276800,"available":742621257728,"internally_reserved":0,"allocated":7530450944,"data_stored":7530006230,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":18941,"internal_metadata":1467463171},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":2508,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":7505186816,"data_stored":7505182744,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":93,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":5741150208,"data_stored":5741150208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":93,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":6373838848,"data_stored":6373834776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8101498880,"data_stored":8101494808,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":5369,"internal_metadata":0}]}} 2026-02-24T07:54:23.757 INFO:tasks.cephadm.ceph_manager.ceph:increase pool size by 10 2026-02-24T07:54:23.757 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph osd pool set unique_pool_3 pg_num 66 2026-02-24T07:54:23.850 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:54:23.847+0000 7f7c30a1e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:54:23.850 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:54:23.848+0000 7f7c30a1e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:54:23.866 INFO:teuthology.orchestra.run.trial035.stdout:osd.0: {} 2026-02-24T07:54:23.866 INFO:teuthology.orchestra.run.trial035.stderr:osd.0: osd_enable_op_tracker = '' 2026-02-24T07:54:23.880 INFO:teuthology.orchestra.run.trial035.stdout:osd.1: {} 2026-02-24T07:54:23.880 INFO:teuthology.orchestra.run.trial035.stderr:osd.1: osd_enable_op_tracker = '' 2026-02-24T07:54:23.892 INFO:teuthology.orchestra.run.trial035.stdout:osd.2: {} 2026-02-24T07:54:23.893 INFO:teuthology.orchestra.run.trial035.stderr:osd.2: osd_enable_op_tracker = '' 2026-02-24T07:54:23.907 INFO:teuthology.orchestra.run.trial035.stdout:osd.3: {} 2026-02-24T07:54:23.908 INFO:teuthology.orchestra.run.trial035.stderr:osd.3: osd_enable_op_tracker = '' 2026-02-24T07:54:23.935 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:54:24.071 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:54:24.069+0000 7fe7f8236640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:54:24.071 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:54:24.069+0000 7fe7f8236640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:54:24.072 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:54:23 trial035 ceph-mon[39284]: from='client.15370 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2026-02-24T07:54:24.072 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:54:23 trial035 ceph-mon[39284]: 5.f scrub starts 2026-02-24T07:54:24.072 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:54:23 trial035 ceph-mon[39284]: pgmap v404: 17 pgs: 17 active+clean; 8.6 GiB data, 31 GiB used, 2.7 TiB / 2.7 TiB avail; 0 B/s wr, 6.11k op/s 2026-02-24T07:54:24.072 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:54:23 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/1820629245' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2026-02-24T07:54:24.211 INFO:teuthology.orchestra.run.trial035.stderr:Error EPERM: crimson_allow_pg_split is false; pg_num increase denied for crimson pool 2026-02-24T07:54:24.248 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-24T07:54:25.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:54:24 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/1892605490' entity='client.admin' cmd={"prefix": "osd pool set", "pool": "unique_pool_3", "var": "pg_num", "val": "66"} : dispatch 2026-02-24T07:54:25.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:54:24 trial035 ceph-mon[39284]: 5.f scrub ok 2026-02-24T07:54:25.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:54:24 trial035 ceph-mon[39284]: 5.4 scrub starts 2026-02-24T07:54:25.973 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2026-02-24T07:54:26.112 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:54:26.176 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:54:25 trial035 ceph-mon[39284]: 5.4 scrub ok 2026-02-24T07:54:26.176 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:54:25 trial035 ceph-mon[39284]: pgmap v405: 17 pgs: 17 active+clean; 8.6 GiB data, 31 GiB used, 2.7 TiB / 2.7 TiB avail; 0 B/s wr, 6.11k op/s 2026-02-24T07:54:26.237 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:54:26.234+0000 7f29a33bb640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:54:26.237 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:54:26.235+0000 7f29a33bb640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:54:26.250 INFO:teuthology.orchestra.run.trial035.stdout:osd.0: {} 2026-02-24T07:54:26.251 INFO:teuthology.orchestra.run.trial035.stderr:osd.0: osd_enable_op_tracker = '' 2026-02-24T07:54:26.263 INFO:teuthology.orchestra.run.trial035.stdout:osd.1: {} 2026-02-24T07:54:26.263 INFO:teuthology.orchestra.run.trial035.stderr:osd.1: osd_enable_op_tracker = '' 2026-02-24T07:54:26.274 INFO:teuthology.orchestra.run.trial035.stdout:osd.2: {} 2026-02-24T07:54:26.274 INFO:teuthology.orchestra.run.trial035.stderr:osd.2: osd_enable_op_tracker = '' 2026-02-24T07:54:26.286 INFO:teuthology.orchestra.run.trial035.stdout:osd.3: {} 2026-02-24T07:54:26.286 INFO:teuthology.orchestra.run.trial035.stderr:osd.3: osd_enable_op_tracker = '' 2026-02-24T07:54:27.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:54:26 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/2396420302' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2026-02-24T07:54:27.205 INFO:journalctl@ceph.mgr.x.trial035.stdout:Feb 24 07:54:26 trial035 ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-mgr-x[39621]: ::ffff:10.20.193.91 - - [24/Feb/2026:07:54:26] "GET /metrics HTTP/1.1" 200 4235 "" "Prometheus/3.6.0" 2026-02-24T07:54:27.356 INFO:journalctl@ceph.prometheus.a.trial091.stdout:Feb 24 07:54:26 trial091 ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-prometheus-a[66007]: time=2026-02-24T07:54:26.973Z level=ERROR source=scrape.go:1630 msg="Failed to determine correct type of scrape target." component="scrape manager" scrape_pool=ceph target=http://trial035:9283/metrics content_type="text/html;charset=utf-8" fallback_media_type="" err="received unsupported Content-Type \"text/html;charset=utf-8\" and no fallback_scrape_protocol specified for target" 2026-02-24T07:54:28.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:54:27 trial035 ceph-mon[39284]: 5.e scrub starts 2026-02-24T07:54:28.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:54:27 trial035 ceph-mon[39284]: 5.e scrub ok 2026-02-24T07:54:28.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:54:27 trial035 ceph-mon[39284]: pgmap v406: 17 pgs: 17 active+clean; 6.1 GiB data, 22 GiB used, 2.7 TiB / 2.7 TiB avail; 0 B/s wr, 9.51k op/s 2026-02-24T07:54:28.328 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2026-02-24T07:54:28.465 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:54:28.587 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:54:28.585+0000 7f409dcdb640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:54:28.588 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:54:28.585+0000 7f409dcdb640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:54:28.601 INFO:teuthology.orchestra.run.trial035.stdout:osd.0: {} 2026-02-24T07:54:28.602 INFO:teuthology.orchestra.run.trial035.stderr:osd.0: osd_enable_op_tracker = '' 2026-02-24T07:54:28.614 INFO:teuthology.orchestra.run.trial035.stdout:osd.1: {} 2026-02-24T07:54:28.615 INFO:teuthology.orchestra.run.trial035.stderr:osd.1: osd_enable_op_tracker = '' 2026-02-24T07:54:28.627 INFO:teuthology.orchestra.run.trial035.stdout:osd.2: {} 2026-02-24T07:54:28.627 INFO:teuthology.orchestra.run.trial035.stderr:osd.2: osd_enable_op_tracker = '' 2026-02-24T07:54:28.639 INFO:teuthology.orchestra.run.trial035.stdout:osd.3: {} 2026-02-24T07:54:28.639 INFO:teuthology.orchestra.run.trial035.stderr:osd.3: osd_enable_op_tracker = '' 2026-02-24T07:54:29.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:54:28 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/2856213363' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2026-02-24T07:54:29.249 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2026-02-24T07:54:29.250 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2026-02-24T07:54:29.250 INFO:tasks.thrashosds.thrasher:Growing pool unique_pool_3 2026-02-24T07:54:29.250 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph pg dump --format=json 2026-02-24T07:54:29.387 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:54:29.730 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:54:29.697+0000 7f9d992bc640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:54:29.730 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:54:29.698+0000 7f9d992bc640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:54:29.845 INFO:teuthology.orchestra.run.trial035.stdout: 2026-02-24T07:54:29.845 INFO:teuthology.orchestra.run.trial035.stderr:dumped all 2026-02-24T07:54:30.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:54:29 trial035 ceph-mon[39284]: pgmap v407: 17 pgs: 17 active+clean; 6.1 GiB data, 22 GiB used, 2.7 TiB / 2.7 TiB avail; 0 B/s wr, 6.37k op/s 2026-02-24T07:54:30.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:54:29 trial035 ceph-mon[39284]: from='client.15380 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2026-02-24T07:54:30.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:54:29 trial035 ceph-mon[39284]: from='mgr.14284 10.20.193.35:0/3681373222' entity='mgr.x' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2026-02-24T07:54:30.266 INFO:teuthology.orchestra.run.trial035.stdout:{"pg_ready":true,"pg_map":{"version":407,"stamp":"2026-02-24T07:54:29.751002+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":6572474936,"num_objects":100282,"num_object_clones":0,"num_object_copies":300846,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":100282,"num_whiteouts":0,"num_read":137,"num_read_kb":241,"num_write":1010995,"num_write_kb":35558567,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":51,"ondisk_log_size":51,"up":51,"acting":51,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":51,"num_osds":4,"num_per_pool_osds":4,"num_per_pool_omap_osds":4,"kb":2936012800,"kb_used":23376216,"kb_used_data":19526216,"kb_used_omap":78,"kb_used_meta":3849649,"kb_avail":2912636584,"statfs":{"total":3006477107200,"available":2982539862016,"internally_reserved":0,"allocated":19994845184,"data_stored":19993073952,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":80816,"internal_metadata":3942040656},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-5009768448,"num_objects":-76443,"num_object_clones":0,"num_object_copies":-229329,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-76443,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":76443,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001743"},"pg_stats":[{"pgid":"5.f","version":"88'63029","reported_seq":63415,"reported_epoch":88,"state":"active+clean","last_fresh":"2026-02-24T07:54:25.758317+0000","last_change":"2026-02-24T07:54:24.268279+0000","last_active":"2026-02-24T07:54:25.758317+0000","last_peered":"2026-02-24T07:54:25.758317+0000","last_clean":"2026-02-24T07:54:25.758317+0000","last_became_active":"2026-02-24T07:51:53.900431+0000","last_became_peered":"2026-02-24T07:51:53.900431+0000","last_unstale":"2026-02-24T07:54:25.758317+0000","last_undegraded":"2026-02-24T07:54:25.758317+0000","last_fullsized":"2026-02-24T07:54:25.758317+0000","mapping_epoch":74,"log_start":"88'63026","ondisk_log_start":"88'63026","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"88'62231","last_scrub_stamp":"2026-02-24T07:54:24.268247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_clean_scrub_stamp":"2026-02-24T07:54:24.268247+0000","objects_scrubbed":7256,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:55:31.871221+0000","scrub_duration":683,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":413073408,"num_objects":6303,"num_object_clones":0,"num_object_copies":18909,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6303,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":63029,"num_write_kb":2218624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.e","version":"88'62911","reported_seq":63199,"reported_epoch":88,"state":"active+clean","last_fresh":"2026-02-24T07:54:25.774576+0000","last_change":"2026-02-24T07:52:56.767335+0000","last_active":"2026-02-24T07:54:25.774576+0000","last_peered":"2026-02-24T07:54:25.774576+0000","last_clean":"2026-02-24T07:54:25.774576+0000","last_became_active":"2026-02-24T07:51:53.899294+0000","last_became_peered":"2026-02-24T07:51:53.899294+0000","last_unstale":"2026-02-24T07:54:25.774576+0000","last_undegraded":"2026-02-24T07:54:25.774576+0000","last_fullsized":"2026-02-24T07:54:25.774576+0000","mapping_epoch":74,"log_start":"88'62908","ondisk_log_start":"88'62908","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"83'24557","last_scrub_stamp":"2026-02-24T07:52:56.767267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_clean_scrub_stamp":"2026-02-24T07:52:56.767267+0000","objects_scrubbed":24118,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"queued for scrub","scrub_duration":2806,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":401276928,"num_objects":6123,"num_object_clones":0,"num_object_copies":18369,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6123,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":62911,"num_write_kb":2209088,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"88'63192","reported_seq":63562,"reported_epoch":88,"state":"active+clean","last_fresh":"2026-02-24T07:54:25.787156+0000","last_change":"2026-02-24T07:53:32.480551+0000","last_active":"2026-02-24T07:54:25.787156+0000","last_peered":"2026-02-24T07:54:25.787156+0000","last_clean":"2026-02-24T07:54:25.787156+0000","last_became_active":"2026-02-24T07:51:53.899772+0000","last_became_peered":"2026-02-24T07:51:53.899772+0000","last_unstale":"2026-02-24T07:54:25.787156+0000","last_undegraded":"2026-02-24T07:54:25.787156+0000","last_fullsized":"2026-02-24T07:54:25.787156+0000","mapping_epoch":74,"log_start":"88'63189","ondisk_log_start":"88'63189","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"84'37904","last_scrub_stamp":"2026-02-24T07:53:32.480499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_clean_scrub_stamp":"2026-02-24T07:53:32.480499+0000","objects_scrubbed":32149,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:54:51.939533+0000","scrub_duration":2742,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":410845208,"num_objects":6270,"num_object_clones":0,"num_object_copies":18810,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6270,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":63192,"num_write_kb":2222721,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,3],"acting":[0,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.c","version":"88'63298","reported_seq":63661,"reported_epoch":88,"state":"active+clean","last_fresh":"2026-02-24T07:54:25.862986+0000","last_change":"2026-02-24T07:53:17.083868+0000","last_active":"2026-02-24T07:54:25.862986+0000","last_peered":"2026-02-24T07:54:25.862986+0000","last_clean":"2026-02-24T07:54:25.862986+0000","last_became_active":"2026-02-24T07:51:53.898285+0000","last_became_peered":"2026-02-24T07:51:53.898285+0000","last_unstale":"2026-02-24T07:54:25.862986+0000","last_undegraded":"2026-02-24T07:54:25.862986+0000","last_fullsized":"2026-02-24T07:54:25.862986+0000","mapping_epoch":74,"log_start":"88'63295","ondisk_log_start":"88'63295","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"84'32067","last_scrub_stamp":"2026-02-24T07:53:17.083821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_clean_scrub_stamp":"2026-02-24T07:53:17.083821+0000","objects_scrubbed":31581,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:54:37.993683+0000","scrub_duration":2856,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":402915328,"num_objects":6148,"num_object_clones":0,"num_object_copies":18444,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6148,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":63298,"num_write_kb":2222272,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.b","version":"88'63167","reported_seq":63547,"reported_epoch":88,"state":"active+clean","last_fresh":"2026-02-24T07:54:25.786727+0000","last_change":"2026-02-24T07:53:30.225127+0000","last_active":"2026-02-24T07:54:25.786727+0000","last_peered":"2026-02-24T07:54:25.786727+0000","last_clean":"2026-02-24T07:54:25.786727+0000","last_became_active":"2026-02-24T07:51:53.899901+0000","last_became_peered":"2026-02-24T07:51:53.899901+0000","last_unstale":"2026-02-24T07:54:25.786727+0000","last_undegraded":"2026-02-24T07:54:25.786727+0000","last_fullsized":"2026-02-24T07:54:25.786727+0000","mapping_epoch":74,"log_start":"88'63164","ondisk_log_start":"88'63164","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"84'36879","last_scrub_stamp":"2026-02-24T07:53:30.225075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_clean_scrub_stamp":"2026-02-24T07:53:30.225075+0000","objects_scrubbed":33279,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:54:35.718496+0000","scrub_duration":3524,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":414908416,"num_objects":6331,"num_object_clones":0,"num_object_copies":18993,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6331,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":63167,"num_write_kb":2223936,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"88'63180","reported_seq":63532,"reported_epoch":88,"state":"active+clean","last_fresh":"2026-02-24T07:54:25.786815+0000","last_change":"2026-02-24T07:53:14.212339+0000","last_active":"2026-02-24T07:54:25.786815+0000","last_peered":"2026-02-24T07:54:25.786815+0000","last_clean":"2026-02-24T07:54:25.786815+0000","last_became_active":"2026-02-24T07:51:53.899549+0000","last_became_peered":"2026-02-24T07:51:53.899549+0000","last_unstale":"2026-02-24T07:54:25.786815+0000","last_undegraded":"2026-02-24T07:54:25.786815+0000","last_fullsized":"2026-02-24T07:54:25.786815+0000","mapping_epoch":74,"log_start":"88'63177","ondisk_log_start":"88'63177","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"84'30986","last_scrub_stamp":"2026-02-24T07:53:14.212265+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_clean_scrub_stamp":"2026-02-24T07:53:14.212265+0000","objects_scrubbed":30490,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:54:35.341423+0000","scrub_duration":3381,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":410517504,"num_objects":6264,"num_object_clones":0,"num_object_copies":18792,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6264,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":63180,"num_write_kb":2222208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,1],"acting":[0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.9","version":"88'63913","reported_seq":64296,"reported_epoch":88,"state":"active+clean","last_fresh":"2026-02-24T07:54:25.772086+0000","last_change":"2026-02-24T07:53:22.113835+0000","last_active":"2026-02-24T07:54:25.772086+0000","last_peered":"2026-02-24T07:54:25.772086+0000","last_clean":"2026-02-24T07:54:25.772086+0000","last_became_active":"2026-02-24T07:51:53.901610+0000","last_became_peered":"2026-02-24T07:51:53.901610+0000","last_unstale":"2026-02-24T07:54:25.772086+0000","last_undegraded":"2026-02-24T07:54:25.772086+0000","last_fullsized":"2026-02-24T07:54:25.772086+0000","mapping_epoch":74,"log_start":"88'63910","ondisk_log_start":"88'63910","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"84'34029","last_scrub_stamp":"2026-02-24T07:53:22.113774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_clean_scrub_stamp":"2026-02-24T07:53:22.113774+0000","objects_scrubbed":33520,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:54:41.903404+0000","scrub_duration":3005,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":419364864,"num_objects":6399,"num_object_clones":0,"num_object_copies":19197,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6399,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":63913,"num_write_kb":2249984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.8","version":"88'63453","reported_seq":63855,"reported_epoch":88,"state":"active+clean","last_fresh":"2026-02-24T07:54:25.782223+0000","last_change":"2026-02-24T07:54:20.727673+0000","last_active":"2026-02-24T07:54:25.782223+0000","last_peered":"2026-02-24T07:54:25.782223+0000","last_clean":"2026-02-24T07:54:25.782223+0000","last_became_active":"2026-02-24T07:51:53.900207+0000","last_became_peered":"2026-02-24T07:51:53.900207+0000","last_unstale":"2026-02-24T07:54:25.782223+0000","last_undegraded":"2026-02-24T07:54:25.782223+0000","last_fullsized":"2026-02-24T07:54:25.782223+0000","mapping_epoch":74,"log_start":"88'63450","ondisk_log_start":"88'63450","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"88'60829","last_scrub_stamp":"2026-02-24T07:54:20.727592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_clean_scrub_stamp":"2026-02-24T07:54:20.727592+0000","objects_scrubbed":9200,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:55:41.278790+0000","scrub_duration":1000,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":417398784,"num_objects":6369,"num_object_clones":0,"num_object_copies":19107,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6369,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":63453,"num_write_kb":2234304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.7","version":"88'62682","reported_seq":63125,"reported_epoch":88,"state":"active+clean","last_fresh":"2026-02-24T07:54:25.768236+0000","last_change":"2026-02-24T07:54:10.924892+0000","last_active":"2026-02-24T07:54:25.768236+0000","last_peered":"2026-02-24T07:54:25.768236+0000","last_clean":"2026-02-24T07:54:25.768236+0000","last_became_active":"2026-02-24T07:51:53.899472+0000","last_became_peered":"2026-02-24T07:51:53.899472+0000","last_unstale":"2026-02-24T07:54:25.768236+0000","last_undegraded":"2026-02-24T07:54:25.768236+0000","last_fullsized":"2026-02-24T07:54:25.768236+0000","mapping_epoch":74,"log_start":"88'62679","ondisk_log_start":"88'62679","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"88'55711","last_scrub_stamp":"2026-02-24T07:54:10.924866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_clean_scrub_stamp":"2026-02-24T07:54:10.924866+0000","objects_scrubbed":13335,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:55:37.296783+0000","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":400818176,"num_objects":6116,"num_object_clones":0,"num_object_copies":18348,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6116,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":62682,"num_write_kb":2201536,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.6","version":"88'62816","reported_seq":63279,"reported_epoch":88,"state":"active+clean","last_fresh":"2026-02-24T07:54:25.861188+0000","last_change":"2026-02-24T07:54:15.288914+0000","last_active":"2026-02-24T07:54:25.861188+0000","last_peered":"2026-02-24T07:54:25.861188+0000","last_clean":"2026-02-24T07:54:25.861188+0000","last_became_active":"2026-02-24T07:51:53.898308+0000","last_became_peered":"2026-02-24T07:51:53.898308+0000","last_unstale":"2026-02-24T07:54:25.861188+0000","last_undegraded":"2026-02-24T07:54:25.861188+0000","last_fullsized":"2026-02-24T07:54:25.861188+0000","mapping_epoch":74,"log_start":"88'62813","ondisk_log_start":"88'62813","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"88'57815","last_scrub_stamp":"2026-02-24T07:54:15.288870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_clean_scrub_stamp":"2026-02-24T07:54:15.288870+0000","objects_scrubbed":11321,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:55:31.761926+0000","scrub_duration":822,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":401735680,"num_objects":6130,"num_object_clones":0,"num_object_copies":18390,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6130,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":62816,"num_write_kb":2206272,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.5","version":"88'63092","reported_seq":63485,"reported_epoch":88,"state":"active+clean","last_fresh":"2026-02-24T07:54:25.786060+0000","last_change":"2026-02-24T07:53:26.781324+0000","last_active":"2026-02-24T07:54:25.786060+0000","last_peered":"2026-02-24T07:54:25.786060+0000","last_clean":"2026-02-24T07:54:25.786060+0000","last_became_active":"2026-02-24T07:51:53.899749+0000","last_became_peered":"2026-02-24T07:51:53.899749+0000","last_unstale":"2026-02-24T07:54:25.786060+0000","last_undegraded":"2026-02-24T07:54:25.786060+0000","last_fullsized":"2026-02-24T07:54:25.786060+0000","mapping_epoch":74,"log_start":"88'63089","ondisk_log_start":"88'63089","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"84'35538","last_scrub_stamp":"2026-02-24T07:53:26.781276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_clean_scrub_stamp":"2026-02-24T07:53:26.781276+0000","objects_scrubbed":34358,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:54:37.440161+0000","scrub_duration":2982,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":413925376,"num_objects":6316,"num_object_clones":0,"num_object_copies":18948,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6316,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":63092,"num_write_kb":2221056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"88'63130","reported_seq":63452,"reported_epoch":88,"state":"active+clean","last_fresh":"2026-02-24T07:54:25.786640+0000","last_change":"2026-02-24T07:53:05.312364+0000","last_active":"2026-02-24T07:54:25.786640+0000","last_peered":"2026-02-24T07:54:25.786640+0000","last_clean":"2026-02-24T07:54:25.786640+0000","last_became_active":"2026-02-24T07:51:53.900042+0000","last_became_peered":"2026-02-24T07:51:53.900042+0000","last_unstale":"2026-02-24T07:54:25.786640+0000","last_undegraded":"2026-02-24T07:54:25.786640+0000","last_fullsized":"2026-02-24T07:54:25.786640+0000","mapping_epoch":74,"log_start":"88'63127","ondisk_log_start":"88'63127","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"84'27738","last_scrub_stamp":"2026-02-24T07:53:05.312312+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_clean_scrub_stamp":"2026-02-24T07:53:05.312312+0000","objects_scrubbed":27245,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:54:32.166518+0000","scrub_duration":2472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":415367168,"num_objects":6338,"num_object_clones":0,"num_object_copies":19014,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6338,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":63130,"num_write_kb":2222976,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.2","version":"88'63832","reported_seq":64200,"reported_epoch":88,"state":"active+clean","last_fresh":"2026-02-24T07:54:25.774772+0000","last_change":"2026-02-24T07:53:17.780799+0000","last_active":"2026-02-24T07:54:25.774772+0000","last_peered":"2026-02-24T07:54:25.774772+0000","last_clean":"2026-02-24T07:54:25.774772+0000","last_became_active":"2026-02-24T07:51:53.900096+0000","last_became_peered":"2026-02-24T07:51:53.900096+0000","last_unstale":"2026-02-24T07:54:25.774772+0000","last_undegraded":"2026-02-24T07:54:25.774772+0000","last_fullsized":"2026-02-24T07:54:25.774772+0000","mapping_epoch":74,"log_start":"88'63829","ondisk_log_start":"88'63829","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"84'32548","last_scrub_stamp":"2026-02-24T07:53:17.780705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_clean_scrub_stamp":"2026-02-24T07:53:17.780705+0000","objects_scrubbed":32030,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:54:35.344780+0000","scrub_duration":2741,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":412221440,"num_objects":6290,"num_object_clones":0,"num_object_copies":18870,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6290,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":63832,"num_write_kb":2243904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"88'62447","reported_seq":62796,"reported_epoch":88,"state":"active+clean","last_fresh":"2026-02-24T07:54:25.774827+0000","last_change":"2026-02-24T07:53:14.051194+0000","last_active":"2026-02-24T07:54:25.774827+0000","last_peered":"2026-02-24T07:54:25.774827+0000","last_clean":"2026-02-24T07:54:25.774827+0000","last_became_active":"2026-02-24T07:51:53.901027+0000","last_became_peered":"2026-02-24T07:51:53.901027+0000","last_unstale":"2026-02-24T07:54:25.774827+0000","last_undegraded":"2026-02-24T07:54:25.774827+0000","last_fullsized":"2026-02-24T07:54:25.774827+0000","mapping_epoch":74,"log_start":"88'62444","ondisk_log_start":"88'62444","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"84'30565","last_scrub_stamp":"2026-02-24T07:53:14.051154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_clean_scrub_stamp":"2026-02-24T07:53:14.051154+0000","objects_scrubbed":30140,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:54:38.448076+0000","scrub_duration":3147,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":414777344,"num_objects":6329,"num_object_clones":0,"num_object_copies":18987,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6329,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":62447,"num_write_kb":2200832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.0","version":"88'63809","reported_seq":64184,"reported_epoch":88,"state":"active+clean","last_fresh":"2026-02-24T07:54:25.785296+0000","last_change":"2026-02-24T07:53:20.014077+0000","last_active":"2026-02-24T07:54:25.785296+0000","last_peered":"2026-02-24T07:54:25.785296+0000","last_clean":"2026-02-24T07:54:25.785296+0000","last_became_active":"2026-02-24T07:51:53.898551+0000","last_became_peered":"2026-02-24T07:51:53.898551+0000","last_unstale":"2026-02-24T07:54:25.785296+0000","last_undegraded":"2026-02-24T07:54:25.785296+0000","last_fullsized":"2026-02-24T07:54:25.785296+0000","mapping_epoch":74,"log_start":"88'63806","ondisk_log_start":"88'63806","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"84'33261","last_scrub_stamp":"2026-02-24T07:53:20.014009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_clean_scrub_stamp":"2026-02-24T07:53:20.014009+0000","objects_scrubbed":32679,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:54:37.178341+0000","scrub_duration":3163,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":407175168,"num_objects":6213,"num_object_clones":0,"num_object_copies":18639,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6213,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":63809,"num_write_kb":2240704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.4","version":"88'62909","reported_seq":63331,"reported_epoch":88,"state":"active+clean","last_fresh":"2026-02-24T07:54:25.755836+0000","last_change":"2026-02-24T07:54:25.250263+0000","last_active":"2026-02-24T07:54:25.755836+0000","last_peered":"2026-02-24T07:54:25.755836+0000","last_clean":"2026-02-24T07:54:25.755836+0000","last_became_active":"2026-02-24T07:51:53.899675+0000","last_became_peered":"2026-02-24T07:51:53.899675+0000","last_unstale":"2026-02-24T07:54:25.755836+0000","last_undegraded":"2026-02-24T07:54:25.755836+0000","last_fullsized":"2026-02-24T07:54:25.755836+0000","mapping_epoch":74,"log_start":"88'62906","ondisk_log_start":"88'62906","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"88'62638","last_scrub_stamp":"2026-02-24T07:54:25.250228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_clean_scrub_stamp":"2026-02-24T07:54:25.250228+0000","objects_scrubbed":6771,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:55:42.796366+0000","scrub_duration":618,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":415563776,"num_objects":6341,"num_object_clones":0,"num_object_copies":19023,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6341,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":62909,"num_write_kb":2216000,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"29'99","reported_seq":373,"reported_epoch":88,"state":"active+clean","last_fresh":"2026-02-24T07:54:08.521371+0000","last_change":"2026-02-24T07:54:08.521371+0000","last_active":"2026-02-24T07:54:08.521371+0000","last_peered":"2026-02-24T07:54:08.521371+0000","last_clean":"2026-02-24T07:54:08.521371+0000","last_became_active":"2026-02-24T07:42:54.186719+0000","last_became_peered":"2026-02-24T07:42:54.186719+0000","last_unstale":"2026-02-24T07:54:08.521371+0000","last_undegraded":"2026-02-24T07:54:08.521371+0000","last_fullsized":"2026-02-24T07:54:08.521371+0000","mapping_epoch":27,"log_start":"29'96","ondisk_log_start":"29'96","created":20,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'99","last_scrub_stamp":"2026-02-24T07:54:08.521342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:42:43.640776+0000","last_clean_scrub_stamp":"2026-02-24T07:54:08.521342+0000","objects_scrubbed":2,"log_size":3,"log_dups_size":96,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:55:15.605551+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":240,"num_write":135,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":16,"stat_sum":{"num_bytes":6571884568,"num_objects":100280,"num_object_clones":0,"num_object_copies":300840,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":100280,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1010860,"num_write_kb":35556417,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":19709571072,"data_stored":19709558856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":5741,"internal_metadata":0},"log_size":48,"ondisk_log_size":48,"up":48,"acting":48,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":240,"num_write":135,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":2508,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":3,"up_from":25,"seq":107374182541,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":6646776,"kb_used_data":5666340,"kb_used_omap":21,"kb_used_meta":980394,"kb_avail":727356424,"statfs":{"total":751619276800,"available":744812978176,"internally_reserved":0,"allocated":5802332160,"data_stored":5801887446,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":21632,"internal_metadata":1003924352},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":19,"seq":81604378767,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":5245176,"kb_used_data":4496868,"kb_used_omap":20,"kb_used_meta":748203,"kb_avail":728758024,"statfs":{"total":751619276800,"available":746248216576,"internally_reserved":0,"allocated":4604792832,"data_stored":4604348118,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":21470,"internal_metadata":766159906},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542288,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":4720048,"kb_used_data":4104156,"kb_used_omap":18,"kb_used_meta":615789,"kb_avail":729283152,"statfs":{"total":751619276800,"available":746785947648,"internally_reserved":0,"allocated":4202655744,"data_stored":4202218654,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":18773,"internal_metadata":630568619},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705811,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":6764216,"kb_used_data":5258852,"kb_used_omap":18,"kb_used_meta":1505261,"kb_avail":727238984,"statfs":{"total":751619276800,"available":744692719616,"internally_reserved":0,"allocated":5385064448,"data_stored":5384619734,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":18941,"internal_metadata":1541387779},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":2508,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":5334700032,"data_stored":5334695960,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":93,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4066246656,"data_stored":4066246656,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":93,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4542042112,"data_stored":4542038040,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":186,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":5766582272,"data_stored":5766578200,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":5369,"internal_metadata":0}]}} 2026-02-24T07:54:30.268 INFO:tasks.cephadm.ceph_manager.ceph:increase pool size by 10 2026-02-24T07:54:30.269 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph osd pool set unique_pool_3 pg_num 76 2026-02-24T07:54:30.406 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:54:30.521 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:54:30.520+0000 7f0044caa640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:54:30.521 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:54:30.520+0000 7f0044caa640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:54:30.659 INFO:teuthology.orchestra.run.trial035.stderr:Error EPERM: crimson_allow_pg_split is false; pg_num increase denied for crimson pool 2026-02-24T07:54:30.679 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2026-02-24T07:54:30.695 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-24T07:54:30.818 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:54:30.939 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:54:30.938+0000 7f512eee5640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:54:30.939 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:54:30.938+0000 7f512eee5640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:54:30.955 INFO:teuthology.orchestra.run.trial035.stdout:osd.0: {} 2026-02-24T07:54:30.955 INFO:teuthology.orchestra.run.trial035.stderr:osd.0: osd_enable_op_tracker = '' 2026-02-24T07:54:30.968 INFO:teuthology.orchestra.run.trial035.stdout:osd.1: {} 2026-02-24T07:54:30.968 INFO:teuthology.orchestra.run.trial035.stderr:osd.1: osd_enable_op_tracker = '' 2026-02-24T07:54:30.982 INFO:teuthology.orchestra.run.trial035.stdout:osd.2: {} 2026-02-24T07:54:30.982 INFO:teuthology.orchestra.run.trial035.stderr:osd.2: osd_enable_op_tracker = '' 2026-02-24T07:54:30.994 INFO:teuthology.orchestra.run.trial035.stdout:osd.3: {} 2026-02-24T07:54:30.994 INFO:teuthology.orchestra.run.trial035.stderr:osd.3: osd_enable_op_tracker = '' 2026-02-24T07:54:31.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:54:30 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/1684612416' entity='client.admin' cmd={"prefix": "osd pool set", "pool": "unique_pool_3", "var": "pg_num", "val": "76"} : dispatch 2026-02-24T07:54:32.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:54:31 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/3446151581' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2026-02-24T07:54:32.205 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:54:31 trial035 ceph-mon[39284]: pgmap v408: 17 pgs: 17 active+clean; 3.7 GiB data, 15 GiB used, 2.7 TiB / 2.7 TiB avail; 0 B/s wr, 9.69k op/s 2026-02-24T07:54:33.032 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2026-02-24T07:54:33.167 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:54:33.284 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:54:33.283+0000 7fa2b6cf9640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:54:33.284 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:54:33.283+0000 7fa2b6cf9640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:54:33.300 INFO:teuthology.orchestra.run.trial035.stdout:osd.0: {} 2026-02-24T07:54:33.301 INFO:teuthology.orchestra.run.trial035.stderr:osd.0: osd_enable_op_tracker = '' 2026-02-24T07:54:33.312 INFO:teuthology.orchestra.run.trial035.stdout:osd.1: {} 2026-02-24T07:54:33.312 INFO:teuthology.orchestra.run.trial035.stderr:osd.1: osd_enable_op_tracker = '' 2026-02-24T07:54:33.324 INFO:teuthology.orchestra.run.trial035.stdout:osd.2: {} 2026-02-24T07:54:33.324 INFO:teuthology.orchestra.run.trial035.stderr:osd.2: osd_enable_op_tracker = '' 2026-02-24T07:54:33.336 INFO:teuthology.orchestra.run.trial035.stdout:osd.3: {} 2026-02-24T07:54:33.336 INFO:teuthology.orchestra.run.trial035.stderr:osd.3: osd_enable_op_tracker = '' 2026-02-24T07:54:33.704 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:54:33 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/4058120868' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2026-02-24T07:54:35.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:54:34 trial035 ceph-mon[39284]: pgmap v409: 17 pgs: 17 active+clean; 3.7 GiB data, 15 GiB used, 2.7 TiB / 2.7 TiB avail; 0 B/s wr, 6.71k op/s 2026-02-24T07:54:35.379 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2026-02-24T07:54:35.520 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:54:35.697 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2026-02-24T07:54:35.697 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2026-02-24T07:54:35.697 INFO:tasks.thrashosds.thrasher:Growing pool unique_pool_3 2026-02-24T07:54:35.697 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph pg dump --format=json 2026-02-24T07:54:35.828 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:54:35.977 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:54:35 trial035 ceph-mon[39284]: 5.3 scrub starts 2026-02-24T07:54:35.977 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:54:35 trial035 ceph-mon[39284]: 5.3 scrub ok 2026-02-24T07:54:36.163 INFO:teuthology.orchestra.run.trial035.stdout:osd.0: {} 2026-02-24T07:54:36.163 INFO:teuthology.orchestra.run.trial035.stdout:osd.1: {} 2026-02-24T07:54:36.163 INFO:teuthology.orchestra.run.trial035.stdout:osd.2: {} 2026-02-24T07:54:36.163 INFO:teuthology.orchestra.run.trial035.stdout:osd.3: {} 2026-02-24T07:54:36.163 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:54:35.967+0000 7fab73130640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:54:36.163 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:54:35.967+0000 7fab73130640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:54:36.163 INFO:teuthology.orchestra.run.trial035.stderr:osd.0: osd_enable_op_tracker = '' 2026-02-24T07:54:36.163 INFO:teuthology.orchestra.run.trial035.stderr:osd.1: osd_enable_op_tracker = '' 2026-02-24T07:54:36.163 INFO:teuthology.orchestra.run.trial035.stderr:osd.2: osd_enable_op_tracker = '' 2026-02-24T07:54:36.163 INFO:teuthology.orchestra.run.trial035.stderr:osd.3: osd_enable_op_tracker = '' 2026-02-24T07:54:36.958 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:54:36.957+0000 7f31e61d1640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:54:36.958 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:54:36.957+0000 7f31e61d1640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:54:37.094 INFO:teuthology.orchestra.run.trial035.stdout: 2026-02-24T07:54:37.095 INFO:teuthology.orchestra.run.trial035.stderr:dumped all 2026-02-24T07:54:37.095 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:54:36 trial035 ceph-mon[39284]: pgmap v410: 17 pgs: 17 active+clean; 3.7 GiB data, 15 GiB used, 2.7 TiB / 2.7 TiB avail; 0 B/s wr, 6.71k op/s 2026-02-24T07:54:37.096 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:54:36 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/3162393482' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2026-02-24T07:54:37.096 INFO:journalctl@ceph.mgr.x.trial035.stdout:Feb 24 07:54:36 trial035 ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-mgr-x[39621]: ::ffff:10.20.193.91 - - [24/Feb/2026:07:54:36] "GET /metrics HTTP/1.1" 200 4233 "" "Prometheus/3.6.0" 2026-02-24T07:54:37.136 INFO:teuthology.orchestra.run.trial035.stdout:{"pg_ready":true,"pg_map":{"version":410,"stamp":"2026-02-24T07:54:35.751783+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":3960865336,"num_objects":60432,"num_object_clones":0,"num_object_copies":181296,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60432,"num_whiteouts":0,"num_read":137,"num_read_kb":241,"num_write":1050845,"num_write_kb":35558567,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":54,"ondisk_log_size":54,"up":51,"acting":51,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":51,"num_osds":4,"num_per_pool_osds":4,"num_per_pool_omap_osds":4,"kb":2936012800,"kb_used":15952152,"kb_used_data":11885640,"kb_used_omap":79,"kb_used_meta":4066224,"kb_avail":2920060648,"statfs":{"total":3006477107200,"available":2990142103552,"internally_reserved":0,"allocated":12170895360,"data_stored":12169124128,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":80903,"internal_metadata":4163814393},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,5],"upper_bound":4},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-5280956416,"num_objects":-80581,"num_object_clones":0,"num_object_copies":-241743,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-80581,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":80581,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001628"},"pg_stats":[{"pgid":"5.f","version":"88'65536","reported_seq":65922,"reported_epoch":88,"state":"active+clean","last_fresh":"2026-02-24T07:54:30.757819+0000","last_change":"2026-02-24T07:54:24.268279+0000","last_active":"2026-02-24T07:54:30.757819+0000","last_peered":"2026-02-24T07:54:30.757819+0000","last_clean":"2026-02-24T07:54:30.757819+0000","last_became_active":"2026-02-24T07:51:53.900431+0000","last_became_peered":"2026-02-24T07:51:53.900431+0000","last_unstale":"2026-02-24T07:54:30.757819+0000","last_undegraded":"2026-02-24T07:54:30.757819+0000","last_fullsized":"2026-02-24T07:54:30.757819+0000","mapping_epoch":74,"log_start":"88'65533","ondisk_log_start":"88'65533","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"88'62231","last_scrub_stamp":"2026-02-24T07:54:24.268247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_clean_scrub_stamp":"2026-02-24T07:54:24.268247+0000","objects_scrubbed":7256,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:55:31.871221+0000","scrub_duration":683,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":248774656,"num_objects":3796,"num_object_clones":0,"num_object_copies":11388,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3796,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":65536,"num_write_kb":2218624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.e","version":"88'65341","reported_seq":65695,"reported_epoch":88,"state":"active+clean","last_fresh":"2026-02-24T07:54:30.770984+0000","last_change":"2026-02-24T07:54:27.217859+0000","last_active":"2026-02-24T07:54:30.770984+0000","last_peered":"2026-02-24T07:54:30.770984+0000","last_clean":"2026-02-24T07:54:30.770984+0000","last_became_active":"2026-02-24T07:51:53.899294+0000","last_became_peered":"2026-02-24T07:51:53.899294+0000","last_unstale":"2026-02-24T07:54:30.770984+0000","last_undegraded":"2026-02-24T07:54:30.770984+0000","last_fullsized":"2026-02-24T07:54:30.770984+0000","mapping_epoch":74,"log_start":"88'65338","ondisk_log_start":"88'65338","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"88'63696","last_scrub_stamp":"2026-02-24T07:54:27.217834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_clean_scrub_stamp":"2026-02-24T07:54:27.217834+0000","objects_scrubbed":5460,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:55:43.881366+0000","scrub_duration":493,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":242024448,"num_objects":3693,"num_object_clones":0,"num_object_copies":11079,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3693,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":65341,"num_write_kb":2209088,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"88'65747","reported_seq":66117,"reported_epoch":88,"state":"active+clean","last_fresh":"2026-02-24T07:54:30.786975+0000","last_change":"2026-02-24T07:53:32.480551+0000","last_active":"2026-02-24T07:54:30.786975+0000","last_peered":"2026-02-24T07:54:30.786975+0000","last_clean":"2026-02-24T07:54:30.786975+0000","last_became_active":"2026-02-24T07:51:53.899772+0000","last_became_peered":"2026-02-24T07:51:53.899772+0000","last_unstale":"2026-02-24T07:54:30.786975+0000","last_undegraded":"2026-02-24T07:54:30.786975+0000","last_fullsized":"2026-02-24T07:54:30.786975+0000","mapping_epoch":74,"log_start":"88'65744","ondisk_log_start":"88'65744","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"84'37904","last_scrub_stamp":"2026-02-24T07:53:32.480499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_clean_scrub_stamp":"2026-02-24T07:53:32.480499+0000","objects_scrubbed":32149,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:54:51.939533+0000","scrub_duration":2742,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":243400728,"num_objects":3715,"num_object_clones":0,"num_object_copies":11145,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3715,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":65747,"num_write_kb":2222721,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,3],"acting":[0,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.c","version":"88'65823","reported_seq":66186,"reported_epoch":88,"state":"active+clean","last_fresh":"2026-02-24T07:54:30.859245+0000","last_change":"2026-02-24T07:53:17.083868+0000","last_active":"2026-02-24T07:54:30.859245+0000","last_peered":"2026-02-24T07:54:30.859245+0000","last_clean":"2026-02-24T07:54:30.859245+0000","last_became_active":"2026-02-24T07:51:53.898285+0000","last_became_peered":"2026-02-24T07:51:53.898285+0000","last_unstale":"2026-02-24T07:54:30.859245+0000","last_undegraded":"2026-02-24T07:54:30.859245+0000","last_fullsized":"2026-02-24T07:54:30.859245+0000","mapping_epoch":74,"log_start":"88'65820","ondisk_log_start":"88'65820","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"84'32067","last_scrub_stamp":"2026-02-24T07:53:17.083821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_clean_scrub_stamp":"2026-02-24T07:53:17.083821+0000","objects_scrubbed":31581,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:54:37.993683+0000","scrub_duration":2856,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":237436928,"num_objects":3623,"num_object_clones":0,"num_object_copies":10869,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3623,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":65823,"num_write_kb":2222272,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.b","version":"88'65644","reported_seq":66024,"reported_epoch":88,"state":"active+clean","last_fresh":"2026-02-24T07:54:30.781059+0000","last_change":"2026-02-24T07:53:30.225127+0000","last_active":"2026-02-24T07:54:30.781059+0000","last_peered":"2026-02-24T07:54:30.781059+0000","last_clean":"2026-02-24T07:54:30.781059+0000","last_became_active":"2026-02-24T07:51:53.899901+0000","last_became_peered":"2026-02-24T07:51:53.899901+0000","last_unstale":"2026-02-24T07:54:30.781059+0000","last_undegraded":"2026-02-24T07:54:30.781059+0000","last_fullsized":"2026-02-24T07:54:30.781059+0000","mapping_epoch":74,"log_start":"88'65641","ondisk_log_start":"88'65641","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"84'36879","last_scrub_stamp":"2026-02-24T07:53:30.225075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_clean_scrub_stamp":"2026-02-24T07:53:30.225075+0000","objects_scrubbed":33279,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:54:35.718496+0000","scrub_duration":3524,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":252575744,"num_objects":3854,"num_object_clones":0,"num_object_copies":11562,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3854,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":65644,"num_write_kb":2223936,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"88'65666","reported_seq":66014,"reported_epoch":88,"state":"active+clean","last_fresh":"2026-02-24T07:54:30.789716+0000","last_change":"2026-02-24T07:53:14.212339+0000","last_active":"2026-02-24T07:54:30.789716+0000","last_peered":"2026-02-24T07:54:30.789716+0000","last_clean":"2026-02-24T07:54:30.789716+0000","last_became_active":"2026-02-24T07:51:53.899549+0000","last_became_peered":"2026-02-24T07:51:53.899549+0000","last_unstale":"2026-02-24T07:54:30.789716+0000","last_undegraded":"2026-02-24T07:54:30.789716+0000","last_fullsized":"2026-02-24T07:54:30.789716+0000","mapping_epoch":74,"log_start":"88'65660","ondisk_log_start":"88'65660","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"84'30986","last_scrub_stamp":"2026-02-24T07:53:14.212265+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_clean_scrub_stamp":"2026-02-24T07:53:14.212265+0000","objects_scrubbed":30490,"log_size":6,"log_dups_size":3000,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:54:35.341423+0000","scrub_duration":3381,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":247595008,"num_objects":3778,"num_object_clones":0,"num_object_copies":11334,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3778,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":65666,"num_write_kb":2222208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,1],"acting":[0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.9","version":"88'66389","reported_seq":66772,"reported_epoch":88,"state":"active+clean","last_fresh":"2026-02-24T07:54:30.771901+0000","last_change":"2026-02-24T07:53:22.113835+0000","last_active":"2026-02-24T07:54:30.771901+0000","last_peered":"2026-02-24T07:54:30.771901+0000","last_clean":"2026-02-24T07:54:30.771901+0000","last_became_active":"2026-02-24T07:51:53.901610+0000","last_became_peered":"2026-02-24T07:51:53.901610+0000","last_unstale":"2026-02-24T07:54:30.771901+0000","last_undegraded":"2026-02-24T07:54:30.771901+0000","last_fullsized":"2026-02-24T07:54:30.771901+0000","mapping_epoch":74,"log_start":"88'66386","ondisk_log_start":"88'66386","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"84'34029","last_scrub_stamp":"2026-02-24T07:53:22.113774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_clean_scrub_stamp":"2026-02-24T07:53:22.113774+0000","objects_scrubbed":33520,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:54:41.903404+0000","scrub_duration":3005,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":257097728,"num_objects":3923,"num_object_clones":0,"num_object_copies":11769,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3923,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":66389,"num_write_kb":2249984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.8","version":"88'65993","reported_seq":66394,"reported_epoch":88,"state":"active+clean","last_fresh":"2026-02-24T07:54:30.785041+0000","last_change":"2026-02-24T07:54:20.727673+0000","last_active":"2026-02-24T07:54:30.785041+0000","last_peered":"2026-02-24T07:54:30.785041+0000","last_clean":"2026-02-24T07:54:30.785041+0000","last_became_active":"2026-02-24T07:51:53.900207+0000","last_became_peered":"2026-02-24T07:51:53.900207+0000","last_unstale":"2026-02-24T07:54:30.785041+0000","last_undegraded":"2026-02-24T07:54:30.785041+0000","last_fullsized":"2026-02-24T07:54:30.785041+0000","mapping_epoch":74,"log_start":"88'65990","ondisk_log_start":"88'65990","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"88'60829","last_scrub_stamp":"2026-02-24T07:54:20.727592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_clean_scrub_stamp":"2026-02-24T07:54:20.727592+0000","objects_scrubbed":9200,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:55:41.278790+0000","scrub_duration":1000,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":250937344,"num_objects":3829,"num_object_clones":0,"num_object_copies":11487,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3829,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":65993,"num_write_kb":2234304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.7","version":"88'65131","reported_seq":65574,"reported_epoch":88,"state":"active+clean","last_fresh":"2026-02-24T07:54:30.775681+0000","last_change":"2026-02-24T07:54:10.924892+0000","last_active":"2026-02-24T07:54:30.775681+0000","last_peered":"2026-02-24T07:54:30.775681+0000","last_clean":"2026-02-24T07:54:30.775681+0000","last_became_active":"2026-02-24T07:51:53.899472+0000","last_became_peered":"2026-02-24T07:51:53.899472+0000","last_unstale":"2026-02-24T07:54:30.775681+0000","last_undegraded":"2026-02-24T07:54:30.775681+0000","last_fullsized":"2026-02-24T07:54:30.775681+0000","mapping_epoch":74,"log_start":"88'65128","ondisk_log_start":"88'65128","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"88'55711","last_scrub_stamp":"2026-02-24T07:54:10.924866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_clean_scrub_stamp":"2026-02-24T07:54:10.924866+0000","objects_scrubbed":13335,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:55:37.296783+0000","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":240320512,"num_objects":3667,"num_object_clones":0,"num_object_copies":11001,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3667,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":65131,"num_write_kb":2201536,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.6","version":"88'65298","reported_seq":65761,"reported_epoch":88,"state":"active+clean","last_fresh":"2026-02-24T07:54:30.862317+0000","last_change":"2026-02-24T07:54:15.288914+0000","last_active":"2026-02-24T07:54:30.862317+0000","last_peered":"2026-02-24T07:54:30.862317+0000","last_clean":"2026-02-24T07:54:30.862317+0000","last_became_active":"2026-02-24T07:51:53.898308+0000","last_became_peered":"2026-02-24T07:51:53.898308+0000","last_unstale":"2026-02-24T07:54:30.862317+0000","last_undegraded":"2026-02-24T07:54:30.862317+0000","last_fullsized":"2026-02-24T07:54:30.862317+0000","mapping_epoch":74,"log_start":"88'65295","ondisk_log_start":"88'65295","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"88'57815","last_scrub_stamp":"2026-02-24T07:54:15.288870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_clean_scrub_stamp":"2026-02-24T07:54:15.288870+0000","objects_scrubbed":11321,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:55:31.761926+0000","scrub_duration":822,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":239075328,"num_objects":3648,"num_object_clones":0,"num_object_copies":10944,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3648,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":65298,"num_write_kb":2206272,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.5","version":"88'65532","reported_seq":65925,"reported_epoch":88,"state":"active+clean","last_fresh":"2026-02-24T07:54:30.784009+0000","last_change":"2026-02-24T07:53:26.781324+0000","last_active":"2026-02-24T07:54:30.784009+0000","last_peered":"2026-02-24T07:54:30.784009+0000","last_clean":"2026-02-24T07:54:30.784009+0000","last_became_active":"2026-02-24T07:51:53.899749+0000","last_became_peered":"2026-02-24T07:51:53.899749+0000","last_unstale":"2026-02-24T07:54:30.784009+0000","last_undegraded":"2026-02-24T07:54:30.784009+0000","last_fullsized":"2026-02-24T07:54:30.784009+0000","mapping_epoch":74,"log_start":"88'65529","ondisk_log_start":"88'65529","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"84'35538","last_scrub_stamp":"2026-02-24T07:53:26.781276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_clean_scrub_stamp":"2026-02-24T07:53:26.781276+0000","objects_scrubbed":34358,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:54:37.440161+0000","scrub_duration":2982,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":254017536,"num_objects":3876,"num_object_clones":0,"num_object_copies":11628,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3876,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":65532,"num_write_kb":2221056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"88'65615","reported_seq":65937,"reported_epoch":88,"state":"active+clean","last_fresh":"2026-02-24T07:54:30.780136+0000","last_change":"2026-02-24T07:53:05.312364+0000","last_active":"2026-02-24T07:54:30.780136+0000","last_peered":"2026-02-24T07:54:30.780136+0000","last_clean":"2026-02-24T07:54:30.780136+0000","last_became_active":"2026-02-24T07:51:53.900042+0000","last_became_peered":"2026-02-24T07:51:53.900042+0000","last_unstale":"2026-02-24T07:54:30.780136+0000","last_undegraded":"2026-02-24T07:54:30.780136+0000","last_fullsized":"2026-02-24T07:54:30.780136+0000","mapping_epoch":74,"log_start":"88'65612","ondisk_log_start":"88'65612","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"84'27738","last_scrub_stamp":"2026-02-24T07:53:05.312312+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_clean_scrub_stamp":"2026-02-24T07:53:05.312312+0000","objects_scrubbed":27245,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:54:32.166518+0000","scrub_duration":2472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":252510208,"num_objects":3853,"num_object_clones":0,"num_object_copies":11559,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3853,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":65615,"num_write_kb":2222976,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.2","version":"88'66304","reported_seq":66672,"reported_epoch":88,"state":"active+clean","last_fresh":"2026-02-24T07:54:30.771119+0000","last_change":"2026-02-24T07:53:17.780799+0000","last_active":"2026-02-24T07:54:30.771119+0000","last_peered":"2026-02-24T07:54:30.771119+0000","last_clean":"2026-02-24T07:54:30.771119+0000","last_became_active":"2026-02-24T07:51:53.900096+0000","last_became_peered":"2026-02-24T07:51:53.900096+0000","last_unstale":"2026-02-24T07:54:30.771119+0000","last_undegraded":"2026-02-24T07:54:30.771119+0000","last_fullsized":"2026-02-24T07:54:30.771119+0000","mapping_epoch":74,"log_start":"88'66301","ondisk_log_start":"88'66301","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"84'32548","last_scrub_stamp":"2026-02-24T07:53:17.780705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_clean_scrub_stamp":"2026-02-24T07:53:17.780705+0000","objects_scrubbed":32030,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:54:35.344780+0000","scrub_duration":2741,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":250216448,"num_objects":3818,"num_object_clones":0,"num_object_copies":11454,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3818,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":66304,"num_write_kb":2243904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"88'64951","reported_seq":65301,"reported_epoch":88,"state":"active+clean","last_fresh":"2026-02-24T07:54:30.774426+0000","last_change":"2026-02-24T07:53:14.051194+0000","last_active":"2026-02-24T07:54:30.774426+0000","last_peered":"2026-02-24T07:54:30.774426+0000","last_clean":"2026-02-24T07:54:30.774426+0000","last_became_active":"2026-02-24T07:51:53.901027+0000","last_became_peered":"2026-02-24T07:51:53.901027+0000","last_unstale":"2026-02-24T07:54:30.774426+0000","last_undegraded":"2026-02-24T07:54:30.774426+0000","last_fullsized":"2026-02-24T07:54:30.774426+0000","mapping_epoch":74,"log_start":"88'64948","ondisk_log_start":"88'64948","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"84'30565","last_scrub_stamp":"2026-02-24T07:53:14.051154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_clean_scrub_stamp":"2026-02-24T07:53:14.051154+0000","objects_scrubbed":30140,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:54:38.448076+0000","scrub_duration":3147,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":250675200,"num_objects":3825,"num_object_clones":0,"num_object_copies":11475,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3825,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":64951,"num_write_kb":2200832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.0","version":"88'66282","reported_seq":66656,"reported_epoch":88,"state":"active+clean","last_fresh":"2026-02-24T07:54:30.787702+0000","last_change":"2026-02-24T07:53:20.014077+0000","last_active":"2026-02-24T07:54:30.787702+0000","last_peered":"2026-02-24T07:54:30.787702+0000","last_clean":"2026-02-24T07:54:30.787702+0000","last_became_active":"2026-02-24T07:51:53.898551+0000","last_became_peered":"2026-02-24T07:51:53.898551+0000","last_unstale":"2026-02-24T07:54:30.787702+0000","last_undegraded":"2026-02-24T07:54:30.787702+0000","last_fullsized":"2026-02-24T07:54:30.787702+0000","mapping_epoch":74,"log_start":"88'66279","ondisk_log_start":"88'66279","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"84'33261","last_scrub_stamp":"2026-02-24T07:53:20.014009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_clean_scrub_stamp":"2026-02-24T07:53:20.014009+0000","objects_scrubbed":32679,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:54:37.178341+0000","scrub_duration":3163,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":245104640,"num_objects":3740,"num_object_clones":0,"num_object_copies":11220,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3740,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":66282,"num_write_kb":2240704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.4","version":"88'65458","reported_seq":65880,"reported_epoch":88,"state":"active+clean","last_fresh":"2026-02-24T07:54:30.759440+0000","last_change":"2026-02-24T07:54:25.250263+0000","last_active":"2026-02-24T07:54:30.759440+0000","last_peered":"2026-02-24T07:54:30.759440+0000","last_clean":"2026-02-24T07:54:30.759440+0000","last_became_active":"2026-02-24T07:51:53.899675+0000","last_became_peered":"2026-02-24T07:51:53.899675+0000","last_unstale":"2026-02-24T07:54:30.759440+0000","last_undegraded":"2026-02-24T07:54:30.759440+0000","last_fullsized":"2026-02-24T07:54:30.759440+0000","mapping_epoch":74,"log_start":"88'65455","ondisk_log_start":"88'65455","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"88'62638","last_scrub_stamp":"2026-02-24T07:54:25.250228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:51:52.890170+0000","last_clean_scrub_stamp":"2026-02-24T07:54:25.250228+0000","objects_scrubbed":6771,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:55:42.796366+0000","scrub_duration":618,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":248512512,"num_objects":3792,"num_object_clones":0,"num_object_copies":11376,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3792,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":65458,"num_write_kb":2216000,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"29'99","reported_seq":373,"reported_epoch":88,"state":"active+clean","last_fresh":"2026-02-24T07:54:08.521371+0000","last_change":"2026-02-24T07:54:08.521371+0000","last_active":"2026-02-24T07:54:08.521371+0000","last_peered":"2026-02-24T07:54:08.521371+0000","last_clean":"2026-02-24T07:54:08.521371+0000","last_became_active":"2026-02-24T07:42:54.186719+0000","last_became_peered":"2026-02-24T07:42:54.186719+0000","last_unstale":"2026-02-24T07:54:08.521371+0000","last_undegraded":"2026-02-24T07:54:08.521371+0000","last_fullsized":"2026-02-24T07:54:08.521371+0000","mapping_epoch":27,"log_start":"29'96","ondisk_log_start":"29'96","created":20,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'99","last_scrub_stamp":"2026-02-24T07:54:08.521342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:42:43.640776+0000","last_clean_scrub_stamp":"2026-02-24T07:54:08.521342+0000","objects_scrubbed":2,"log_size":3,"log_dups_size":96,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:55:15.605551+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":240,"num_write":135,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":16,"stat_sum":{"num_bytes":3960274968,"num_objects":60430,"num_object_clones":0,"num_object_copies":181290,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60430,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1050710,"num_write_kb":35556417,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":11874021376,"data_stored":11874009160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":5828,"internal_metadata":0},"log_size":51,"ondisk_log_size":51,"up":48,"acting":48,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":240,"num_write":135,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":2364,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":3,"up_from":25,"seq":107374182542,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":4465336,"kb_used_data":3430756,"kb_used_omap":21,"kb_used_meta":1034538,"kb_avail":729537864,"statfs":{"total":751619276800,"available":747046772736,"internally_reserved":0,"allocated":3513094144,"data_stored":3512649430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":21719,"internal_metadata":1059367721},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":19,"seq":81604378768,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":3518200,"kb_used_data":2715748,"kb_used_omap":20,"kb_used_meta":802347,"kb_avail":730485000,"statfs":{"total":751619276800,"available":748016640000,"internally_reserved":0,"allocated":2780925952,"data_stored":2780481238,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":21470,"internal_metadata":821603362},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542289,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":3198128,"kb_used_data":2528156,"kb_used_omap":18,"kb_used_meta":669933,"kb_avail":730805072,"statfs":{"total":751619276800,"available":748344393728,"internally_reserved":0,"allocated":2588831744,"data_stored":2588394654,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":18773,"internal_metadata":686012075},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705812,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":4770488,"kb_used_data":3210980,"kb_used_omap":18,"kb_used_meta":1559405,"kb_avail":729232712,"statfs":{"total":751619276800,"available":746734297088,"internally_reserved":0,"allocated":3288043520,"data_stored":3287598806,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":18941,"internal_metadata":1596831235},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,5],"upper_bound":4},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":2364,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":3214610432,"data_stored":3214606360,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":93,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":2440626176,"data_stored":2440626176,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":93,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":2737377280,"data_stored":2737373208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":186,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":3481407488,"data_stored":3481403416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":5456,"internal_metadata":0}]}} 2026-02-24T07:54:37.137 INFO:tasks.cephadm.ceph_manager.ceph:increase pool size by 10 2026-02-24T07:54:37.138 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph osd pool set unique_pool_3 pg_num 86 2026-02-24T07:54:37.275 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:54:37.356 INFO:journalctl@ceph.prometheus.a.trial091.stdout:Feb 24 07:54:36 trial091 ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-prometheus-a[66007]: time=2026-02-24T07:54:36.973Z level=ERROR source=scrape.go:1630 msg="Failed to determine correct type of scrape target." component="scrape manager" scrape_pool=ceph target=http://trial035:9283/metrics content_type="text/html;charset=utf-8" fallback_media_type="" err="received unsupported Content-Type \"text/html;charset=utf-8\" and no fallback_scrape_protocol specified for target" 2026-02-24T07:54:37.403 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:54:37.402+0000 7f5ae908f640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:54:37.403 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:54:37.402+0000 7f5ae908f640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:54:37.544 INFO:teuthology.orchestra.run.trial035.stderr:Error EPERM: crimson_allow_pg_split is false; pg_num increase denied for crimson pool 2026-02-24T07:54:37.585 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-24T07:54:38.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:54:37 trial035 ceph-mon[39284]: 5.a scrub starts 2026-02-24T07:54:38.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:54:37 trial035 ceph-mon[39284]: 5.2 scrub starts 2026-02-24T07:54:38.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:54:37 trial035 ceph-mon[39284]: 5.a scrub ok 2026-02-24T07:54:38.205 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:54:37 trial035 ceph-mon[39284]: 5.2 scrub ok 2026-02-24T07:54:38.205 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:54:37 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/3595987953' entity='client.admin' cmd={"prefix": "osd pool set", "pool": "unique_pool_3", "var": "pg_num", "val": "86"} : dispatch 2026-02-24T07:54:38.568 INFO:tasks.radosbench.radosbench.0.trial035.stdout:Removed 555569 objects 2026-02-24T07:54:38.568 INFO:tasks.radosbench.radosbench.0.trial035.stdout:Clean up completed and total clean up time :73.5676 2026-02-24T07:54:38.769 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2026-02-24T07:54:38.888 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:54:38.988 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:54:38.986+0000 7f9965147640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:54:38.988 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:54:38.987+0000 7f9965147640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:54:38.999 INFO:teuthology.orchestra.run.trial035.stdout:osd.0: {} 2026-02-24T07:54:38.999 INFO:teuthology.orchestra.run.trial035.stderr:osd.0: osd_enable_op_tracker = '' 2026-02-24T07:54:39.010 INFO:teuthology.orchestra.run.trial035.stdout:osd.1: {} 2026-02-24T07:54:39.010 INFO:teuthology.orchestra.run.trial035.stderr:osd.1: osd_enable_op_tracker = '' 2026-02-24T07:54:39.021 INFO:teuthology.orchestra.run.trial035.stdout:osd.2: {} 2026-02-24T07:54:39.021 INFO:teuthology.orchestra.run.trial035.stderr:osd.2: osd_enable_op_tracker = '' 2026-02-24T07:54:39.031 INFO:teuthology.orchestra.run.trial035.stdout:osd.3: {} 2026-02-24T07:54:39.031 INFO:teuthology.orchestra.run.trial035.stderr:osd.3: osd_enable_op_tracker = '' 2026-02-24T07:54:39.045 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:54:38 trial035 ceph-mon[39284]: from='client.15390 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2026-02-24T07:54:39.045 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:54:38 trial035 ceph-mon[39284]: pgmap v411: 17 pgs: 17 active+clean; 1.2 GiB data, 7.8 GiB used, 2.7 TiB / 2.7 TiB avail; 0 B/s wr, 10.18k op/s 2026-02-24T07:54:39.678 INFO:tasks.cephadm.ceph_manager.ceph:removing pool_name unique_pool_3 2026-02-24T07:54:39.678 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph osd pool rm unique_pool_3 unique_pool_3 --yes-i-really-really-mean-it 2026-02-24T07:54:39.796 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:54:39.896 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:54:39.894+0000 7fd4f6e9d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:54:39.896 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:54:39.895+0000 7fd4f6e9d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:54:39.954 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:54:39 trial035 ceph-mon[39284]: 5.b scrub starts 2026-02-24T07:54:39.954 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:54:39 trial035 ceph-mon[39284]: 5.b scrub ok 2026-02-24T07:54:39.954 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:54:39 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/3087028371' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2026-02-24T07:54:40.915 INFO:teuthology.orchestra.run.trial035.stderr:pool 'unique_pool_3' removed 2026-02-24T07:54:40.974 INFO:teuthology.task.full_sequential:In full_sequential, running task radosbench... 2026-02-24T07:54:40.975 INFO:tasks.radosbench:Beginning radosbench... 2026-02-24T07:54:40.975 INFO:tasks.cephadm.ceph_manager.ceph:creating pool_name unique_pool_4 2026-02-24T07:54:40.975 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph osd pool create unique_pool_4 16 2026-02-24T07:54:41.068 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2026-02-24T07:54:41.102 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:54:41.167 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:54:40 trial035 ceph-mon[39284]: 5.c scrub starts 2026-02-24T07:54:41.167 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:54:40 trial035 ceph-mon[39284]: 5.c scrub ok 2026-02-24T07:54:41.168 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:54:40 trial035 ceph-mon[39284]: 5.1 scrub starts 2026-02-24T07:54:41.168 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:54:40 trial035 ceph-mon[39284]: 5.1 scrub ok 2026-02-24T07:54:41.168 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:54:40 trial035 ceph-mon[39284]: pgmap v412: 17 pgs: 17 active+clean; 1.2 GiB data, 7.8 GiB used, 2.7 TiB / 2.7 TiB avail; 0 B/s wr, 6.78k op/s 2026-02-24T07:54:41.168 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:54:40 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/32504357' entity='client.admin' cmd={"prefix": "osd pool rm", "pool": "unique_pool_3", "pool2": "unique_pool_3", "yes_i_really_really_mean_it": true} : dispatch 2026-02-24T07:54:41.168 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:54:40 trial035 ceph-mon[39284]: purged_snaps scrub starts 2026-02-24T07:54:41.168 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:54:40 trial035 ceph-mon[39284]: purged_snaps scrub ok 2026-02-24T07:54:41.194 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:54:41.220 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:54:41.217+0000 7f766e739640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:54:41.220 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:54:41.218+0000 7f766e739640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:54:41.307 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:54:41.305+0000 7fe2808ad640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:54:41.308 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:54:41.305+0000 7fe2808ad640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:54:41.320 INFO:teuthology.orchestra.run.trial035.stdout:osd.0: {} 2026-02-24T07:54:41.320 INFO:teuthology.orchestra.run.trial035.stderr:osd.0: osd_enable_op_tracker = '' 2026-02-24T07:54:41.331 INFO:teuthology.orchestra.run.trial035.stdout:osd.1: {} 2026-02-24T07:54:41.331 INFO:teuthology.orchestra.run.trial035.stderr:osd.1: osd_enable_op_tracker = '' 2026-02-24T07:54:41.343 INFO:teuthology.orchestra.run.trial035.stdout:osd.2: {} 2026-02-24T07:54:41.343 INFO:teuthology.orchestra.run.trial035.stderr:osd.2: osd_enable_op_tracker = '' 2026-02-24T07:54:41.355 INFO:teuthology.orchestra.run.trial035.stdout:osd.3: {} 2026-02-24T07:54:41.355 INFO:teuthology.orchestra.run.trial035.stderr:osd.3: osd_enable_op_tracker = '' 2026-02-24T07:54:41.921 INFO:teuthology.orchestra.run.trial035.stderr:pool 'unique_pool_4' created 2026-02-24T07:54:41.962 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph osd pool application enable unique_pool_4 rados --yes-i-really-mean-it || true 2026-02-24T07:54:42.077 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:54:42.192 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:54:41 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/32504357' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "unique_pool_3", "pool2": "unique_pool_3", "yes_i_really_really_mean_it": true}]': finished 2026-02-24T07:54:42.193 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:54:41 trial035 ceph-mon[39284]: osdmap e89: 4 total, 4 up, 4 in 2026-02-24T07:54:42.193 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:54:41 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/3937647682' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2026-02-24T07:54:42.193 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:54:41 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/3494534432' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "unique_pool_4", "pg_num": 16} : dispatch 2026-02-24T07:54:42.433 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:54:42.406+0000 7f216ebf0640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:54:42.433 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:54:42.406+0000 7f216ebf0640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:54:42.586 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2026-02-24T07:54:42.586 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2026-02-24T07:54:42.921 INFO:teuthology.orchestra.run.trial035.stderr:enabled application 'rados' on pool 'unique_pool_4' 2026-02-24T07:54:43.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:54:42 trial035 ceph-mon[39284]: pgmap v414: 1 pgs: 1 active+clean; 577 KiB data, 3.8 GiB used, 2.7 TiB / 2.7 TiB avail 2026-02-24T07:54:43.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:54:42 trial035 ceph-mon[39284]: purged_snaps scrub starts 2026-02-24T07:54:43.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:54:42 trial035 ceph-mon[39284]: purged_snaps scrub ok 2026-02-24T07:54:43.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:54:42 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/3494534432' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "unique_pool_4", "pg_num": 16}]': finished 2026-02-24T07:54:43.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:54:42 trial035 ceph-mon[39284]: osdmap e90: 4 total, 4 up, 4 in 2026-02-24T07:54:43.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:54:42 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/1988027071' entity='client.admin' cmd={"prefix": "osd pool application enable", "pool": "unique_pool_4", "app": "rados", "yes_i_really_mean_it": true} : dispatch 2026-02-24T07:54:43.392 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2026-02-24T07:54:43.510 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:54:43.609 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:54:43.606+0000 7f7c3fa8a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:54:43.609 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:54:43.607+0000 7f7c3fa8a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:54:43.621 INFO:teuthology.orchestra.run.trial035.stdout:osd.0: {} 2026-02-24T07:54:43.621 INFO:teuthology.orchestra.run.trial035.stderr:osd.0: osd_enable_op_tracker = '' 2026-02-24T07:54:43.631 INFO:teuthology.orchestra.run.trial035.stdout:osd.1: {} 2026-02-24T07:54:43.631 INFO:teuthology.orchestra.run.trial035.stderr:osd.1: osd_enable_op_tracker = '' 2026-02-24T07:54:43.642 INFO:teuthology.orchestra.run.trial035.stdout:osd.2: {} 2026-02-24T07:54:43.642 INFO:teuthology.orchestra.run.trial035.stderr:osd.2: osd_enable_op_tracker = '' 2026-02-24T07:54:43.652 INFO:teuthology.orchestra.run.trial035.stdout:osd.3: {} 2026-02-24T07:54:43.652 INFO:teuthology.orchestra.run.trial035.stderr:osd.3: osd_enable_op_tracker = '' 2026-02-24T07:54:43.964 DEBUG:teuthology.orchestra.run.trial035:> /bin/sh -c 'adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --no-log-to-stderr --name client.0 -t 16 -b 65536 --object-size 65536 -p unique_pool_4 bench 90 write' 2026-02-24T07:54:43.965 INFO:tasks.thrashosds.thrasher:Growing pool unique_pool_4 2026-02-24T07:54:43.966 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph pg dump --format=json 2026-02-24T07:54:43.968 INFO:tasks.radosbench:joining radosbench (timing out after 3000s) 2026-02-24T07:54:43.968 INFO:teuthology.orchestra.run:waiting for 3000 2026-02-24T07:54:44.014 INFO:tasks.radosbench.radosbench.0.trial035.stdout:hints = 1 2026-02-24T07:54:44.015 INFO:tasks.radosbench.radosbench.0.trial035.stdout:Maintaining 16 concurrent writes of 65536 bytes to objects of size 65536 for up to 90 seconds or 0 objects 2026-02-24T07:54:44.015 INFO:tasks.radosbench.radosbench.0.trial035.stdout:Object prefix: benchmark_data_trial035_135156 2026-02-24T07:54:44.015 INFO:tasks.radosbench.radosbench.0.trial035.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2026-02-24T07:54:44.015 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 0 0 0 0 0 0 - 0 2026-02-24T07:54:44.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:54:43 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/1988027071' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "unique_pool_4", "app": "rados", "yes_i_really_mean_it": true}]': finished 2026-02-24T07:54:44.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:54:43 trial035 ceph-mon[39284]: osdmap e91: 4 total, 4 up, 4 in 2026-02-24T07:54:44.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:54:43 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/2850894177' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2026-02-24T07:54:44.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:54:43 trial035 ceph-mon[39284]: pgmap v417: 17 pgs: 16 unknown, 1 active+clean; 577 KiB data, 3.8 GiB used, 2.7 TiB / 2.7 TiB avail 2026-02-24T07:54:44.210 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:54:44.729 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:54:44.713+0000 7fa963683640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:54:44.730 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:54:44.713+0000 7fa963683640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:54:44.854 INFO:teuthology.orchestra.run.trial035.stdout: 2026-02-24T07:54:44.855 INFO:teuthology.orchestra.run.trial035.stderr:dumped all 2026-02-24T07:54:44.917 INFO:teuthology.orchestra.run.trial035.stdout:{"pg_ready":true,"pg_map":{"version":418,"stamp":"2026-02-24T07:54:43.926442+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":240,"num_write":135,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":3,"acting":3,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":51,"num_osds":4,"num_per_pool_osds":4,"num_per_pool_omap_osds":4,"kb":2936012800,"kb_used":3980556,"kb_used_data":4412,"kb_used_omap":80,"kb_used_meta":3975983,"kb_avail":2932032244,"statfs":{"total":3006477107200,"available":3002401017856,"internally_reserved":0,"allocated":4517888,"data_stored":2758872,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":82101,"internal_metadata":4071407435},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-1235484696,"num_objects":-18853,"num_object_clones":0,"num_object_copies":-56559,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-18853,"num_whiteouts":0,"num_read":-1,"num_read_kb":-1,"num_write":-1092287,"num_write_kb":-35556417,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.174148"},"pg_stats":[{"pgid":"6.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2026-02-24T07:54:41.918128+0000","last_change":"2026-02-24T07:54:41.918128+0000","last_active":"2026-02-24T07:54:41.918128+0000","last_peered":"2026-02-24T07:54:41.918128+0000","last_clean":"2026-02-24T07:54:41.918128+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2026-02-24T07:54:41.918128+0000","last_undegraded":"2026-02-24T07:54:41.918128+0000","last_fullsized":"2026-02-24T07:54:41.918128+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_clean_scrub_stamp":"2026-02-24T07:54:41.918128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2026-02-24T07:54:41.918128+0000","last_change":"2026-02-24T07:54:41.918128+0000","last_active":"2026-02-24T07:54:41.918128+0000","last_peered":"2026-02-24T07:54:41.918128+0000","last_clean":"2026-02-24T07:54:41.918128+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2026-02-24T07:54:41.918128+0000","last_undegraded":"2026-02-24T07:54:41.918128+0000","last_fullsized":"2026-02-24T07:54:41.918128+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_clean_scrub_stamp":"2026-02-24T07:54:41.918128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2026-02-24T07:54:41.918128+0000","last_change":"2026-02-24T07:54:41.918128+0000","last_active":"2026-02-24T07:54:41.918128+0000","last_peered":"2026-02-24T07:54:41.918128+0000","last_clean":"2026-02-24T07:54:41.918128+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2026-02-24T07:54:41.918128+0000","last_undegraded":"2026-02-24T07:54:41.918128+0000","last_fullsized":"2026-02-24T07:54:41.918128+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_clean_scrub_stamp":"2026-02-24T07:54:41.918128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2026-02-24T07:54:41.918128+0000","last_change":"2026-02-24T07:54:41.918128+0000","last_active":"2026-02-24T07:54:41.918128+0000","last_peered":"2026-02-24T07:54:41.918128+0000","last_clean":"2026-02-24T07:54:41.918128+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2026-02-24T07:54:41.918128+0000","last_undegraded":"2026-02-24T07:54:41.918128+0000","last_fullsized":"2026-02-24T07:54:41.918128+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_clean_scrub_stamp":"2026-02-24T07:54:41.918128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2026-02-24T07:54:41.918128+0000","last_change":"2026-02-24T07:54:41.918128+0000","last_active":"2026-02-24T07:54:41.918128+0000","last_peered":"2026-02-24T07:54:41.918128+0000","last_clean":"2026-02-24T07:54:41.918128+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2026-02-24T07:54:41.918128+0000","last_undegraded":"2026-02-24T07:54:41.918128+0000","last_fullsized":"2026-02-24T07:54:41.918128+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_clean_scrub_stamp":"2026-02-24T07:54:41.918128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2026-02-24T07:54:41.918128+0000","last_change":"2026-02-24T07:54:41.918128+0000","last_active":"2026-02-24T07:54:41.918128+0000","last_peered":"2026-02-24T07:54:41.918128+0000","last_clean":"2026-02-24T07:54:41.918128+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2026-02-24T07:54:41.918128+0000","last_undegraded":"2026-02-24T07:54:41.918128+0000","last_fullsized":"2026-02-24T07:54:41.918128+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_clean_scrub_stamp":"2026-02-24T07:54:41.918128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2026-02-24T07:54:41.918128+0000","last_change":"2026-02-24T07:54:41.918128+0000","last_active":"2026-02-24T07:54:41.918128+0000","last_peered":"2026-02-24T07:54:41.918128+0000","last_clean":"2026-02-24T07:54:41.918128+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2026-02-24T07:54:41.918128+0000","last_undegraded":"2026-02-24T07:54:41.918128+0000","last_fullsized":"2026-02-24T07:54:41.918128+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_clean_scrub_stamp":"2026-02-24T07:54:41.918128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2026-02-24T07:54:41.918128+0000","last_change":"2026-02-24T07:54:41.918128+0000","last_active":"2026-02-24T07:54:41.918128+0000","last_peered":"2026-02-24T07:54:41.918128+0000","last_clean":"2026-02-24T07:54:41.918128+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2026-02-24T07:54:41.918128+0000","last_undegraded":"2026-02-24T07:54:41.918128+0000","last_fullsized":"2026-02-24T07:54:41.918128+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_clean_scrub_stamp":"2026-02-24T07:54:41.918128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2026-02-24T07:54:41.918128+0000","last_change":"2026-02-24T07:54:41.918128+0000","last_active":"2026-02-24T07:54:41.918128+0000","last_peered":"2026-02-24T07:54:41.918128+0000","last_clean":"2026-02-24T07:54:41.918128+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2026-02-24T07:54:41.918128+0000","last_undegraded":"2026-02-24T07:54:41.918128+0000","last_fullsized":"2026-02-24T07:54:41.918128+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_clean_scrub_stamp":"2026-02-24T07:54:41.918128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2026-02-24T07:54:41.918128+0000","last_change":"2026-02-24T07:54:41.918128+0000","last_active":"2026-02-24T07:54:41.918128+0000","last_peered":"2026-02-24T07:54:41.918128+0000","last_clean":"2026-02-24T07:54:41.918128+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2026-02-24T07:54:41.918128+0000","last_undegraded":"2026-02-24T07:54:41.918128+0000","last_fullsized":"2026-02-24T07:54:41.918128+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_clean_scrub_stamp":"2026-02-24T07:54:41.918128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2026-02-24T07:54:41.918128+0000","last_change":"2026-02-24T07:54:41.918128+0000","last_active":"2026-02-24T07:54:41.918128+0000","last_peered":"2026-02-24T07:54:41.918128+0000","last_clean":"2026-02-24T07:54:41.918128+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2026-02-24T07:54:41.918128+0000","last_undegraded":"2026-02-24T07:54:41.918128+0000","last_fullsized":"2026-02-24T07:54:41.918128+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_clean_scrub_stamp":"2026-02-24T07:54:41.918128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2026-02-24T07:54:41.918128+0000","last_change":"2026-02-24T07:54:41.918128+0000","last_active":"2026-02-24T07:54:41.918128+0000","last_peered":"2026-02-24T07:54:41.918128+0000","last_clean":"2026-02-24T07:54:41.918128+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2026-02-24T07:54:41.918128+0000","last_undegraded":"2026-02-24T07:54:41.918128+0000","last_fullsized":"2026-02-24T07:54:41.918128+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_clean_scrub_stamp":"2026-02-24T07:54:41.918128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2026-02-24T07:54:41.918128+0000","last_change":"2026-02-24T07:54:41.918128+0000","last_active":"2026-02-24T07:54:41.918128+0000","last_peered":"2026-02-24T07:54:41.918128+0000","last_clean":"2026-02-24T07:54:41.918128+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2026-02-24T07:54:41.918128+0000","last_undegraded":"2026-02-24T07:54:41.918128+0000","last_fullsized":"2026-02-24T07:54:41.918128+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_clean_scrub_stamp":"2026-02-24T07:54:41.918128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2026-02-24T07:54:41.918128+0000","last_change":"2026-02-24T07:54:41.918128+0000","last_active":"2026-02-24T07:54:41.918128+0000","last_peered":"2026-02-24T07:54:41.918128+0000","last_clean":"2026-02-24T07:54:41.918128+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2026-02-24T07:54:41.918128+0000","last_undegraded":"2026-02-24T07:54:41.918128+0000","last_fullsized":"2026-02-24T07:54:41.918128+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_clean_scrub_stamp":"2026-02-24T07:54:41.918128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2026-02-24T07:54:41.918128+0000","last_change":"2026-02-24T07:54:41.918128+0000","last_active":"2026-02-24T07:54:41.918128+0000","last_peered":"2026-02-24T07:54:41.918128+0000","last_clean":"2026-02-24T07:54:41.918128+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2026-02-24T07:54:41.918128+0000","last_undegraded":"2026-02-24T07:54:41.918128+0000","last_fullsized":"2026-02-24T07:54:41.918128+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_clean_scrub_stamp":"2026-02-24T07:54:41.918128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2026-02-24T07:54:41.918128+0000","last_change":"2026-02-24T07:54:41.918128+0000","last_active":"2026-02-24T07:54:41.918128+0000","last_peered":"2026-02-24T07:54:41.918128+0000","last_clean":"2026-02-24T07:54:41.918128+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2026-02-24T07:54:41.918128+0000","last_undegraded":"2026-02-24T07:54:41.918128+0000","last_fullsized":"2026-02-24T07:54:41.918128+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_clean_scrub_stamp":"2026-02-24T07:54:41.918128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"29'99","reported_seq":373,"reported_epoch":88,"state":"active+clean","last_fresh":"2026-02-24T07:54:08.521371+0000","last_change":"2026-02-24T07:54:08.521371+0000","last_active":"2026-02-24T07:54:08.521371+0000","last_peered":"2026-02-24T07:54:08.521371+0000","last_clean":"2026-02-24T07:54:08.521371+0000","last_became_active":"2026-02-24T07:42:54.186719+0000","last_became_peered":"2026-02-24T07:42:54.186719+0000","last_unstale":"2026-02-24T07:54:08.521371+0000","last_undegraded":"2026-02-24T07:54:08.521371+0000","last_fullsized":"2026-02-24T07:54:08.521371+0000","mapping_epoch":27,"log_start":"29'96","ondisk_log_start":"29'96","created":20,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'99","last_scrub_stamp":"2026-02-24T07:54:08.521342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:42:43.640776+0000","last_clean_scrub_stamp":"2026-02-24T07:54:08.521342+0000","objects_scrubbed":2,"log_size":3,"log_dups_size":96,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:55:15.605551+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":240,"num_write":135,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":16,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":240,"num_write":135,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":2400,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":3,"up_from":25,"seq":107374182544,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":1126068,"kb_used_data":1248,"kb_used_omap":22,"kb_used_meta":1124777,"kb_avail":732877132,"statfs":{"total":751619276800,"available":750466183168,"internally_reserved":0,"allocated":1277952,"data_stored":837310,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":22552,"internal_metadata":1151772648},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":19,"seq":81604378770,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":875828,"kb_used_data":1248,"kb_used_omap":20,"kb_used_meta":874539,"kb_avail":733127372,"statfs":{"total":751619276800,"available":750722428928,"internally_reserved":0,"allocated":1277952,"data_stored":837310,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":21470,"internal_metadata":895527970},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542291,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":327728,"kb_used_data":668,"kb_used_omap":18,"kb_used_meta":327021,"kb_avail":733675472,"statfs":{"total":751619276800,"available":751283683328,"internally_reserved":0,"allocated":684032,"data_stored":246942,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":18866,"internal_metadata":334870094},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705814,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":1650932,"kb_used_data":1248,"kb_used_omap":18,"kb_used_meta":1649645,"kb_avail":732352268,"statfs":{"total":751619276800,"available":749928722432,"internally_reserved":0,"allocated":1277952,"data_stored":837310,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19213,"internal_metadata":1689236723},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":2400,"internal_metadata":0}]}} 2026-02-24T07:54:44.917 INFO:tasks.cephadm.ceph_manager.ceph:increase pool size by 10 2026-02-24T07:54:44.918 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph osd pool set unique_pool_4 pg_num 26 2026-02-24T07:54:45.015 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 1 16 6125 6109 381.783 381.812 0.00221714 0.00261442 2026-02-24T07:54:45.062 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:54:45.122 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:54:44 trial035 ceph-mon[39284]: osdmap e92: 4 total, 4 up, 4 in 2026-02-24T07:54:45.181 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:54:44 trial035 ceph-mon[39284]: from='client.15409 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2026-02-24T07:54:45.181 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:54:44 trial035 ceph-mon[39284]: from='mgr.14284 10.20.193.35:0/3681373222' entity='mgr.x' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2026-02-24T07:54:45.198 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:54:45.196+0000 7f32eed32640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:54:45.198 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:54:45.196+0000 7f32eed32640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:54:45.331 INFO:teuthology.orchestra.run.trial035.stderr:Error EPERM: crimson_allow_pg_split is false; pg_num increase denied for crimson pool 2026-02-24T07:54:45.369 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-24T07:54:46.015 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 2 16 13766 13750 429.654 477.562 0.00253957 0.00232522 2026-02-24T07:54:46.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:54:45 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/47928250' entity='client.admin' cmd={"prefix": "osd pool set", "pool": "unique_pool_4", "var": "pg_num", "val": "26"} : dispatch 2026-02-24T07:54:46.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:54:45 trial035 ceph-mon[39284]: pgmap v419: 17 pgs: 16 unknown, 1 active+clean; 577 KiB data, 3.8 GiB used, 2.7 TiB / 2.7 TiB avail 2026-02-24T07:54:46.299 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2026-02-24T07:54:46.446 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:54:46.561 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:54:46.559+0000 7f2b1aacb640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:54:46.561 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:54:46.559+0000 7f2b1aacb640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:54:46.577 INFO:teuthology.orchestra.run.trial035.stdout:osd.0: {} 2026-02-24T07:54:46.577 INFO:teuthology.orchestra.run.trial035.stderr:osd.0: osd_enable_op_tracker = '' 2026-02-24T07:54:46.593 INFO:teuthology.orchestra.run.trial035.stdout:osd.1: {} 2026-02-24T07:54:46.594 INFO:teuthology.orchestra.run.trial035.stderr:osd.1: osd_enable_op_tracker = '' 2026-02-24T07:54:46.606 INFO:teuthology.orchestra.run.trial035.stdout:osd.2: {} 2026-02-24T07:54:46.606 INFO:teuthology.orchestra.run.trial035.stderr:osd.2: osd_enable_op_tracker = '' 2026-02-24T07:54:46.622 INFO:teuthology.orchestra.run.trial035.stdout:osd.3: {} 2026-02-24T07:54:46.622 INFO:teuthology.orchestra.run.trial035.stderr:osd.3: osd_enable_op_tracker = '' 2026-02-24T07:54:47.015 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 3 16 21563 21547 448.859 487.312 0.0020788 0.00222573 2026-02-24T07:54:47.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:54:46 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/2995785970' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2026-02-24T07:54:47.204 INFO:journalctl@ceph.mgr.x.trial035.stdout:Feb 24 07:54:46 trial035 ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-mgr-x[39621]: ::ffff:10.20.193.91 - - [24/Feb/2026:07:54:46] "GET /metrics HTTP/1.1" 200 4156 "" "Prometheus/3.6.0" 2026-02-24T07:54:47.266 INFO:journalctl@ceph.prometheus.a.trial091.stdout:Feb 24 07:54:46 trial091 ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-prometheus-a[66007]: time=2026-02-24T07:54:46.973Z level=ERROR source=scrape.go:1630 msg="Failed to determine correct type of scrape target." component="scrape manager" scrape_pool=ceph target=http://trial035:9283/metrics content_type="text/html;charset=utf-8" fallback_media_type="" err="received unsupported Content-Type \"text/html;charset=utf-8\" and no fallback_scrape_protocol specified for target" 2026-02-24T07:54:48.015 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 4 16 29302 29286 457.556 483.688 0.00135358 0.00218363 2026-02-24T07:54:48.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:54:47 trial035 ceph-mon[39284]: pgmap v420: 17 pgs: 17 active+clean; 761 MiB data, 5.8 GiB used, 2.7 TiB / 2.7 TiB avail; 127 MiB/s wr, 2.03k op/s 2026-02-24T07:54:48.670 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2026-02-24T07:54:48.829 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:54:48.964 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:54:48.962+0000 7f489c217640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:54:48.965 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:54:48.962+0000 7f489c217640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:54:48.981 INFO:teuthology.orchestra.run.trial035.stdout:osd.0: {} 2026-02-24T07:54:48.981 INFO:teuthology.orchestra.run.trial035.stderr:osd.0: osd_enable_op_tracker = '' 2026-02-24T07:54:49.000 INFO:teuthology.orchestra.run.trial035.stdout:osd.1: {} 2026-02-24T07:54:49.000 INFO:teuthology.orchestra.run.trial035.stderr:osd.1: osd_enable_op_tracker = '' 2026-02-24T07:54:49.013 INFO:teuthology.orchestra.run.trial035.stdout:osd.2: {} 2026-02-24T07:54:49.013 INFO:teuthology.orchestra.run.trial035.stderr:osd.2: osd_enable_op_tracker = '' 2026-02-24T07:54:49.015 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 5 16 37028 37012 462.611 482.875 0.00193429 0.0021599 2026-02-24T07:54:49.027 INFO:teuthology.orchestra.run.trial035.stdout:osd.3: {} 2026-02-24T07:54:49.028 INFO:teuthology.orchestra.run.trial035.stderr:osd.3: osd_enable_op_tracker = '' 2026-02-24T07:54:50.015 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 6 16 44777 44761 466.222 484.312 0.0027364 0.00214344 2026-02-24T07:54:50.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:54:49 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/1329265792' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2026-02-24T07:54:50.371 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2026-02-24T07:54:50.371 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2026-02-24T07:54:50.371 INFO:tasks.thrashosds.thrasher:Growing pool unique_pool_4 2026-02-24T07:54:50.371 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph pg dump --format=json 2026-02-24T07:54:50.512 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:54:50.631 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:54:50.629+0000 7f9a8f987640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:54:50.631 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:54:50.629+0000 7f9a8f987640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:54:50.784 INFO:teuthology.orchestra.run.trial035.stdout: 2026-02-24T07:54:50.784 INFO:teuthology.orchestra.run.trial035.stderr:dumped all 2026-02-24T07:54:50.831 INFO:teuthology.orchestra.run.trial035.stdout:{"pg_ready":true,"pg_map":{"version":421,"stamp":"2026-02-24T07:54:49.753592+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":798360096,"num_objects":12175,"num_object_clones":0,"num_object_copies":36525,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12175,"num_whiteouts":0,"num_read":136,"num_read_kb":240,"num_write":12308,"num_write_kb":781222,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":51,"ondisk_log_size":51,"up":51,"acting":51,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":39,"num_osds":4,"num_per_pool_osds":4,"num_per_pool_omap_osds":4,"kb":2936012800,"kb_used":6049868,"kb_used_data":2001532,"kb_used_omap":90,"kb_used_meta":4048165,"kb_avail":2929962932,"statfs":{"total":3006477107200,"available":3000282042368,"internally_reserved":0,"allocated":2049568768,"data_stored":2047687912,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":92257,"internal_metadata":4145321887},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":797769728,"num_objects":12173,"num_object_clones":0,"num_object_copies":36519,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12173,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12173,"num_write_kb":779072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.833201"},"pg_stats":[{"pgid":"6.f","version":"92'811","reported_seq":825,"reported_epoch":92,"state":"active+clean","last_fresh":"2026-02-24T07:54:45.862772+0000","last_change":"2026-02-24T07:54:42.927271+0000","last_active":"2026-02-24T07:54:45.862772+0000","last_peered":"2026-02-24T07:54:45.862772+0000","last_clean":"2026-02-24T07:54:45.862772+0000","last_became_active":"2026-02-24T07:54:42.927080+0000","last_became_peered":"2026-02-24T07:54:42.927080+0000","last_unstale":"2026-02-24T07:54:45.862772+0000","last_undegraded":"2026-02-24T07:54:45.862772+0000","last_fullsized":"2026-02-24T07:54:45.862772+0000","mapping_epoch":90,"log_start":"92'808","ondisk_log_start":"92'808","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_clean_scrub_stamp":"2026-02-24T07:54:41.918128+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":808,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:56:00.801546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":53149696,"num_objects":811,"num_object_clones":0,"num_object_copies":2433,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":811,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":811,"num_write_kb":51904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.e","version":"92'719","reported_seq":731,"reported_epoch":92,"state":"active+clean","last_fresh":"2026-02-24T07:54:45.760752+0000","last_change":"2026-02-24T07:54:42.929104+0000","last_active":"2026-02-24T07:54:45.760752+0000","last_peered":"2026-02-24T07:54:45.760752+0000","last_clean":"2026-02-24T07:54:45.760752+0000","last_became_active":"2026-02-24T07:54:42.928748+0000","last_became_peered":"2026-02-24T07:54:42.928748+0000","last_unstale":"2026-02-24T07:54:45.760752+0000","last_undegraded":"2026-02-24T07:54:45.760752+0000","last_fullsized":"2026-02-24T07:54:45.760752+0000","mapping_epoch":90,"log_start":"92'716","ondisk_log_start":"92'716","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_clean_scrub_stamp":"2026-02-24T07:54:41.918128+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":716,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:55:53.670906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":47120384,"num_objects":719,"num_object_clones":0,"num_object_copies":2157,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":719,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":719,"num_write_kb":46016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.d","version":"92'727","reported_seq":741,"reported_epoch":92,"state":"active+clean","last_fresh":"2026-02-24T07:54:45.754107+0000","last_change":"2026-02-24T07:54:42.927535+0000","last_active":"2026-02-24T07:54:45.754107+0000","last_peered":"2026-02-24T07:54:45.754107+0000","last_clean":"2026-02-24T07:54:45.754107+0000","last_became_active":"2026-02-24T07:54:42.926886+0000","last_became_peered":"2026-02-24T07:54:42.926886+0000","last_unstale":"2026-02-24T07:54:45.754107+0000","last_undegraded":"2026-02-24T07:54:45.754107+0000","last_fullsized":"2026-02-24T07:54:45.754107+0000","mapping_epoch":90,"log_start":"92'724","ondisk_log_start":"92'724","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_clean_scrub_stamp":"2026-02-24T07:54:41.918128+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":724,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:55:43.220815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":47644672,"num_objects":727,"num_object_clones":0,"num_object_copies":2181,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":727,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":727,"num_write_kb":46528,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.c","version":"92'770","reported_seq":782,"reported_epoch":92,"state":"active+clean","last_fresh":"2026-02-24T07:54:45.864481+0000","last_change":"2026-02-24T07:54:42.928482+0000","last_active":"2026-02-24T07:54:45.864481+0000","last_peered":"2026-02-24T07:54:45.864481+0000","last_clean":"2026-02-24T07:54:45.864481+0000","last_became_active":"2026-02-24T07:54:42.928238+0000","last_became_peered":"2026-02-24T07:54:42.928238+0000","last_unstale":"2026-02-24T07:54:45.864481+0000","last_undegraded":"2026-02-24T07:54:45.864481+0000","last_fullsized":"2026-02-24T07:54:45.864481+0000","mapping_epoch":90,"log_start":"92'767","ondisk_log_start":"92'767","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_clean_scrub_stamp":"2026-02-24T07:54:41.918128+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":767,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:56:09.181158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":50462720,"num_objects":770,"num_object_clones":0,"num_object_copies":2310,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":770,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":770,"num_write_kb":49280,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.b","version":"92'765","reported_seq":778,"reported_epoch":92,"state":"active+clean","last_fresh":"2026-02-24T07:54:45.865937+0000","last_change":"2026-02-24T07:54:42.927223+0000","last_active":"2026-02-24T07:54:45.865937+0000","last_peered":"2026-02-24T07:54:45.865937+0000","last_clean":"2026-02-24T07:54:45.865937+0000","last_became_active":"2026-02-24T07:54:42.926965+0000","last_became_peered":"2026-02-24T07:54:42.926965+0000","last_unstale":"2026-02-24T07:54:45.865937+0000","last_undegraded":"2026-02-24T07:54:45.865937+0000","last_fullsized":"2026-02-24T07:54:45.865937+0000","mapping_epoch":90,"log_start":"92'762","ondisk_log_start":"92'762","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_clean_scrub_stamp":"2026-02-24T07:54:41.918128+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":762,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:56:04.661233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":50135040,"num_objects":765,"num_object_clones":0,"num_object_copies":2295,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":765,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":765,"num_write_kb":48960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.a","version":"92'753","reported_seq":766,"reported_epoch":92,"state":"active+clean","last_fresh":"2026-02-24T07:54:45.758258+0000","last_change":"2026-02-24T07:54:42.926926+0000","last_active":"2026-02-24T07:54:45.758258+0000","last_peered":"2026-02-24T07:54:45.758258+0000","last_clean":"2026-02-24T07:54:45.758258+0000","last_became_active":"2026-02-24T07:54:42.926477+0000","last_became_peered":"2026-02-24T07:54:42.926477+0000","last_unstale":"2026-02-24T07:54:45.758258+0000","last_undegraded":"2026-02-24T07:54:45.758258+0000","last_fullsized":"2026-02-24T07:54:45.758258+0000","mapping_epoch":90,"log_start":"92'750","ondisk_log_start":"92'750","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_clean_scrub_stamp":"2026-02-24T07:54:41.918128+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":750,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:55:46.420689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":49348608,"num_objects":753,"num_object_clones":0,"num_object_copies":2259,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":753,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":753,"num_write_kb":48192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"92'746","reported_seq":759,"reported_epoch":92,"state":"active+clean","last_fresh":"2026-02-24T07:54:45.791137+0000","last_change":"2026-02-24T07:54:42.926317+0000","last_active":"2026-02-24T07:54:45.791137+0000","last_peered":"2026-02-24T07:54:45.791137+0000","last_clean":"2026-02-24T07:54:45.791137+0000","last_became_active":"2026-02-24T07:54:42.926136+0000","last_became_peered":"2026-02-24T07:54:42.926136+0000","last_unstale":"2026-02-24T07:54:45.791137+0000","last_undegraded":"2026-02-24T07:54:45.791137+0000","last_fullsized":"2026-02-24T07:54:45.791137+0000","mapping_epoch":90,"log_start":"92'743","ondisk_log_start":"92'743","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_clean_scrub_stamp":"2026-02-24T07:54:41.918128+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":743,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:55:45.176781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":48889856,"num_objects":746,"num_object_clones":0,"num_object_copies":2238,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":746,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":746,"num_write_kb":47744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,1],"acting":[0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.8","version":"92'763","reported_seq":777,"reported_epoch":92,"state":"active+clean","last_fresh":"2026-02-24T07:54:45.776250+0000","last_change":"2026-02-24T07:54:42.929353+0000","last_active":"2026-02-24T07:54:45.776250+0000","last_peered":"2026-02-24T07:54:45.776250+0000","last_clean":"2026-02-24T07:54:45.776250+0000","last_became_active":"2026-02-24T07:54:42.929078+0000","last_became_peered":"2026-02-24T07:54:42.929078+0000","last_unstale":"2026-02-24T07:54:45.776250+0000","last_undegraded":"2026-02-24T07:54:45.776250+0000","last_fullsized":"2026-02-24T07:54:45.776250+0000","mapping_epoch":90,"log_start":"92'760","ondisk_log_start":"92'760","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_clean_scrub_stamp":"2026-02-24T07:54:41.918128+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":760,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:55:48.303915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":50003968,"num_objects":763,"num_object_clones":0,"num_object_copies":2289,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":763,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":763,"num_write_kb":48832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.6","version":"92'819","reported_seq":833,"reported_epoch":92,"state":"active+clean","last_fresh":"2026-02-24T07:54:45.863967+0000","last_change":"2026-02-24T07:54:42.926358+0000","last_active":"2026-02-24T07:54:45.863967+0000","last_peered":"2026-02-24T07:54:45.863967+0000","last_clean":"2026-02-24T07:54:45.863967+0000","last_became_active":"2026-02-24T07:54:42.926045+0000","last_became_peered":"2026-02-24T07:54:42.926045+0000","last_unstale":"2026-02-24T07:54:45.863967+0000","last_undegraded":"2026-02-24T07:54:45.863967+0000","last_fullsized":"2026-02-24T07:54:45.863967+0000","mapping_epoch":90,"log_start":"92'816","ondisk_log_start":"92'816","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_clean_scrub_stamp":"2026-02-24T07:54:41.918128+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":816,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:56:09.126690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":53673984,"num_objects":819,"num_object_clones":0,"num_object_copies":2457,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":819,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":819,"num_write_kb":52416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.5","version":"92'717","reported_seq":731,"reported_epoch":92,"state":"active+clean","last_fresh":"2026-02-24T07:54:45.760142+0000","last_change":"2026-02-24T07:54:42.928143+0000","last_active":"2026-02-24T07:54:45.760142+0000","last_peered":"2026-02-24T07:54:45.760142+0000","last_clean":"2026-02-24T07:54:45.760142+0000","last_became_active":"2026-02-24T07:54:42.927535+0000","last_became_peered":"2026-02-24T07:54:42.927535+0000","last_unstale":"2026-02-24T07:54:45.760142+0000","last_undegraded":"2026-02-24T07:54:45.760142+0000","last_fullsized":"2026-02-24T07:54:45.760142+0000","mapping_epoch":90,"log_start":"92'714","ondisk_log_start":"92'714","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_clean_scrub_stamp":"2026-02-24T07:54:41.918128+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":714,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:55:53.650972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46989312,"num_objects":717,"num_object_clones":0,"num_object_copies":2151,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":717,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":717,"num_write_kb":45888,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.4","version":"92'821","reported_seq":835,"reported_epoch":92,"state":"active+clean","last_fresh":"2026-02-24T07:54:45.865668+0000","last_change":"2026-02-24T07:54:42.928344+0000","last_active":"2026-02-24T07:54:45.865668+0000","last_peered":"2026-02-24T07:54:45.865668+0000","last_clean":"2026-02-24T07:54:45.865668+0000","last_became_active":"2026-02-24T07:54:42.928011+0000","last_became_peered":"2026-02-24T07:54:42.928011+0000","last_unstale":"2026-02-24T07:54:45.865668+0000","last_undegraded":"2026-02-24T07:54:45.865668+0000","last_fullsized":"2026-02-24T07:54:45.865668+0000","mapping_epoch":90,"log_start":"92'818","ondisk_log_start":"92'818","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_clean_scrub_stamp":"2026-02-24T07:54:41.918128+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":818,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:55:43.506947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":53805056,"num_objects":821,"num_object_clones":0,"num_object_copies":2463,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":821,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":821,"num_write_kb":52544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.3","version":"92'731","reported_seq":744,"reported_epoch":92,"state":"active+clean","last_fresh":"2026-02-24T07:54:45.759889+0000","last_change":"2026-02-24T07:54:42.928536+0000","last_active":"2026-02-24T07:54:45.759889+0000","last_peered":"2026-02-24T07:54:45.759889+0000","last_clean":"2026-02-24T07:54:45.759889+0000","last_became_active":"2026-02-24T07:54:42.927730+0000","last_became_peered":"2026-02-24T07:54:42.927730+0000","last_unstale":"2026-02-24T07:54:45.759889+0000","last_undegraded":"2026-02-24T07:54:45.759889+0000","last_fullsized":"2026-02-24T07:54:45.759889+0000","mapping_epoch":90,"log_start":"92'728","ondisk_log_start":"92'728","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_clean_scrub_stamp":"2026-02-24T07:54:41.918128+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":728,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:55:58.987317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":47906816,"num_objects":731,"num_object_clones":0,"num_object_copies":2193,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":731,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":731,"num_write_kb":46784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.2","version":"92'741","reported_seq":754,"reported_epoch":92,"state":"active+clean","last_fresh":"2026-02-24T07:54:45.776860+0000","last_change":"2026-02-24T07:54:42.930452+0000","last_active":"2026-02-24T07:54:45.776860+0000","last_peered":"2026-02-24T07:54:45.776860+0000","last_clean":"2026-02-24T07:54:45.776860+0000","last_became_active":"2026-02-24T07:54:42.930247+0000","last_became_peered":"2026-02-24T07:54:42.930247+0000","last_unstale":"2026-02-24T07:54:45.776860+0000","last_undegraded":"2026-02-24T07:54:45.776860+0000","last_fullsized":"2026-02-24T07:54:45.776860+0000","mapping_epoch":90,"log_start":"92'738","ondisk_log_start":"92'738","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_clean_scrub_stamp":"2026-02-24T07:54:41.918128+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":738,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:55:49.096947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":48562176,"num_objects":741,"num_object_clones":0,"num_object_copies":2223,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":741,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":741,"num_write_kb":47424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1","version":"92'797","reported_seq":811,"reported_epoch":92,"state":"active+clean","last_fresh":"2026-02-24T07:54:45.865872+0000","last_change":"2026-02-24T07:54:42.926619+0000","last_active":"2026-02-24T07:54:45.865872+0000","last_peered":"2026-02-24T07:54:45.865872+0000","last_clean":"2026-02-24T07:54:45.865872+0000","last_became_active":"2026-02-24T07:54:42.926442+0000","last_became_peered":"2026-02-24T07:54:42.926442+0000","last_unstale":"2026-02-24T07:54:45.865872+0000","last_undegraded":"2026-02-24T07:54:45.865872+0000","last_fullsized":"2026-02-24T07:54:45.865872+0000","mapping_epoch":90,"log_start":"92'794","ondisk_log_start":"92'794","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_clean_scrub_stamp":"2026-02-24T07:54:41.918128+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":794,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:56:03.125413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":52232192,"num_objects":797,"num_object_clones":0,"num_object_copies":2391,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":797,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":797,"num_write_kb":51008,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.0","version":"92'775","reported_seq":789,"reported_epoch":92,"state":"active+clean","last_fresh":"2026-02-24T07:54:45.788685+0000","last_change":"2026-02-24T07:54:42.927178+0000","last_active":"2026-02-24T07:54:45.788685+0000","last_peered":"2026-02-24T07:54:45.788685+0000","last_clean":"2026-02-24T07:54:45.788685+0000","last_became_active":"2026-02-24T07:54:42.926910+0000","last_became_peered":"2026-02-24T07:54:42.926910+0000","last_unstale":"2026-02-24T07:54:45.788685+0000","last_undegraded":"2026-02-24T07:54:45.788685+0000","last_fullsized":"2026-02-24T07:54:45.788685+0000","mapping_epoch":90,"log_start":"92'772","ondisk_log_start":"92'772","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_clean_scrub_stamp":"2026-02-24T07:54:41.918128+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":772,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:55:58.002045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":50790400,"num_objects":775,"num_object_clones":0,"num_object_copies":2325,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":775,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":775,"num_write_kb":49600,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.7","version":"92'718","reported_seq":730,"reported_epoch":92,"state":"active+clean","last_fresh":"2026-02-24T07:54:45.776653+0000","last_change":"2026-02-24T07:54:42.930348+0000","last_active":"2026-02-24T07:54:45.776653+0000","last_peered":"2026-02-24T07:54:45.776653+0000","last_clean":"2026-02-24T07:54:45.776653+0000","last_became_active":"2026-02-24T07:54:42.930142+0000","last_became_peered":"2026-02-24T07:54:42.930142+0000","last_unstale":"2026-02-24T07:54:45.776653+0000","last_undegraded":"2026-02-24T07:54:45.776653+0000","last_fullsized":"2026-02-24T07:54:45.776653+0000","mapping_epoch":90,"log_start":"92'715","ondisk_log_start":"92'715","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_clean_scrub_stamp":"2026-02-24T07:54:41.918128+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":715,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:55:56.704334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":47054848,"num_objects":718,"num_object_clones":0,"num_object_copies":2154,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":718,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":718,"num_write_kb":45952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"29'99","reported_seq":381,"reported_epoch":92,"state":"active+clean","last_fresh":"2026-02-24T07:54:44.016757+0000","last_change":"2026-02-24T07:54:08.521371+0000","last_active":"2026-02-24T07:54:44.016757+0000","last_peered":"2026-02-24T07:54:44.016757+0000","last_clean":"2026-02-24T07:54:44.016757+0000","last_became_active":"2026-02-24T07:42:54.186719+0000","last_became_peered":"2026-02-24T07:42:54.186719+0000","last_unstale":"2026-02-24T07:54:44.016757+0000","last_undegraded":"2026-02-24T07:54:44.016757+0000","last_fullsized":"2026-02-24T07:54:44.016757+0000","mapping_epoch":27,"log_start":"29'96","ondisk_log_start":"29'96","created":20,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'99","last_scrub_stamp":"2026-02-24T07:54:08.521342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:42:43.640776+0000","last_clean_scrub_stamp":"2026-02-24T07:54:08.521342+0000","objects_scrubbed":2,"log_size":3,"log_dups_size":96,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:55:15.605551+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":240,"num_write":135,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":16,"stat_sum":{"num_bytes":797769728,"num_objects":12173,"num_object_clones":0,"num_object_copies":36519,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12173,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12173,"num_write_kb":779072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2384658432,"data_stored":2384658432,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":48,"ondisk_log_size":48,"up":48,"acting":48,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":240,"num_write":135,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":2446,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":3,"up_from":25,"seq":107374182545,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":1603364,"kb_used_data":460496,"kb_used_omap":24,"kb_used_meta":1142823,"kb_avail":732399836,"statfs":{"total":751619276800,"available":749977432064,"internally_reserved":0,"allocated":471547904,"data_stored":471076802,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":25297,"internal_metadata":1170251055},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":19,"seq":81604378771,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":1418852,"kb_used_data":526224,"kb_used_omap":22,"kb_used_meta":892585,"kb_avail":732584348,"statfs":{"total":751619276800,"available":750166372352,"internally_reserved":0,"allocated":538853376,"data_stored":538382274,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":23488,"internal_metadata":914007104},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542292,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":823840,"kb_used_data":496780,"kb_used_omap":21,"kb_used_meta":327018,"kb_avail":733179360,"statfs":{"total":751619276800,"available":750775664640,"internally_reserved":0,"allocated":508702720,"data_stored":508235170,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":21532,"internal_metadata":334867428},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705815,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":2203812,"kb_used_data":518032,"kb_used_omap":21,"kb_used_meta":1685738,"kb_avail":731799388,"statfs":{"total":751619276800,"available":749362573312,"internally_reserved":0,"allocated":530464768,"data_stored":529993666,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":21940,"internal_metadata":1726196300},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":2446,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":594018304,"data_stored":594018304,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":725221376,"data_stored":725221376,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":577372160,"data_stored":577372160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":488046592,"data_stored":488046592,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-24T07:54:50.833 INFO:tasks.cephadm.ceph_manager.ceph:increase pool size by 10 2026-02-24T07:54:50.833 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph osd pool set unique_pool_4 pg_num 36 2026-02-24T07:54:50.975 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:54:51.015 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 7 16 52468 52452 468.283 480.688 0.0017795 0.00213393 2026-02-24T07:54:51.039 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:54:50 trial035 ceph-mon[39284]: purged_snaps scrub starts 2026-02-24T07:54:51.039 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:54:50 trial035 ceph-mon[39284]: purged_snaps scrub ok 2026-02-24T07:54:51.040 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:54:50 trial035 ceph-mon[39284]: pgmap v421: 17 pgs: 17 active+clean; 761 MiB data, 5.8 GiB used, 2.7 TiB / 2.7 TiB avail; 97 MiB/s wr, 1.55k op/s 2026-02-24T07:54:51.072 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2026-02-24T07:54:51.108 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:54:51.106+0000 7f42baac0640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:54:51.109 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:54:51.106+0000 7f42baac0640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:54:51.223 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:54:51.253 INFO:teuthology.orchestra.run.trial035.stderr:Error EPERM: crimson_allow_pg_split is false; pg_num increase denied for crimson pool 2026-02-24T07:54:51.307 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-24T07:54:51.347 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:54:51.344+0000 7fe897809640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:54:51.347 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:54:51.345+0000 7fe897809640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:54:51.363 INFO:teuthology.orchestra.run.trial035.stdout:osd.0: {} 2026-02-24T07:54:51.364 INFO:teuthology.orchestra.run.trial035.stderr:osd.0: osd_enable_op_tracker = '' 2026-02-24T07:54:51.380 INFO:teuthology.orchestra.run.trial035.stdout:osd.1: {} 2026-02-24T07:54:51.380 INFO:teuthology.orchestra.run.trial035.stderr:osd.1: osd_enable_op_tracker = '' 2026-02-24T07:54:51.392 INFO:teuthology.orchestra.run.trial035.stdout:osd.2: {} 2026-02-24T07:54:51.393 INFO:teuthology.orchestra.run.trial035.stderr:osd.2: osd_enable_op_tracker = '' 2026-02-24T07:54:51.406 INFO:teuthology.orchestra.run.trial035.stdout:osd.3: {} 2026-02-24T07:54:51.407 INFO:teuthology.orchestra.run.trial035.stderr:osd.3: osd_enable_op_tracker = '' 2026-02-24T07:54:52.015 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 8 16 60322 60306 471.102 490.875 0.00156272 0.00212101 2026-02-24T07:54:52.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:54:51 trial035 ceph-mon[39284]: from='client.15417 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2026-02-24T07:54:52.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:54:51 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/3914734332' entity='client.admin' cmd={"prefix": "osd pool set", "pool": "unique_pool_4", "var": "pg_num", "val": "36"} : dispatch 2026-02-24T07:54:52.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:54:51 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/2840834359' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2026-02-24T07:54:53.016 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 9 16 67958 67942 471.781 477.25 0.00267831 0.00211817 2026-02-24T07:54:53.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:54:52 trial035 ceph-mon[39284]: pgmap v422: 17 pgs: 17 active+clean; 3.1 GiB data, 12 GiB used, 2.7 TiB / 2.7 TiB avail; 360 MiB/s wr, 5.77k op/s 2026-02-24T07:54:53.451 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2026-02-24T07:54:53.602 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:54:53.729 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:54:53.726+0000 7f3512dd6640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:54:53.729 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:54:53.726+0000 7f3512dd6640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:54:53.745 INFO:teuthology.orchestra.run.trial035.stdout:osd.0: {} 2026-02-24T07:54:53.746 INFO:teuthology.orchestra.run.trial035.stderr:osd.0: osd_enable_op_tracker = '' 2026-02-24T07:54:53.763 INFO:teuthology.orchestra.run.trial035.stdout:osd.1: {} 2026-02-24T07:54:53.764 INFO:teuthology.orchestra.run.trial035.stderr:osd.1: osd_enable_op_tracker = '' 2026-02-24T07:54:53.777 INFO:teuthology.orchestra.run.trial035.stdout:osd.2: {} 2026-02-24T07:54:53.777 INFO:teuthology.orchestra.run.trial035.stderr:osd.2: osd_enable_op_tracker = '' 2026-02-24T07:54:53.791 INFO:teuthology.orchestra.run.trial035.stdout:osd.3: {} 2026-02-24T07:54:53.791 INFO:teuthology.orchestra.run.trial035.stderr:osd.3: osd_enable_op_tracker = '' 2026-02-24T07:54:54.016 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 10 16 75753 75737 473.318 487.188 0.00229235 0.00211134 2026-02-24T07:54:54.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:54:53 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/2167830225' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2026-02-24T07:54:55.016 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 11 15 83416 83401 473.831 479 0.00143412 0.00210907 2026-02-24T07:54:55.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:54:54 trial035 ceph-mon[39284]: pgmap v423: 17 pgs: 17 active+clean; 3.1 GiB data, 12 GiB used, 2.7 TiB / 2.7 TiB avail; 318 MiB/s wr, 5.09k op/s 2026-02-24T07:54:55.838 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2026-02-24T07:54:55.990 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:54:56.016 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 12 15 91027 91012 473.983 475.688 0.00243826 0.00210833 2026-02-24T07:54:56.116 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:54:56.113+0000 7fe0faef4640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:54:56.116 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:54:56.114+0000 7fe0faef4640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:54:56.131 INFO:teuthology.orchestra.run.trial035.stdout:osd.0: {} 2026-02-24T07:54:56.131 INFO:teuthology.orchestra.run.trial035.stderr:osd.0: osd_enable_op_tracker = '' 2026-02-24T07:54:56.149 INFO:teuthology.orchestra.run.trial035.stdout:osd.1: {} 2026-02-24T07:54:56.149 INFO:teuthology.orchestra.run.trial035.stderr:osd.1: osd_enable_op_tracker = '' 2026-02-24T07:54:56.166 INFO:teuthology.orchestra.run.trial035.stdout:osd.2: {} 2026-02-24T07:54:56.166 INFO:teuthology.orchestra.run.trial035.stderr:osd.2: osd_enable_op_tracker = '' 2026-02-24T07:54:56.181 INFO:teuthology.orchestra.run.trial035.stdout:osd.3: {} 2026-02-24T07:54:56.181 INFO:teuthology.orchestra.run.trial035.stderr:osd.3: osd_enable_op_tracker = '' 2026-02-24T07:54:56.308 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2026-02-24T07:54:56.308 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2026-02-24T07:54:56.308 INFO:tasks.thrashosds.thrasher:trim_stale_osdmaps 2026-02-24T07:54:56.308 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph tell osd.0 'trim stale osdmaps' 2026-02-24T07:54:56.456 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:54:56.591 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:54:56.588+0000 7fe2689e5640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:54:56.591 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:54:56.589+0000 7fe2689e5640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:54:56.612 INFO:teuthology.orchestra.run.trial035.stderr:no valid command found; 10 closest matches: 2026-02-24T07:54:56.612 INFO:teuthology.orchestra.run.trial035.stderr:0 2026-02-24T07:54:56.612 INFO:teuthology.orchestra.run.trial035.stderr:1 2026-02-24T07:54:56.612 INFO:teuthology.orchestra.run.trial035.stderr:2 2026-02-24T07:54:56.612 INFO:teuthology.orchestra.run.trial035.stderr:abort 2026-02-24T07:54:56.612 INFO:teuthology.orchestra.run.trial035.stderr:assert 2026-02-24T07:54:56.613 INFO:teuthology.orchestra.run.trial035.stderr:bench [] [] [] [] 2026-02-24T07:54:56.613 INFO:teuthology.orchestra.run.trial035.stderr:bluefs debug_inject_read_zeros 2026-02-24T07:54:56.613 INFO:teuthology.orchestra.run.trial035.stderr:bluefs files list 2026-02-24T07:54:56.613 INFO:teuthology.orchestra.run.trial035.stderr:bluefs stats 2026-02-24T07:54:56.613 INFO:teuthology.orchestra.run.trial035.stderr:bluestore allocator dump block 2026-02-24T07:54:56.613 INFO:teuthology.orchestra.run.trial035.stderr:Error EINVAL: invalid command 2026-02-24T07:54:56.647 DEBUG:teuthology.orchestra.run:got remote process result: 22 2026-02-24T07:54:56.648 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2026-02-24T07:54:56.648 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph tell osd.1 'trim stale osdmaps' 2026-02-24T07:54:56.794 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:54:56.926 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:54:56.923+0000 7f86f53ce640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:54:56.926 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:54:56.924+0000 7f86f53ce640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:54:56.948 INFO:teuthology.orchestra.run.trial035.stderr:no valid command found; 10 closest matches: 2026-02-24T07:54:56.948 INFO:teuthology.orchestra.run.trial035.stderr:0 2026-02-24T07:54:56.948 INFO:teuthology.orchestra.run.trial035.stderr:1 2026-02-24T07:54:56.948 INFO:teuthology.orchestra.run.trial035.stderr:2 2026-02-24T07:54:56.948 INFO:teuthology.orchestra.run.trial035.stderr:abort 2026-02-24T07:54:56.949 INFO:teuthology.orchestra.run.trial035.stderr:assert 2026-02-24T07:54:56.949 INFO:teuthology.orchestra.run.trial035.stderr:bench [] [] [] [] 2026-02-24T07:54:56.949 INFO:teuthology.orchestra.run.trial035.stderr:bluefs debug_inject_read_zeros 2026-02-24T07:54:56.949 INFO:teuthology.orchestra.run.trial035.stderr:bluefs files list 2026-02-24T07:54:56.949 INFO:teuthology.orchestra.run.trial035.stderr:bluefs stats 2026-02-24T07:54:56.949 INFO:teuthology.orchestra.run.trial035.stderr:bluestore allocator dump block 2026-02-24T07:54:56.949 INFO:teuthology.orchestra.run.trial035.stderr:Error EINVAL: invalid command 2026-02-24T07:54:56.984 DEBUG:teuthology.orchestra.run:got remote process result: 22 2026-02-24T07:54:56.984 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2026-02-24T07:54:56.985 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph tell osd.2 'trim stale osdmaps' 2026-02-24T07:54:57.016 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 13 15 98573 98558 473.799 471.625 0.0020859 0.00210922 2026-02-24T07:54:57.136 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:54:57.161 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:54:56 trial035 ceph-mon[39284]: pgmap v424: 17 pgs: 17 active+clean; 3.1 GiB data, 12 GiB used, 2.7 TiB / 2.7 TiB avail; 269 MiB/s wr, 4.31k op/s 2026-02-24T07:54:57.161 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:54:56 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/1329175965' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2026-02-24T07:54:57.161 INFO:journalctl@ceph.mgr.x.trial035.stdout:Feb 24 07:54:56 trial035 ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-mgr-x[39621]: ::ffff:10.20.193.91 - - [24/Feb/2026:07:54:56] "GET /metrics HTTP/1.1" 200 4156 "" "Prometheus/3.6.0" 2026-02-24T07:54:57.272 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:54:57.270+0000 7fa8feaae640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:54:57.273 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:54:57.271+0000 7fa8feaae640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:54:57.295 INFO:teuthology.orchestra.run.trial035.stderr:no valid command found; 10 closest matches: 2026-02-24T07:54:57.295 INFO:teuthology.orchestra.run.trial035.stderr:0 2026-02-24T07:54:57.295 INFO:teuthology.orchestra.run.trial035.stderr:1 2026-02-24T07:54:57.296 INFO:teuthology.orchestra.run.trial035.stderr:2 2026-02-24T07:54:57.296 INFO:teuthology.orchestra.run.trial035.stderr:abort 2026-02-24T07:54:57.296 INFO:teuthology.orchestra.run.trial035.stderr:assert 2026-02-24T07:54:57.296 INFO:teuthology.orchestra.run.trial035.stderr:bench [] [] [] [] 2026-02-24T07:54:57.296 INFO:teuthology.orchestra.run.trial035.stderr:bluefs debug_inject_read_zeros 2026-02-24T07:54:57.296 INFO:teuthology.orchestra.run.trial035.stderr:bluefs files list 2026-02-24T07:54:57.296 INFO:teuthology.orchestra.run.trial035.stderr:bluefs stats 2026-02-24T07:54:57.296 INFO:teuthology.orchestra.run.trial035.stderr:bluestore allocator dump block 2026-02-24T07:54:57.296 INFO:teuthology.orchestra.run.trial035.stderr:Error EINVAL: invalid command 2026-02-24T07:54:57.335 DEBUG:teuthology.orchestra.run:got remote process result: 22 2026-02-24T07:54:57.335 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2026-02-24T07:54:57.336 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph tell osd.3 'trim stale osdmaps' 2026-02-24T07:54:57.356 INFO:journalctl@ceph.prometheus.a.trial091.stdout:Feb 24 07:54:56 trial091 ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-prometheus-a[66007]: time=2026-02-24T07:54:56.973Z level=ERROR source=scrape.go:1630 msg="Failed to determine correct type of scrape target." component="scrape manager" scrape_pool=ceph target=http://trial035:9283/metrics content_type="text/html;charset=utf-8" fallback_media_type="" err="received unsupported Content-Type \"text/html;charset=utf-8\" and no fallback_scrape_protocol specified for target" 2026-02-24T07:54:57.530 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:54:57.657 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:54:57.655+0000 7fa90b478640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:54:57.658 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:54:57.655+0000 7fa90b478640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:54:57.679 INFO:teuthology.orchestra.run.trial035.stderr:no valid command found; 10 closest matches: 2026-02-24T07:54:57.679 INFO:teuthology.orchestra.run.trial035.stderr:0 2026-02-24T07:54:57.679 INFO:teuthology.orchestra.run.trial035.stderr:1 2026-02-24T07:54:57.679 INFO:teuthology.orchestra.run.trial035.stderr:2 2026-02-24T07:54:57.679 INFO:teuthology.orchestra.run.trial035.stderr:abort 2026-02-24T07:54:57.679 INFO:teuthology.orchestra.run.trial035.stderr:assert 2026-02-24T07:54:57.679 INFO:teuthology.orchestra.run.trial035.stderr:bench [] [] [] [] 2026-02-24T07:54:57.679 INFO:teuthology.orchestra.run.trial035.stderr:bluefs debug_inject_read_zeros 2026-02-24T07:54:57.680 INFO:teuthology.orchestra.run.trial035.stderr:bluefs files list 2026-02-24T07:54:57.680 INFO:teuthology.orchestra.run.trial035.stderr:bluefs stats 2026-02-24T07:54:57.680 INFO:teuthology.orchestra.run.trial035.stderr:bluestore allocator dump block 2026-02-24T07:54:57.680 INFO:teuthology.orchestra.run.trial035.stderr:Error EINVAL: invalid command 2026-02-24T07:54:57.713 DEBUG:teuthology.orchestra.run:got remote process result: 22 2026-02-24T07:54:57.714 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2026-02-24T07:54:58.016 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 14 15 106748 106733 476.449 510.938 0.00177361 0.0020975 2026-02-24T07:54:58.231 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2026-02-24T07:54:58.382 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:54:58.515 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:54:58.512+0000 7ff699a5a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:54:58.515 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:54:58.512+0000 7ff699a5a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:54:58.530 INFO:teuthology.orchestra.run.trial035.stdout:osd.0: {} 2026-02-24T07:54:58.530 INFO:teuthology.orchestra.run.trial035.stderr:osd.0: osd_enable_op_tracker = '' 2026-02-24T07:54:58.547 INFO:teuthology.orchestra.run.trial035.stdout:osd.1: {} 2026-02-24T07:54:58.548 INFO:teuthology.orchestra.run.trial035.stderr:osd.1: osd_enable_op_tracker = '' 2026-02-24T07:54:58.561 INFO:teuthology.orchestra.run.trial035.stdout:osd.2: {} 2026-02-24T07:54:58.561 INFO:teuthology.orchestra.run.trial035.stderr:osd.2: osd_enable_op_tracker = '' 2026-02-24T07:54:58.577 INFO:teuthology.orchestra.run.trial035.stdout:osd.3: {} 2026-02-24T07:54:58.578 INFO:teuthology.orchestra.run.trial035.stderr:osd.3: osd_enable_op_tracker = '' 2026-02-24T07:54:59.016 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 15 15 114964 114949 478.916 513.5 0.00193178 0.00208667 2026-02-24T07:54:59.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:54:58 trial035 ceph-mon[39284]: pgmap v425: 17 pgs: 17 active+clean; 5.5 GiB data, 20 GiB used, 2.7 TiB / 2.7 TiB avail; 466 MiB/s wr, 7.46k op/s 2026-02-24T07:54:59.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:54:58 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/3653301724' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2026-02-24T07:55:00.016 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 16 16 122701 122685 479.2 483.5 0.00180756 0.00208545 2026-02-24T07:55:00.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:54:59 trial035 ceph-mon[39284]: from='mgr.14284 10.20.193.35:0/3681373222' entity='mgr.x' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2026-02-24T07:55:00.624 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2026-02-24T07:55:00.767 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:55:01.016 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 17 16 130270 130254 478.837 473.062 0.00171807 0.00208706 2026-02-24T07:55:01.148 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:55:01.106+0000 7f9ac8608640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:55:01.148 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:55:01.107+0000 7f9ac8608640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:55:01.148 INFO:teuthology.orchestra.run.trial035.stderr:osd.0: osd_enable_op_tracker = '' 2026-02-24T07:55:01.148 INFO:teuthology.orchestra.run.trial035.stderr:osd.1: osd_enable_op_tracker = '' 2026-02-24T07:55:01.148 INFO:teuthology.orchestra.run.trial035.stdout:osd.0: {} 2026-02-24T07:55:01.148 INFO:teuthology.orchestra.run.trial035.stdout:osd.1: {} 2026-02-24T07:55:01.161 INFO:teuthology.orchestra.run.trial035.stdout:osd.2: {} 2026-02-24T07:55:01.161 INFO:teuthology.orchestra.run.trial035.stderr:osd.2: osd_enable_op_tracker = '' 2026-02-24T07:55:01.174 INFO:teuthology.orchestra.run.trial035.stdout:osd.3: {} 2026-02-24T07:55:01.174 INFO:teuthology.orchestra.run.trial035.stderr:osd.3: osd_enable_op_tracker = '' 2026-02-24T07:55:01.174 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:55:00 trial035 ceph-mon[39284]: pgmap v426: 17 pgs: 17 active+clean; 5.5 GiB data, 20 GiB used, 2.7 TiB / 2.7 TiB avail; 403 MiB/s wr, 6.44k op/s 2026-02-24T07:55:01.954 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:55:01 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/1619176973' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2026-02-24T07:55:01.954 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:55:01 trial035 ceph-mon[39284]: pgmap v427: 17 pgs: 17 active+clean; 7.9 GiB data, 28 GiB used, 2.7 TiB / 2.7 TiB avail; 608 MiB/s wr, 9.73k op/s 2026-02-24T07:55:02.016 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 18 16 135906 135890 471.803 352.25 0.00147167 0.00211817 2026-02-24T07:55:02.715 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2026-02-24T07:55:02.715 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph osd reweight 0 1 2026-02-24T07:55:02.860 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:55:03.016 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 19 16 143521 143505 472.019 475.938 0.00177806 0.0021172 2026-02-24T07:55:03.316 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:55:03.256+0000 7f8de4261640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:55:03.316 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:55:03.256+0000 7f8de4261640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:55:03.457 INFO:teuthology.orchestra.run.trial035.stderr:reweighted osd.0 to 1 (10000) 2026-02-24T07:55:03.704 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:55:03 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/921229821' entity='client.admin' cmd={"prefix": "osd reweight", "id": 0, "weight": 1.0} : dispatch 2026-02-24T07:55:03.728 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2026-02-24T07:55:04.016 INFO:tasks.radosbench.radosbench.0.trial035.stdout:2026-02-24T07:55:04.016217+0000 min lat: 0.000735688 max lat: 0.0424949 avg lat: 0.00214326 2026-02-24T07:55:04.017 INFO:tasks.radosbench.radosbench.0.trial035.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2026-02-24T07:55:04.017 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 20 16 149009 148993 465.567 343 0.014715 0.00214326 2026-02-24T07:55:04.176 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph osd reweight 1 1 2026-02-24T07:55:04.198 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:55:04.475 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:55:04.720 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:55:04 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/921229821' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 0, "weight": 1.0}]': finished 2026-02-24T07:55:04.720 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:55:04 trial035 ceph-mon[39284]: osdmap e93: 4 total, 4 up, 4 in 2026-02-24T07:55:04.720 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:55:04 trial035 ceph-mon[39284]: pgmap v429: 17 pgs: 17 active+clean; 7.9 GiB data, 28 GiB used, 2.7 TiB / 2.7 TiB avail; 488 MiB/s wr, 7.80k op/s 2026-02-24T07:55:04.726 INFO:teuthology.orchestra.run.trial035.stdout:osd.0: {} 2026-02-24T07:55:04.726 INFO:teuthology.orchestra.run.trial035.stdout:osd.1: {} 2026-02-24T07:55:04.726 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:55:04.676+0000 7fafd4b43640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:55:04.727 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:55:04.676+0000 7fafd4b43640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:55:04.727 INFO:teuthology.orchestra.run.trial035.stderr:osd.0: osd_enable_op_tracker = '' 2026-02-24T07:55:04.727 INFO:teuthology.orchestra.run.trial035.stderr:osd.1: osd_enable_op_tracker = '' 2026-02-24T07:55:04.734 INFO:teuthology.orchestra.run.trial035.stdout:osd.2: {} 2026-02-24T07:55:04.734 INFO:teuthology.orchestra.run.trial035.stderr:osd.2: osd_enable_op_tracker = '' 2026-02-24T07:55:04.748 INFO:teuthology.orchestra.run.trial035.stdout:osd.3: {} 2026-02-24T07:55:04.748 INFO:teuthology.orchestra.run.trial035.stderr:osd.3: osd_enable_op_tracker = '' 2026-02-24T07:55:05.017 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 21 16 154737 154721 460.443 358 0.00785051 0.00216948 2026-02-24T07:55:05.652 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:55:05.637+0000 7fd21c474640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:55:05.652 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:55:05.637+0000 7fd21c474640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:55:05.704 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:55:05 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/2834889956' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2026-02-24T07:55:06.017 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 22 15 161099 161084 457.589 397.688 0.00184795 0.00218405 2026-02-24T07:55:06.465 INFO:teuthology.orchestra.run.trial035.stderr:reweighted osd.1 to 1 (10000) 2026-02-24T07:55:06.954 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:55:06 trial035 ceph-mon[39284]: pgmap v430: 17 pgs: 17 active+clean; 7.9 GiB data, 28 GiB used, 2.7 TiB / 2.7 TiB avail; 488 MiB/s wr, 7.80k op/s 2026-02-24T07:55:06.954 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:55:06 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/817037530' entity='client.admin' cmd={"prefix": "osd reweight", "id": 1, "weight": 1.0} : dispatch 2026-02-24T07:55:07.016 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 23 16 167323 167307 454.603 388.938 0.00243808 0.00219839 2026-02-24T07:55:07.032 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph osd reweight 2 1 2026-02-24T07:55:07.147 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2026-02-24T07:55:07.187 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:55:07.307 INFO:journalctl@ceph.mgr.x.trial035.stdout:Feb 24 07:55:06 trial035 ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-mgr-x[39621]: ::ffff:10.20.193.91 - - [24/Feb/2026:07:55:06] "GET /metrics HTTP/1.1" 200 4210 "" "Prometheus/3.6.0" 2026-02-24T07:55:07.346 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:55:07.356 INFO:journalctl@ceph.prometheus.a.trial091.stdout:Feb 24 07:55:06 trial091 ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-prometheus-a[66007]: time=2026-02-24T07:55:06.973Z level=ERROR source=scrape.go:1630 msg="Failed to determine correct type of scrape target." component="scrape manager" scrape_pool=ceph target=http://trial035:9283/metrics content_type="text/html;charset=utf-8" fallback_media_type="" err="received unsupported Content-Type \"text/html;charset=utf-8\" and no fallback_scrape_protocol specified for target" 2026-02-24T07:55:07.570 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:55:07 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/817037530' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 1, "weight": 1.0}]': finished 2026-02-24T07:55:07.570 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:55:07 trial035 ceph-mon[39284]: osdmap e94: 4 total, 4 up, 4 in 2026-02-24T07:55:07.651 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:55:07.644+0000 7fe073097640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:55:07.652 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:55:07.644+0000 7fe073097640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:55:07.888 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:55:07.886+0000 7f5f6e01e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:55:07.888 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:55:07.886+0000 7f5f6e01e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:55:07.903 INFO:teuthology.orchestra.run.trial035.stdout:osd.0: {} 2026-02-24T07:55:07.904 INFO:teuthology.orchestra.run.trial035.stderr:osd.0: osd_enable_op_tracker = '' 2026-02-24T07:55:07.918 INFO:teuthology.orchestra.run.trial035.stdout:osd.1: {} 2026-02-24T07:55:07.918 INFO:teuthology.orchestra.run.trial035.stderr:osd.1: osd_enable_op_tracker = '' 2026-02-24T07:55:07.931 INFO:teuthology.orchestra.run.trial035.stdout:osd.2: {} 2026-02-24T07:55:07.932 INFO:teuthology.orchestra.run.trial035.stderr:osd.2: osd_enable_op_tracker = '' 2026-02-24T07:55:07.945 INFO:teuthology.orchestra.run.trial035.stdout:osd.3: {} 2026-02-24T07:55:07.946 INFO:teuthology.orchestra.run.trial035.stderr:osd.3: osd_enable_op_tracker = '' 2026-02-24T07:55:08.016 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 24 16 173752 173736 452.402 401.812 0.00250543 0.0022091 2026-02-24T07:55:08.468 INFO:teuthology.orchestra.run.trial035.stderr:reweighted osd.2 to 1 (10000) 2026-02-24T07:55:08.516 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph osd reweight 3 1 2026-02-24T07:55:08.717 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:55:08.743 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:55:08 trial035 ceph-mon[39284]: pgmap v432: 17 pgs: 17 active+clean; 9.7 GiB data, 33 GiB used, 2.7 TiB / 2.7 TiB avail; 546 MiB/s wr, 8.74k op/s 2026-02-24T07:55:08.743 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:55:08 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/3366930697' entity='client.admin' cmd={"prefix": "osd reweight", "id": 2, "weight": 1.0} : dispatch 2026-02-24T07:55:08.744 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:55:08 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/15752530' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2026-02-24T07:55:08.848 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:55:08.846+0000 7fc8279ca640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:55:08.848 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:55:08.847+0000 7fc8279ca640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:55:09.017 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 25 16 181379 181363 453.372 476.688 0.00192021 0.00220436 2026-02-24T07:55:09.469 INFO:teuthology.orchestra.run.trial035.stderr:reweighted osd.3 to 1 (10000) 2026-02-24T07:55:09.514 INFO:tasks.cephadm.ceph_manager.ceph:waiting for recovery to complete 2026-02-24T07:55:09.515 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph pg dump --format=json 2026-02-24T07:55:09.664 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:55:09.730 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:55:09 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/3366930697' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 2, "weight": 1.0}]': finished 2026-02-24T07:55:09.730 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:55:09 trial035 ceph-mon[39284]: osdmap e95: 4 total, 4 up, 4 in 2026-02-24T07:55:09.730 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:55:09 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/482114618' entity='client.admin' cmd={"prefix": "osd reweight", "id": 3, "weight": 1.0} : dispatch 2026-02-24T07:55:09.800 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:55:09.799+0000 7f5b48241640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:55:09.800 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:55:09.799+0000 7f5b48241640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:55:09.942 INFO:teuthology.orchestra.run.trial035.stdout: 2026-02-24T07:55:09.943 INFO:teuthology.orchestra.run.trial035.stderr:dumped all 2026-02-24T07:55:09.993 INFO:teuthology.orchestra.run.trial035.stdout:{"pg_ready":true,"pg_map":{"version":435,"stamp":"2026-02-24T07:55:09.756327+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":10446897696,"num_objects":159400,"num_object_clones":0,"num_object_copies":478200,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":159400,"num_whiteouts":0,"num_read":136,"num_read_kb":240,"num_write":159533,"num_write_kb":10203622,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":52,"ondisk_log_size":52,"up":51,"acting":51,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":51,"num_osds":4,"num_per_pool_osds":4,"num_per_pool_omap_osds":4,"kb":2936012800,"kb_used":35093692,"kb_used_data":30199788,"kb_used_omap":92,"kb_used_meta":4893667,"kb_avail":2900919108,"statfs":{"total":3006477107200,"available":2970541166592,"internally_reserved":0,"allocated":30924582912,"data_stored":30922670540,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":94875,"internal_metadata":5011115365},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[10,1,0,0,1,3],"upper_bound":64},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1994260480,"num_objects":30430,"num_object_clones":0,"num_object_copies":91290,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30430,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":30430,"num_write_kb":1947520,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.000789"},"pg_stats":[{"pgid":"6.f","version":"93'9952","reported_seq":9968,"reported_epoch":93,"state":"active+clean","last_fresh":"2026-02-24T07:55:05.864879+0000","last_change":"2026-02-24T07:54:42.927271+0000","last_active":"2026-02-24T07:55:05.864879+0000","last_peered":"2026-02-24T07:55:05.864879+0000","last_clean":"2026-02-24T07:55:05.864879+0000","last_became_active":"2026-02-24T07:54:42.927080+0000","last_became_peered":"2026-02-24T07:54:42.927080+0000","last_unstale":"2026-02-24T07:55:05.864879+0000","last_undegraded":"2026-02-24T07:55:05.864879+0000","last_fullsized":"2026-02-24T07:55:05.864879+0000","mapping_epoch":90,"log_start":"93'9949","ondisk_log_start":"93'9949","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_clean_scrub_stamp":"2026-02-24T07:54:41.918128+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:56:00.801546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":652214272,"num_objects":9952,"num_object_clones":0,"num_object_copies":29856,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9952,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9952,"num_write_kb":636928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.e","version":"93'9997","reported_seq":10013,"reported_epoch":93,"state":"active+clean","last_fresh":"2026-02-24T07:55:05.763307+0000","last_change":"2026-02-24T07:54:42.929104+0000","last_active":"2026-02-24T07:55:05.763307+0000","last_peered":"2026-02-24T07:55:05.763307+0000","last_clean":"2026-02-24T07:55:05.763307+0000","last_became_active":"2026-02-24T07:54:42.928748+0000","last_became_peered":"2026-02-24T07:54:42.928748+0000","last_unstale":"2026-02-24T07:55:05.763307+0000","last_undegraded":"2026-02-24T07:55:05.763307+0000","last_fullsized":"2026-02-24T07:55:05.763307+0000","mapping_epoch":90,"log_start":"93'9993","ondisk_log_start":"93'9993","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_clean_scrub_stamp":"2026-02-24T07:54:41.918128+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":3000,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:55:53.670906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":655163392,"num_objects":9997,"num_object_clones":0,"num_object_copies":29991,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9997,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9997,"num_write_kb":639808,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.d","version":"93'9977","reported_seq":9993,"reported_epoch":93,"state":"active+clean","last_fresh":"2026-02-24T07:55:05.759388+0000","last_change":"2026-02-24T07:54:42.927535+0000","last_active":"2026-02-24T07:55:05.759388+0000","last_peered":"2026-02-24T07:55:05.759388+0000","last_clean":"2026-02-24T07:55:05.759388+0000","last_became_active":"2026-02-24T07:54:42.926886+0000","last_became_peered":"2026-02-24T07:54:42.926886+0000","last_unstale":"2026-02-24T07:55:05.759388+0000","last_undegraded":"2026-02-24T07:55:05.759388+0000","last_fullsized":"2026-02-24T07:55:05.759388+0000","mapping_epoch":90,"log_start":"93'9974","ondisk_log_start":"93'9974","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_clean_scrub_stamp":"2026-02-24T07:54:41.918128+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:55:43.220815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":653852672,"num_objects":9977,"num_object_clones":0,"num_object_copies":29931,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9977,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9977,"num_write_kb":638528,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.c","version":"93'10014","reported_seq":10030,"reported_epoch":93,"state":"active+clean","last_fresh":"2026-02-24T07:55:05.860544+0000","last_change":"2026-02-24T07:54:42.928482+0000","last_active":"2026-02-24T07:55:05.860544+0000","last_peered":"2026-02-24T07:55:05.860544+0000","last_clean":"2026-02-24T07:55:05.860544+0000","last_became_active":"2026-02-24T07:54:42.928238+0000","last_became_peered":"2026-02-24T07:54:42.928238+0000","last_unstale":"2026-02-24T07:55:05.860544+0000","last_undegraded":"2026-02-24T07:55:05.860544+0000","last_fullsized":"2026-02-24T07:55:05.860544+0000","mapping_epoch":90,"log_start":"93'10011","ondisk_log_start":"93'10011","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_clean_scrub_stamp":"2026-02-24T07:54:41.918128+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:56:09.181158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":656277504,"num_objects":10014,"num_object_clones":0,"num_object_copies":30042,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10014,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10014,"num_write_kb":640896,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.b","version":"93'9916","reported_seq":9932,"reported_epoch":93,"state":"active+clean","last_fresh":"2026-02-24T07:55:05.861977+0000","last_change":"2026-02-24T07:54:42.927223+0000","last_active":"2026-02-24T07:55:05.861977+0000","last_peered":"2026-02-24T07:55:05.861977+0000","last_clean":"2026-02-24T07:55:05.861977+0000","last_became_active":"2026-02-24T07:54:42.926965+0000","last_became_peered":"2026-02-24T07:54:42.926965+0000","last_unstale":"2026-02-24T07:55:05.861977+0000","last_undegraded":"2026-02-24T07:55:05.861977+0000","last_fullsized":"2026-02-24T07:55:05.861977+0000","mapping_epoch":90,"log_start":"93'9913","ondisk_log_start":"93'9913","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_clean_scrub_stamp":"2026-02-24T07:54:41.918128+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:56:04.661233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":649854976,"num_objects":9916,"num_object_clones":0,"num_object_copies":29748,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9916,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9916,"num_write_kb":634624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.a","version":"93'9837","reported_seq":9853,"reported_epoch":93,"state":"active+clean","last_fresh":"2026-02-24T07:55:05.761189+0000","last_change":"2026-02-24T07:54:42.926926+0000","last_active":"2026-02-24T07:55:05.761189+0000","last_peered":"2026-02-24T07:55:05.761189+0000","last_clean":"2026-02-24T07:55:05.761189+0000","last_became_active":"2026-02-24T07:54:42.926477+0000","last_became_peered":"2026-02-24T07:54:42.926477+0000","last_unstale":"2026-02-24T07:55:05.761189+0000","last_undegraded":"2026-02-24T07:55:05.761189+0000","last_fullsized":"2026-02-24T07:55:05.761189+0000","mapping_epoch":90,"log_start":"93'9834","ondisk_log_start":"93'9834","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_clean_scrub_stamp":"2026-02-24T07:54:41.918128+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:55:46.420689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":644677632,"num_objects":9837,"num_object_clones":0,"num_object_copies":29511,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9837,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9837,"num_write_kb":629568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"93'9889","reported_seq":9905,"reported_epoch":93,"state":"active+clean","last_fresh":"2026-02-24T07:55:05.792421+0000","last_change":"2026-02-24T07:54:42.926317+0000","last_active":"2026-02-24T07:55:05.792421+0000","last_peered":"2026-02-24T07:55:05.792421+0000","last_clean":"2026-02-24T07:55:05.792421+0000","last_became_active":"2026-02-24T07:54:42.926136+0000","last_became_peered":"2026-02-24T07:54:42.926136+0000","last_unstale":"2026-02-24T07:55:05.792421+0000","last_undegraded":"2026-02-24T07:55:05.792421+0000","last_fullsized":"2026-02-24T07:55:05.792421+0000","mapping_epoch":90,"log_start":"93'9886","ondisk_log_start":"93'9886","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_clean_scrub_stamp":"2026-02-24T07:54:41.918128+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:55:45.176781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":648085504,"num_objects":9889,"num_object_clones":0,"num_object_copies":29667,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9889,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9889,"num_write_kb":632896,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,1],"acting":[0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.8","version":"93'9905","reported_seq":9921,"reported_epoch":93,"state":"active+clean","last_fresh":"2026-02-24T07:55:05.774340+0000","last_change":"2026-02-24T07:54:42.929353+0000","last_active":"2026-02-24T07:55:05.774340+0000","last_peered":"2026-02-24T07:55:05.774340+0000","last_clean":"2026-02-24T07:55:05.774340+0000","last_became_active":"2026-02-24T07:54:42.929078+0000","last_became_peered":"2026-02-24T07:54:42.929078+0000","last_unstale":"2026-02-24T07:55:05.774340+0000","last_undegraded":"2026-02-24T07:55:05.774340+0000","last_fullsized":"2026-02-24T07:55:05.774340+0000","mapping_epoch":90,"log_start":"93'9902","ondisk_log_start":"93'9902","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_clean_scrub_stamp":"2026-02-24T07:54:41.918128+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:55:48.303915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":649134080,"num_objects":9905,"num_object_clones":0,"num_object_copies":29715,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9905,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9905,"num_write_kb":633920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.6","version":"93'10039","reported_seq":10053,"reported_epoch":93,"state":"active+clean","last_fresh":"2026-02-24T07:55:05.863532+0000","last_change":"2026-02-24T07:54:42.926358+0000","last_active":"2026-02-24T07:55:05.863532+0000","last_peered":"2026-02-24T07:55:05.863532+0000","last_clean":"2026-02-24T07:55:05.863532+0000","last_became_active":"2026-02-24T07:54:42.926045+0000","last_became_peered":"2026-02-24T07:54:42.926045+0000","last_unstale":"2026-02-24T07:55:05.863532+0000","last_undegraded":"2026-02-24T07:55:05.863532+0000","last_fullsized":"2026-02-24T07:55:05.863532+0000","mapping_epoch":90,"log_start":"93'10036","ondisk_log_start":"93'10036","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_clean_scrub_stamp":"2026-02-24T07:54:41.918128+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:56:09.126690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":657915904,"num_objects":10039,"num_object_clones":0,"num_object_copies":30117,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10039,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10039,"num_write_kb":642496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.5","version":"93'9967","reported_seq":9982,"reported_epoch":93,"state":"active+clean","last_fresh":"2026-02-24T07:55:05.762842+0000","last_change":"2026-02-24T07:54:42.928143+0000","last_active":"2026-02-24T07:55:05.762842+0000","last_peered":"2026-02-24T07:55:05.762842+0000","last_clean":"2026-02-24T07:55:05.762842+0000","last_became_active":"2026-02-24T07:54:42.927535+0000","last_became_peered":"2026-02-24T07:54:42.927535+0000","last_unstale":"2026-02-24T07:55:05.762842+0000","last_undegraded":"2026-02-24T07:55:05.762842+0000","last_fullsized":"2026-02-24T07:55:05.762842+0000","mapping_epoch":90,"log_start":"93'9964","ondisk_log_start":"93'9964","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_clean_scrub_stamp":"2026-02-24T07:54:41.918128+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:55:53.650972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":653197312,"num_objects":9967,"num_object_clones":0,"num_object_copies":29901,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9967,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9967,"num_write_kb":637888,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.4","version":"93'10028","reported_seq":10042,"reported_epoch":93,"state":"active+clean","last_fresh":"2026-02-24T07:55:05.865876+0000","last_change":"2026-02-24T07:54:42.928344+0000","last_active":"2026-02-24T07:55:05.865876+0000","last_peered":"2026-02-24T07:55:05.865876+0000","last_clean":"2026-02-24T07:55:05.865876+0000","last_became_active":"2026-02-24T07:54:42.928011+0000","last_became_peered":"2026-02-24T07:54:42.928011+0000","last_unstale":"2026-02-24T07:55:05.865876+0000","last_undegraded":"2026-02-24T07:55:05.865876+0000","last_fullsized":"2026-02-24T07:55:05.865876+0000","mapping_epoch":90,"log_start":"93'10025","ondisk_log_start":"93'10025","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_clean_scrub_stamp":"2026-02-24T07:54:41.918128+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:55:43.506947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":657195008,"num_objects":10028,"num_object_clones":0,"num_object_copies":30084,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10028,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10028,"num_write_kb":641792,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.3","version":"93'10020","reported_seq":10036,"reported_epoch":93,"state":"active+clean","last_fresh":"2026-02-24T07:55:05.757695+0000","last_change":"2026-02-24T07:54:42.928536+0000","last_active":"2026-02-24T07:55:05.757695+0000","last_peered":"2026-02-24T07:55:05.757695+0000","last_clean":"2026-02-24T07:55:05.757695+0000","last_became_active":"2026-02-24T07:54:42.927730+0000","last_became_peered":"2026-02-24T07:54:42.927730+0000","last_unstale":"2026-02-24T07:55:05.757695+0000","last_undegraded":"2026-02-24T07:55:05.757695+0000","last_fullsized":"2026-02-24T07:55:05.757695+0000","mapping_epoch":90,"log_start":"93'10017","ondisk_log_start":"93'10017","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_clean_scrub_stamp":"2026-02-24T07:54:41.918128+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:55:58.987317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":656670720,"num_objects":10020,"num_object_clones":0,"num_object_copies":30060,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10020,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10020,"num_write_kb":641280,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.2","version":"93'9970","reported_seq":9986,"reported_epoch":93,"state":"active+clean","last_fresh":"2026-02-24T07:55:05.776065+0000","last_change":"2026-02-24T07:54:42.930452+0000","last_active":"2026-02-24T07:55:05.776065+0000","last_peered":"2026-02-24T07:55:05.776065+0000","last_clean":"2026-02-24T07:55:05.776065+0000","last_became_active":"2026-02-24T07:54:42.930247+0000","last_became_peered":"2026-02-24T07:54:42.930247+0000","last_unstale":"2026-02-24T07:55:05.776065+0000","last_undegraded":"2026-02-24T07:55:05.776065+0000","last_fullsized":"2026-02-24T07:55:05.776065+0000","mapping_epoch":90,"log_start":"93'9967","ondisk_log_start":"93'9967","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_clean_scrub_stamp":"2026-02-24T07:54:41.918128+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:55:49.096947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":653393920,"num_objects":9970,"num_object_clones":0,"num_object_copies":29910,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9970,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9970,"num_write_kb":638080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1","version":"93'10047","reported_seq":10063,"reported_epoch":93,"state":"active+clean","last_fresh":"2026-02-24T07:55:05.861654+0000","last_change":"2026-02-24T07:54:42.926619+0000","last_active":"2026-02-24T07:55:05.861654+0000","last_peered":"2026-02-24T07:55:05.861654+0000","last_clean":"2026-02-24T07:55:05.861654+0000","last_became_active":"2026-02-24T07:54:42.926442+0000","last_became_peered":"2026-02-24T07:54:42.926442+0000","last_unstale":"2026-02-24T07:55:05.861654+0000","last_undegraded":"2026-02-24T07:55:05.861654+0000","last_fullsized":"2026-02-24T07:55:05.861654+0000","mapping_epoch":90,"log_start":"93'10044","ondisk_log_start":"93'10044","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_clean_scrub_stamp":"2026-02-24T07:54:41.918128+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:56:03.125413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":658440192,"num_objects":10047,"num_object_clones":0,"num_object_copies":30141,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10047,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10047,"num_write_kb":643008,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.0","version":"93'9880","reported_seq":9896,"reported_epoch":93,"state":"active+clean","last_fresh":"2026-02-24T07:55:05.789947+0000","last_change":"2026-02-24T07:54:42.927178+0000","last_active":"2026-02-24T07:55:05.789947+0000","last_peered":"2026-02-24T07:55:05.789947+0000","last_clean":"2026-02-24T07:55:05.789947+0000","last_became_active":"2026-02-24T07:54:42.926910+0000","last_became_peered":"2026-02-24T07:54:42.926910+0000","last_unstale":"2026-02-24T07:55:05.789947+0000","last_undegraded":"2026-02-24T07:55:05.789947+0000","last_fullsized":"2026-02-24T07:55:05.789947+0000","mapping_epoch":90,"log_start":"93'9877","ondisk_log_start":"93'9877","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_clean_scrub_stamp":"2026-02-24T07:54:41.918128+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:55:58.002045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":647495680,"num_objects":9880,"num_object_clones":0,"num_object_copies":29640,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9880,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9880,"num_write_kb":632320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.7","version":"93'9960","reported_seq":9976,"reported_epoch":93,"state":"active+clean","last_fresh":"2026-02-24T07:55:05.775405+0000","last_change":"2026-02-24T07:54:42.930348+0000","last_active":"2026-02-24T07:55:05.775405+0000","last_peered":"2026-02-24T07:55:05.775405+0000","last_clean":"2026-02-24T07:55:05.775405+0000","last_became_active":"2026-02-24T07:54:42.930142+0000","last_became_peered":"2026-02-24T07:54:42.930142+0000","last_unstale":"2026-02-24T07:55:05.775405+0000","last_undegraded":"2026-02-24T07:55:05.775405+0000","last_fullsized":"2026-02-24T07:55:05.775405+0000","mapping_epoch":90,"log_start":"93'9957","ondisk_log_start":"93'9957","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_clean_scrub_stamp":"2026-02-24T07:54:41.918128+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:55:56.704334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":652738560,"num_objects":9960,"num_object_clones":0,"num_object_copies":29880,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9960,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9960,"num_write_kb":637440,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"29'99","reported_seq":384,"reported_epoch":93,"state":"active+clean","last_fresh":"2026-02-24T07:55:03.457261+0000","last_change":"2026-02-24T07:54:08.521371+0000","last_active":"2026-02-24T07:55:03.457261+0000","last_peered":"2026-02-24T07:55:03.457261+0000","last_clean":"2026-02-24T07:55:03.457261+0000","last_became_active":"2026-02-24T07:42:54.186719+0000","last_became_peered":"2026-02-24T07:42:54.186719+0000","last_unstale":"2026-02-24T07:55:03.457261+0000","last_undegraded":"2026-02-24T07:55:03.457261+0000","last_fullsized":"2026-02-24T07:55:03.457261+0000","mapping_epoch":27,"log_start":"29'96","ondisk_log_start":"29'96","created":20,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'99","last_scrub_stamp":"2026-02-24T07:54:08.521342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:42:43.640776+0000","last_clean_scrub_stamp":"2026-02-24T07:54:08.521342+0000","objects_scrubbed":2,"log_size":3,"log_dups_size":96,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:55:15.605551+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":240,"num_write":135,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":16,"stat_sum":{"num_bytes":10446307328,"num_objects":159398,"num_object_clones":0,"num_object_copies":478194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":159398,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":159398,"num_write_kb":10201472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":31335186432,"data_stored":31335186432,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":49,"ondisk_log_size":49,"up":48,"acting":48,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":240,"num_write":135,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":2468,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":3,"up_from":25,"seq":107374182549,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":7817392,"kb_used_data":6313564,"kb_used_omap":25,"kb_used_meta":1503782,"kb_avail":726185808,"statfs":{"total":751619276800,"available":743614267392,"internally_reserved":0,"allocated":6465089536,"data_stored":6464610555,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":25943,"internal_metadata":1539873449},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":19,"seq":81604378775,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":8879728,"kb_used_data":7553884,"kb_used_omap":22,"kb_used_meta":1325801,"kb_avail":725123472,"statfs":{"total":751619276800,"available":742526435328,"internally_reserved":0,"allocated":7735177216,"data_stored":7734698235,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":23488,"internal_metadata":1357620288},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,1,3],"upper_bound":64},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542296,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":9545964,"kb_used_data":8709336,"kb_used_omap":21,"kb_used_meta":836522,"kb_avail":724457236,"statfs":{"total":751619276800,"available":741844209664,"internally_reserved":0,"allocated":8918360064,"data_stored":8917884635,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":22188,"internal_metadata":856598868},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[5,1],"upper_bound":4},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705819,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":8850608,"kb_used_data":7623004,"kb_used_omap":22,"kb_used_meta":1227561,"kb_avail":725152592,"statfs":{"total":751619276800,"available":742556254208,"internally_reserved":0,"allocated":7805956096,"data_stored":7805477115,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":23256,"internal_metadata":1257022760},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[5],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":2468,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":7830503424,"data_stored":7830503424,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":9163440128,"data_stored":9163440128,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":7819427840,"data_stored":7819427840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":6521815040,"data_stored":6521815040,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-24T07:55:09.995 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph pg dump --format=json 2026-02-24T07:55:09.995 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2026-02-24T07:55:10.017 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 26 15 189029 189014 454.325 478.188 0.00259125 0.00219971 2026-02-24T07:55:10.146 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:55:10.156 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:55:10.282 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:55:10.281+0000 7f62a3d58640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:55:10.283 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:55:10.281+0000 7f62a3d58640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:55:10.283 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:55:10.282+0000 7f75fec31640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:55:10.284 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:55:10.283+0000 7f75fec31640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:55:10.301 INFO:teuthology.orchestra.run.trial035.stdout:osd.0: {} 2026-02-24T07:55:10.302 INFO:teuthology.orchestra.run.trial035.stderr:osd.0: osd_enable_op_tracker = '' 2026-02-24T07:55:10.318 INFO:teuthology.orchestra.run.trial035.stdout:osd.1: {} 2026-02-24T07:55:10.318 INFO:teuthology.orchestra.run.trial035.stderr:osd.1: osd_enable_op_tracker = '' 2026-02-24T07:55:10.331 INFO:teuthology.orchestra.run.trial035.stdout:osd.2: {} 2026-02-24T07:55:10.331 INFO:teuthology.orchestra.run.trial035.stderr:osd.2: osd_enable_op_tracker = '' 2026-02-24T07:55:10.345 INFO:teuthology.orchestra.run.trial035.stdout:osd.3: {} 2026-02-24T07:55:10.345 INFO:teuthology.orchestra.run.trial035.stderr:osd.3: osd_enable_op_tracker = '' 2026-02-24T07:55:10.449 INFO:teuthology.orchestra.run.trial035.stdout: 2026-02-24T07:55:10.450 INFO:teuthology.orchestra.run.trial035.stderr:dumped all 2026-02-24T07:55:10.503 INFO:teuthology.orchestra.run.trial035.stdout:{"pg_ready":true,"pg_map":{"version":435,"stamp":"2026-02-24T07:55:09.756327+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":10446897696,"num_objects":159400,"num_object_clones":0,"num_object_copies":478200,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":159400,"num_whiteouts":0,"num_read":136,"num_read_kb":240,"num_write":159533,"num_write_kb":10203622,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":52,"ondisk_log_size":52,"up":51,"acting":51,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":51,"num_osds":4,"num_per_pool_osds":4,"num_per_pool_omap_osds":4,"kb":2936012800,"kb_used":35093692,"kb_used_data":30199788,"kb_used_omap":92,"kb_used_meta":4893667,"kb_avail":2900919108,"statfs":{"total":3006477107200,"available":2970541166592,"internally_reserved":0,"allocated":30924582912,"data_stored":30922670540,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":94875,"internal_metadata":5011115365},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[10,1,0,0,1,3],"upper_bound":64},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1994260480,"num_objects":30430,"num_object_clones":0,"num_object_copies":91290,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30430,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":30430,"num_write_kb":1947520,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.000789"},"pg_stats":[{"pgid":"6.f","version":"93'9952","reported_seq":9968,"reported_epoch":93,"state":"active+clean","last_fresh":"2026-02-24T07:55:05.864879+0000","last_change":"2026-02-24T07:54:42.927271+0000","last_active":"2026-02-24T07:55:05.864879+0000","last_peered":"2026-02-24T07:55:05.864879+0000","last_clean":"2026-02-24T07:55:05.864879+0000","last_became_active":"2026-02-24T07:54:42.927080+0000","last_became_peered":"2026-02-24T07:54:42.927080+0000","last_unstale":"2026-02-24T07:55:05.864879+0000","last_undegraded":"2026-02-24T07:55:05.864879+0000","last_fullsized":"2026-02-24T07:55:05.864879+0000","mapping_epoch":90,"log_start":"93'9949","ondisk_log_start":"93'9949","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_clean_scrub_stamp":"2026-02-24T07:54:41.918128+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:56:00.801546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":652214272,"num_objects":9952,"num_object_clones":0,"num_object_copies":29856,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9952,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9952,"num_write_kb":636928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.e","version":"93'9997","reported_seq":10013,"reported_epoch":93,"state":"active+clean","last_fresh":"2026-02-24T07:55:05.763307+0000","last_change":"2026-02-24T07:54:42.929104+0000","last_active":"2026-02-24T07:55:05.763307+0000","last_peered":"2026-02-24T07:55:05.763307+0000","last_clean":"2026-02-24T07:55:05.763307+0000","last_became_active":"2026-02-24T07:54:42.928748+0000","last_became_peered":"2026-02-24T07:54:42.928748+0000","last_unstale":"2026-02-24T07:55:05.763307+0000","last_undegraded":"2026-02-24T07:55:05.763307+0000","last_fullsized":"2026-02-24T07:55:05.763307+0000","mapping_epoch":90,"log_start":"93'9993","ondisk_log_start":"93'9993","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_clean_scrub_stamp":"2026-02-24T07:54:41.918128+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":3000,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:55:53.670906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":655163392,"num_objects":9997,"num_object_clones":0,"num_object_copies":29991,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9997,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9997,"num_write_kb":639808,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.d","version":"93'9977","reported_seq":9993,"reported_epoch":93,"state":"active+clean","last_fresh":"2026-02-24T07:55:05.759388+0000","last_change":"2026-02-24T07:54:42.927535+0000","last_active":"2026-02-24T07:55:05.759388+0000","last_peered":"2026-02-24T07:55:05.759388+0000","last_clean":"2026-02-24T07:55:05.759388+0000","last_became_active":"2026-02-24T07:54:42.926886+0000","last_became_peered":"2026-02-24T07:54:42.926886+0000","last_unstale":"2026-02-24T07:55:05.759388+0000","last_undegraded":"2026-02-24T07:55:05.759388+0000","last_fullsized":"2026-02-24T07:55:05.759388+0000","mapping_epoch":90,"log_start":"93'9974","ondisk_log_start":"93'9974","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_clean_scrub_stamp":"2026-02-24T07:54:41.918128+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:55:43.220815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":653852672,"num_objects":9977,"num_object_clones":0,"num_object_copies":29931,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9977,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9977,"num_write_kb":638528,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.c","version":"93'10014","reported_seq":10030,"reported_epoch":93,"state":"active+clean","last_fresh":"2026-02-24T07:55:05.860544+0000","last_change":"2026-02-24T07:54:42.928482+0000","last_active":"2026-02-24T07:55:05.860544+0000","last_peered":"2026-02-24T07:55:05.860544+0000","last_clean":"2026-02-24T07:55:05.860544+0000","last_became_active":"2026-02-24T07:54:42.928238+0000","last_became_peered":"2026-02-24T07:54:42.928238+0000","last_unstale":"2026-02-24T07:55:05.860544+0000","last_undegraded":"2026-02-24T07:55:05.860544+0000","last_fullsized":"2026-02-24T07:55:05.860544+0000","mapping_epoch":90,"log_start":"93'10011","ondisk_log_start":"93'10011","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_clean_scrub_stamp":"2026-02-24T07:54:41.918128+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:56:09.181158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":656277504,"num_objects":10014,"num_object_clones":0,"num_object_copies":30042,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10014,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10014,"num_write_kb":640896,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.b","version":"93'9916","reported_seq":9932,"reported_epoch":93,"state":"active+clean","last_fresh":"2026-02-24T07:55:05.861977+0000","last_change":"2026-02-24T07:54:42.927223+0000","last_active":"2026-02-24T07:55:05.861977+0000","last_peered":"2026-02-24T07:55:05.861977+0000","last_clean":"2026-02-24T07:55:05.861977+0000","last_became_active":"2026-02-24T07:54:42.926965+0000","last_became_peered":"2026-02-24T07:54:42.926965+0000","last_unstale":"2026-02-24T07:55:05.861977+0000","last_undegraded":"2026-02-24T07:55:05.861977+0000","last_fullsized":"2026-02-24T07:55:05.861977+0000","mapping_epoch":90,"log_start":"93'9913","ondisk_log_start":"93'9913","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_clean_scrub_stamp":"2026-02-24T07:54:41.918128+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:56:04.661233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":649854976,"num_objects":9916,"num_object_clones":0,"num_object_copies":29748,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9916,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9916,"num_write_kb":634624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.a","version":"93'9837","reported_seq":9853,"reported_epoch":93,"state":"active+clean","last_fresh":"2026-02-24T07:55:05.761189+0000","last_change":"2026-02-24T07:54:42.926926+0000","last_active":"2026-02-24T07:55:05.761189+0000","last_peered":"2026-02-24T07:55:05.761189+0000","last_clean":"2026-02-24T07:55:05.761189+0000","last_became_active":"2026-02-24T07:54:42.926477+0000","last_became_peered":"2026-02-24T07:54:42.926477+0000","last_unstale":"2026-02-24T07:55:05.761189+0000","last_undegraded":"2026-02-24T07:55:05.761189+0000","last_fullsized":"2026-02-24T07:55:05.761189+0000","mapping_epoch":90,"log_start":"93'9834","ondisk_log_start":"93'9834","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_clean_scrub_stamp":"2026-02-24T07:54:41.918128+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:55:46.420689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":644677632,"num_objects":9837,"num_object_clones":0,"num_object_copies":29511,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9837,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9837,"num_write_kb":629568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"93'9889","reported_seq":9905,"reported_epoch":93,"state":"active+clean","last_fresh":"2026-02-24T07:55:05.792421+0000","last_change":"2026-02-24T07:54:42.926317+0000","last_active":"2026-02-24T07:55:05.792421+0000","last_peered":"2026-02-24T07:55:05.792421+0000","last_clean":"2026-02-24T07:55:05.792421+0000","last_became_active":"2026-02-24T07:54:42.926136+0000","last_became_peered":"2026-02-24T07:54:42.926136+0000","last_unstale":"2026-02-24T07:55:05.792421+0000","last_undegraded":"2026-02-24T07:55:05.792421+0000","last_fullsized":"2026-02-24T07:55:05.792421+0000","mapping_epoch":90,"log_start":"93'9886","ondisk_log_start":"93'9886","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_clean_scrub_stamp":"2026-02-24T07:54:41.918128+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:55:45.176781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":648085504,"num_objects":9889,"num_object_clones":0,"num_object_copies":29667,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9889,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9889,"num_write_kb":632896,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,1],"acting":[0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.8","version":"93'9905","reported_seq":9921,"reported_epoch":93,"state":"active+clean","last_fresh":"2026-02-24T07:55:05.774340+0000","last_change":"2026-02-24T07:54:42.929353+0000","last_active":"2026-02-24T07:55:05.774340+0000","last_peered":"2026-02-24T07:55:05.774340+0000","last_clean":"2026-02-24T07:55:05.774340+0000","last_became_active":"2026-02-24T07:54:42.929078+0000","last_became_peered":"2026-02-24T07:54:42.929078+0000","last_unstale":"2026-02-24T07:55:05.774340+0000","last_undegraded":"2026-02-24T07:55:05.774340+0000","last_fullsized":"2026-02-24T07:55:05.774340+0000","mapping_epoch":90,"log_start":"93'9902","ondisk_log_start":"93'9902","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_clean_scrub_stamp":"2026-02-24T07:54:41.918128+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:55:48.303915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":649134080,"num_objects":9905,"num_object_clones":0,"num_object_copies":29715,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9905,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9905,"num_write_kb":633920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.6","version":"93'10039","reported_seq":10053,"reported_epoch":93,"state":"active+clean","last_fresh":"2026-02-24T07:55:05.863532+0000","last_change":"2026-02-24T07:54:42.926358+0000","last_active":"2026-02-24T07:55:05.863532+0000","last_peered":"2026-02-24T07:55:05.863532+0000","last_clean":"2026-02-24T07:55:05.863532+0000","last_became_active":"2026-02-24T07:54:42.926045+0000","last_became_peered":"2026-02-24T07:54:42.926045+0000","last_unstale":"2026-02-24T07:55:05.863532+0000","last_undegraded":"2026-02-24T07:55:05.863532+0000","last_fullsized":"2026-02-24T07:55:05.863532+0000","mapping_epoch":90,"log_start":"93'10036","ondisk_log_start":"93'10036","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_clean_scrub_stamp":"2026-02-24T07:54:41.918128+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:56:09.126690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":657915904,"num_objects":10039,"num_object_clones":0,"num_object_copies":30117,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10039,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10039,"num_write_kb":642496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.5","version":"93'9967","reported_seq":9982,"reported_epoch":93,"state":"active+clean","last_fresh":"2026-02-24T07:55:05.762842+0000","last_change":"2026-02-24T07:54:42.928143+0000","last_active":"2026-02-24T07:55:05.762842+0000","last_peered":"2026-02-24T07:55:05.762842+0000","last_clean":"2026-02-24T07:55:05.762842+0000","last_became_active":"2026-02-24T07:54:42.927535+0000","last_became_peered":"2026-02-24T07:54:42.927535+0000","last_unstale":"2026-02-24T07:55:05.762842+0000","last_undegraded":"2026-02-24T07:55:05.762842+0000","last_fullsized":"2026-02-24T07:55:05.762842+0000","mapping_epoch":90,"log_start":"93'9964","ondisk_log_start":"93'9964","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_clean_scrub_stamp":"2026-02-24T07:54:41.918128+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:55:53.650972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":653197312,"num_objects":9967,"num_object_clones":0,"num_object_copies":29901,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9967,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9967,"num_write_kb":637888,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.4","version":"93'10028","reported_seq":10042,"reported_epoch":93,"state":"active+clean","last_fresh":"2026-02-24T07:55:05.865876+0000","last_change":"2026-02-24T07:54:42.928344+0000","last_active":"2026-02-24T07:55:05.865876+0000","last_peered":"2026-02-24T07:55:05.865876+0000","last_clean":"2026-02-24T07:55:05.865876+0000","last_became_active":"2026-02-24T07:54:42.928011+0000","last_became_peered":"2026-02-24T07:54:42.928011+0000","last_unstale":"2026-02-24T07:55:05.865876+0000","last_undegraded":"2026-02-24T07:55:05.865876+0000","last_fullsized":"2026-02-24T07:55:05.865876+0000","mapping_epoch":90,"log_start":"93'10025","ondisk_log_start":"93'10025","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_clean_scrub_stamp":"2026-02-24T07:54:41.918128+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:55:43.506947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":657195008,"num_objects":10028,"num_object_clones":0,"num_object_copies":30084,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10028,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10028,"num_write_kb":641792,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.3","version":"93'10020","reported_seq":10036,"reported_epoch":93,"state":"active+clean","last_fresh":"2026-02-24T07:55:05.757695+0000","last_change":"2026-02-24T07:54:42.928536+0000","last_active":"2026-02-24T07:55:05.757695+0000","last_peered":"2026-02-24T07:55:05.757695+0000","last_clean":"2026-02-24T07:55:05.757695+0000","last_became_active":"2026-02-24T07:54:42.927730+0000","last_became_peered":"2026-02-24T07:54:42.927730+0000","last_unstale":"2026-02-24T07:55:05.757695+0000","last_undegraded":"2026-02-24T07:55:05.757695+0000","last_fullsized":"2026-02-24T07:55:05.757695+0000","mapping_epoch":90,"log_start":"93'10017","ondisk_log_start":"93'10017","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_clean_scrub_stamp":"2026-02-24T07:54:41.918128+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:55:58.987317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":656670720,"num_objects":10020,"num_object_clones":0,"num_object_copies":30060,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10020,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10020,"num_write_kb":641280,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.2","version":"93'9970","reported_seq":9986,"reported_epoch":93,"state":"active+clean","last_fresh":"2026-02-24T07:55:05.776065+0000","last_change":"2026-02-24T07:54:42.930452+0000","last_active":"2026-02-24T07:55:05.776065+0000","last_peered":"2026-02-24T07:55:05.776065+0000","last_clean":"2026-02-24T07:55:05.776065+0000","last_became_active":"2026-02-24T07:54:42.930247+0000","last_became_peered":"2026-02-24T07:54:42.930247+0000","last_unstale":"2026-02-24T07:55:05.776065+0000","last_undegraded":"2026-02-24T07:55:05.776065+0000","last_fullsized":"2026-02-24T07:55:05.776065+0000","mapping_epoch":90,"log_start":"93'9967","ondisk_log_start":"93'9967","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_clean_scrub_stamp":"2026-02-24T07:54:41.918128+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:55:49.096947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":653393920,"num_objects":9970,"num_object_clones":0,"num_object_copies":29910,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9970,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9970,"num_write_kb":638080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1","version":"93'10047","reported_seq":10063,"reported_epoch":93,"state":"active+clean","last_fresh":"2026-02-24T07:55:05.861654+0000","last_change":"2026-02-24T07:54:42.926619+0000","last_active":"2026-02-24T07:55:05.861654+0000","last_peered":"2026-02-24T07:55:05.861654+0000","last_clean":"2026-02-24T07:55:05.861654+0000","last_became_active":"2026-02-24T07:54:42.926442+0000","last_became_peered":"2026-02-24T07:54:42.926442+0000","last_unstale":"2026-02-24T07:55:05.861654+0000","last_undegraded":"2026-02-24T07:55:05.861654+0000","last_fullsized":"2026-02-24T07:55:05.861654+0000","mapping_epoch":90,"log_start":"93'10044","ondisk_log_start":"93'10044","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_clean_scrub_stamp":"2026-02-24T07:54:41.918128+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:56:03.125413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":658440192,"num_objects":10047,"num_object_clones":0,"num_object_copies":30141,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10047,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10047,"num_write_kb":643008,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.0","version":"93'9880","reported_seq":9896,"reported_epoch":93,"state":"active+clean","last_fresh":"2026-02-24T07:55:05.789947+0000","last_change":"2026-02-24T07:54:42.927178+0000","last_active":"2026-02-24T07:55:05.789947+0000","last_peered":"2026-02-24T07:55:05.789947+0000","last_clean":"2026-02-24T07:55:05.789947+0000","last_became_active":"2026-02-24T07:54:42.926910+0000","last_became_peered":"2026-02-24T07:54:42.926910+0000","last_unstale":"2026-02-24T07:55:05.789947+0000","last_undegraded":"2026-02-24T07:55:05.789947+0000","last_fullsized":"2026-02-24T07:55:05.789947+0000","mapping_epoch":90,"log_start":"93'9877","ondisk_log_start":"93'9877","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_clean_scrub_stamp":"2026-02-24T07:54:41.918128+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:55:58.002045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":647495680,"num_objects":9880,"num_object_clones":0,"num_object_copies":29640,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9880,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9880,"num_write_kb":632320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.7","version":"93'9960","reported_seq":9976,"reported_epoch":93,"state":"active+clean","last_fresh":"2026-02-24T07:55:05.775405+0000","last_change":"2026-02-24T07:54:42.930348+0000","last_active":"2026-02-24T07:55:05.775405+0000","last_peered":"2026-02-24T07:55:05.775405+0000","last_clean":"2026-02-24T07:55:05.775405+0000","last_became_active":"2026-02-24T07:54:42.930142+0000","last_became_peered":"2026-02-24T07:54:42.930142+0000","last_unstale":"2026-02-24T07:55:05.775405+0000","last_undegraded":"2026-02-24T07:55:05.775405+0000","last_fullsized":"2026-02-24T07:55:05.775405+0000","mapping_epoch":90,"log_start":"93'9957","ondisk_log_start":"93'9957","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_clean_scrub_stamp":"2026-02-24T07:54:41.918128+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:55:56.704334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":652738560,"num_objects":9960,"num_object_clones":0,"num_object_copies":29880,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9960,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9960,"num_write_kb":637440,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"29'99","reported_seq":384,"reported_epoch":93,"state":"active+clean","last_fresh":"2026-02-24T07:55:03.457261+0000","last_change":"2026-02-24T07:54:08.521371+0000","last_active":"2026-02-24T07:55:03.457261+0000","last_peered":"2026-02-24T07:55:03.457261+0000","last_clean":"2026-02-24T07:55:03.457261+0000","last_became_active":"2026-02-24T07:42:54.186719+0000","last_became_peered":"2026-02-24T07:42:54.186719+0000","last_unstale":"2026-02-24T07:55:03.457261+0000","last_undegraded":"2026-02-24T07:55:03.457261+0000","last_fullsized":"2026-02-24T07:55:03.457261+0000","mapping_epoch":27,"log_start":"29'96","ondisk_log_start":"29'96","created":20,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'99","last_scrub_stamp":"2026-02-24T07:54:08.521342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:42:43.640776+0000","last_clean_scrub_stamp":"2026-02-24T07:54:08.521342+0000","objects_scrubbed":2,"log_size":3,"log_dups_size":96,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:55:15.605551+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":240,"num_write":135,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":16,"stat_sum":{"num_bytes":10446307328,"num_objects":159398,"num_object_clones":0,"num_object_copies":478194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":159398,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":159398,"num_write_kb":10201472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":31335186432,"data_stored":31335186432,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":49,"ondisk_log_size":49,"up":48,"acting":48,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":240,"num_write":135,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":2468,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":3,"up_from":25,"seq":107374182549,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":7817392,"kb_used_data":6313564,"kb_used_omap":25,"kb_used_meta":1503782,"kb_avail":726185808,"statfs":{"total":751619276800,"available":743614267392,"internally_reserved":0,"allocated":6465089536,"data_stored":6464610555,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":25943,"internal_metadata":1539873449},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":19,"seq":81604378775,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":8879728,"kb_used_data":7553884,"kb_used_omap":22,"kb_used_meta":1325801,"kb_avail":725123472,"statfs":{"total":751619276800,"available":742526435328,"internally_reserved":0,"allocated":7735177216,"data_stored":7734698235,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":23488,"internal_metadata":1357620288},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,1,3],"upper_bound":64},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542296,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":9545964,"kb_used_data":8709336,"kb_used_omap":21,"kb_used_meta":836522,"kb_avail":724457236,"statfs":{"total":751619276800,"available":741844209664,"internally_reserved":0,"allocated":8918360064,"data_stored":8917884635,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":22188,"internal_metadata":856598868},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[5,1],"upper_bound":4},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705819,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":8850608,"kb_used_data":7623004,"kb_used_omap":22,"kb_used_meta":1227561,"kb_avail":725152592,"statfs":{"total":751619276800,"available":742556254208,"internally_reserved":0,"allocated":7805956096,"data_stored":7805477115,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":23256,"internal_metadata":1257022760},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[5],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":2468,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":7830503424,"data_stored":7830503424,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":9163440128,"data_stored":9163440128,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":7819427840,"data_stored":7819427840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":6521815040,"data_stored":6521815040,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-24T07:55:10.505 INFO:tasks.cephadm.ceph_manager.ceph:recovered! 2026-02-24T07:55:10.506 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2026-02-24T07:55:10.506 INFO:tasks.thrashosds.thrasher:Growing pool unique_pool_4 2026-02-24T07:55:10.506 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph pg dump --format=json 2026-02-24T07:55:10.532 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:55:10 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/482114618' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 3, "weight": 1.0}]': finished 2026-02-24T07:55:10.533 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:55:10 trial035 ceph-mon[39284]: osdmap e96: 4 total, 4 up, 4 in 2026-02-24T07:55:10.533 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:55:10 trial035 ceph-mon[39284]: pgmap v435: 17 pgs: 17 active+clean; 9.7 GiB data, 33 GiB used, 2.7 TiB / 2.7 TiB avail; 317 MiB/s wr, 5.07k op/s 2026-02-24T07:55:10.533 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:55:10 trial035 ceph-mon[39284]: from='mgr.14284 10.20.193.35:0/3681373222' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2026-02-24T07:55:10.533 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:55:10 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/277194952' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2026-02-24T07:55:10.653 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:55:10.786 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:55:10.785+0000 7f6b7a3e0640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:55:10.787 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:55:10.785+0000 7f6b7a3e0640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:55:10.937 INFO:teuthology.orchestra.run.trial035.stdout: 2026-02-24T07:55:10.937 INFO:teuthology.orchestra.run.trial035.stderr:dumped all 2026-02-24T07:55:10.984 INFO:teuthology.orchestra.run.trial035.stdout:{"pg_ready":true,"pg_map":{"version":435,"stamp":"2026-02-24T07:55:09.756327+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":10446897696,"num_objects":159400,"num_object_clones":0,"num_object_copies":478200,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":159400,"num_whiteouts":0,"num_read":136,"num_read_kb":240,"num_write":159533,"num_write_kb":10203622,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":52,"ondisk_log_size":52,"up":51,"acting":51,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":51,"num_osds":4,"num_per_pool_osds":4,"num_per_pool_omap_osds":4,"kb":2936012800,"kb_used":35093692,"kb_used_data":30199788,"kb_used_omap":92,"kb_used_meta":4893667,"kb_avail":2900919108,"statfs":{"total":3006477107200,"available":2970541166592,"internally_reserved":0,"allocated":30924582912,"data_stored":30922670540,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":94875,"internal_metadata":5011115365},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[10,1,0,0,1,3],"upper_bound":64},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1994260480,"num_objects":30430,"num_object_clones":0,"num_object_copies":91290,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30430,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":30430,"num_write_kb":1947520,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.000789"},"pg_stats":[{"pgid":"6.f","version":"93'9952","reported_seq":9968,"reported_epoch":93,"state":"active+clean","last_fresh":"2026-02-24T07:55:05.864879+0000","last_change":"2026-02-24T07:54:42.927271+0000","last_active":"2026-02-24T07:55:05.864879+0000","last_peered":"2026-02-24T07:55:05.864879+0000","last_clean":"2026-02-24T07:55:05.864879+0000","last_became_active":"2026-02-24T07:54:42.927080+0000","last_became_peered":"2026-02-24T07:54:42.927080+0000","last_unstale":"2026-02-24T07:55:05.864879+0000","last_undegraded":"2026-02-24T07:55:05.864879+0000","last_fullsized":"2026-02-24T07:55:05.864879+0000","mapping_epoch":90,"log_start":"93'9949","ondisk_log_start":"93'9949","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_clean_scrub_stamp":"2026-02-24T07:54:41.918128+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:56:00.801546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":652214272,"num_objects":9952,"num_object_clones":0,"num_object_copies":29856,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9952,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9952,"num_write_kb":636928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.e","version":"93'9997","reported_seq":10013,"reported_epoch":93,"state":"active+clean","last_fresh":"2026-02-24T07:55:05.763307+0000","last_change":"2026-02-24T07:54:42.929104+0000","last_active":"2026-02-24T07:55:05.763307+0000","last_peered":"2026-02-24T07:55:05.763307+0000","last_clean":"2026-02-24T07:55:05.763307+0000","last_became_active":"2026-02-24T07:54:42.928748+0000","last_became_peered":"2026-02-24T07:54:42.928748+0000","last_unstale":"2026-02-24T07:55:05.763307+0000","last_undegraded":"2026-02-24T07:55:05.763307+0000","last_fullsized":"2026-02-24T07:55:05.763307+0000","mapping_epoch":90,"log_start":"93'9993","ondisk_log_start":"93'9993","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_clean_scrub_stamp":"2026-02-24T07:54:41.918128+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":3000,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:55:53.670906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":655163392,"num_objects":9997,"num_object_clones":0,"num_object_copies":29991,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9997,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9997,"num_write_kb":639808,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.d","version":"93'9977","reported_seq":9993,"reported_epoch":93,"state":"active+clean","last_fresh":"2026-02-24T07:55:05.759388+0000","last_change":"2026-02-24T07:54:42.927535+0000","last_active":"2026-02-24T07:55:05.759388+0000","last_peered":"2026-02-24T07:55:05.759388+0000","last_clean":"2026-02-24T07:55:05.759388+0000","last_became_active":"2026-02-24T07:54:42.926886+0000","last_became_peered":"2026-02-24T07:54:42.926886+0000","last_unstale":"2026-02-24T07:55:05.759388+0000","last_undegraded":"2026-02-24T07:55:05.759388+0000","last_fullsized":"2026-02-24T07:55:05.759388+0000","mapping_epoch":90,"log_start":"93'9974","ondisk_log_start":"93'9974","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_clean_scrub_stamp":"2026-02-24T07:54:41.918128+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:55:43.220815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":653852672,"num_objects":9977,"num_object_clones":0,"num_object_copies":29931,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9977,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9977,"num_write_kb":638528,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.c","version":"93'10014","reported_seq":10030,"reported_epoch":93,"state":"active+clean","last_fresh":"2026-02-24T07:55:05.860544+0000","last_change":"2026-02-24T07:54:42.928482+0000","last_active":"2026-02-24T07:55:05.860544+0000","last_peered":"2026-02-24T07:55:05.860544+0000","last_clean":"2026-02-24T07:55:05.860544+0000","last_became_active":"2026-02-24T07:54:42.928238+0000","last_became_peered":"2026-02-24T07:54:42.928238+0000","last_unstale":"2026-02-24T07:55:05.860544+0000","last_undegraded":"2026-02-24T07:55:05.860544+0000","last_fullsized":"2026-02-24T07:55:05.860544+0000","mapping_epoch":90,"log_start":"93'10011","ondisk_log_start":"93'10011","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_clean_scrub_stamp":"2026-02-24T07:54:41.918128+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:56:09.181158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":656277504,"num_objects":10014,"num_object_clones":0,"num_object_copies":30042,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10014,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10014,"num_write_kb":640896,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.b","version":"93'9916","reported_seq":9932,"reported_epoch":93,"state":"active+clean","last_fresh":"2026-02-24T07:55:05.861977+0000","last_change":"2026-02-24T07:54:42.927223+0000","last_active":"2026-02-24T07:55:05.861977+0000","last_peered":"2026-02-24T07:55:05.861977+0000","last_clean":"2026-02-24T07:55:05.861977+0000","last_became_active":"2026-02-24T07:54:42.926965+0000","last_became_peered":"2026-02-24T07:54:42.926965+0000","last_unstale":"2026-02-24T07:55:05.861977+0000","last_undegraded":"2026-02-24T07:55:05.861977+0000","last_fullsized":"2026-02-24T07:55:05.861977+0000","mapping_epoch":90,"log_start":"93'9913","ondisk_log_start":"93'9913","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_clean_scrub_stamp":"2026-02-24T07:54:41.918128+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:56:04.661233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":649854976,"num_objects":9916,"num_object_clones":0,"num_object_copies":29748,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9916,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9916,"num_write_kb":634624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.a","version":"93'9837","reported_seq":9853,"reported_epoch":93,"state":"active+clean","last_fresh":"2026-02-24T07:55:05.761189+0000","last_change":"2026-02-24T07:54:42.926926+0000","last_active":"2026-02-24T07:55:05.761189+0000","last_peered":"2026-02-24T07:55:05.761189+0000","last_clean":"2026-02-24T07:55:05.761189+0000","last_became_active":"2026-02-24T07:54:42.926477+0000","last_became_peered":"2026-02-24T07:54:42.926477+0000","last_unstale":"2026-02-24T07:55:05.761189+0000","last_undegraded":"2026-02-24T07:55:05.761189+0000","last_fullsized":"2026-02-24T07:55:05.761189+0000","mapping_epoch":90,"log_start":"93'9834","ondisk_log_start":"93'9834","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_clean_scrub_stamp":"2026-02-24T07:54:41.918128+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:55:46.420689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":644677632,"num_objects":9837,"num_object_clones":0,"num_object_copies":29511,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9837,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9837,"num_write_kb":629568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"93'9889","reported_seq":9905,"reported_epoch":93,"state":"active+clean","last_fresh":"2026-02-24T07:55:05.792421+0000","last_change":"2026-02-24T07:54:42.926317+0000","last_active":"2026-02-24T07:55:05.792421+0000","last_peered":"2026-02-24T07:55:05.792421+0000","last_clean":"2026-02-24T07:55:05.792421+0000","last_became_active":"2026-02-24T07:54:42.926136+0000","last_became_peered":"2026-02-24T07:54:42.926136+0000","last_unstale":"2026-02-24T07:55:05.792421+0000","last_undegraded":"2026-02-24T07:55:05.792421+0000","last_fullsized":"2026-02-24T07:55:05.792421+0000","mapping_epoch":90,"log_start":"93'9886","ondisk_log_start":"93'9886","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_clean_scrub_stamp":"2026-02-24T07:54:41.918128+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:55:45.176781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":648085504,"num_objects":9889,"num_object_clones":0,"num_object_copies":29667,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9889,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9889,"num_write_kb":632896,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,1],"acting":[0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.8","version":"93'9905","reported_seq":9921,"reported_epoch":93,"state":"active+clean","last_fresh":"2026-02-24T07:55:05.774340+0000","last_change":"2026-02-24T07:54:42.929353+0000","last_active":"2026-02-24T07:55:05.774340+0000","last_peered":"2026-02-24T07:55:05.774340+0000","last_clean":"2026-02-24T07:55:05.774340+0000","last_became_active":"2026-02-24T07:54:42.929078+0000","last_became_peered":"2026-02-24T07:54:42.929078+0000","last_unstale":"2026-02-24T07:55:05.774340+0000","last_undegraded":"2026-02-24T07:55:05.774340+0000","last_fullsized":"2026-02-24T07:55:05.774340+0000","mapping_epoch":90,"log_start":"93'9902","ondisk_log_start":"93'9902","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_clean_scrub_stamp":"2026-02-24T07:54:41.918128+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:55:48.303915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":649134080,"num_objects":9905,"num_object_clones":0,"num_object_copies":29715,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9905,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9905,"num_write_kb":633920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.6","version":"93'10039","reported_seq":10053,"reported_epoch":93,"state":"active+clean","last_fresh":"2026-02-24T07:55:05.863532+0000","last_change":"2026-02-24T07:54:42.926358+0000","last_active":"2026-02-24T07:55:05.863532+0000","last_peered":"2026-02-24T07:55:05.863532+0000","last_clean":"2026-02-24T07:55:05.863532+0000","last_became_active":"2026-02-24T07:54:42.926045+0000","last_became_peered":"2026-02-24T07:54:42.926045+0000","last_unstale":"2026-02-24T07:55:05.863532+0000","last_undegraded":"2026-02-24T07:55:05.863532+0000","last_fullsized":"2026-02-24T07:55:05.863532+0000","mapping_epoch":90,"log_start":"93'10036","ondisk_log_start":"93'10036","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_clean_scrub_stamp":"2026-02-24T07:54:41.918128+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:56:09.126690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":657915904,"num_objects":10039,"num_object_clones":0,"num_object_copies":30117,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10039,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10039,"num_write_kb":642496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.5","version":"93'9967","reported_seq":9982,"reported_epoch":93,"state":"active+clean","last_fresh":"2026-02-24T07:55:05.762842+0000","last_change":"2026-02-24T07:54:42.928143+0000","last_active":"2026-02-24T07:55:05.762842+0000","last_peered":"2026-02-24T07:55:05.762842+0000","last_clean":"2026-02-24T07:55:05.762842+0000","last_became_active":"2026-02-24T07:54:42.927535+0000","last_became_peered":"2026-02-24T07:54:42.927535+0000","last_unstale":"2026-02-24T07:55:05.762842+0000","last_undegraded":"2026-02-24T07:55:05.762842+0000","last_fullsized":"2026-02-24T07:55:05.762842+0000","mapping_epoch":90,"log_start":"93'9964","ondisk_log_start":"93'9964","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_clean_scrub_stamp":"2026-02-24T07:54:41.918128+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:55:53.650972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":653197312,"num_objects":9967,"num_object_clones":0,"num_object_copies":29901,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9967,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9967,"num_write_kb":637888,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.4","version":"93'10028","reported_seq":10042,"reported_epoch":93,"state":"active+clean","last_fresh":"2026-02-24T07:55:05.865876+0000","last_change":"2026-02-24T07:54:42.928344+0000","last_active":"2026-02-24T07:55:05.865876+0000","last_peered":"2026-02-24T07:55:05.865876+0000","last_clean":"2026-02-24T07:55:05.865876+0000","last_became_active":"2026-02-24T07:54:42.928011+0000","last_became_peered":"2026-02-24T07:54:42.928011+0000","last_unstale":"2026-02-24T07:55:05.865876+0000","last_undegraded":"2026-02-24T07:55:05.865876+0000","last_fullsized":"2026-02-24T07:55:05.865876+0000","mapping_epoch":90,"log_start":"93'10025","ondisk_log_start":"93'10025","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_clean_scrub_stamp":"2026-02-24T07:54:41.918128+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:55:43.506947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":657195008,"num_objects":10028,"num_object_clones":0,"num_object_copies":30084,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10028,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10028,"num_write_kb":641792,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.3","version":"93'10020","reported_seq":10036,"reported_epoch":93,"state":"active+clean","last_fresh":"2026-02-24T07:55:05.757695+0000","last_change":"2026-02-24T07:54:42.928536+0000","last_active":"2026-02-24T07:55:05.757695+0000","last_peered":"2026-02-24T07:55:05.757695+0000","last_clean":"2026-02-24T07:55:05.757695+0000","last_became_active":"2026-02-24T07:54:42.927730+0000","last_became_peered":"2026-02-24T07:54:42.927730+0000","last_unstale":"2026-02-24T07:55:05.757695+0000","last_undegraded":"2026-02-24T07:55:05.757695+0000","last_fullsized":"2026-02-24T07:55:05.757695+0000","mapping_epoch":90,"log_start":"93'10017","ondisk_log_start":"93'10017","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_clean_scrub_stamp":"2026-02-24T07:54:41.918128+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:55:58.987317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":656670720,"num_objects":10020,"num_object_clones":0,"num_object_copies":30060,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10020,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10020,"num_write_kb":641280,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.2","version":"93'9970","reported_seq":9986,"reported_epoch":93,"state":"active+clean","last_fresh":"2026-02-24T07:55:05.776065+0000","last_change":"2026-02-24T07:54:42.930452+0000","last_active":"2026-02-24T07:55:05.776065+0000","last_peered":"2026-02-24T07:55:05.776065+0000","last_clean":"2026-02-24T07:55:05.776065+0000","last_became_active":"2026-02-24T07:54:42.930247+0000","last_became_peered":"2026-02-24T07:54:42.930247+0000","last_unstale":"2026-02-24T07:55:05.776065+0000","last_undegraded":"2026-02-24T07:55:05.776065+0000","last_fullsized":"2026-02-24T07:55:05.776065+0000","mapping_epoch":90,"log_start":"93'9967","ondisk_log_start":"93'9967","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_clean_scrub_stamp":"2026-02-24T07:54:41.918128+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:55:49.096947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":653393920,"num_objects":9970,"num_object_clones":0,"num_object_copies":29910,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9970,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9970,"num_write_kb":638080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1","version":"93'10047","reported_seq":10063,"reported_epoch":93,"state":"active+clean","last_fresh":"2026-02-24T07:55:05.861654+0000","last_change":"2026-02-24T07:54:42.926619+0000","last_active":"2026-02-24T07:55:05.861654+0000","last_peered":"2026-02-24T07:55:05.861654+0000","last_clean":"2026-02-24T07:55:05.861654+0000","last_became_active":"2026-02-24T07:54:42.926442+0000","last_became_peered":"2026-02-24T07:54:42.926442+0000","last_unstale":"2026-02-24T07:55:05.861654+0000","last_undegraded":"2026-02-24T07:55:05.861654+0000","last_fullsized":"2026-02-24T07:55:05.861654+0000","mapping_epoch":90,"log_start":"93'10044","ondisk_log_start":"93'10044","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_clean_scrub_stamp":"2026-02-24T07:54:41.918128+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:56:03.125413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":658440192,"num_objects":10047,"num_object_clones":0,"num_object_copies":30141,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10047,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10047,"num_write_kb":643008,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.0","version":"93'9880","reported_seq":9896,"reported_epoch":93,"state":"active+clean","last_fresh":"2026-02-24T07:55:05.789947+0000","last_change":"2026-02-24T07:54:42.927178+0000","last_active":"2026-02-24T07:55:05.789947+0000","last_peered":"2026-02-24T07:55:05.789947+0000","last_clean":"2026-02-24T07:55:05.789947+0000","last_became_active":"2026-02-24T07:54:42.926910+0000","last_became_peered":"2026-02-24T07:54:42.926910+0000","last_unstale":"2026-02-24T07:55:05.789947+0000","last_undegraded":"2026-02-24T07:55:05.789947+0000","last_fullsized":"2026-02-24T07:55:05.789947+0000","mapping_epoch":90,"log_start":"93'9877","ondisk_log_start":"93'9877","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_clean_scrub_stamp":"2026-02-24T07:54:41.918128+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:55:58.002045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":647495680,"num_objects":9880,"num_object_clones":0,"num_object_copies":29640,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9880,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9880,"num_write_kb":632320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.7","version":"93'9960","reported_seq":9976,"reported_epoch":93,"state":"active+clean","last_fresh":"2026-02-24T07:55:05.775405+0000","last_change":"2026-02-24T07:54:42.930348+0000","last_active":"2026-02-24T07:55:05.775405+0000","last_peered":"2026-02-24T07:55:05.775405+0000","last_clean":"2026-02-24T07:55:05.775405+0000","last_became_active":"2026-02-24T07:54:42.930142+0000","last_became_peered":"2026-02-24T07:54:42.930142+0000","last_unstale":"2026-02-24T07:55:05.775405+0000","last_undegraded":"2026-02-24T07:55:05.775405+0000","last_fullsized":"2026-02-24T07:55:05.775405+0000","mapping_epoch":90,"log_start":"93'9957","ondisk_log_start":"93'9957","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_clean_scrub_stamp":"2026-02-24T07:54:41.918128+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:55:56.704334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":652738560,"num_objects":9960,"num_object_clones":0,"num_object_copies":29880,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9960,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9960,"num_write_kb":637440,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"29'99","reported_seq":384,"reported_epoch":93,"state":"active+clean","last_fresh":"2026-02-24T07:55:03.457261+0000","last_change":"2026-02-24T07:54:08.521371+0000","last_active":"2026-02-24T07:55:03.457261+0000","last_peered":"2026-02-24T07:55:03.457261+0000","last_clean":"2026-02-24T07:55:03.457261+0000","last_became_active":"2026-02-24T07:42:54.186719+0000","last_became_peered":"2026-02-24T07:42:54.186719+0000","last_unstale":"2026-02-24T07:55:03.457261+0000","last_undegraded":"2026-02-24T07:55:03.457261+0000","last_fullsized":"2026-02-24T07:55:03.457261+0000","mapping_epoch":27,"log_start":"29'96","ondisk_log_start":"29'96","created":20,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'99","last_scrub_stamp":"2026-02-24T07:54:08.521342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:42:43.640776+0000","last_clean_scrub_stamp":"2026-02-24T07:54:08.521342+0000","objects_scrubbed":2,"log_size":3,"log_dups_size":96,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:55:15.605551+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":240,"num_write":135,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":16,"stat_sum":{"num_bytes":10446307328,"num_objects":159398,"num_object_clones":0,"num_object_copies":478194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":159398,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":159398,"num_write_kb":10201472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":31335186432,"data_stored":31335186432,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":49,"ondisk_log_size":49,"up":48,"acting":48,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":240,"num_write":135,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":2468,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":3,"up_from":25,"seq":107374182549,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":7817392,"kb_used_data":6313564,"kb_used_omap":25,"kb_used_meta":1503782,"kb_avail":726185808,"statfs":{"total":751619276800,"available":743614267392,"internally_reserved":0,"allocated":6465089536,"data_stored":6464610555,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":25943,"internal_metadata":1539873449},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":19,"seq":81604378775,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":8879728,"kb_used_data":7553884,"kb_used_omap":22,"kb_used_meta":1325801,"kb_avail":725123472,"statfs":{"total":751619276800,"available":742526435328,"internally_reserved":0,"allocated":7735177216,"data_stored":7734698235,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":23488,"internal_metadata":1357620288},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,1,3],"upper_bound":64},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542296,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":9545964,"kb_used_data":8709336,"kb_used_omap":21,"kb_used_meta":836522,"kb_avail":724457236,"statfs":{"total":751619276800,"available":741844209664,"internally_reserved":0,"allocated":8918360064,"data_stored":8917884635,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":22188,"internal_metadata":856598868},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[5,1],"upper_bound":4},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705819,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":8850608,"kb_used_data":7623004,"kb_used_omap":22,"kb_used_meta":1227561,"kb_avail":725152592,"statfs":{"total":751619276800,"available":742556254208,"internally_reserved":0,"allocated":7805956096,"data_stored":7805477115,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":23256,"internal_metadata":1257022760},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[5],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":2468,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":7830503424,"data_stored":7830503424,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":9163440128,"data_stored":9163440128,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":7819427840,"data_stored":7819427840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":6521815040,"data_stored":6521815040,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-24T07:55:10.986 INFO:tasks.cephadm.ceph_manager.ceph:increase pool size by 10 2026-02-24T07:55:10.987 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph osd pool set unique_pool_4 pg_num 46 2026-02-24T07:55:11.017 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 27 16 196190 196174 454.071 447.5 0.00164708 0.00220095 2026-02-24T07:55:11.163 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:55:11.300 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:55:11.299+0000 7ffa36b27640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:55:11.300 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:55:11.299+0000 7ffa36b27640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:55:11.458 INFO:teuthology.orchestra.run.trial035.stderr:Error EPERM: crimson_allow_pg_split is false; pg_num increase denied for crimson pool 2026-02-24T07:55:11.504 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-24T07:55:11.704 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:55:11 trial035 ceph-mon[39284]: from='client.15451 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2026-02-24T07:55:11.705 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:55:11 trial035 ceph-mon[39284]: from='client.15455 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2026-02-24T07:55:11.705 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:55:11 trial035 ceph-mon[39284]: from='mgr.14284 10.20.193.35:0/3681373222' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-24T07:55:11.705 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:55:11 trial035 ceph-mon[39284]: from='mgr.14284 10.20.193.35:0/3681373222' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2026-02-24T07:55:11.705 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:55:11 trial035 ceph-mon[39284]: from='mgr.14284 10.20.193.35:0/3681373222' entity='mgr.x' 2026-02-24T07:55:11.705 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:55:11 trial035 ceph-mon[39284]: from='mgr.14284 10.20.193.35:0/3681373222' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2026-02-24T07:55:11.705 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:55:11 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/2944744874' entity='client.admin' cmd={"prefix": "osd pool set", "pool": "unique_pool_4", "var": "pg_num", "val": "46"} : dispatch 2026-02-24T07:55:12.017 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 28 16 203931 203915 455.132 483.812 0.0016288 0.00219584 2026-02-24T07:55:12.393 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2026-02-24T07:55:12.541 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:55:12.668 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:55:12.666+0000 7efd2444d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:55:12.668 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:55:12.667+0000 7efd2444d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:55:12.683 INFO:teuthology.orchestra.run.trial035.stdout:osd.0: {} 2026-02-24T07:55:12.683 INFO:teuthology.orchestra.run.trial035.stderr:osd.0: osd_enable_op_tracker = '' 2026-02-24T07:55:12.684 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:55:12 trial035 ceph-mon[39284]: from='client.15457 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2026-02-24T07:55:12.684 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:55:12 trial035 ceph-mon[39284]: pgmap v436: 17 pgs: 17 active+clean; 12 GiB data, 40 GiB used, 2.7 TiB / 2.7 TiB avail; 684 MiB/s wr, 10.94k op/s 2026-02-24T07:55:12.701 INFO:teuthology.orchestra.run.trial035.stdout:osd.1: {} 2026-02-24T07:55:12.701 INFO:teuthology.orchestra.run.trial035.stderr:osd.1: osd_enable_op_tracker = '' 2026-02-24T07:55:12.715 INFO:teuthology.orchestra.run.trial035.stdout:osd.2: {} 2026-02-24T07:55:12.716 INFO:teuthology.orchestra.run.trial035.stderr:osd.2: osd_enable_op_tracker = '' 2026-02-24T07:55:12.733 INFO:teuthology.orchestra.run.trial035.stdout:osd.3: {} 2026-02-24T07:55:12.734 INFO:teuthology.orchestra.run.trial035.stderr:osd.3: osd_enable_op_tracker = '' 2026-02-24T07:55:13.017 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 29 16 211660 211644 456.094 483.062 0.00240395 0.00219118 2026-02-24T07:55:13.954 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:55:13 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/2851184755' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2026-02-24T07:55:14.017 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 30 16 219280 219264 456.764 476.25 0.00158261 0.00218798 2026-02-24T07:55:14.780 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2026-02-24T07:55:14.933 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:55:15.017 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 31 16 227438 227422 458.476 509.875 0.00196458 0.00217982 2026-02-24T07:55:15.055 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:55:15.052+0000 7f1bffa72640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:55:15.055 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:55:15.053+0000 7f1bffa72640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:55:15.070 INFO:teuthology.orchestra.run.trial035.stdout:osd.0: {} 2026-02-24T07:55:15.071 INFO:teuthology.orchestra.run.trial035.stderr:osd.0: osd_enable_op_tracker = '' 2026-02-24T07:55:15.071 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:55:14 trial035 ceph-mon[39284]: pgmap v437: 17 pgs: 17 active+clean; 12 GiB data, 40 GiB used, 2.7 TiB / 2.7 TiB avail; 302 MiB/s wr, 4.83k op/s 2026-02-24T07:55:15.071 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:55:14 trial035 ceph-mon[39284]: from='mgr.14284 10.20.193.35:0/3681373222' entity='mgr.x' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2026-02-24T07:55:15.086 INFO:teuthology.orchestra.run.trial035.stdout:osd.1: {} 2026-02-24T07:55:15.086 INFO:teuthology.orchestra.run.trial035.stderr:osd.1: osd_enable_op_tracker = '' 2026-02-24T07:55:15.103 INFO:teuthology.orchestra.run.trial035.stdout:osd.2: {} 2026-02-24T07:55:15.103 INFO:teuthology.orchestra.run.trial035.stderr:osd.2: osd_enable_op_tracker = '' 2026-02-24T07:55:15.116 INFO:teuthology.orchestra.run.trial035.stdout:osd.3: {} 2026-02-24T07:55:15.116 INFO:teuthology.orchestra.run.trial035.stderr:osd.3: osd_enable_op_tracker = '' 2026-02-24T07:55:16.017 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 32 15 235745 235730 460.374 519.25 0.00126195 0.00217085 2026-02-24T07:55:16.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:55:15 trial035 ceph-mon[39284]: purged_snaps scrub starts 2026-02-24T07:55:16.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:55:15 trial035 ceph-mon[39284]: purged_snaps scrub ok 2026-02-24T07:55:16.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:55:15 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/1196994366' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2026-02-24T07:55:16.505 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2026-02-24T07:55:16.506 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2026-02-24T07:55:16.506 INFO:tasks.thrashosds.thrasher:Growing pool .mgr 2026-02-24T07:55:16.506 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph pg dump --format=json 2026-02-24T07:55:16.657 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:55:16.777 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:55:16.775+0000 7f6fbe8de640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:55:16.777 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:55:16.775+0000 7f6fbe8de640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:55:16.927 INFO:teuthology.orchestra.run.trial035.stdout: 2026-02-24T07:55:16.928 INFO:teuthology.orchestra.run.trial035.stderr:dumped all 2026-02-24T07:55:17.018 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 33 16 243635 243619 461.363 493.062 0.00114445 0.00216567 2026-02-24T07:55:17.077 INFO:teuthology.orchestra.run.trial035.stdout:{"pg_ready":true,"pg_map":{"version":438,"stamp":"2026-02-24T07:55:15.757183+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":12754944544,"num_objects":194618,"num_object_clones":0,"num_object_copies":583854,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":194618,"num_whiteouts":0,"num_read":136,"num_read_kb":240,"num_write":194751,"num_write_kb":12457574,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":52,"ondisk_log_size":52,"up":51,"acting":51,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":51,"num_osds":4,"num_per_pool_osds":4,"num_per_pool_omap_osds":4,"kb":2936012800,"kb_used":42185996,"kb_used_data":36877052,"kb_used_omap":102,"kb_used_meta":5308761,"kb_avail":2893826804,"statfs":{"total":3006477107200,"available":2963278647296,"internally_reserved":0,"allocated":37762101248,"data_stored":37760094328,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":104646,"internal_metadata":5436172090},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[15,7,2],"upper_bound":8},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2308046848,"num_objects":35218,"num_object_clones":0,"num_object_copies":105654,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35218,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":35218,"num_write_kb":2253952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001144"},"pg_stats":[{"pgid":"6.f","version":"96'12110","reported_seq":12131,"reported_epoch":96,"state":"active+clean","last_fresh":"2026-02-24T07:55:10.868305+0000","last_change":"2026-02-24T07:54:42.927271+0000","last_active":"2026-02-24T07:55:10.868305+0000","last_peered":"2026-02-24T07:55:10.868305+0000","last_clean":"2026-02-24T07:55:10.868305+0000","last_became_active":"2026-02-24T07:54:42.927080+0000","last_became_peered":"2026-02-24T07:54:42.927080+0000","last_unstale":"2026-02-24T07:55:10.868305+0000","last_undegraded":"2026-02-24T07:55:10.868305+0000","last_fullsized":"2026-02-24T07:55:10.868305+0000","mapping_epoch":90,"log_start":"96'12107","ondisk_log_start":"96'12107","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_clean_scrub_stamp":"2026-02-24T07:54:41.918128+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:56:00.801546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":793640960,"num_objects":12110,"num_object_clones":0,"num_object_copies":36330,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12110,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12110,"num_write_kb":775040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.e","version":"96'12249","reported_seq":12269,"reported_epoch":96,"state":"active+clean","last_fresh":"2026-02-24T07:55:10.761986+0000","last_change":"2026-02-24T07:54:42.929104+0000","last_active":"2026-02-24T07:55:10.761986+0000","last_peered":"2026-02-24T07:55:10.761986+0000","last_clean":"2026-02-24T07:55:10.761986+0000","last_became_active":"2026-02-24T07:54:42.928748+0000","last_became_peered":"2026-02-24T07:54:42.928748+0000","last_unstale":"2026-02-24T07:55:10.761986+0000","last_undegraded":"2026-02-24T07:55:10.761986+0000","last_fullsized":"2026-02-24T07:55:10.761986+0000","mapping_epoch":90,"log_start":"96'12246","ondisk_log_start":"96'12246","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_clean_scrub_stamp":"2026-02-24T07:54:41.918128+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:55:53.670906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":802750464,"num_objects":12249,"num_object_clones":0,"num_object_copies":36747,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12249,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12249,"num_write_kb":783936,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.d","version":"96'12158","reported_seq":12180,"reported_epoch":96,"state":"active+clean","last_fresh":"2026-02-24T07:55:10.759333+0000","last_change":"2026-02-24T07:54:42.927535+0000","last_active":"2026-02-24T07:55:10.759333+0000","last_peered":"2026-02-24T07:55:10.759333+0000","last_clean":"2026-02-24T07:55:10.759333+0000","last_became_active":"2026-02-24T07:54:42.926886+0000","last_became_peered":"2026-02-24T07:54:42.926886+0000","last_unstale":"2026-02-24T07:55:10.759333+0000","last_undegraded":"2026-02-24T07:55:10.759333+0000","last_fullsized":"2026-02-24T07:55:10.759333+0000","mapping_epoch":90,"log_start":"96'12155","ondisk_log_start":"96'12155","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_clean_scrub_stamp":"2026-02-24T07:54:41.918128+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:55:43.220815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":796786688,"num_objects":12158,"num_object_clones":0,"num_object_copies":36474,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12158,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12158,"num_write_kb":778112,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.c","version":"96'12278","reported_seq":12300,"reported_epoch":96,"state":"active+clean","last_fresh":"2026-02-24T07:55:10.866442+0000","last_change":"2026-02-24T07:54:42.928482+0000","last_active":"2026-02-24T07:55:10.866442+0000","last_peered":"2026-02-24T07:55:10.866442+0000","last_clean":"2026-02-24T07:55:10.866442+0000","last_became_active":"2026-02-24T07:54:42.928238+0000","last_became_peered":"2026-02-24T07:54:42.928238+0000","last_unstale":"2026-02-24T07:55:10.866442+0000","last_undegraded":"2026-02-24T07:55:10.866442+0000","last_fullsized":"2026-02-24T07:55:10.866442+0000","mapping_epoch":90,"log_start":"96'12275","ondisk_log_start":"96'12275","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_clean_scrub_stamp":"2026-02-24T07:54:41.918128+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:56:09.181158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":804651008,"num_objects":12278,"num_object_clones":0,"num_object_copies":36834,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12278,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12278,"num_write_kb":785792,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.b","version":"96'12141","reported_seq":12162,"reported_epoch":96,"state":"active+clean","last_fresh":"2026-02-24T07:55:10.862293+0000","last_change":"2026-02-24T07:54:42.927223+0000","last_active":"2026-02-24T07:55:10.862293+0000","last_peered":"2026-02-24T07:55:10.862293+0000","last_clean":"2026-02-24T07:55:10.862293+0000","last_became_active":"2026-02-24T07:54:42.926965+0000","last_became_peered":"2026-02-24T07:54:42.926965+0000","last_unstale":"2026-02-24T07:55:10.862293+0000","last_undegraded":"2026-02-24T07:55:10.862293+0000","last_fullsized":"2026-02-24T07:55:10.862293+0000","mapping_epoch":90,"log_start":"96'12138","ondisk_log_start":"96'12138","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_clean_scrub_stamp":"2026-02-24T07:54:41.918128+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:56:04.661233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":795672576,"num_objects":12141,"num_object_clones":0,"num_object_copies":36423,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12141,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12141,"num_write_kb":777024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.a","version":"96'11921","reported_seq":11943,"reported_epoch":96,"state":"active+clean","last_fresh":"2026-02-24T07:55:10.757859+0000","last_change":"2026-02-24T07:54:42.926926+0000","last_active":"2026-02-24T07:55:10.757859+0000","last_peered":"2026-02-24T07:55:10.757859+0000","last_clean":"2026-02-24T07:55:10.757859+0000","last_became_active":"2026-02-24T07:54:42.926477+0000","last_became_peered":"2026-02-24T07:54:42.926477+0000","last_unstale":"2026-02-24T07:55:10.757859+0000","last_undegraded":"2026-02-24T07:55:10.757859+0000","last_fullsized":"2026-02-24T07:55:10.757859+0000","mapping_epoch":90,"log_start":"96'11918","ondisk_log_start":"96'11918","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_clean_scrub_stamp":"2026-02-24T07:54:41.918128+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:55:46.420689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":781254656,"num_objects":11921,"num_object_clones":0,"num_object_copies":35763,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11921,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11921,"num_write_kb":762944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"96'12127","reported_seq":12149,"reported_epoch":96,"state":"active+clean","last_fresh":"2026-02-24T07:55:10.791673+0000","last_change":"2026-02-24T07:54:42.926317+0000","last_active":"2026-02-24T07:55:10.791673+0000","last_peered":"2026-02-24T07:55:10.791673+0000","last_clean":"2026-02-24T07:55:10.791673+0000","last_became_active":"2026-02-24T07:54:42.926136+0000","last_became_peered":"2026-02-24T07:54:42.926136+0000","last_unstale":"2026-02-24T07:55:10.791673+0000","last_undegraded":"2026-02-24T07:55:10.791673+0000","last_fullsized":"2026-02-24T07:55:10.791673+0000","mapping_epoch":90,"log_start":"96'12124","ondisk_log_start":"96'12124","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_clean_scrub_stamp":"2026-02-24T07:54:41.918128+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:55:45.176781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":794755072,"num_objects":12127,"num_object_clones":0,"num_object_copies":36381,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12127,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12127,"num_write_kb":776128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,1],"acting":[0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.8","version":"96'12140","reported_seq":12162,"reported_epoch":96,"state":"active+clean","last_fresh":"2026-02-24T07:55:10.779874+0000","last_change":"2026-02-24T07:54:42.929353+0000","last_active":"2026-02-24T07:55:10.779874+0000","last_peered":"2026-02-24T07:55:10.779874+0000","last_clean":"2026-02-24T07:55:10.779874+0000","last_became_active":"2026-02-24T07:54:42.929078+0000","last_became_peered":"2026-02-24T07:54:42.929078+0000","last_unstale":"2026-02-24T07:55:10.779874+0000","last_undegraded":"2026-02-24T07:55:10.779874+0000","last_fullsized":"2026-02-24T07:55:10.779874+0000","mapping_epoch":90,"log_start":"96'12137","ondisk_log_start":"96'12137","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_clean_scrub_stamp":"2026-02-24T07:54:41.918128+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:55:48.303915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":795607040,"num_objects":12140,"num_object_clones":0,"num_object_copies":36420,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12140,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12140,"num_write_kb":776960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.6","version":"96'12181","reported_seq":12203,"reported_epoch":96,"state":"active+clean","last_fresh":"2026-02-24T07:55:10.867972+0000","last_change":"2026-02-24T07:54:42.926358+0000","last_active":"2026-02-24T07:55:10.867972+0000","last_peered":"2026-02-24T07:55:10.867972+0000","last_clean":"2026-02-24T07:55:10.867972+0000","last_became_active":"2026-02-24T07:54:42.926045+0000","last_became_peered":"2026-02-24T07:54:42.926045+0000","last_unstale":"2026-02-24T07:55:10.867972+0000","last_undegraded":"2026-02-24T07:55:10.867972+0000","last_fullsized":"2026-02-24T07:55:10.867972+0000","mapping_epoch":90,"log_start":"96'12178","ondisk_log_start":"96'12178","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_clean_scrub_stamp":"2026-02-24T07:54:41.918128+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:56:09.126690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":798294016,"num_objects":12181,"num_object_clones":0,"num_object_copies":36543,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12181,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12181,"num_write_kb":779584,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.5","version":"96'12216","reported_seq":12238,"reported_epoch":96,"state":"active+clean","last_fresh":"2026-02-24T07:55:10.756500+0000","last_change":"2026-02-24T07:54:42.928143+0000","last_active":"2026-02-24T07:55:10.756500+0000","last_peered":"2026-02-24T07:55:10.756500+0000","last_clean":"2026-02-24T07:55:10.756500+0000","last_became_active":"2026-02-24T07:54:42.927535+0000","last_became_peered":"2026-02-24T07:54:42.927535+0000","last_unstale":"2026-02-24T07:55:10.756500+0000","last_undegraded":"2026-02-24T07:55:10.756500+0000","last_fullsized":"2026-02-24T07:55:10.756500+0000","mapping_epoch":90,"log_start":"96'12213","ondisk_log_start":"96'12213","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_clean_scrub_stamp":"2026-02-24T07:54:41.918128+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:55:53.650972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":800587776,"num_objects":12216,"num_object_clones":0,"num_object_copies":36648,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12216,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12216,"num_write_kb":781824,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.4","version":"96'12253","reported_seq":12275,"reported_epoch":96,"state":"active+clean","last_fresh":"2026-02-24T07:55:10.864382+0000","last_change":"2026-02-24T07:54:42.928344+0000","last_active":"2026-02-24T07:55:10.864382+0000","last_peered":"2026-02-24T07:55:10.864382+0000","last_clean":"2026-02-24T07:55:10.864382+0000","last_became_active":"2026-02-24T07:54:42.928011+0000","last_became_peered":"2026-02-24T07:54:42.928011+0000","last_unstale":"2026-02-24T07:55:10.864382+0000","last_undegraded":"2026-02-24T07:55:10.864382+0000","last_fullsized":"2026-02-24T07:55:10.864382+0000","mapping_epoch":90,"log_start":"96'12250","ondisk_log_start":"96'12250","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_clean_scrub_stamp":"2026-02-24T07:54:41.918128+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:55:43.506947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":803012608,"num_objects":12253,"num_object_clones":0,"num_object_copies":36759,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12253,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12253,"num_write_kb":784192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.3","version":"96'12296","reported_seq":12318,"reported_epoch":96,"state":"active+clean","last_fresh":"2026-02-24T07:55:10.759212+0000","last_change":"2026-02-24T07:54:42.928536+0000","last_active":"2026-02-24T07:55:10.759212+0000","last_peered":"2026-02-24T07:55:10.759212+0000","last_clean":"2026-02-24T07:55:10.759212+0000","last_became_active":"2026-02-24T07:54:42.927730+0000","last_became_peered":"2026-02-24T07:54:42.927730+0000","last_unstale":"2026-02-24T07:55:10.759212+0000","last_undegraded":"2026-02-24T07:55:10.759212+0000","last_fullsized":"2026-02-24T07:55:10.759212+0000","mapping_epoch":90,"log_start":"96'12293","ondisk_log_start":"96'12293","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_clean_scrub_stamp":"2026-02-24T07:54:41.918128+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:55:58.987317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":805830656,"num_objects":12296,"num_object_clones":0,"num_object_copies":36888,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12296,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12296,"num_write_kb":786944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.2","version":"96'12158","reported_seq":12180,"reported_epoch":96,"state":"active+clean","last_fresh":"2026-02-24T07:55:10.778313+0000","last_change":"2026-02-24T07:54:42.930452+0000","last_active":"2026-02-24T07:55:10.778313+0000","last_peered":"2026-02-24T07:55:10.778313+0000","last_clean":"2026-02-24T07:55:10.778313+0000","last_became_active":"2026-02-24T07:54:42.930247+0000","last_became_peered":"2026-02-24T07:54:42.930247+0000","last_unstale":"2026-02-24T07:55:10.778313+0000","last_undegraded":"2026-02-24T07:55:10.778313+0000","last_fullsized":"2026-02-24T07:55:10.778313+0000","mapping_epoch":90,"log_start":"96'12155","ondisk_log_start":"96'12155","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_clean_scrub_stamp":"2026-02-24T07:54:41.918128+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:55:49.096947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":796786688,"num_objects":12158,"num_object_clones":0,"num_object_copies":36474,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12158,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12158,"num_write_kb":778112,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1","version":"96'12244","reported_seq":12263,"reported_epoch":96,"state":"active+clean","last_fresh":"2026-02-24T07:55:10.868077+0000","last_change":"2026-02-24T07:54:42.926619+0000","last_active":"2026-02-24T07:55:10.868077+0000","last_peered":"2026-02-24T07:55:10.868077+0000","last_clean":"2026-02-24T07:55:10.868077+0000","last_became_active":"2026-02-24T07:54:42.926442+0000","last_became_peered":"2026-02-24T07:54:42.926442+0000","last_unstale":"2026-02-24T07:55:10.868077+0000","last_undegraded":"2026-02-24T07:55:10.868077+0000","last_fullsized":"2026-02-24T07:55:10.868077+0000","mapping_epoch":90,"log_start":"96'12240","ondisk_log_start":"96'12240","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_clean_scrub_stamp":"2026-02-24T07:54:41.918128+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":3000,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:56:03.125413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":802422784,"num_objects":12244,"num_object_clones":0,"num_object_copies":36732,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12244,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12244,"num_write_kb":783616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.0","version":"96'12019","reported_seq":12041,"reported_epoch":96,"state":"active+clean","last_fresh":"2026-02-24T07:55:10.794003+0000","last_change":"2026-02-24T07:54:42.927178+0000","last_active":"2026-02-24T07:55:10.794003+0000","last_peered":"2026-02-24T07:55:10.794003+0000","last_clean":"2026-02-24T07:55:10.794003+0000","last_became_active":"2026-02-24T07:54:42.926910+0000","last_became_peered":"2026-02-24T07:54:42.926910+0000","last_unstale":"2026-02-24T07:55:10.794003+0000","last_undegraded":"2026-02-24T07:55:10.794003+0000","last_fullsized":"2026-02-24T07:55:10.794003+0000","mapping_epoch":90,"log_start":"96'12016","ondisk_log_start":"96'12016","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_clean_scrub_stamp":"2026-02-24T07:54:41.918128+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:55:58.002045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":787677184,"num_objects":12019,"num_object_clones":0,"num_object_copies":36057,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12019,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12019,"num_write_kb":769216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.7","version":"96'12125","reported_seq":12147,"reported_epoch":96,"state":"active+clean","last_fresh":"2026-02-24T07:55:10.774495+0000","last_change":"2026-02-24T07:54:42.930348+0000","last_active":"2026-02-24T07:55:10.774495+0000","last_peered":"2026-02-24T07:55:10.774495+0000","last_clean":"2026-02-24T07:55:10.774495+0000","last_became_active":"2026-02-24T07:54:42.930142+0000","last_became_peered":"2026-02-24T07:54:42.930142+0000","last_unstale":"2026-02-24T07:55:10.774495+0000","last_undegraded":"2026-02-24T07:55:10.774495+0000","last_fullsized":"2026-02-24T07:55:10.774495+0000","mapping_epoch":90,"log_start":"96'12122","ondisk_log_start":"96'12122","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_clean_scrub_stamp":"2026-02-24T07:54:41.918128+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:55:56.704334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":794624000,"num_objects":12125,"num_object_clones":0,"num_object_copies":36375,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12125,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12125,"num_write_kb":776000,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"29'99","reported_seq":390,"reported_epoch":96,"state":"active+clean","last_fresh":"2026-02-24T07:55:09.469651+0000","last_change":"2026-02-24T07:54:08.521371+0000","last_active":"2026-02-24T07:55:09.469651+0000","last_peered":"2026-02-24T07:55:09.469651+0000","last_clean":"2026-02-24T07:55:09.469651+0000","last_became_active":"2026-02-24T07:42:54.186719+0000","last_became_peered":"2026-02-24T07:42:54.186719+0000","last_unstale":"2026-02-24T07:55:09.469651+0000","last_undegraded":"2026-02-24T07:55:09.469651+0000","last_fullsized":"2026-02-24T07:55:09.469651+0000","mapping_epoch":27,"log_start":"29'96","ondisk_log_start":"29'96","created":20,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'99","last_scrub_stamp":"2026-02-24T07:54:08.521342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:42:43.640776+0000","last_clean_scrub_stamp":"2026-02-24T07:54:08.521342+0000","objects_scrubbed":2,"log_size":3,"log_dups_size":96,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:55:15.605551+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":240,"num_write":135,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":16,"stat_sum":{"num_bytes":12754354176,"num_objects":194616,"num_object_clones":0,"num_object_copies":583848,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":194616,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":194616,"num_write_kb":12455424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":38261358592,"data_stored":38261358592,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":49,"ondisk_log_size":49,"up":48,"acting":48,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":240,"num_write":135,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":2526,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":3,"up_from":25,"seq":107374182550,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":9272276,"kb_used_data":7696256,"kb_used_omap":27,"kb_used_meta":1575972,"kb_avail":724730924,"statfs":{"total":751619276800,"available":742124466176,"internally_reserved":0,"allocated":7880966144,"data_stored":7880463526,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":27905,"internal_metadata":1613796095},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1],"upper_bound":4},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":19,"seq":81604378776,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":10594324,"kb_used_data":9196288,"kb_used_omap":26,"kb_used_meta":1397989,"kb_avail":723408876,"statfs":{"total":751619276800,"available":740770689024,"internally_reserved":0,"allocated":9416998912,"data_stored":9416496294,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":27400,"internal_metadata":1431540984},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[4,3,1],"upper_bound":8},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542297,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":11621776,"kb_used_data":10676924,"kb_used_omap":23,"kb_used_meta":944808,"kb_avail":722381424,"statfs":{"total":751619276800,"available":739718578176,"internally_reserved":0,"allocated":10933170176,"data_stored":10932671110,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":24144,"internal_metadata":967483824},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[5,3],"upper_bound":4},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705820,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":10697620,"kb_used_data":9307584,"kb_used_omap":24,"kb_used_meta":1389991,"kb_avail":723305580,"statfs":{"total":751619276800,"available":740664913920,"internally_reserved":0,"allocated":9530966016,"data_stored":9530463398,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":25197,"internal_metadata":1423351187},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[5,0,1],"upper_bound":8},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":2526,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":9561309184,"data_stored":9561309184,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":11181359104,"data_stored":11181359104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":9550168064,"data_stored":9550168064,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":7968522240,"data_stored":7968522240,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-24T07:55:17.081 INFO:tasks.cephadm.ceph_manager.ceph:increase pool size by 10 2026-02-24T07:55:17.082 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph osd pool set .mgr pg_num 371 2026-02-24T07:55:17.164 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2026-02-24T07:55:17.190 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:55:16 trial035 ceph-mon[39284]: pgmap v438: 17 pgs: 17 active+clean; 12 GiB data, 40 GiB used, 2.7 TiB / 2.7 TiB avail; 275 MiB/s wr, 4.40k op/s 2026-02-24T07:55:17.190 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:55:16 trial035 ceph-mon[39284]: 1.0 scrub starts 2026-02-24T07:55:17.190 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:55:16 trial035 ceph-mon[39284]: 1.0 scrub ok 2026-02-24T07:55:17.190 INFO:journalctl@ceph.mgr.x.trial035.stdout:Feb 24 07:55:16 trial035 ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-mgr-x[39621]: ::ffff:10.20.193.91 - - [24/Feb/2026:07:55:16] "GET /metrics HTTP/1.1" 200 4213 "" "Prometheus/3.6.0" 2026-02-24T07:55:17.233 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:55:17.264 INFO:journalctl@ceph.prometheus.a.trial091.stdout:Feb 24 07:55:16 trial091 ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-prometheus-a[66007]: time=2026-02-24T07:55:16.973Z level=ERROR source=scrape.go:1630 msg="Failed to determine correct type of scrape target." component="scrape manager" scrape_pool=ceph target=http://trial035:9283/metrics content_type="text/html;charset=utf-8" fallback_media_type="" err="received unsupported Content-Type \"text/html;charset=utf-8\" and no fallback_scrape_protocol specified for target" 2026-02-24T07:55:17.308 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:55:17.373 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:55:17.371+0000 7f7eea21c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:55:17.373 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:55:17.372+0000 7f7eea21c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:55:17.437 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:55:17.434+0000 7f9153023640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:55:17.437 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:55:17.435+0000 7f9153023640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:55:17.452 INFO:teuthology.orchestra.run.trial035.stdout:osd.0: {} 2026-02-24T07:55:17.453 INFO:teuthology.orchestra.run.trial035.stderr:osd.0: osd_enable_op_tracker = '' 2026-02-24T07:55:17.470 INFO:teuthology.orchestra.run.trial035.stdout:osd.1: {} 2026-02-24T07:55:17.471 INFO:teuthology.orchestra.run.trial035.stderr:osd.1: osd_enable_op_tracker = '' 2026-02-24T07:55:17.485 INFO:teuthology.orchestra.run.trial035.stdout:osd.2: {} 2026-02-24T07:55:17.486 INFO:teuthology.orchestra.run.trial035.stderr:osd.2: osd_enable_op_tracker = '' 2026-02-24T07:55:17.500 INFO:teuthology.orchestra.run.trial035.stdout:osd.3: {} 2026-02-24T07:55:17.500 INFO:teuthology.orchestra.run.trial035.stderr:osd.3: osd_enable_op_tracker = '' 2026-02-24T07:55:17.521 INFO:teuthology.orchestra.run.trial035.stderr:Error EPERM: crimson_allow_pg_split is false; pg_num increase denied for crimson pool 2026-02-24T07:55:17.586 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-24T07:55:18.018 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 34 16 251197 251181 461.694 472.625 0.00278137 0.00216464 2026-02-24T07:55:18.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:55:17 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/2242944262' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2026-02-24T07:55:18.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:55:17 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/2694413971' entity='client.admin' cmd={"prefix": "osd pool set", "pool": ".mgr", "var": "pg_num", "val": "371"} : dispatch 2026-02-24T07:55:19.018 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 35 16 258654 258638 461.817 466.062 0.00202144 0.00216408 2026-02-24T07:55:19.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:55:18 trial035 ceph-mon[39284]: from='client.15465 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2026-02-24T07:55:19.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:55:18 trial035 ceph-mon[39284]: pgmap v439: 17 pgs: 17 active+clean; 14 GiB data, 47 GiB used, 2.7 TiB / 2.7 TiB avail; 502 MiB/s wr, 8.03k op/s 2026-02-24T07:55:19.547 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2026-02-24T07:55:19.692 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:55:20.018 INFO:teuthology.orchestra.run.trial035.stdout:osd.0: {} 2026-02-24T07:55:20.018 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:55:19.974+0000 7fc3caad8640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:55:20.019 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:55:19.974+0000 7fc3caad8640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:55:20.019 INFO:teuthology.orchestra.run.trial035.stderr:osd.0: osd_enable_op_tracker = '' 2026-02-24T07:55:20.019 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 36 16 265581 265565 461.014 432.938 0.00196222 0.00216784 2026-02-24T07:55:20.020 INFO:teuthology.orchestra.run.trial035.stdout:osd.1: {} 2026-02-24T07:55:20.020 INFO:teuthology.orchestra.run.trial035.stderr:osd.1: osd_enable_op_tracker = '' 2026-02-24T07:55:20.032 INFO:teuthology.orchestra.run.trial035.stdout:osd.2: {} 2026-02-24T07:55:20.033 INFO:teuthology.orchestra.run.trial035.stderr:osd.2: osd_enable_op_tracker = '' 2026-02-24T07:55:20.045 INFO:teuthology.orchestra.run.trial035.stdout:osd.3: {} 2026-02-24T07:55:20.045 INFO:teuthology.orchestra.run.trial035.stderr:osd.3: osd_enable_op_tracker = '' 2026-02-24T07:55:21.018 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 37 16 272613 272597 460.431 439.5 0.0011677 0.00217057 2026-02-24T07:55:21.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:55:20 trial035 ceph-mon[39284]: pgmap v440: 17 pgs: 17 active+clean; 14 GiB data, 47 GiB used, 2.7 TiB / 2.7 TiB avail; 453 MiB/s wr, 7.25k op/s 2026-02-24T07:55:21.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:55:20 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/579902220' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2026-02-24T07:55:22.018 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 38 16 280510 280494 461.302 493.562 0.00186777 0.00216648 2026-02-24T07:55:22.322 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2026-02-24T07:55:22.467 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:55:22.588 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2026-02-24T07:55:22.588 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2026-02-24T07:55:22.589 INFO:tasks.thrashosds.thrasher:Growing pool .mgr 2026-02-24T07:55:22.589 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph pg dump --format=json 2026-02-24T07:55:22.694 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:55:22.677+0000 7f1549544640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:55:22.694 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:55:22.677+0000 7f1549544640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:55:22.695 INFO:teuthology.orchestra.run.trial035.stderr:osd.0: osd_enable_op_tracker = '' 2026-02-24T07:55:22.695 INFO:teuthology.orchestra.run.trial035.stdout:osd.0: {} 2026-02-24T07:55:22.718 INFO:teuthology.orchestra.run.trial035.stdout:osd.1: {} 2026-02-24T07:55:22.718 INFO:teuthology.orchestra.run.trial035.stderr:osd.1: osd_enable_op_tracker = '' 2026-02-24T07:55:22.730 INFO:teuthology.orchestra.run.trial035.stdout:osd.2: {} 2026-02-24T07:55:22.731 INFO:teuthology.orchestra.run.trial035.stderr:osd.2: osd_enable_op_tracker = '' 2026-02-24T07:55:22.731 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:55:22.742 INFO:teuthology.orchestra.run.trial035.stdout:osd.3: {} 2026-02-24T07:55:22.743 INFO:teuthology.orchestra.run.trial035.stderr:osd.3: osd_enable_op_tracker = '' 2026-02-24T07:55:23.018 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 39 16 287043 287027 459.943 408.312 0.00112371 0.00217132 2026-02-24T07:55:23.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:55:22 trial035 ceph-mon[39284]: pgmap v441: 17 pgs: 17 active+clean; 17 GiB data, 55 GiB used, 2.7 TiB / 2.7 TiB avail; 581 MiB/s wr, 9.30k op/s 2026-02-24T07:55:23.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:55:22 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/370199968' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2026-02-24T07:55:23.785 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:55:23.733+0000 7ffa51ddf640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:55:23.785 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:55:23.733+0000 7ffa51ddf640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:55:23.887 INFO:teuthology.orchestra.run.trial035.stdout: 2026-02-24T07:55:23.887 INFO:teuthology.orchestra.run.trial035.stderr:dumped all 2026-02-24T07:55:24.018 INFO:tasks.radosbench.radosbench.0.trial035.stdout:2026-02-24T07:55:24.017813+0000 min lat: 0.000728555 max lat: 0.0462542 avg lat: 0.00218689 2026-02-24T07:55:24.018 INFO:tasks.radosbench.radosbench.0.trial035.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2026-02-24T07:55:24.019 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 40 15 292520 292505 457.003 342.375 0.00159115 0.00218689 2026-02-24T07:55:24.521 INFO:teuthology.orchestra.run.trial035.stdout:{"pg_ready":true,"pg_map":{"version":442,"stamp":"2026-02-24T07:55:23.758278+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":17760256544,"num_objects":270993,"num_object_clones":0,"num_object_copies":812979,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":270993,"num_whiteouts":0,"num_read":136,"num_read_kb":240,"num_write":271126,"num_write_kb":17345574,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":51,"ondisk_log_size":51,"up":51,"acting":51,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":51,"num_osds":4,"num_per_pool_osds":4,"num_per_pool_omap_osds":4,"kb":2936012800,"kb_used":57228428,"kb_used_data":51594812,"kb_used_omap":93,"kb_used_meta":5633378,"kb_avail":2878784372,"statfs":{"total":3006477107200,"available":2947875196928,"internally_reserved":0,"allocated":52833087488,"data_stored":52831080568,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":95701,"internal_metadata":5768579627},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[9,3],"upper_bound":4},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":5005312000,"num_objects":76375,"num_object_clones":0,"num_object_copies":229125,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":76375,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":76375,"num_write_kb":4888000,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001683"},"pg_stats":[{"pgid":"6.f","version":"96'16894","reported_seq":16916,"reported_epoch":96,"state":"active+clean","last_fresh":"2026-02-24T07:55:20.866256+0000","last_change":"2026-02-24T07:54:42.927271+0000","last_active":"2026-02-24T07:55:20.866256+0000","last_peered":"2026-02-24T07:55:20.866256+0000","last_clean":"2026-02-24T07:55:20.866256+0000","last_became_active":"2026-02-24T07:54:42.927080+0000","last_became_peered":"2026-02-24T07:54:42.927080+0000","last_unstale":"2026-02-24T07:55:20.866256+0000","last_undegraded":"2026-02-24T07:55:20.866256+0000","last_fullsized":"2026-02-24T07:55:20.866256+0000","mapping_epoch":90,"log_start":"96'16891","ondisk_log_start":"96'16891","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_clean_scrub_stamp":"2026-02-24T07:54:41.918128+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:56:00.801546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1107165184,"num_objects":16894,"num_object_clones":0,"num_object_copies":50682,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16894,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16894,"num_write_kb":1081216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.e","version":"96'16962","reported_seq":16983,"reported_epoch":96,"state":"active+clean","last_fresh":"2026-02-24T07:55:20.757478+0000","last_change":"2026-02-24T07:54:42.929104+0000","last_active":"2026-02-24T07:55:20.757478+0000","last_peered":"2026-02-24T07:55:20.757478+0000","last_clean":"2026-02-24T07:55:20.757478+0000","last_became_active":"2026-02-24T07:54:42.928748+0000","last_became_peered":"2026-02-24T07:54:42.928748+0000","last_unstale":"2026-02-24T07:55:20.757478+0000","last_undegraded":"2026-02-24T07:55:20.757478+0000","last_fullsized":"2026-02-24T07:55:20.757478+0000","mapping_epoch":90,"log_start":"96'16959","ondisk_log_start":"96'16959","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_clean_scrub_stamp":"2026-02-24T07:54:41.918128+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:55:53.670906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1111621632,"num_objects":16962,"num_object_clones":0,"num_object_copies":50886,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16962,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16962,"num_write_kb":1085568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.d","version":"96'16967","reported_seq":16988,"reported_epoch":96,"state":"active+clean","last_fresh":"2026-02-24T07:55:20.765528+0000","last_change":"2026-02-24T07:54:42.927535+0000","last_active":"2026-02-24T07:55:20.765528+0000","last_peered":"2026-02-24T07:55:20.765528+0000","last_clean":"2026-02-24T07:55:20.765528+0000","last_became_active":"2026-02-24T07:54:42.926886+0000","last_became_peered":"2026-02-24T07:54:42.926886+0000","last_unstale":"2026-02-24T07:55:20.765528+0000","last_undegraded":"2026-02-24T07:55:20.765528+0000","last_fullsized":"2026-02-24T07:55:20.765528+0000","mapping_epoch":90,"log_start":"96'16964","ondisk_log_start":"96'16964","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_clean_scrub_stamp":"2026-02-24T07:54:41.918128+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:55:43.220815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1111949312,"num_objects":16967,"num_object_clones":0,"num_object_copies":50901,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16967,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16967,"num_write_kb":1085888,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.c","version":"96'17049","reported_seq":17071,"reported_epoch":96,"state":"active+clean","last_fresh":"2026-02-24T07:55:20.867598+0000","last_change":"2026-02-24T07:54:42.928482+0000","last_active":"2026-02-24T07:55:20.867598+0000","last_peered":"2026-02-24T07:55:20.867598+0000","last_clean":"2026-02-24T07:55:20.867598+0000","last_became_active":"2026-02-24T07:54:42.928238+0000","last_became_peered":"2026-02-24T07:54:42.928238+0000","last_unstale":"2026-02-24T07:55:20.867598+0000","last_undegraded":"2026-02-24T07:55:20.867598+0000","last_fullsized":"2026-02-24T07:55:20.867598+0000","mapping_epoch":90,"log_start":"96'17046","ondisk_log_start":"96'17046","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_clean_scrub_stamp":"2026-02-24T07:54:41.918128+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:56:09.181158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1117323264,"num_objects":17049,"num_object_clones":0,"num_object_copies":51147,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17049,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17049,"num_write_kb":1091136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.b","version":"96'16983","reported_seq":17005,"reported_epoch":96,"state":"active+clean","last_fresh":"2026-02-24T07:55:20.865130+0000","last_change":"2026-02-24T07:54:42.927223+0000","last_active":"2026-02-24T07:55:20.865130+0000","last_peered":"2026-02-24T07:55:20.865130+0000","last_clean":"2026-02-24T07:55:20.865130+0000","last_became_active":"2026-02-24T07:54:42.926965+0000","last_became_peered":"2026-02-24T07:54:42.926965+0000","last_unstale":"2026-02-24T07:55:20.865130+0000","last_undegraded":"2026-02-24T07:55:20.865130+0000","last_fullsized":"2026-02-24T07:55:20.865130+0000","mapping_epoch":90,"log_start":"96'16980","ondisk_log_start":"96'16980","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_clean_scrub_stamp":"2026-02-24T07:54:41.918128+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:56:04.661233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1112997888,"num_objects":16983,"num_object_clones":0,"num_object_copies":50949,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16983,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16983,"num_write_kb":1086912,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.a","version":"96'16690","reported_seq":16712,"reported_epoch":96,"state":"active+clean","last_fresh":"2026-02-24T07:55:20.763799+0000","last_change":"2026-02-24T07:54:42.926926+0000","last_active":"2026-02-24T07:55:20.763799+0000","last_peered":"2026-02-24T07:55:20.763799+0000","last_clean":"2026-02-24T07:55:20.763799+0000","last_became_active":"2026-02-24T07:54:42.926477+0000","last_became_peered":"2026-02-24T07:54:42.926477+0000","last_unstale":"2026-02-24T07:55:20.763799+0000","last_undegraded":"2026-02-24T07:55:20.763799+0000","last_fullsized":"2026-02-24T07:55:20.763799+0000","mapping_epoch":90,"log_start":"96'16687","ondisk_log_start":"96'16687","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_clean_scrub_stamp":"2026-02-24T07:54:41.918128+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:55:46.420689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1093795840,"num_objects":16690,"num_object_clones":0,"num_object_copies":50070,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16690,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16690,"num_write_kb":1068160,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"96'16967","reported_seq":16989,"reported_epoch":96,"state":"active+clean","last_fresh":"2026-02-24T07:55:20.788210+0000","last_change":"2026-02-24T07:54:42.926317+0000","last_active":"2026-02-24T07:55:20.788210+0000","last_peered":"2026-02-24T07:55:20.788210+0000","last_clean":"2026-02-24T07:55:20.788210+0000","last_became_active":"2026-02-24T07:54:42.926136+0000","last_became_peered":"2026-02-24T07:54:42.926136+0000","last_unstale":"2026-02-24T07:55:20.788210+0000","last_undegraded":"2026-02-24T07:55:20.788210+0000","last_fullsized":"2026-02-24T07:55:20.788210+0000","mapping_epoch":90,"log_start":"96'16964","ondisk_log_start":"96'16964","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_clean_scrub_stamp":"2026-02-24T07:54:41.918128+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:55:45.176781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1111949312,"num_objects":16967,"num_object_clones":0,"num_object_copies":50901,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16967,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16967,"num_write_kb":1085888,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,1],"acting":[0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.8","version":"96'16862","reported_seq":16884,"reported_epoch":96,"state":"active+clean","last_fresh":"2026-02-24T07:55:20.778284+0000","last_change":"2026-02-24T07:54:42.929353+0000","last_active":"2026-02-24T07:55:20.778284+0000","last_peered":"2026-02-24T07:55:20.778284+0000","last_clean":"2026-02-24T07:55:20.778284+0000","last_became_active":"2026-02-24T07:54:42.929078+0000","last_became_peered":"2026-02-24T07:54:42.929078+0000","last_unstale":"2026-02-24T07:55:20.778284+0000","last_undegraded":"2026-02-24T07:55:20.778284+0000","last_fullsized":"2026-02-24T07:55:20.778284+0000","mapping_epoch":90,"log_start":"96'16859","ondisk_log_start":"96'16859","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_clean_scrub_stamp":"2026-02-24T07:54:41.918128+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:55:48.303915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1105068032,"num_objects":16862,"num_object_clones":0,"num_object_copies":50586,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16862,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16862,"num_write_kb":1079168,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.6","version":"96'16928","reported_seq":16950,"reported_epoch":96,"state":"active+clean","last_fresh":"2026-02-24T07:55:20.860482+0000","last_change":"2026-02-24T07:54:42.926358+0000","last_active":"2026-02-24T07:55:20.860482+0000","last_peered":"2026-02-24T07:55:20.860482+0000","last_clean":"2026-02-24T07:55:20.860482+0000","last_became_active":"2026-02-24T07:54:42.926045+0000","last_became_peered":"2026-02-24T07:54:42.926045+0000","last_unstale":"2026-02-24T07:55:20.860482+0000","last_undegraded":"2026-02-24T07:55:20.860482+0000","last_fullsized":"2026-02-24T07:55:20.860482+0000","mapping_epoch":90,"log_start":"96'16925","ondisk_log_start":"96'16925","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_clean_scrub_stamp":"2026-02-24T07:54:41.918128+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:56:09.126690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1109393408,"num_objects":16928,"num_object_clones":0,"num_object_copies":50784,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16928,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16928,"num_write_kb":1083392,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.5","version":"96'16994","reported_seq":17016,"reported_epoch":96,"state":"active+clean","last_fresh":"2026-02-24T07:55:20.764273+0000","last_change":"2026-02-24T07:54:42.928143+0000","last_active":"2026-02-24T07:55:20.764273+0000","last_peered":"2026-02-24T07:55:20.764273+0000","last_clean":"2026-02-24T07:55:20.764273+0000","last_became_active":"2026-02-24T07:54:42.927535+0000","last_became_peered":"2026-02-24T07:54:42.927535+0000","last_unstale":"2026-02-24T07:55:20.764273+0000","last_undegraded":"2026-02-24T07:55:20.764273+0000","last_fullsized":"2026-02-24T07:55:20.764273+0000","mapping_epoch":90,"log_start":"96'16991","ondisk_log_start":"96'16991","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_clean_scrub_stamp":"2026-02-24T07:54:41.918128+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:55:53.650972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1113718784,"num_objects":16994,"num_object_clones":0,"num_object_copies":50982,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16994,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16994,"num_write_kb":1087616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.4","version":"96'17094","reported_seq":17116,"reported_epoch":96,"state":"active+clean","last_fresh":"2026-02-24T07:55:20.865049+0000","last_change":"2026-02-24T07:54:42.928344+0000","last_active":"2026-02-24T07:55:20.865049+0000","last_peered":"2026-02-24T07:55:20.865049+0000","last_clean":"2026-02-24T07:55:20.865049+0000","last_became_active":"2026-02-24T07:54:42.928011+0000","last_became_peered":"2026-02-24T07:54:42.928011+0000","last_unstale":"2026-02-24T07:55:20.865049+0000","last_undegraded":"2026-02-24T07:55:20.865049+0000","last_fullsized":"2026-02-24T07:55:20.865049+0000","mapping_epoch":90,"log_start":"96'17091","ondisk_log_start":"96'17091","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_clean_scrub_stamp":"2026-02-24T07:54:41.918128+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:55:43.506947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1120272384,"num_objects":17094,"num_object_clones":0,"num_object_copies":51282,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17094,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17094,"num_write_kb":1094016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.3","version":"96'17055","reported_seq":17077,"reported_epoch":96,"state":"active+clean","last_fresh":"2026-02-24T07:55:20.765469+0000","last_change":"2026-02-24T07:54:42.928536+0000","last_active":"2026-02-24T07:55:20.765469+0000","last_peered":"2026-02-24T07:55:20.765469+0000","last_clean":"2026-02-24T07:55:20.765469+0000","last_became_active":"2026-02-24T07:54:42.927730+0000","last_became_peered":"2026-02-24T07:54:42.927730+0000","last_unstale":"2026-02-24T07:55:20.765469+0000","last_undegraded":"2026-02-24T07:55:20.765469+0000","last_fullsized":"2026-02-24T07:55:20.765469+0000","mapping_epoch":90,"log_start":"96'17052","ondisk_log_start":"96'17052","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_clean_scrub_stamp":"2026-02-24T07:54:41.918128+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:55:58.987317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1117716480,"num_objects":17055,"num_object_clones":0,"num_object_copies":51165,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17055,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17055,"num_write_kb":1091520,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.2","version":"96'16869","reported_seq":16891,"reported_epoch":96,"state":"active+clean","last_fresh":"2026-02-24T07:55:20.773415+0000","last_change":"2026-02-24T07:54:42.930452+0000","last_active":"2026-02-24T07:55:20.773415+0000","last_peered":"2026-02-24T07:55:20.773415+0000","last_clean":"2026-02-24T07:55:20.773415+0000","last_became_active":"2026-02-24T07:54:42.930247+0000","last_became_peered":"2026-02-24T07:54:42.930247+0000","last_unstale":"2026-02-24T07:55:20.773415+0000","last_undegraded":"2026-02-24T07:55:20.773415+0000","last_fullsized":"2026-02-24T07:55:20.773415+0000","mapping_epoch":90,"log_start":"96'16866","ondisk_log_start":"96'16866","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_clean_scrub_stamp":"2026-02-24T07:54:41.918128+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:55:49.096947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1105526784,"num_objects":16869,"num_object_clones":0,"num_object_copies":50607,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16869,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16869,"num_write_kb":1079616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1","version":"96'16988","reported_seq":17010,"reported_epoch":96,"state":"active+clean","last_fresh":"2026-02-24T07:55:20.869606+0000","last_change":"2026-02-24T07:54:42.926619+0000","last_active":"2026-02-24T07:55:20.869606+0000","last_peered":"2026-02-24T07:55:20.869606+0000","last_clean":"2026-02-24T07:55:20.869606+0000","last_became_active":"2026-02-24T07:54:42.926442+0000","last_became_peered":"2026-02-24T07:54:42.926442+0000","last_unstale":"2026-02-24T07:55:20.869606+0000","last_undegraded":"2026-02-24T07:55:20.869606+0000","last_fullsized":"2026-02-24T07:55:20.869606+0000","mapping_epoch":90,"log_start":"96'16985","ondisk_log_start":"96'16985","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_clean_scrub_stamp":"2026-02-24T07:54:41.918128+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:56:03.125413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1113325568,"num_objects":16988,"num_object_clones":0,"num_object_copies":50964,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16988,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16988,"num_write_kb":1087232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.0","version":"96'16711","reported_seq":16733,"reported_epoch":96,"state":"active+clean","last_fresh":"2026-02-24T07:55:20.793851+0000","last_change":"2026-02-24T07:54:42.927178+0000","last_active":"2026-02-24T07:55:20.793851+0000","last_peered":"2026-02-24T07:55:20.793851+0000","last_clean":"2026-02-24T07:55:20.793851+0000","last_became_active":"2026-02-24T07:54:42.926910+0000","last_became_peered":"2026-02-24T07:54:42.926910+0000","last_unstale":"2026-02-24T07:55:20.793851+0000","last_undegraded":"2026-02-24T07:55:20.793851+0000","last_fullsized":"2026-02-24T07:55:20.793851+0000","mapping_epoch":90,"log_start":"96'16708","ondisk_log_start":"96'16708","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_clean_scrub_stamp":"2026-02-24T07:54:41.918128+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:55:58.002045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1095172096,"num_objects":16711,"num_object_clones":0,"num_object_copies":50133,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16711,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16711,"num_write_kb":1069504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.7","version":"96'16978","reported_seq":17000,"reported_epoch":96,"state":"active+clean","last_fresh":"2026-02-24T07:55:20.780743+0000","last_change":"2026-02-24T07:54:42.930348+0000","last_active":"2026-02-24T07:55:20.780743+0000","last_peered":"2026-02-24T07:55:20.780743+0000","last_clean":"2026-02-24T07:55:20.780743+0000","last_became_active":"2026-02-24T07:54:42.930142+0000","last_became_peered":"2026-02-24T07:54:42.930142+0000","last_unstale":"2026-02-24T07:55:20.780743+0000","last_undegraded":"2026-02-24T07:55:20.780743+0000","last_fullsized":"2026-02-24T07:55:20.780743+0000","mapping_epoch":90,"log_start":"96'16975","ondisk_log_start":"96'16975","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_clean_scrub_stamp":"2026-02-24T07:54:41.918128+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:55:56.704334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1112670208,"num_objects":16978,"num_object_clones":0,"num_object_copies":50934,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16978,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16978,"num_write_kb":1086592,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"29'99","reported_seq":398,"reported_epoch":96,"state":"active+clean","last_fresh":"2026-02-24T07:55:16.398160+0000","last_change":"2026-02-24T07:55:16.398160+0000","last_active":"2026-02-24T07:55:16.398160+0000","last_peered":"2026-02-24T07:55:16.398160+0000","last_clean":"2026-02-24T07:55:16.398160+0000","last_became_active":"2026-02-24T07:42:54.186719+0000","last_became_peered":"2026-02-24T07:42:54.186719+0000","last_unstale":"2026-02-24T07:55:16.398160+0000","last_undegraded":"2026-02-24T07:55:16.398160+0000","last_fullsized":"2026-02-24T07:55:16.398160+0000","mapping_epoch":27,"log_start":"29'96","ondisk_log_start":"29'96","created":20,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'99","last_scrub_stamp":"2026-02-24T07:55:16.398131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:42:43.640776+0000","last_clean_scrub_stamp":"2026-02-24T07:55:16.398131+0000","objects_scrubbed":2,"log_size":3,"log_dups_size":96,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:56:32.925996+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":240,"num_write":135,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":16,"stat_sum":{"num_bytes":17759666176,"num_objects":270991,"num_object_clones":0,"num_object_copies":812973,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":270991,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":270991,"num_write_kb":17343424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":53274738688,"data_stored":53274738688,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":48,"ondisk_log_size":48,"up":48,"acting":48,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":240,"num_write":135,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1744,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":3,"up_from":25,"seq":107374182552,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":11928340,"kb_used_data":10731520,"kb_used_omap":17,"kb_used_meta":1196782,"kb_avail":722074860,"statfs":{"total":751619276800,"available":739404656640,"internally_reserved":0,"allocated":10989076480,"data_stored":10988573862,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":18320,"internal_metadata":1225504880},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":19,"seq":81604378778,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":14528852,"kb_used_data":12878144,"kb_used_omap":26,"kb_used_meta":1650661,"kb_avail":719474348,"statfs":{"total":751619276800,"available":736741732352,"internally_reserved":0,"allocated":13187219456,"data_stored":13186716838,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":27400,"internal_metadata":1690277112},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[6,1],"upper_bound":4},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542299,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":16215376,"kb_used_data":15017788,"kb_used_omap":24,"kb_used_meta":1197479,"kb_avail":717787824,"statfs":{"total":751619276800,"available":735014731776,"internally_reserved":0,"allocated":15378214912,"data_stored":15377715846,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":24784,"internal_metadata":1226219312},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,2],"upper_bound":4},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705822,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":14555860,"kb_used_data":12967360,"kb_used_omap":24,"kb_used_meta":1588455,"kb_avail":719447340,"statfs":{"total":751619276800,"available":736714076160,"internally_reserved":0,"allocated":13278576640,"data_stored":13278074022,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":25197,"internal_metadata":1626578323},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1744,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":13305970688,"data_stored":13305970688,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":15564931072,"data_stored":15564931072,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":13304856576,"data_stored":13304856576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":11098980352,"data_stored":11098980352,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-24T07:55:24.524 INFO:tasks.cephadm.ceph_manager.ceph:increase pool size by 10 2026-02-24T07:55:24.524 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph osd pool set .mgr pg_num 381 2026-02-24T07:55:24.663 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:55:25.018 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 41 16 298485 298469 454.947 372.75 0.00157916 0.00219678 2026-02-24T07:55:25.077 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:55:24 trial035 ceph-mon[39284]: pgmap v442: 17 pgs: 17 active+clean; 17 GiB data, 55 GiB used, 2.7 TiB / 2.7 TiB avail; 398 MiB/s wr, 6.36k op/s 2026-02-24T07:55:25.078 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:55:24 trial035 ceph-mon[39284]: from='client.15475 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2026-02-24T07:55:25.087 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:55:25.075+0000 7f0f4e0dc640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:55:25.087 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:55:25.076+0000 7f0f4e0dc640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:55:25.223 INFO:teuthology.orchestra.run.trial035.stderr:Error EPERM: crimson_allow_pg_split is false; pg_num increase denied for crimson pool 2026-02-24T07:55:25.419 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2026-02-24T07:55:25.842 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-24T07:55:25.884 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:55:26.018 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 42 15 304151 304136 452.548 354.188 0.00237554 0.00220841 2026-02-24T07:55:26.151 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:55:25 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/4150024601' entity='client.admin' cmd={"prefix": "osd pool set", "pool": ".mgr", "var": "pg_num", "val": "381"} : dispatch 2026-02-24T07:55:26.374 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:55:26.355+0000 7fa5df39a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:55:26.375 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:55:26.355+0000 7fa5df39a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:55:26.375 INFO:teuthology.orchestra.run.trial035.stderr:osd.0: osd_enable_op_tracker = '' 2026-02-24T07:55:26.375 INFO:teuthology.orchestra.run.trial035.stdout:osd.0: {} 2026-02-24T07:55:26.388 INFO:teuthology.orchestra.run.trial035.stdout:osd.1: {} 2026-02-24T07:55:26.389 INFO:teuthology.orchestra.run.trial035.stderr:osd.1: osd_enable_op_tracker = '' 2026-02-24T07:55:26.401 INFO:teuthology.orchestra.run.trial035.stdout:osd.2: {} 2026-02-24T07:55:26.402 INFO:teuthology.orchestra.run.trial035.stderr:osd.2: osd_enable_op_tracker = '' 2026-02-24T07:55:26.416 INFO:teuthology.orchestra.run.trial035.stdout:osd.3: {} 2026-02-24T07:55:26.416 INFO:teuthology.orchestra.run.trial035.stderr:osd.3: osd_enable_op_tracker = '' 2026-02-24T07:55:27.018 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 43 15 309936 309921 450.431 361.562 0.00195603 0.00221881 2026-02-24T07:55:27.043 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:55:26 trial035 ceph-mon[39284]: pgmap v443: 17 pgs: 17 active+clean; 17 GiB data, 55 GiB used, 2.7 TiB / 2.7 TiB avail; 398 MiB/s wr, 6.36k op/s 2026-02-24T07:55:27.043 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:55:26 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/1994078440' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2026-02-24T07:55:27.043 INFO:journalctl@ceph.mgr.x.trial035.stdout:Feb 24 07:55:26 trial035 ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-mgr-x[39621]: ::ffff:10.20.193.91 - - [24/Feb/2026:07:55:26] "GET /metrics HTTP/1.1" 200 4213 "" "Prometheus/3.6.0" 2026-02-24T07:55:27.356 INFO:journalctl@ceph.prometheus.a.trial091.stdout:Feb 24 07:55:26 trial091 ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-prometheus-a[66007]: time=2026-02-24T07:55:26.973Z level=ERROR source=scrape.go:1630 msg="Failed to determine correct type of scrape target." component="scrape manager" scrape_pool=ceph target=http://trial035:9283/metrics content_type="text/html;charset=utf-8" fallback_media_type="" err="received unsupported Content-Type \"text/html;charset=utf-8\" and no fallback_scrape_protocol specified for target" 2026-02-24T07:55:28.018 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 44 16 318245 318229 451.994 519.25 0.00118776 0.00221113 2026-02-24T07:55:28.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:55:27 trial035 ceph-mon[39284]: pgmap v444: 17 pgs: 17 active+clean; 18 GiB data, 60 GiB used, 2.7 TiB / 2.7 TiB avail; 564 MiB/s wr, 9.02k op/s 2026-02-24T07:55:29.018 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 45 16 326465 326449 453.365 513.75 0.00193323 0.00220445 2026-02-24T07:55:29.045 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2026-02-24T07:55:29.198 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:55:29.327 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:55:29.324+0000 7f809fd81640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:55:29.327 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:55:29.325+0000 7f809fd81640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:55:29.342 INFO:teuthology.orchestra.run.trial035.stdout:osd.0: {} 2026-02-24T07:55:29.342 INFO:teuthology.orchestra.run.trial035.stderr:osd.0: osd_enable_op_tracker = '' 2026-02-24T07:55:29.357 INFO:teuthology.orchestra.run.trial035.stdout:osd.1: {} 2026-02-24T07:55:29.357 INFO:teuthology.orchestra.run.trial035.stderr:osd.1: osd_enable_op_tracker = '' 2026-02-24T07:55:29.370 INFO:teuthology.orchestra.run.trial035.stdout:osd.2: {} 2026-02-24T07:55:29.371 INFO:teuthology.orchestra.run.trial035.stderr:osd.2: osd_enable_op_tracker = '' 2026-02-24T07:55:29.385 INFO:teuthology.orchestra.run.trial035.stdout:osd.3: {} 2026-02-24T07:55:29.386 INFO:teuthology.orchestra.run.trial035.stderr:osd.3: osd_enable_op_tracker = '' 2026-02-24T07:55:30.018 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 46 16 334514 334498 454.445 503.062 0.00185135 0.0021992 2026-02-24T07:55:30.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:55:29 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/926010908' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2026-02-24T07:55:30.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:55:29 trial035 ceph-mon[39284]: from='mgr.14284 10.20.193.35:0/3681373222' entity='mgr.x' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2026-02-24T07:55:30.843 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2026-02-24T07:55:30.843 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2026-02-24T07:55:30.844 INFO:tasks.thrashosds.thrasher:Growing pool unique_pool_4 2026-02-24T07:55:30.844 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph pg dump --format=json 2026-02-24T07:55:30.994 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:55:31.018 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 47 16 342025 342009 454.763 469.438 0.00186798 0.00219769 2026-02-24T07:55:31.123 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:55:31.120+0000 7fe9a943d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:55:31.123 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:55:31.120+0000 7fe9a943d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:55:31.123 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:55:30 trial035 ceph-mon[39284]: pgmap v445: 17 pgs: 17 active+clean; 18 GiB data, 60 GiB used, 2.7 TiB / 2.7 TiB avail; 359 MiB/s wr, 5.74k op/s 2026-02-24T07:55:31.265 INFO:teuthology.orchestra.run.trial035.stdout: 2026-02-24T07:55:31.265 INFO:teuthology.orchestra.run.trial035.stderr:dumped all 2026-02-24T07:55:31.314 INFO:teuthology.orchestra.run.trial035.stdout:{"pg_ready":true,"pg_map":{"version":445,"stamp":"2026-02-24T07:55:29.759128+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19850789408,"num_objects":302892,"num_object_clones":0,"num_object_copies":908676,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":302892,"num_whiteouts":0,"num_read":136,"num_read_kb":240,"num_write":303025,"num_write_kb":19387110,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":51,"ondisk_log_size":51,"up":51,"acting":51,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":51,"num_osds":4,"num_per_pool_osds":4,"num_per_pool_omap_osds":4,"kb":2936012800,"kb_used":63182412,"kb_used_data":57819516,"kb_used_omap":84,"kb_used_meta":5362731,"kb_avail":2872830388,"statfs":{"total":3006477107200,"available":2941778317312,"internally_reserved":0,"allocated":59207184384,"data_stored":59205177464,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":86301,"internal_metadata":5491437283},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,0,6,4],"upper_bound":64},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":4516675584,"num_objects":68919,"num_object_clones":0,"num_object_copies":206757,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":68919,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":68919,"num_write_kb":4410816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001690"},"pg_stats":[{"pgid":"6.f","version":"96'18944","reported_seq":18966,"reported_epoch":96,"state":"active+clean","last_fresh":"2026-02-24T07:55:25.869779+0000","last_change":"2026-02-24T07:54:42.927271+0000","last_active":"2026-02-24T07:55:25.869779+0000","last_peered":"2026-02-24T07:55:25.869779+0000","last_clean":"2026-02-24T07:55:25.869779+0000","last_became_active":"2026-02-24T07:54:42.927080+0000","last_became_peered":"2026-02-24T07:54:42.927080+0000","last_unstale":"2026-02-24T07:55:25.869779+0000","last_undegraded":"2026-02-24T07:55:25.869779+0000","last_fullsized":"2026-02-24T07:55:25.869779+0000","mapping_epoch":90,"log_start":"96'18941","ondisk_log_start":"96'18941","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_clean_scrub_stamp":"2026-02-24T07:54:41.918128+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:56:00.801546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1241513984,"num_objects":18944,"num_object_clones":0,"num_object_copies":56832,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18944,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18944,"num_write_kb":1212416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.e","version":"96'18913","reported_seq":18935,"reported_epoch":96,"state":"active+clean","last_fresh":"2026-02-24T07:55:25.762868+0000","last_change":"2026-02-24T07:54:42.929104+0000","last_active":"2026-02-24T07:55:25.762868+0000","last_peered":"2026-02-24T07:55:25.762868+0000","last_clean":"2026-02-24T07:55:25.762868+0000","last_became_active":"2026-02-24T07:54:42.928748+0000","last_became_peered":"2026-02-24T07:54:42.928748+0000","last_unstale":"2026-02-24T07:55:25.762868+0000","last_undegraded":"2026-02-24T07:55:25.762868+0000","last_fullsized":"2026-02-24T07:55:25.762868+0000","mapping_epoch":90,"log_start":"96'18910","ondisk_log_start":"96'18910","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_clean_scrub_stamp":"2026-02-24T07:54:41.918128+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:55:53.670906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1239482368,"num_objects":18913,"num_object_clones":0,"num_object_copies":56739,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18913,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18913,"num_write_kb":1210432,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.d","version":"96'18924","reported_seq":18946,"reported_epoch":96,"state":"active+clean","last_fresh":"2026-02-24T07:55:25.760811+0000","last_change":"2026-02-24T07:54:42.927535+0000","last_active":"2026-02-24T07:55:25.760811+0000","last_peered":"2026-02-24T07:55:25.760811+0000","last_clean":"2026-02-24T07:55:25.760811+0000","last_became_active":"2026-02-24T07:54:42.926886+0000","last_became_peered":"2026-02-24T07:54:42.926886+0000","last_unstale":"2026-02-24T07:55:25.760811+0000","last_undegraded":"2026-02-24T07:55:25.760811+0000","last_fullsized":"2026-02-24T07:55:25.760811+0000","mapping_epoch":90,"log_start":"96'18921","ondisk_log_start":"96'18921","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_clean_scrub_stamp":"2026-02-24T07:54:41.918128+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:55:43.220815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1240203264,"num_objects":18924,"num_object_clones":0,"num_object_copies":56772,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18924,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18924,"num_write_kb":1211136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.c","version":"96'19023","reported_seq":19045,"reported_epoch":96,"state":"active+clean","last_fresh":"2026-02-24T07:55:25.858453+0000","last_change":"2026-02-24T07:54:42.928482+0000","last_active":"2026-02-24T07:55:25.858453+0000","last_peered":"2026-02-24T07:55:25.858453+0000","last_clean":"2026-02-24T07:55:25.858453+0000","last_became_active":"2026-02-24T07:54:42.928238+0000","last_became_peered":"2026-02-24T07:54:42.928238+0000","last_unstale":"2026-02-24T07:55:25.858453+0000","last_undegraded":"2026-02-24T07:55:25.858453+0000","last_fullsized":"2026-02-24T07:55:25.858453+0000","mapping_epoch":90,"log_start":"96'19020","ondisk_log_start":"96'19020","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_clean_scrub_stamp":"2026-02-24T07:54:41.918128+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:56:09.181158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1246691328,"num_objects":19023,"num_object_clones":0,"num_object_copies":57069,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19023,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19023,"num_write_kb":1217472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.b","version":"96'18931","reported_seq":18953,"reported_epoch":96,"state":"active+clean","last_fresh":"2026-02-24T07:55:25.867229+0000","last_change":"2026-02-24T07:54:42.927223+0000","last_active":"2026-02-24T07:55:25.867229+0000","last_peered":"2026-02-24T07:55:25.867229+0000","last_clean":"2026-02-24T07:55:25.867229+0000","last_became_active":"2026-02-24T07:54:42.926965+0000","last_became_peered":"2026-02-24T07:54:42.926965+0000","last_unstale":"2026-02-24T07:55:25.867229+0000","last_undegraded":"2026-02-24T07:55:25.867229+0000","last_fullsized":"2026-02-24T07:55:25.867229+0000","mapping_epoch":90,"log_start":"96'18928","ondisk_log_start":"96'18928","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_clean_scrub_stamp":"2026-02-24T07:54:41.918128+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:56:04.661233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1240662016,"num_objects":18931,"num_object_clones":0,"num_object_copies":56793,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18931,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18931,"num_write_kb":1211584,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.a","version":"96'18708","reported_seq":18730,"reported_epoch":96,"state":"active+clean","last_fresh":"2026-02-24T07:55:25.765016+0000","last_change":"2026-02-24T07:54:42.926926+0000","last_active":"2026-02-24T07:55:25.765016+0000","last_peered":"2026-02-24T07:55:25.765016+0000","last_clean":"2026-02-24T07:55:25.765016+0000","last_became_active":"2026-02-24T07:54:42.926477+0000","last_became_peered":"2026-02-24T07:54:42.926477+0000","last_unstale":"2026-02-24T07:55:25.765016+0000","last_undegraded":"2026-02-24T07:55:25.765016+0000","last_fullsized":"2026-02-24T07:55:25.765016+0000","mapping_epoch":90,"log_start":"96'18705","ondisk_log_start":"96'18705","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_clean_scrub_stamp":"2026-02-24T07:54:41.918128+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:55:46.420689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1226047488,"num_objects":18708,"num_object_clones":0,"num_object_copies":56124,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18708,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18708,"num_write_kb":1197312,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"96'18936","reported_seq":18958,"reported_epoch":96,"state":"active+clean","last_fresh":"2026-02-24T07:55:25.796186+0000","last_change":"2026-02-24T07:54:42.926317+0000","last_active":"2026-02-24T07:55:25.796186+0000","last_peered":"2026-02-24T07:55:25.796186+0000","last_clean":"2026-02-24T07:55:25.796186+0000","last_became_active":"2026-02-24T07:54:42.926136+0000","last_became_peered":"2026-02-24T07:54:42.926136+0000","last_unstale":"2026-02-24T07:55:25.796186+0000","last_undegraded":"2026-02-24T07:55:25.796186+0000","last_fullsized":"2026-02-24T07:55:25.796186+0000","mapping_epoch":90,"log_start":"96'18933","ondisk_log_start":"96'18933","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_clean_scrub_stamp":"2026-02-24T07:54:41.918128+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:55:45.176781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1240989696,"num_objects":18936,"num_object_clones":0,"num_object_copies":56808,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18936,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18936,"num_write_kb":1211904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,1],"acting":[0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.8","version":"96'18891","reported_seq":18913,"reported_epoch":96,"state":"active+clean","last_fresh":"2026-02-24T07:55:25.778438+0000","last_change":"2026-02-24T07:54:42.929353+0000","last_active":"2026-02-24T07:55:25.778438+0000","last_peered":"2026-02-24T07:55:25.778438+0000","last_clean":"2026-02-24T07:55:25.778438+0000","last_became_active":"2026-02-24T07:54:42.929078+0000","last_became_peered":"2026-02-24T07:54:42.929078+0000","last_unstale":"2026-02-24T07:55:25.778438+0000","last_undegraded":"2026-02-24T07:55:25.778438+0000","last_fullsized":"2026-02-24T07:55:25.778438+0000","mapping_epoch":90,"log_start":"96'18888","ondisk_log_start":"96'18888","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_clean_scrub_stamp":"2026-02-24T07:54:41.918128+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:55:48.303915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1238040576,"num_objects":18891,"num_object_clones":0,"num_object_copies":56673,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18891,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18891,"num_write_kb":1209024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.6","version":"96'18916","reported_seq":18938,"reported_epoch":96,"state":"active+clean","last_fresh":"2026-02-24T07:55:25.867536+0000","last_change":"2026-02-24T07:54:42.926358+0000","last_active":"2026-02-24T07:55:25.867536+0000","last_peered":"2026-02-24T07:55:25.867536+0000","last_clean":"2026-02-24T07:55:25.867536+0000","last_became_active":"2026-02-24T07:54:42.926045+0000","last_became_peered":"2026-02-24T07:54:42.926045+0000","last_unstale":"2026-02-24T07:55:25.867536+0000","last_undegraded":"2026-02-24T07:55:25.867536+0000","last_fullsized":"2026-02-24T07:55:25.867536+0000","mapping_epoch":90,"log_start":"96'18913","ondisk_log_start":"96'18913","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_clean_scrub_stamp":"2026-02-24T07:54:41.918128+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:56:09.126690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1239678976,"num_objects":18916,"num_object_clones":0,"num_object_copies":56748,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18916,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18916,"num_write_kb":1210624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.5","version":"96'18940","reported_seq":18962,"reported_epoch":96,"state":"active+clean","last_fresh":"2026-02-24T07:55:25.761348+0000","last_change":"2026-02-24T07:54:42.928143+0000","last_active":"2026-02-24T07:55:25.761348+0000","last_peered":"2026-02-24T07:55:25.761348+0000","last_clean":"2026-02-24T07:55:25.761348+0000","last_became_active":"2026-02-24T07:54:42.927535+0000","last_became_peered":"2026-02-24T07:54:42.927535+0000","last_unstale":"2026-02-24T07:55:25.761348+0000","last_undegraded":"2026-02-24T07:55:25.761348+0000","last_fullsized":"2026-02-24T07:55:25.761348+0000","mapping_epoch":90,"log_start":"96'18937","ondisk_log_start":"96'18937","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_clean_scrub_stamp":"2026-02-24T07:54:41.918128+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:55:53.650972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1241251840,"num_objects":18940,"num_object_clones":0,"num_object_copies":56820,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18940,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18940,"num_write_kb":1212160,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.4","version":"96'19110","reported_seq":19132,"reported_epoch":96,"state":"active+clean","last_fresh":"2026-02-24T07:55:25.867613+0000","last_change":"2026-02-24T07:54:42.928344+0000","last_active":"2026-02-24T07:55:25.867613+0000","last_peered":"2026-02-24T07:55:25.867613+0000","last_clean":"2026-02-24T07:55:25.867613+0000","last_became_active":"2026-02-24T07:54:42.928011+0000","last_became_peered":"2026-02-24T07:54:42.928011+0000","last_unstale":"2026-02-24T07:55:25.867613+0000","last_undegraded":"2026-02-24T07:55:25.867613+0000","last_fullsized":"2026-02-24T07:55:25.867613+0000","mapping_epoch":90,"log_start":"96'19107","ondisk_log_start":"96'19107","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_clean_scrub_stamp":"2026-02-24T07:54:41.918128+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:55:43.506947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1252392960,"num_objects":19110,"num_object_clones":0,"num_object_copies":57330,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19110,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19110,"num_write_kb":1223040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.3","version":"96'19060","reported_seq":19082,"reported_epoch":96,"state":"active+clean","last_fresh":"2026-02-24T07:55:25.763725+0000","last_change":"2026-02-24T07:54:42.928536+0000","last_active":"2026-02-24T07:55:25.763725+0000","last_peered":"2026-02-24T07:55:25.763725+0000","last_clean":"2026-02-24T07:55:25.763725+0000","last_became_active":"2026-02-24T07:54:42.927730+0000","last_became_peered":"2026-02-24T07:54:42.927730+0000","last_unstale":"2026-02-24T07:55:25.763725+0000","last_undegraded":"2026-02-24T07:55:25.763725+0000","last_fullsized":"2026-02-24T07:55:25.763725+0000","mapping_epoch":90,"log_start":"96'19057","ondisk_log_start":"96'19057","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_clean_scrub_stamp":"2026-02-24T07:54:41.918128+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:55:58.987317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1249116160,"num_objects":19060,"num_object_clones":0,"num_object_copies":57180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19060,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19060,"num_write_kb":1219840,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.2","version":"96'18932","reported_seq":18954,"reported_epoch":96,"state":"active+clean","last_fresh":"2026-02-24T07:55:25.778182+0000","last_change":"2026-02-24T07:54:42.930452+0000","last_active":"2026-02-24T07:55:25.778182+0000","last_peered":"2026-02-24T07:55:25.778182+0000","last_clean":"2026-02-24T07:55:25.778182+0000","last_became_active":"2026-02-24T07:54:42.930247+0000","last_became_peered":"2026-02-24T07:54:42.930247+0000","last_unstale":"2026-02-24T07:55:25.778182+0000","last_undegraded":"2026-02-24T07:55:25.778182+0000","last_fullsized":"2026-02-24T07:55:25.778182+0000","mapping_epoch":90,"log_start":"96'18929","ondisk_log_start":"96'18929","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_clean_scrub_stamp":"2026-02-24T07:54:41.918128+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:55:49.096947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1240727552,"num_objects":18932,"num_object_clones":0,"num_object_copies":56796,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18932,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18932,"num_write_kb":1211648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1","version":"96'19005","reported_seq":19027,"reported_epoch":96,"state":"active+clean","last_fresh":"2026-02-24T07:55:25.865160+0000","last_change":"2026-02-24T07:54:42.926619+0000","last_active":"2026-02-24T07:55:25.865160+0000","last_peered":"2026-02-24T07:55:25.865160+0000","last_clean":"2026-02-24T07:55:25.865160+0000","last_became_active":"2026-02-24T07:54:42.926442+0000","last_became_peered":"2026-02-24T07:54:42.926442+0000","last_unstale":"2026-02-24T07:55:25.865160+0000","last_undegraded":"2026-02-24T07:55:25.865160+0000","last_fullsized":"2026-02-24T07:55:25.865160+0000","mapping_epoch":90,"log_start":"96'19002","ondisk_log_start":"96'19002","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_clean_scrub_stamp":"2026-02-24T07:54:41.918128+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:56:03.125413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1245511680,"num_objects":19005,"num_object_clones":0,"num_object_copies":57015,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19005,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19005,"num_write_kb":1216320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.0","version":"96'18667","reported_seq":18689,"reported_epoch":96,"state":"active+clean","last_fresh":"2026-02-24T07:55:25.796138+0000","last_change":"2026-02-24T07:54:42.927178+0000","last_active":"2026-02-24T07:55:25.796138+0000","last_peered":"2026-02-24T07:55:25.796138+0000","last_clean":"2026-02-24T07:55:25.796138+0000","last_became_active":"2026-02-24T07:54:42.926910+0000","last_became_peered":"2026-02-24T07:54:42.926910+0000","last_unstale":"2026-02-24T07:55:25.796138+0000","last_undegraded":"2026-02-24T07:55:25.796138+0000","last_fullsized":"2026-02-24T07:55:25.796138+0000","mapping_epoch":90,"log_start":"96'18664","ondisk_log_start":"96'18664","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_clean_scrub_stamp":"2026-02-24T07:54:41.918128+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:55:58.002045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1223360512,"num_objects":18667,"num_object_clones":0,"num_object_copies":56001,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18667,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18667,"num_write_kb":1194688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.7","version":"96'18990","reported_seq":19011,"reported_epoch":96,"state":"active+clean","last_fresh":"2026-02-24T07:55:25.780877+0000","last_change":"2026-02-24T07:54:42.930348+0000","last_active":"2026-02-24T07:55:25.780877+0000","last_peered":"2026-02-24T07:55:25.780877+0000","last_clean":"2026-02-24T07:55:25.780877+0000","last_became_active":"2026-02-24T07:54:42.930142+0000","last_became_peered":"2026-02-24T07:54:42.930142+0000","last_unstale":"2026-02-24T07:55:25.780877+0000","last_undegraded":"2026-02-24T07:55:25.780877+0000","last_fullsized":"2026-02-24T07:55:25.780877+0000","mapping_epoch":90,"log_start":"96'18987","ondisk_log_start":"96'18987","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_clean_scrub_stamp":"2026-02-24T07:54:41.918128+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:55:56.704334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1244528640,"num_objects":18990,"num_object_clones":0,"num_object_copies":56970,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18990,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18990,"num_write_kb":1215360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"29'99","reported_seq":398,"reported_epoch":96,"state":"active+clean","last_fresh":"2026-02-24T07:55:16.398160+0000","last_change":"2026-02-24T07:55:16.398160+0000","last_active":"2026-02-24T07:55:16.398160+0000","last_peered":"2026-02-24T07:55:16.398160+0000","last_clean":"2026-02-24T07:55:16.398160+0000","last_became_active":"2026-02-24T07:42:54.186719+0000","last_became_peered":"2026-02-24T07:42:54.186719+0000","last_unstale":"2026-02-24T07:55:16.398160+0000","last_undegraded":"2026-02-24T07:55:16.398160+0000","last_fullsized":"2026-02-24T07:55:16.398160+0000","mapping_epoch":27,"log_start":"29'96","ondisk_log_start":"29'96","created":20,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'99","last_scrub_stamp":"2026-02-24T07:55:16.398131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:42:43.640776+0000","last_clean_scrub_stamp":"2026-02-24T07:55:16.398131+0000","objects_scrubbed":2,"log_size":3,"log_dups_size":96,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:56:32.925996+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":240,"num_write":135,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":16,"stat_sum":{"num_bytes":19850199040,"num_objects":302890,"num_object_clones":0,"num_object_copies":908670,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":302890,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":302890,"num_write_kb":19384960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":59544436736,"data_stored":59544436736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":48,"ondisk_log_size":48,"up":48,"acting":48,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":240,"num_write":135,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1744,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":3,"up_from":25,"seq":107374182553,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":13314708,"kb_used_data":12045696,"kb_used_omap":17,"kb_used_meta":1268974,"kb_avail":720688492,"statfs":{"total":751619276800,"available":737985015808,"internally_reserved":0,"allocated":12334792704,"data_stored":12334290086,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":18320,"internal_metadata":1299429488},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":19,"seq":81604378779,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":15528212,"kb_used_data":14454976,"kb_used_omap":17,"kb_used_meta":1073198,"kb_avail":718474988,"statfs":{"total":751619276800,"available":735718387712,"internally_reserved":0,"allocated":14801895424,"data_stored":14801392806,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":18000,"internal_metadata":1098955184},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,4],"upper_bound":64},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542300,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":18133008,"kb_used_data":16827196,"kb_used_omap":24,"kb_used_meta":1305767,"kb_avail":715870192,"statfs":{"total":751619276800,"available":733051076608,"internally_reserved":0,"allocated":17231048704,"data_stored":17230549638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":24784,"internal_metadata":1337106224},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,6],"upper_bound":32},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705823,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":16206484,"kb_used_data":14491648,"kb_used_omap":24,"kb_used_meta":1714791,"kb_avail":717796716,"statfs":{"total":751619276800,"available":735023837184,"internally_reserved":0,"allocated":14839447552,"data_stored":14838944934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":25197,"internal_metadata":1755946387},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1744,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":14870904832,"data_stored":14870904832,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":17391222784,"data_stored":17391222784,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":14871494656,"data_stored":14871494656,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12410814464,"data_stored":12410814464,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-24T07:55:31.316 INFO:tasks.cephadm.ceph_manager.ceph:increase pool size by 10 2026-02-24T07:55:31.316 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph osd pool set unique_pool_4 pg_num 56 2026-02-24T07:55:31.433 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2026-02-24T07:55:31.470 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:55:31.577 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:55:31.615 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:55:31.612+0000 7f5937b40640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:55:31.615 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:55:31.613+0000 7f5937b40640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:55:31.704 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:55:31.702+0000 7f0011347640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:55:31.704 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:55:31.702+0000 7f0011347640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:55:31.721 INFO:teuthology.orchestra.run.trial035.stdout:osd.0: {} 2026-02-24T07:55:31.721 INFO:teuthology.orchestra.run.trial035.stderr:osd.0: osd_enable_op_tracker = '' 2026-02-24T07:55:31.735 INFO:teuthology.orchestra.run.trial035.stdout:osd.1: {} 2026-02-24T07:55:31.735 INFO:teuthology.orchestra.run.trial035.stderr:osd.1: osd_enable_op_tracker = '' 2026-02-24T07:55:31.750 INFO:teuthology.orchestra.run.trial035.stdout:osd.2: {} 2026-02-24T07:55:31.750 INFO:teuthology.orchestra.run.trial035.stderr:osd.2: osd_enable_op_tracker = '' 2026-02-24T07:55:31.765 INFO:teuthology.orchestra.run.trial035.stdout:osd.3: {} 2026-02-24T07:55:31.766 INFO:teuthology.orchestra.run.trial035.stderr:osd.3: osd_enable_op_tracker = '' 2026-02-24T07:55:31.772 INFO:teuthology.orchestra.run.trial035.stderr:Error EPERM: crimson_allow_pg_split is false; pg_num increase denied for crimson pool 2026-02-24T07:55:31.820 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-24T07:55:32.019 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 48 16 349589 349573 455.137 472.75 0.00171036 0.00219586 2026-02-24T07:55:32.021 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:55:31 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/3643500302' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2026-02-24T07:55:32.021 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:55:31 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/3516367056' entity='client.admin' cmd={"prefix": "osd pool set", "pool": "unique_pool_4", "var": "pg_num", "val": "56"} : dispatch 2026-02-24T07:55:33.019 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 49 16 357421 357405 455.838 489.5 0.00211338 0.00219248 2026-02-24T07:55:33.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:55:32 trial035 ceph-mon[39284]: from='client.15483 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2026-02-24T07:55:33.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:55:32 trial035 ceph-mon[39284]: pgmap v446: 17 pgs: 17 active+clean; 21 GiB data, 67 GiB used, 2.7 TiB / 2.7 TiB avail; 555 MiB/s wr, 8.87k op/s 2026-02-24T07:55:33.819 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2026-02-24T07:55:33.971 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:55:34.019 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 50 16 365360 365344 456.644 496.188 0.0030051 0.00218862 2026-02-24T07:55:34.368 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:55:34.362+0000 7f8fdec31640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:55:34.369 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:55:34.362+0000 7f8fdec31640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:55:34.381 INFO:teuthology.orchestra.run.trial035.stdout:osd.0: {} 2026-02-24T07:55:34.382 INFO:teuthology.orchestra.run.trial035.stderr:osd.0: osd_enable_op_tracker = '' 2026-02-24T07:55:34.399 INFO:teuthology.orchestra.run.trial035.stdout:osd.1: {} 2026-02-24T07:55:34.399 INFO:teuthology.orchestra.run.trial035.stderr:osd.1: osd_enable_op_tracker = '' 2026-02-24T07:55:34.412 INFO:teuthology.orchestra.run.trial035.stdout:osd.2: {} 2026-02-24T07:55:34.413 INFO:teuthology.orchestra.run.trial035.stderr:osd.2: osd_enable_op_tracker = '' 2026-02-24T07:55:34.427 INFO:teuthology.orchestra.run.trial035.stdout:osd.3: {} 2026-02-24T07:55:34.428 INFO:teuthology.orchestra.run.trial035.stderr:osd.3: osd_enable_op_tracker = '' 2026-02-24T07:55:34.954 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:55:34 trial035 ceph-mon[39284]: pgmap v447: 17 pgs: 17 active+clean; 21 GiB data, 67 GiB used, 2.7 TiB / 2.7 TiB avail; 362 MiB/s wr, 5.79k op/s 2026-02-24T07:55:34.954 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:55:34 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/689923237' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2026-02-24T07:55:35.019 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 51 16 370754 370738 454.3 337.125 0.00121369 0.0021999 2026-02-24T07:55:36.019 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 52 16 378344 378328 454.685 474.375 0.00192892 0.00219804 2026-02-24T07:55:36.821 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2026-02-24T07:55:36.821 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2026-02-24T07:55:36.822 INFO:tasks.thrashosds.thrasher:Growing pool .mgr 2026-02-24T07:55:36.822 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph pg dump --format=json 2026-02-24T07:55:36.970 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:55:37.001 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2026-02-24T07:55:37.019 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 53 16 386174 386158 455.339 489.375 0.00143026 0.0021949 2026-02-24T07:55:37.106 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:55:37.105+0000 7f8068efb640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:55:37.107 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:55:36 trial035 ceph-mon[39284]: pgmap v448: 17 pgs: 17 active+clean; 21 GiB data, 67 GiB used, 2.7 TiB / 2.7 TiB avail; 362 MiB/s wr, 5.79k op/s 2026-02-24T07:55:37.107 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:55:37.106+0000 7f8068efb640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:55:37.107 INFO:journalctl@ceph.mgr.x.trial035.stdout:Feb 24 07:55:36 trial035 ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-mgr-x[39621]: ::ffff:10.20.193.91 - - [24/Feb/2026:07:55:36] "GET /metrics HTTP/1.1" 200 4220 "" "Prometheus/3.6.0" 2026-02-24T07:55:37.160 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:55:37.256 INFO:teuthology.orchestra.run.trial035.stdout: 2026-02-24T07:55:37.257 INFO:teuthology.orchestra.run.trial035.stderr:dumped all 2026-02-24T07:55:37.304 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:55:37.303+0000 7f27aed3d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:55:37.304 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:55:37.303+0000 7f27aed3d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:55:37.309 INFO:teuthology.orchestra.run.trial035.stdout:{"pg_ready":true,"pg_map":{"version":448,"stamp":"2026-02-24T07:55:35.759971+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":22312976928,"num_objects":340462,"num_object_clones":0,"num_object_copies":1021386,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":340462,"num_whiteouts":0,"num_read":136,"num_read_kb":240,"num_write":340595,"num_write_kb":21791590,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":52,"ondisk_log_size":52,"up":51,"acting":51,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":51,"num_osds":4,"num_per_pool_osds":4,"num_per_pool_omap_osds":4,"kb":2936012800,"kb_used":70078732,"kb_used_data":64806012,"kb_used_omap":75,"kb_used_meta":5272564,"kb_avail":2865934068,"statfs":{"total":3006477107200,"available":2934716485632,"internally_reserved":0,"allocated":66361356288,"data_stored":66359349368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":76806,"internal_metadata":5399106554},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[15,3,1],"upper_bound":8},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":4552720384,"num_objects":69469,"num_object_clones":0,"num_object_copies":208407,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":69469,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":69469,"num_write_kb":4446016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001693"},"pg_stats":[{"pgid":"6.f","version":"96'21321","reported_seq":21343,"reported_epoch":96,"state":"active+clean","last_fresh":"2026-02-24T07:55:30.863334+0000","last_change":"2026-02-24T07:54:42.927271+0000","last_active":"2026-02-24T07:55:30.863334+0000","last_peered":"2026-02-24T07:55:30.863334+0000","last_clean":"2026-02-24T07:55:30.863334+0000","last_became_active":"2026-02-24T07:54:42.927080+0000","last_became_peered":"2026-02-24T07:54:42.927080+0000","last_unstale":"2026-02-24T07:55:30.863334+0000","last_undegraded":"2026-02-24T07:55:30.863334+0000","last_fullsized":"2026-02-24T07:55:30.863334+0000","mapping_epoch":90,"log_start":"96'21318","ondisk_log_start":"96'21318","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_clean_scrub_stamp":"2026-02-24T07:54:41.918128+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:56:00.801546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1397293056,"num_objects":21321,"num_object_clones":0,"num_object_copies":63963,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21321,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21321,"num_write_kb":1364544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.e","version":"96'21312","reported_seq":21334,"reported_epoch":96,"state":"active+clean","last_fresh":"2026-02-24T07:55:30.764364+0000","last_change":"2026-02-24T07:54:42.929104+0000","last_active":"2026-02-24T07:55:30.764364+0000","last_peered":"2026-02-24T07:55:30.764364+0000","last_clean":"2026-02-24T07:55:30.764364+0000","last_became_active":"2026-02-24T07:54:42.928748+0000","last_became_peered":"2026-02-24T07:54:42.928748+0000","last_unstale":"2026-02-24T07:55:30.764364+0000","last_undegraded":"2026-02-24T07:55:30.764364+0000","last_fullsized":"2026-02-24T07:55:30.764364+0000","mapping_epoch":90,"log_start":"96'21309","ondisk_log_start":"96'21309","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_clean_scrub_stamp":"2026-02-24T07:54:41.918128+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:55:53.670906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1396703232,"num_objects":21312,"num_object_clones":0,"num_object_copies":63936,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21312,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21312,"num_write_kb":1363968,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.d","version":"96'21254","reported_seq":21276,"reported_epoch":96,"state":"active+clean","last_fresh":"2026-02-24T07:55:30.765888+0000","last_change":"2026-02-24T07:54:42.927535+0000","last_active":"2026-02-24T07:55:30.765888+0000","last_peered":"2026-02-24T07:55:30.765888+0000","last_clean":"2026-02-24T07:55:30.765888+0000","last_became_active":"2026-02-24T07:54:42.926886+0000","last_became_peered":"2026-02-24T07:54:42.926886+0000","last_unstale":"2026-02-24T07:55:30.765888+0000","last_undegraded":"2026-02-24T07:55:30.765888+0000","last_fullsized":"2026-02-24T07:55:30.765888+0000","mapping_epoch":90,"log_start":"96'21251","ondisk_log_start":"96'21251","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_clean_scrub_stamp":"2026-02-24T07:54:41.918128+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:55:43.220815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1392902144,"num_objects":21254,"num_object_clones":0,"num_object_copies":63762,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21254,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21254,"num_write_kb":1360256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.c","version":"96'21353","reported_seq":21374,"reported_epoch":96,"state":"active+clean","last_fresh":"2026-02-24T07:55:30.868909+0000","last_change":"2026-02-24T07:54:42.928482+0000","last_active":"2026-02-24T07:55:30.868909+0000","last_peered":"2026-02-24T07:55:30.868909+0000","last_clean":"2026-02-24T07:55:30.868909+0000","last_became_active":"2026-02-24T07:54:42.928238+0000","last_became_peered":"2026-02-24T07:54:42.928238+0000","last_unstale":"2026-02-24T07:55:30.868909+0000","last_undegraded":"2026-02-24T07:55:30.868909+0000","last_fullsized":"2026-02-24T07:55:30.868909+0000","mapping_epoch":90,"log_start":"96'21350","ondisk_log_start":"96'21350","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_clean_scrub_stamp":"2026-02-24T07:54:41.918128+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:56:09.181158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1399390208,"num_objects":21353,"num_object_clones":0,"num_object_copies":64059,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21353,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21353,"num_write_kb":1366592,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.b","version":"96'21192","reported_seq":21214,"reported_epoch":96,"state":"active+clean","last_fresh":"2026-02-24T07:55:30.866269+0000","last_change":"2026-02-24T07:54:42.927223+0000","last_active":"2026-02-24T07:55:30.866269+0000","last_peered":"2026-02-24T07:55:30.866269+0000","last_clean":"2026-02-24T07:55:30.866269+0000","last_became_active":"2026-02-24T07:54:42.926965+0000","last_became_peered":"2026-02-24T07:54:42.926965+0000","last_unstale":"2026-02-24T07:55:30.866269+0000","last_undegraded":"2026-02-24T07:55:30.866269+0000","last_fullsized":"2026-02-24T07:55:30.866269+0000","mapping_epoch":90,"log_start":"96'21189","ondisk_log_start":"96'21189","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_clean_scrub_stamp":"2026-02-24T07:54:41.918128+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:56:04.661233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1388838912,"num_objects":21192,"num_object_clones":0,"num_object_copies":63576,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21192,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21192,"num_write_kb":1356288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.a","version":"96'21094","reported_seq":21115,"reported_epoch":96,"state":"active+clean","last_fresh":"2026-02-24T07:55:30.761307+0000","last_change":"2026-02-24T07:54:42.926926+0000","last_active":"2026-02-24T07:55:30.761307+0000","last_peered":"2026-02-24T07:55:30.761307+0000","last_clean":"2026-02-24T07:55:30.761307+0000","last_became_active":"2026-02-24T07:54:42.926477+0000","last_became_peered":"2026-02-24T07:54:42.926477+0000","last_unstale":"2026-02-24T07:55:30.761307+0000","last_undegraded":"2026-02-24T07:55:30.761307+0000","last_fullsized":"2026-02-24T07:55:30.761307+0000","mapping_epoch":90,"log_start":"96'21091","ondisk_log_start":"96'21091","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_clean_scrub_stamp":"2026-02-24T07:54:41.918128+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:55:46.420689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1382416384,"num_objects":21094,"num_object_clones":0,"num_object_copies":63282,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21094,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21094,"num_write_kb":1350016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"96'21296","reported_seq":21318,"reported_epoch":96,"state":"active+clean","last_fresh":"2026-02-24T07:55:30.796695+0000","last_change":"2026-02-24T07:54:42.926317+0000","last_active":"2026-02-24T07:55:30.796695+0000","last_peered":"2026-02-24T07:55:30.796695+0000","last_clean":"2026-02-24T07:55:30.796695+0000","last_became_active":"2026-02-24T07:54:42.926136+0000","last_became_peered":"2026-02-24T07:54:42.926136+0000","last_unstale":"2026-02-24T07:55:30.796695+0000","last_undegraded":"2026-02-24T07:55:30.796695+0000","last_fullsized":"2026-02-24T07:55:30.796695+0000","mapping_epoch":90,"log_start":"96'21293","ondisk_log_start":"96'21293","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_clean_scrub_stamp":"2026-02-24T07:54:41.918128+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:55:45.176781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1395654656,"num_objects":21296,"num_object_clones":0,"num_object_copies":63888,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21296,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21296,"num_write_kb":1362944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,1],"acting":[0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.8","version":"96'21239","reported_seq":21261,"reported_epoch":96,"state":"active+clean","last_fresh":"2026-02-24T07:55:30.779708+0000","last_change":"2026-02-24T07:54:42.929353+0000","last_active":"2026-02-24T07:55:30.779708+0000","last_peered":"2026-02-24T07:55:30.779708+0000","last_clean":"2026-02-24T07:55:30.779708+0000","last_became_active":"2026-02-24T07:54:42.929078+0000","last_became_peered":"2026-02-24T07:54:42.929078+0000","last_unstale":"2026-02-24T07:55:30.779708+0000","last_undegraded":"2026-02-24T07:55:30.779708+0000","last_fullsized":"2026-02-24T07:55:30.779708+0000","mapping_epoch":90,"log_start":"96'21236","ondisk_log_start":"96'21236","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_clean_scrub_stamp":"2026-02-24T07:54:41.918128+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:55:48.303915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1391919104,"num_objects":21239,"num_object_clones":0,"num_object_copies":63717,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21239,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21239,"num_write_kb":1359296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.6","version":"96'21274","reported_seq":21296,"reported_epoch":96,"state":"active+clean","last_fresh":"2026-02-24T07:55:30.868770+0000","last_change":"2026-02-24T07:54:42.926358+0000","last_active":"2026-02-24T07:55:30.868770+0000","last_peered":"2026-02-24T07:55:30.868770+0000","last_clean":"2026-02-24T07:55:30.868770+0000","last_became_active":"2026-02-24T07:54:42.926045+0000","last_became_peered":"2026-02-24T07:54:42.926045+0000","last_unstale":"2026-02-24T07:55:30.868770+0000","last_undegraded":"2026-02-24T07:55:30.868770+0000","last_fullsized":"2026-02-24T07:55:30.868770+0000","mapping_epoch":90,"log_start":"96'21271","ondisk_log_start":"96'21271","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_clean_scrub_stamp":"2026-02-24T07:54:41.918128+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:56:09.126690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1394212864,"num_objects":21274,"num_object_clones":0,"num_object_copies":63822,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21274,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21274,"num_write_kb":1361536,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.5","version":"96'21259","reported_seq":21280,"reported_epoch":96,"state":"active+clean","last_fresh":"2026-02-24T07:55:30.766362+0000","last_change":"2026-02-24T07:54:42.928143+0000","last_active":"2026-02-24T07:55:30.766362+0000","last_peered":"2026-02-24T07:55:30.766362+0000","last_clean":"2026-02-24T07:55:30.766362+0000","last_became_active":"2026-02-24T07:54:42.927535+0000","last_became_peered":"2026-02-24T07:54:42.927535+0000","last_unstale":"2026-02-24T07:55:30.766362+0000","last_undegraded":"2026-02-24T07:55:30.766362+0000","last_fullsized":"2026-02-24T07:55:30.766362+0000","mapping_epoch":90,"log_start":"96'21256","ondisk_log_start":"96'21256","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_clean_scrub_stamp":"2026-02-24T07:54:41.918128+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:55:53.650972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1393229824,"num_objects":21259,"num_object_clones":0,"num_object_copies":63777,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21259,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21259,"num_write_kb":1360576,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.4","version":"96'21480","reported_seq":21502,"reported_epoch":96,"state":"active+clean","last_fresh":"2026-02-24T07:55:30.867538+0000","last_change":"2026-02-24T07:54:42.928344+0000","last_active":"2026-02-24T07:55:30.867538+0000","last_peered":"2026-02-24T07:55:30.867538+0000","last_clean":"2026-02-24T07:55:30.867538+0000","last_became_active":"2026-02-24T07:54:42.928011+0000","last_became_peered":"2026-02-24T07:54:42.928011+0000","last_unstale":"2026-02-24T07:55:30.867538+0000","last_undegraded":"2026-02-24T07:55:30.867538+0000","last_fullsized":"2026-02-24T07:55:30.867538+0000","mapping_epoch":90,"log_start":"96'21477","ondisk_log_start":"96'21477","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_clean_scrub_stamp":"2026-02-24T07:54:41.918128+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:55:43.506947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1407713280,"num_objects":21480,"num_object_clones":0,"num_object_copies":64440,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21480,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21480,"num_write_kb":1374720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.3","version":"96'21386","reported_seq":21408,"reported_epoch":96,"state":"active+clean","last_fresh":"2026-02-24T07:55:30.758501+0000","last_change":"2026-02-24T07:54:42.928536+0000","last_active":"2026-02-24T07:55:30.758501+0000","last_peered":"2026-02-24T07:55:30.758501+0000","last_clean":"2026-02-24T07:55:30.758501+0000","last_became_active":"2026-02-24T07:54:42.927730+0000","last_became_peered":"2026-02-24T07:54:42.927730+0000","last_unstale":"2026-02-24T07:55:30.758501+0000","last_undegraded":"2026-02-24T07:55:30.758501+0000","last_fullsized":"2026-02-24T07:55:30.758501+0000","mapping_epoch":90,"log_start":"96'21383","ondisk_log_start":"96'21383","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_clean_scrub_stamp":"2026-02-24T07:54:41.918128+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:55:58.987317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1401552896,"num_objects":21386,"num_object_clones":0,"num_object_copies":64158,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21386,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21386,"num_write_kb":1368704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.2","version":"96'21252","reported_seq":21274,"reported_epoch":96,"state":"active+clean","last_fresh":"2026-02-24T07:55:30.776732+0000","last_change":"2026-02-24T07:54:42.930452+0000","last_active":"2026-02-24T07:55:30.776732+0000","last_peered":"2026-02-24T07:55:30.776732+0000","last_clean":"2026-02-24T07:55:30.776732+0000","last_became_active":"2026-02-24T07:54:42.930247+0000","last_became_peered":"2026-02-24T07:54:42.930247+0000","last_unstale":"2026-02-24T07:55:30.776732+0000","last_undegraded":"2026-02-24T07:55:30.776732+0000","last_fullsized":"2026-02-24T07:55:30.776732+0000","mapping_epoch":90,"log_start":"96'21249","ondisk_log_start":"96'21249","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_clean_scrub_stamp":"2026-02-24T07:54:41.918128+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:55:49.096947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1392771072,"num_objects":21252,"num_object_clones":0,"num_object_copies":63756,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21252,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21252,"num_write_kb":1360128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1","version":"96'21357","reported_seq":21379,"reported_epoch":96,"state":"active+clean","last_fresh":"2026-02-24T07:55:30.871433+0000","last_change":"2026-02-24T07:54:42.926619+0000","last_active":"2026-02-24T07:55:30.871433+0000","last_peered":"2026-02-24T07:55:30.871433+0000","last_clean":"2026-02-24T07:55:30.871433+0000","last_became_active":"2026-02-24T07:54:42.926442+0000","last_became_peered":"2026-02-24T07:54:42.926442+0000","last_unstale":"2026-02-24T07:55:30.871433+0000","last_undegraded":"2026-02-24T07:55:30.871433+0000","last_fullsized":"2026-02-24T07:55:30.871433+0000","mapping_epoch":90,"log_start":"96'21353","ondisk_log_start":"96'21353","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_clean_scrub_stamp":"2026-02-24T07:54:41.918128+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":3000,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:56:03.125413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1399652352,"num_objects":21357,"num_object_clones":0,"num_object_copies":64071,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21357,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21357,"num_write_kb":1366848,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.0","version":"96'21057","reported_seq":21079,"reported_epoch":96,"state":"active+clean","last_fresh":"2026-02-24T07:55:30.792521+0000","last_change":"2026-02-24T07:54:42.927178+0000","last_active":"2026-02-24T07:55:30.792521+0000","last_peered":"2026-02-24T07:55:30.792521+0000","last_clean":"2026-02-24T07:55:30.792521+0000","last_became_active":"2026-02-24T07:54:42.926910+0000","last_became_peered":"2026-02-24T07:54:42.926910+0000","last_unstale":"2026-02-24T07:55:30.792521+0000","last_undegraded":"2026-02-24T07:55:30.792521+0000","last_fullsized":"2026-02-24T07:55:30.792521+0000","mapping_epoch":90,"log_start":"96'21054","ondisk_log_start":"96'21054","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_clean_scrub_stamp":"2026-02-24T07:54:41.918128+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:55:58.002045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1379991552,"num_objects":21057,"num_object_clones":0,"num_object_copies":63171,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21057,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21057,"num_write_kb":1347648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.7","version":"96'21334","reported_seq":21356,"reported_epoch":96,"state":"active+clean","last_fresh":"2026-02-24T07:55:30.776918+0000","last_change":"2026-02-24T07:54:42.930348+0000","last_active":"2026-02-24T07:55:30.776918+0000","last_peered":"2026-02-24T07:55:30.776918+0000","last_clean":"2026-02-24T07:55:30.776918+0000","last_became_active":"2026-02-24T07:54:42.930142+0000","last_became_peered":"2026-02-24T07:54:42.930142+0000","last_unstale":"2026-02-24T07:55:30.776918+0000","last_undegraded":"2026-02-24T07:55:30.776918+0000","last_fullsized":"2026-02-24T07:55:30.776918+0000","mapping_epoch":90,"log_start":"96'21331","ondisk_log_start":"96'21331","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_clean_scrub_stamp":"2026-02-24T07:54:41.918128+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:55:56.704334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1398145024,"num_objects":21334,"num_object_clones":0,"num_object_copies":64002,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21334,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21334,"num_write_kb":1365376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"29'99","reported_seq":398,"reported_epoch":96,"state":"active+clean","last_fresh":"2026-02-24T07:55:16.398160+0000","last_change":"2026-02-24T07:55:16.398160+0000","last_active":"2026-02-24T07:55:16.398160+0000","last_peered":"2026-02-24T07:55:16.398160+0000","last_clean":"2026-02-24T07:55:16.398160+0000","last_became_active":"2026-02-24T07:42:54.186719+0000","last_became_peered":"2026-02-24T07:42:54.186719+0000","last_unstale":"2026-02-24T07:55:16.398160+0000","last_undegraded":"2026-02-24T07:55:16.398160+0000","last_fullsized":"2026-02-24T07:55:16.398160+0000","mapping_epoch":27,"log_start":"29'96","ondisk_log_start":"29'96","created":20,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'99","last_scrub_stamp":"2026-02-24T07:55:16.398131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:42:43.640776+0000","last_clean_scrub_stamp":"2026-02-24T07:55:16.398131+0000","objects_scrubbed":2,"log_size":3,"log_dups_size":96,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:56:32.925996+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":240,"num_write":135,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":16,"stat_sum":{"num_bytes":22312386560,"num_objects":340460,"num_object_clones":0,"num_object_copies":1021380,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":340460,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":340460,"num_write_kb":21789440,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":66933489664,"data_stored":66933489664,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":49,"ondisk_log_size":49,"up":48,"acting":48,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":240,"num_write":135,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1744,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":3,"up_from":25,"seq":107374182554,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":14852692,"kb_used_data":13511488,"kb_used_omap":17,"kb_used_meta":1341166,"kb_avail":719150508,"statfs":{"total":751619276800,"available":736410120192,"internally_reserved":0,"allocated":13835763712,"data_stored":13835261094,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":18320,"internal_metadata":1373354096},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":19,"seq":81604378780,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":17380244,"kb_used_data":16198720,"kb_used_omap":17,"kb_used_meta":1181486,"kb_avail":716622956,"statfs":{"total":751619276800,"available":733821906944,"internally_reserved":0,"allocated":16587489280,"data_stored":16586986662,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":18000,"internal_metadata":1209842096},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[8,1],"upper_bound":4},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542301,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":20302096,"kb_used_data":18851900,"kb_used_omap":24,"kb_used_meta":1450151,"kb_avail":713701104,"statfs":{"total":751619276800,"available":730829930496,"internally_reserved":0,"allocated":19304345600,"data_stored":19303846534,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":24784,"internal_metadata":1484955440},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[6,2,1],"upper_bound":8},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705824,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":17543700,"kb_used_data":16243904,"kb_used_omap":15,"kb_used_meta":1299760,"kb_avail":716459500,"statfs":{"total":751619276800,"available":733654528000,"internally_reserved":0,"allocated":16633757696,"data_stored":16633255078,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":15702,"internal_metadata":1330954922},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1744,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16722624512,"data_stored":16722624512,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":19550830592,"data_stored":19550830592,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16717512704,"data_stored":16717512704,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":13942521856,"data_stored":13942521856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-24T07:55:37.310 INFO:tasks.cephadm.ceph_manager.ceph:increase pool size by 10 2026-02-24T07:55:37.310 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph osd pool set .mgr pg_num 391 2026-02-24T07:55:37.322 INFO:teuthology.orchestra.run.trial035.stdout:osd.0: {} 2026-02-24T07:55:37.322 INFO:teuthology.orchestra.run.trial035.stderr:osd.0: osd_enable_op_tracker = '' 2026-02-24T07:55:37.335 INFO:teuthology.orchestra.run.trial035.stdout:osd.1: {} 2026-02-24T07:55:37.336 INFO:teuthology.orchestra.run.trial035.stderr:osd.1: osd_enable_op_tracker = '' 2026-02-24T07:55:37.350 INFO:teuthology.orchestra.run.trial035.stdout:osd.2: {} 2026-02-24T07:55:37.350 INFO:teuthology.orchestra.run.trial035.stderr:osd.2: osd_enable_op_tracker = '' 2026-02-24T07:55:37.356 INFO:journalctl@ceph.prometheus.a.trial091.stdout:Feb 24 07:55:36 trial091 ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-prometheus-a[66007]: time=2026-02-24T07:55:36.973Z level=ERROR source=scrape.go:1630 msg="Failed to determine correct type of scrape target." component="scrape manager" scrape_pool=ceph target=http://trial035:9283/metrics content_type="text/html;charset=utf-8" fallback_media_type="" err="received unsupported Content-Type \"text/html;charset=utf-8\" and no fallback_scrape_protocol specified for target" 2026-02-24T07:55:37.368 INFO:teuthology.orchestra.run.trial035.stdout:osd.3: {} 2026-02-24T07:55:37.369 INFO:teuthology.orchestra.run.trial035.stderr:osd.3: osd_enable_op_tracker = '' 2026-02-24T07:55:37.476 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:55:37.601 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:55:37.600+0000 7f9c0dd5c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:55:37.601 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:55:37.601+0000 7f9c0dd5c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:55:37.750 INFO:teuthology.orchestra.run.trial035.stderr:Error EPERM: crimson_allow_pg_split is false; pg_num increase denied for crimson pool 2026-02-24T07:55:37.790 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-24T07:55:38.003 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:55:37 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/2783784611' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2026-02-24T07:55:38.003 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:55:37 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/2044506945' entity='client.admin' cmd={"prefix": "osd pool set", "pool": ".mgr", "var": "pg_num", "val": "391"} : dispatch 2026-02-24T07:55:38.019 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 54 16 393872 393856 455.816 481.125 0.00233053 0.0021926 2026-02-24T07:55:39.019 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 55 16 401453 401437 456.142 473.812 0.0018077 0.00219103 2026-02-24T07:55:39.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:55:38 trial035 ceph-mon[39284]: from='client.15491 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2026-02-24T07:55:39.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:55:38 trial035 ceph-mon[39284]: pgmap v449: 17 pgs: 17 active+clean; 23 GiB data, 74 GiB used, 2.7 TiB / 2.7 TiB avail; 550 MiB/s wr, 8.81k op/s 2026-02-24T07:55:39.419 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2026-02-24T07:55:39.567 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:55:39.904 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:55:39.857+0000 7fc5a75e0640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:55:39.904 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:55:39.857+0000 7fc5a75e0640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:55:39.904 INFO:teuthology.orchestra.run.trial035.stderr:osd.0: osd_enable_op_tracker = '' 2026-02-24T07:55:39.904 INFO:teuthology.orchestra.run.trial035.stderr:osd.1: osd_enable_op_tracker = '' 2026-02-24T07:55:39.904 INFO:teuthology.orchestra.run.trial035.stdout:osd.0: {} 2026-02-24T07:55:39.905 INFO:teuthology.orchestra.run.trial035.stdout:osd.1: {} 2026-02-24T07:55:39.906 INFO:teuthology.orchestra.run.trial035.stdout:osd.2: {} 2026-02-24T07:55:39.906 INFO:teuthology.orchestra.run.trial035.stderr:osd.2: osd_enable_op_tracker = '' 2026-02-24T07:55:39.919 INFO:teuthology.orchestra.run.trial035.stdout:osd.3: {} 2026-02-24T07:55:39.919 INFO:teuthology.orchestra.run.trial035.stderr:osd.3: osd_enable_op_tracker = '' 2026-02-24T07:55:40.019 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 56 16 407642 407626 454.904 386.812 0.000883854 0.0021962 2026-02-24T07:55:40.167 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:55:39 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/3281758850' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2026-02-24T07:55:41.019 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 57 16 415258 415242 455.273 476 0.00222318 0.00219522 2026-02-24T07:55:41.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:55:40 trial035 ceph-mon[39284]: pgmap v450: 17 pgs: 17 active+clean; 23 GiB data, 74 GiB used, 2.7 TiB / 2.7 TiB avail; 384 MiB/s wr, 6.15k op/s 2026-02-24T07:55:42.019 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 58 15 423266 423251 456.053 500.562 0.00276195 0.00219146 2026-02-24T07:55:42.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:55:41 trial035 ceph-mon[39284]: pgmap v451: 17 pgs: 17 active+clean; 25 GiB data, 81 GiB used, 2.7 TiB / 2.7 TiB avail; 576 MiB/s wr, 9.22k op/s 2026-02-24T07:55:42.207 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2026-02-24T07:55:42.354 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:55:42.480 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:55:42.478+0000 7f3840191640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:55:42.480 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:55:42.478+0000 7f3840191640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:55:42.495 INFO:teuthology.orchestra.run.trial035.stdout:osd.0: {} 2026-02-24T07:55:42.495 INFO:teuthology.orchestra.run.trial035.stderr:osd.0: osd_enable_op_tracker = '' 2026-02-24T07:55:42.512 INFO:teuthology.orchestra.run.trial035.stdout:osd.1: {} 2026-02-24T07:55:42.512 INFO:teuthology.orchestra.run.trial035.stderr:osd.1: osd_enable_op_tracker = '' 2026-02-24T07:55:42.526 INFO:teuthology.orchestra.run.trial035.stdout:osd.2: {} 2026-02-24T07:55:42.526 INFO:teuthology.orchestra.run.trial035.stderr:osd.2: osd_enable_op_tracker = '' 2026-02-24T07:55:42.539 INFO:teuthology.orchestra.run.trial035.stdout:osd.3: {} 2026-02-24T07:55:42.540 INFO:teuthology.orchestra.run.trial035.stderr:osd.3: osd_enable_op_tracker = '' 2026-02-24T07:55:42.792 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2026-02-24T07:55:42.792 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2026-02-24T07:55:42.792 INFO:tasks.thrashosds.thrasher:fixing pg num pool unique_pool_4 2026-02-24T07:55:42.793 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph pg dump --format=json 2026-02-24T07:55:42.939 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:55:43.019 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 59 16 431402 431386 456.941 508.438 0.00221145 0.00218719 2026-02-24T07:55:43.067 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:55:43.066+0000 7f736bb63640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:55:43.067 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:55:43.066+0000 7f736bb63640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:55:43.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:55:42 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/540340776' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2026-02-24T07:55:43.212 INFO:teuthology.orchestra.run.trial035.stdout: 2026-02-24T07:55:43.212 INFO:teuthology.orchestra.run.trial035.stderr:dumped all 2026-02-24T07:55:43.259 INFO:teuthology.orchestra.run.trial035.stdout:{"pg_ready":true,"pg_map":{"version":451,"stamp":"2026-02-24T07:55:41.760875+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":27103265312,"num_objects":413556,"num_object_clones":0,"num_object_copies":1240668,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":413556,"num_whiteouts":0,"num_read":136,"num_read_kb":240,"num_write":413689,"num_write_kb":26469606,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":51,"ondisk_log_size":51,"up":51,"acting":51,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":51,"num_osds":4,"num_per_pool_osds":4,"num_per_pool_omap_osds":4,"kb":2936012800,"kb_used":84964684,"kb_used_data":78825276,"kb_used_omap":75,"kb_used_meta":6139188,"kb_avail":2851048116,"statfs":{"total":3006477107200,"available":2919473270784,"internally_reserved":0,"allocated":80717082624,"data_stored":80715075704,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":76806,"internal_metadata":6286529530},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,3,0,0,2],"upper_bound":32},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":7252475904,"num_objects":110664,"num_object_clones":0,"num_object_copies":331992,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":110664,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":110664,"num_write_kb":7082496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001747"},"pg_stats":[{"pgid":"6.f","version":"96'25877","reported_seq":25897,"reported_epoch":96,"state":"active+clean","last_fresh":"2026-02-24T07:55:40.869714+0000","last_change":"2026-02-24T07:54:42.927271+0000","last_active":"2026-02-24T07:55:40.869714+0000","last_peered":"2026-02-24T07:55:40.869714+0000","last_clean":"2026-02-24T07:55:40.869714+0000","last_became_active":"2026-02-24T07:54:42.927080+0000","last_became_peered":"2026-02-24T07:54:42.927080+0000","last_unstale":"2026-02-24T07:55:40.869714+0000","last_undegraded":"2026-02-24T07:55:40.869714+0000","last_fullsized":"2026-02-24T07:55:40.869714+0000","mapping_epoch":90,"log_start":"96'25874","ondisk_log_start":"96'25874","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_clean_scrub_stamp":"2026-02-24T07:54:41.918128+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:56:00.801546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1695875072,"num_objects":25877,"num_object_clones":0,"num_object_copies":77631,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25877,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":25877,"num_write_kb":1656128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.e","version":"96'25845","reported_seq":25867,"reported_epoch":96,"state":"active+clean","last_fresh":"2026-02-24T07:55:40.768076+0000","last_change":"2026-02-24T07:54:42.929104+0000","last_active":"2026-02-24T07:55:40.768076+0000","last_peered":"2026-02-24T07:55:40.768076+0000","last_clean":"2026-02-24T07:55:40.768076+0000","last_became_active":"2026-02-24T07:54:42.928748+0000","last_became_peered":"2026-02-24T07:54:42.928748+0000","last_unstale":"2026-02-24T07:55:40.768076+0000","last_undegraded":"2026-02-24T07:55:40.768076+0000","last_fullsized":"2026-02-24T07:55:40.768076+0000","mapping_epoch":90,"log_start":"96'25842","ondisk_log_start":"96'25842","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_clean_scrub_stamp":"2026-02-24T07:54:41.918128+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:55:53.670906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1693777920,"num_objects":25845,"num_object_clones":0,"num_object_copies":77535,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25845,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":25845,"num_write_kb":1654080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.d","version":"96'25852","reported_seq":25874,"reported_epoch":96,"state":"active+clean","last_fresh":"2026-02-24T07:55:40.767912+0000","last_change":"2026-02-24T07:54:42.927535+0000","last_active":"2026-02-24T07:55:40.767912+0000","last_peered":"2026-02-24T07:55:40.767912+0000","last_clean":"2026-02-24T07:55:40.767912+0000","last_became_active":"2026-02-24T07:54:42.926886+0000","last_became_peered":"2026-02-24T07:54:42.926886+0000","last_unstale":"2026-02-24T07:55:40.767912+0000","last_undegraded":"2026-02-24T07:55:40.767912+0000","last_fullsized":"2026-02-24T07:55:40.767912+0000","mapping_epoch":90,"log_start":"96'25849","ondisk_log_start":"96'25849","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_clean_scrub_stamp":"2026-02-24T07:54:41.918128+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:55:43.220815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1694236672,"num_objects":25852,"num_object_clones":0,"num_object_copies":77556,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25852,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":25852,"num_write_kb":1654528,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.c","version":"96'25953","reported_seq":25975,"reported_epoch":96,"state":"active+clean","last_fresh":"2026-02-24T07:55:40.873033+0000","last_change":"2026-02-24T07:54:42.928482+0000","last_active":"2026-02-24T07:55:40.873033+0000","last_peered":"2026-02-24T07:55:40.873033+0000","last_clean":"2026-02-24T07:55:40.873033+0000","last_became_active":"2026-02-24T07:54:42.928238+0000","last_became_peered":"2026-02-24T07:54:42.928238+0000","last_unstale":"2026-02-24T07:55:40.873033+0000","last_undegraded":"2026-02-24T07:55:40.873033+0000","last_fullsized":"2026-02-24T07:55:40.873033+0000","mapping_epoch":90,"log_start":"96'25950","ondisk_log_start":"96'25950","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_clean_scrub_stamp":"2026-02-24T07:54:41.918128+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:56:09.181158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1700855808,"num_objects":25953,"num_object_clones":0,"num_object_copies":77859,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25953,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":25953,"num_write_kb":1660992,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.b","version":"96'25692","reported_seq":25714,"reported_epoch":96,"state":"active+clean","last_fresh":"2026-02-24T07:55:40.871947+0000","last_change":"2026-02-24T07:54:42.927223+0000","last_active":"2026-02-24T07:55:40.871947+0000","last_peered":"2026-02-24T07:55:40.871947+0000","last_clean":"2026-02-24T07:55:40.871947+0000","last_became_active":"2026-02-24T07:54:42.926965+0000","last_became_peered":"2026-02-24T07:54:42.926965+0000","last_unstale":"2026-02-24T07:55:40.871947+0000","last_undegraded":"2026-02-24T07:55:40.871947+0000","last_fullsized":"2026-02-24T07:55:40.871947+0000","mapping_epoch":90,"log_start":"96'25689","ondisk_log_start":"96'25689","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_clean_scrub_stamp":"2026-02-24T07:54:41.918128+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:56:04.661233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1683750912,"num_objects":25692,"num_object_clones":0,"num_object_copies":77076,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25692,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":25692,"num_write_kb":1644288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.a","version":"96'25643","reported_seq":25665,"reported_epoch":96,"state":"active+clean","last_fresh":"2026-02-24T07:55:40.766396+0000","last_change":"2026-02-24T07:54:42.926926+0000","last_active":"2026-02-24T07:55:40.766396+0000","last_peered":"2026-02-24T07:55:40.766396+0000","last_clean":"2026-02-24T07:55:40.766396+0000","last_became_active":"2026-02-24T07:54:42.926477+0000","last_became_peered":"2026-02-24T07:54:42.926477+0000","last_unstale":"2026-02-24T07:55:40.766396+0000","last_undegraded":"2026-02-24T07:55:40.766396+0000","last_fullsized":"2026-02-24T07:55:40.766396+0000","mapping_epoch":90,"log_start":"96'25640","ondisk_log_start":"96'25640","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_clean_scrub_stamp":"2026-02-24T07:54:41.918128+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:55:46.420689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1680539648,"num_objects":25643,"num_object_clones":0,"num_object_copies":76929,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25643,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":25643,"num_write_kb":1641152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"96'25903","reported_seq":25923,"reported_epoch":96,"state":"active+clean","last_fresh":"2026-02-24T07:55:40.798530+0000","last_change":"2026-02-24T07:54:42.926317+0000","last_active":"2026-02-24T07:55:40.798530+0000","last_peered":"2026-02-24T07:55:40.798530+0000","last_clean":"2026-02-24T07:55:40.798530+0000","last_became_active":"2026-02-24T07:54:42.926136+0000","last_became_peered":"2026-02-24T07:54:42.926136+0000","last_unstale":"2026-02-24T07:55:40.798530+0000","last_undegraded":"2026-02-24T07:55:40.798530+0000","last_fullsized":"2026-02-24T07:55:40.798530+0000","mapping_epoch":90,"log_start":"96'25900","ondisk_log_start":"96'25900","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_clean_scrub_stamp":"2026-02-24T07:54:41.918128+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:55:45.176781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1697579008,"num_objects":25903,"num_object_clones":0,"num_object_copies":77709,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25903,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":25903,"num_write_kb":1657792,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,1],"acting":[0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.8","version":"96'25904","reported_seq":25926,"reported_epoch":96,"state":"active+clean","last_fresh":"2026-02-24T07:55:40.782927+0000","last_change":"2026-02-24T07:54:42.929353+0000","last_active":"2026-02-24T07:55:40.782927+0000","last_peered":"2026-02-24T07:55:40.782927+0000","last_clean":"2026-02-24T07:55:40.782927+0000","last_became_active":"2026-02-24T07:54:42.929078+0000","last_became_peered":"2026-02-24T07:54:42.929078+0000","last_unstale":"2026-02-24T07:55:40.782927+0000","last_undegraded":"2026-02-24T07:55:40.782927+0000","last_fullsized":"2026-02-24T07:55:40.782927+0000","mapping_epoch":90,"log_start":"96'25901","ondisk_log_start":"96'25901","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_clean_scrub_stamp":"2026-02-24T07:54:41.918128+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:55:48.303915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1697644544,"num_objects":25904,"num_object_clones":0,"num_object_copies":77712,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25904,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":25904,"num_write_kb":1657856,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.6","version":"96'25867","reported_seq":25889,"reported_epoch":96,"state":"active+clean","last_fresh":"2026-02-24T07:55:40.871324+0000","last_change":"2026-02-24T07:54:42.926358+0000","last_active":"2026-02-24T07:55:40.871324+0000","last_peered":"2026-02-24T07:55:40.871324+0000","last_clean":"2026-02-24T07:55:40.871324+0000","last_became_active":"2026-02-24T07:54:42.926045+0000","last_became_peered":"2026-02-24T07:54:42.926045+0000","last_unstale":"2026-02-24T07:55:40.871324+0000","last_undegraded":"2026-02-24T07:55:40.871324+0000","last_fullsized":"2026-02-24T07:55:40.871324+0000","mapping_epoch":90,"log_start":"96'25864","ondisk_log_start":"96'25864","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_clean_scrub_stamp":"2026-02-24T07:54:41.918128+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:56:09.126690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1695219712,"num_objects":25867,"num_object_clones":0,"num_object_copies":77601,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25867,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":25867,"num_write_kb":1655488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.5","version":"96'25781","reported_seq":25803,"reported_epoch":96,"state":"active+clean","last_fresh":"2026-02-24T07:55:40.764047+0000","last_change":"2026-02-24T07:54:42.928143+0000","last_active":"2026-02-24T07:55:40.764047+0000","last_peered":"2026-02-24T07:55:40.764047+0000","last_clean":"2026-02-24T07:55:40.764047+0000","last_became_active":"2026-02-24T07:54:42.927535+0000","last_became_peered":"2026-02-24T07:54:42.927535+0000","last_unstale":"2026-02-24T07:55:40.764047+0000","last_undegraded":"2026-02-24T07:55:40.764047+0000","last_fullsized":"2026-02-24T07:55:40.764047+0000","mapping_epoch":90,"log_start":"96'25778","ondisk_log_start":"96'25778","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_clean_scrub_stamp":"2026-02-24T07:54:41.918128+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:55:53.650972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1689583616,"num_objects":25781,"num_object_clones":0,"num_object_copies":77343,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25781,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":25781,"num_write_kb":1649984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.4","version":"96'26003","reported_seq":26025,"reported_epoch":96,"state":"active+clean","last_fresh":"2026-02-24T07:55:40.871100+0000","last_change":"2026-02-24T07:54:42.928344+0000","last_active":"2026-02-24T07:55:40.871100+0000","last_peered":"2026-02-24T07:55:40.871100+0000","last_clean":"2026-02-24T07:55:40.871100+0000","last_became_active":"2026-02-24T07:54:42.928011+0000","last_became_peered":"2026-02-24T07:54:42.928011+0000","last_unstale":"2026-02-24T07:55:40.871100+0000","last_undegraded":"2026-02-24T07:55:40.871100+0000","last_fullsized":"2026-02-24T07:55:40.871100+0000","mapping_epoch":90,"log_start":"96'26000","ondisk_log_start":"96'26000","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_clean_scrub_stamp":"2026-02-24T07:54:41.918128+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:55:43.506947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1704132608,"num_objects":26003,"num_object_clones":0,"num_object_copies":78009,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26003,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":26003,"num_write_kb":1664192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.3","version":"96'25991","reported_seq":26013,"reported_epoch":96,"state":"active+clean","last_fresh":"2026-02-24T07:55:40.767840+0000","last_change":"2026-02-24T07:54:42.928536+0000","last_active":"2026-02-24T07:55:40.767840+0000","last_peered":"2026-02-24T07:55:40.767840+0000","last_clean":"2026-02-24T07:55:40.767840+0000","last_became_active":"2026-02-24T07:54:42.927730+0000","last_became_peered":"2026-02-24T07:54:42.927730+0000","last_unstale":"2026-02-24T07:55:40.767840+0000","last_undegraded":"2026-02-24T07:55:40.767840+0000","last_fullsized":"2026-02-24T07:55:40.767840+0000","mapping_epoch":90,"log_start":"96'25988","ondisk_log_start":"96'25988","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_clean_scrub_stamp":"2026-02-24T07:54:41.918128+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:55:58.987317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1703346176,"num_objects":25991,"num_object_clones":0,"num_object_copies":77973,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25991,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":25991,"num_write_kb":1663424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.2","version":"96'25720","reported_seq":25741,"reported_epoch":96,"state":"active+clean","last_fresh":"2026-02-24T07:55:40.782575+0000","last_change":"2026-02-24T07:54:42.930452+0000","last_active":"2026-02-24T07:55:40.782575+0000","last_peered":"2026-02-24T07:55:40.782575+0000","last_clean":"2026-02-24T07:55:40.782575+0000","last_became_active":"2026-02-24T07:54:42.930247+0000","last_became_peered":"2026-02-24T07:54:42.930247+0000","last_unstale":"2026-02-24T07:55:40.782575+0000","last_undegraded":"2026-02-24T07:55:40.782575+0000","last_fullsized":"2026-02-24T07:55:40.782575+0000","mapping_epoch":90,"log_start":"96'25717","ondisk_log_start":"96'25717","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_clean_scrub_stamp":"2026-02-24T07:54:41.918128+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:55:49.096947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1685585920,"num_objects":25720,"num_object_clones":0,"num_object_copies":77160,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25720,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":25720,"num_write_kb":1646080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1","version":"96'25943","reported_seq":25965,"reported_epoch":96,"state":"active+clean","last_fresh":"2026-02-24T07:55:40.861102+0000","last_change":"2026-02-24T07:54:42.926619+0000","last_active":"2026-02-24T07:55:40.861102+0000","last_peered":"2026-02-24T07:55:40.861102+0000","last_clean":"2026-02-24T07:55:40.861102+0000","last_became_active":"2026-02-24T07:54:42.926442+0000","last_became_peered":"2026-02-24T07:54:42.926442+0000","last_unstale":"2026-02-24T07:55:40.861102+0000","last_undegraded":"2026-02-24T07:55:40.861102+0000","last_fullsized":"2026-02-24T07:55:40.861102+0000","mapping_epoch":90,"log_start":"96'25940","ondisk_log_start":"96'25940","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_clean_scrub_stamp":"2026-02-24T07:54:41.918128+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:56:03.125413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1700200448,"num_objects":25943,"num_object_clones":0,"num_object_copies":77829,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25943,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":25943,"num_write_kb":1660352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.0","version":"96'25636","reported_seq":25658,"reported_epoch":96,"state":"active+clean","last_fresh":"2026-02-24T07:55:40.796422+0000","last_change":"2026-02-24T07:54:42.927178+0000","last_active":"2026-02-24T07:55:40.796422+0000","last_peered":"2026-02-24T07:55:40.796422+0000","last_clean":"2026-02-24T07:55:40.796422+0000","last_became_active":"2026-02-24T07:54:42.926910+0000","last_became_peered":"2026-02-24T07:54:42.926910+0000","last_unstale":"2026-02-24T07:55:40.796422+0000","last_undegraded":"2026-02-24T07:55:40.796422+0000","last_fullsized":"2026-02-24T07:55:40.796422+0000","mapping_epoch":90,"log_start":"96'25633","ondisk_log_start":"96'25633","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_clean_scrub_stamp":"2026-02-24T07:54:41.918128+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:55:58.002045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1680080896,"num_objects":25636,"num_object_clones":0,"num_object_copies":76908,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25636,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":25636,"num_write_kb":1640704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.7","version":"96'25944","reported_seq":25966,"reported_epoch":96,"state":"active+clean","last_fresh":"2026-02-24T07:55:40.779085+0000","last_change":"2026-02-24T07:54:42.930348+0000","last_active":"2026-02-24T07:55:40.779085+0000","last_peered":"2026-02-24T07:55:40.779085+0000","last_clean":"2026-02-24T07:55:40.779085+0000","last_became_active":"2026-02-24T07:54:42.930142+0000","last_became_peered":"2026-02-24T07:54:42.930142+0000","last_unstale":"2026-02-24T07:55:40.779085+0000","last_undegraded":"2026-02-24T07:55:40.779085+0000","last_fullsized":"2026-02-24T07:55:40.779085+0000","mapping_epoch":90,"log_start":"96'25941","ondisk_log_start":"96'25941","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_clean_scrub_stamp":"2026-02-24T07:54:41.918128+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:55:56.704334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1700265984,"num_objects":25944,"num_object_clones":0,"num_object_copies":77832,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25944,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":25944,"num_write_kb":1660416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"29'99","reported_seq":398,"reported_epoch":96,"state":"active+clean","last_fresh":"2026-02-24T07:55:16.398160+0000","last_change":"2026-02-24T07:55:16.398160+0000","last_active":"2026-02-24T07:55:16.398160+0000","last_peered":"2026-02-24T07:55:16.398160+0000","last_clean":"2026-02-24T07:55:16.398160+0000","last_became_active":"2026-02-24T07:42:54.186719+0000","last_became_peered":"2026-02-24T07:42:54.186719+0000","last_unstale":"2026-02-24T07:55:16.398160+0000","last_undegraded":"2026-02-24T07:55:16.398160+0000","last_fullsized":"2026-02-24T07:55:16.398160+0000","mapping_epoch":27,"log_start":"29'96","ondisk_log_start":"29'96","created":20,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'99","last_scrub_stamp":"2026-02-24T07:55:16.398131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:42:43.640776+0000","last_clean_scrub_stamp":"2026-02-24T07:55:16.398131+0000","objects_scrubbed":2,"log_size":3,"log_dups_size":96,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:56:32.925996+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":240,"num_write":135,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":16,"stat_sum":{"num_bytes":27102674944,"num_objects":413554,"num_object_clones":0,"num_object_copies":1240662,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":413554,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":413554,"num_write_kb":26467456,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":81303633920,"data_stored":81303633920,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":48,"ondisk_log_size":48,"up":48,"acting":48,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":240,"num_write":135,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1744,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":3,"up_from":25,"seq":107374182556,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":17915220,"kb_used_data":16375488,"kb_used_omap":17,"kb_used_meta":1539694,"kb_avail":716087980,"statfs":{"total":751619276800,"available":733274091520,"internally_reserved":0,"allocated":16768499712,"data_stored":16767997094,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":18320,"internal_metadata":1576646768},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,2],"upper_bound":32},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":19,"seq":81604378782,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":21088916,"kb_used_data":19690816,"kb_used_omap":17,"kb_used_meta":1398062,"kb_avail":712914284,"statfs":{"total":751619276800,"available":730024226816,"internally_reserved":0,"allocated":20163395584,"data_stored":20162892966,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":18000,"internal_metadata":1431615920},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542303,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":24684880,"kb_used_data":22981628,"kb_used_omap":24,"kb_used_meta":1703143,"kb_avail":709318320,"statfs":{"total":751619276800,"available":726341959680,"internally_reserved":0,"allocated":23533187072,"data_stored":23532688006,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":24784,"internal_metadata":1744019248},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705826,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":21275668,"kb_used_data":19777344,"kb_used_omap":15,"kb_used_meta":1498288,"kb_avail":712727532,"statfs":{"total":751619276800,"available":729832992768,"internally_reserved":0,"allocated":20252000256,"data_stored":20251497638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":15702,"internal_metadata":1534247594},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,3],"upper_bound":4},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1744,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":20315570176,"data_stored":20315570176,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":23754244096,"data_stored":23754244096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":20296302592,"data_stored":20296302592,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16937517056,"data_stored":16937517056,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-24T07:55:43.261 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph osd pool set unique_pool_4 pgp_num 56 2026-02-24T07:55:43.410 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:55:43.540 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:55:43.539+0000 7f347158e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:55:43.540 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:55:43.539+0000 7f347158e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:55:43.696 INFO:teuthology.orchestra.run.trial035.stderr:Error EPERM: crimson_allow_pg_split is false; pgp_num increase denied 2026-02-24T07:55:43.740 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-24T07:55:44.019 INFO:tasks.radosbench.radosbench.0.trial035.stdout:2026-02-24T07:55:44.019396+0000 min lat: 0.000728555 max lat: 0.0462542 avg lat: 0.00218101 2026-02-24T07:55:44.019 INFO:tasks.radosbench.radosbench.0.trial035.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2026-02-24T07:55:44.020 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 60 15 439959 439944 458.239 534.875 0.00166836 0.00218101 2026-02-24T07:55:44.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:55:43 trial035 ceph-mon[39284]: from='client.15501 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2026-02-24T07:55:44.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:55:43 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/534244391' entity='client.admin' cmd={"prefix": "osd pool set", "pool": "unique_pool_4", "var": "pgp_num", "val": "56"} : dispatch 2026-02-24T07:55:44.205 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:55:43 trial035 ceph-mon[39284]: pgmap v452: 17 pgs: 17 active+clean; 25 GiB data, 81 GiB used, 2.7 TiB / 2.7 TiB avail; 381 MiB/s wr, 6.09k op/s 2026-02-24T07:55:44.583 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2026-02-24T07:55:44.728 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:55:44.866 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:55:44.864+0000 7fe07faa0640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:55:44.866 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:55:44.865+0000 7fe07faa0640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:55:44.883 INFO:teuthology.orchestra.run.trial035.stdout:osd.0: {} 2026-02-24T07:55:44.883 INFO:teuthology.orchestra.run.trial035.stderr:osd.0: osd_enable_op_tracker = '' 2026-02-24T07:55:44.896 INFO:teuthology.orchestra.run.trial035.stdout:osd.1: {} 2026-02-24T07:55:44.897 INFO:teuthology.orchestra.run.trial035.stderr:osd.1: osd_enable_op_tracker = '' 2026-02-24T07:55:44.912 INFO:teuthology.orchestra.run.trial035.stdout:osd.2: {} 2026-02-24T07:55:44.912 INFO:teuthology.orchestra.run.trial035.stderr:osd.2: osd_enable_op_tracker = '' 2026-02-24T07:55:44.926 INFO:teuthology.orchestra.run.trial035.stdout:osd.3: {} 2026-02-24T07:55:44.927 INFO:teuthology.orchestra.run.trial035.stderr:osd.3: osd_enable_op_tracker = '' 2026-02-24T07:55:45.020 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 61 16 447609 447593 458.563 478.062 0.00203335 0.00217942 2026-02-24T07:55:45.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:55:44 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/575609114' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2026-02-24T07:55:45.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:55:44 trial035 ceph-mon[39284]: from='mgr.14284 10.20.193.35:0/3681373222' entity='mgr.x' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2026-02-24T07:55:46.020 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 62 16 453974 453958 457.582 397.812 0.0017151 0.00218412 2026-02-24T07:55:46.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:55:45 trial035 ceph-mon[39284]: 6.4 scrub starts 2026-02-24T07:55:46.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:55:45 trial035 ceph-mon[39284]: 6.d scrub starts 2026-02-24T07:55:46.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:55:45 trial035 ceph-mon[39284]: pgmap v453: 17 pgs: 17 active+clean; 25 GiB data, 81 GiB used, 2.7 TiB / 2.7 TiB avail; 381 MiB/s wr, 6.09k op/s 2026-02-24T07:55:46.971 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2026-02-24T07:55:47.020 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 63 16 459559 459543 455.859 349.062 0.00188292 0.00219237 2026-02-24T07:55:47.134 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:55:47.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:55:46 trial035 ceph-mon[39284]: 6.9 scrub starts 2026-02-24T07:55:47.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:55:46 trial035 ceph-mon[39284]: 6.a scrub starts 2026-02-24T07:55:47.205 INFO:journalctl@ceph.mgr.x.trial035.stdout:Feb 24 07:55:46 trial035 ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-mgr-x[39621]: ::ffff:10.20.193.91 - - [24/Feb/2026:07:55:46] "GET /metrics HTTP/1.1" 200 4214 "" "Prometheus/3.6.0" 2026-02-24T07:55:47.264 INFO:journalctl@ceph.prometheus.a.trial091.stdout:Feb 24 07:55:46 trial091 ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-prometheus-a[66007]: time=2026-02-24T07:55:46.973Z level=ERROR source=scrape.go:1630 msg="Failed to determine correct type of scrape target." component="scrape manager" scrape_pool=ceph target=http://trial035:9283/metrics content_type="text/html;charset=utf-8" fallback_media_type="" err="received unsupported Content-Type \"text/html;charset=utf-8\" and no fallback_scrape_protocol specified for target" 2026-02-24T07:55:47.513 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:55:47.461+0000 7fb3209db640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:55:47.513 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:55:47.462+0000 7fb3209db640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:55:47.513 INFO:teuthology.orchestra.run.trial035.stderr:osd.0: osd_enable_op_tracker = '' 2026-02-24T07:55:47.513 INFO:teuthology.orchestra.run.trial035.stderr:osd.1: osd_enable_op_tracker = '' 2026-02-24T07:55:47.513 INFO:teuthology.orchestra.run.trial035.stdout:osd.0: {} 2026-02-24T07:55:47.513 INFO:teuthology.orchestra.run.trial035.stdout:osd.1: {} 2026-02-24T07:55:47.518 INFO:teuthology.orchestra.run.trial035.stdout:osd.2: {} 2026-02-24T07:55:47.518 INFO:teuthology.orchestra.run.trial035.stderr:osd.2: osd_enable_op_tracker = '' 2026-02-24T07:55:47.533 INFO:teuthology.orchestra.run.trial035.stdout:osd.3: {} 2026-02-24T07:55:47.533 INFO:teuthology.orchestra.run.trial035.stderr:osd.3: osd_enable_op_tracker = '' 2026-02-24T07:55:48.020 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 64 16 463225 463209 452.316 229.125 0.00152812 0.00220955 2026-02-24T07:55:48.028 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:55:47 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/1930633923' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2026-02-24T07:55:48.028 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:55:47 trial035 ceph-mon[39284]: pgmap v454: 17 pgs: 3 active+clean+scrubbing, 14 active+clean; 28 GiB data, 89 GiB used, 2.6 TiB / 2.7 TiB avail; 585 MiB/s wr, 9.37k op/s 2026-02-24T07:55:48.741 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2026-02-24T07:55:48.742 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2026-02-24T07:55:48.742 INFO:tasks.thrashosds.thrasher:trim_stale_osdmaps 2026-02-24T07:55:48.742 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph tell osd.0 'trim stale osdmaps' 2026-02-24T07:55:48.885 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:55:49.016 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:55:49.014+0000 7fc6cb780640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:55:49.016 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:55:49.014+0000 7fc6cb780640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:55:49.020 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 65 16 469742 469726 451.623 407.312 0.00160095 0.00221298 2026-02-24T07:55:49.042 INFO:teuthology.orchestra.run.trial035.stderr:no valid command found; 10 closest matches: 2026-02-24T07:55:49.042 INFO:teuthology.orchestra.run.trial035.stderr:0 2026-02-24T07:55:49.042 INFO:teuthology.orchestra.run.trial035.stderr:1 2026-02-24T07:55:49.043 INFO:teuthology.orchestra.run.trial035.stderr:2 2026-02-24T07:55:49.043 INFO:teuthology.orchestra.run.trial035.stderr:abort 2026-02-24T07:55:49.043 INFO:teuthology.orchestra.run.trial035.stderr:assert 2026-02-24T07:55:49.043 INFO:teuthology.orchestra.run.trial035.stderr:bench [] [] [] [] 2026-02-24T07:55:49.043 INFO:teuthology.orchestra.run.trial035.stderr:bluefs debug_inject_read_zeros 2026-02-24T07:55:49.043 INFO:teuthology.orchestra.run.trial035.stderr:bluefs files list 2026-02-24T07:55:49.043 INFO:teuthology.orchestra.run.trial035.stderr:bluefs stats 2026-02-24T07:55:49.043 INFO:teuthology.orchestra.run.trial035.stderr:bluestore allocator dump block 2026-02-24T07:55:49.043 INFO:teuthology.orchestra.run.trial035.stderr:Error EINVAL: invalid command 2026-02-24T07:55:49.044 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:55:48 trial035 ceph-mon[39284]: 6.4 scrub ok 2026-02-24T07:55:49.044 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:55:48 trial035 ceph-mon[39284]: 6.d scrub ok 2026-02-24T07:55:49.077 DEBUG:teuthology.orchestra.run:got remote process result: 22 2026-02-24T07:55:49.078 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2026-02-24T07:55:49.078 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph tell osd.1 'trim stale osdmaps' 2026-02-24T07:55:49.224 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:55:49.341 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:55:49.339+0000 7ff2be8aa640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:55:49.341 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:55:49.339+0000 7ff2be8aa640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:55:49.367 INFO:teuthology.orchestra.run.trial035.stderr:no valid command found; 10 closest matches: 2026-02-24T07:55:49.367 INFO:teuthology.orchestra.run.trial035.stderr:0 2026-02-24T07:55:49.367 INFO:teuthology.orchestra.run.trial035.stderr:1 2026-02-24T07:55:49.367 INFO:teuthology.orchestra.run.trial035.stderr:2 2026-02-24T07:55:49.367 INFO:teuthology.orchestra.run.trial035.stderr:abort 2026-02-24T07:55:49.367 INFO:teuthology.orchestra.run.trial035.stderr:assert 2026-02-24T07:55:49.367 INFO:teuthology.orchestra.run.trial035.stderr:bench [] [] [] [] 2026-02-24T07:55:49.367 INFO:teuthology.orchestra.run.trial035.stderr:bluefs debug_inject_read_zeros 2026-02-24T07:55:49.367 INFO:teuthology.orchestra.run.trial035.stderr:bluefs files list 2026-02-24T07:55:49.367 INFO:teuthology.orchestra.run.trial035.stderr:bluefs stats 2026-02-24T07:55:49.367 INFO:teuthology.orchestra.run.trial035.stderr:bluestore allocator dump block 2026-02-24T07:55:49.367 INFO:teuthology.orchestra.run.trial035.stderr:Error EINVAL: invalid command 2026-02-24T07:55:49.402 DEBUG:teuthology.orchestra.run:got remote process result: 22 2026-02-24T07:55:49.403 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2026-02-24T07:55:49.403 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph tell osd.2 'trim stale osdmaps' 2026-02-24T07:55:49.551 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:55:49.679 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:55:49.678+0000 7f26d62eb640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:55:49.680 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:55:49.678+0000 7f26d62eb640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:55:49.703 INFO:teuthology.orchestra.run.trial035.stderr:no valid command found; 10 closest matches: 2026-02-24T07:55:49.703 INFO:teuthology.orchestra.run.trial035.stderr:0 2026-02-24T07:55:49.703 INFO:teuthology.orchestra.run.trial035.stderr:1 2026-02-24T07:55:49.703 INFO:teuthology.orchestra.run.trial035.stderr:2 2026-02-24T07:55:49.703 INFO:teuthology.orchestra.run.trial035.stderr:abort 2026-02-24T07:55:49.703 INFO:teuthology.orchestra.run.trial035.stderr:assert 2026-02-24T07:55:49.703 INFO:teuthology.orchestra.run.trial035.stderr:bench [] [] [] [] 2026-02-24T07:55:49.703 INFO:teuthology.orchestra.run.trial035.stderr:bluefs debug_inject_read_zeros 2026-02-24T07:55:49.703 INFO:teuthology.orchestra.run.trial035.stderr:bluefs files list 2026-02-24T07:55:49.703 INFO:teuthology.orchestra.run.trial035.stderr:bluefs stats 2026-02-24T07:55:49.703 INFO:teuthology.orchestra.run.trial035.stderr:bluestore allocator dump block 2026-02-24T07:55:49.703 INFO:teuthology.orchestra.run.trial035.stderr:Error EINVAL: invalid command 2026-02-24T07:55:49.739 DEBUG:teuthology.orchestra.run:got remote process result: 22 2026-02-24T07:55:49.740 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2026-02-24T07:55:49.740 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph tell osd.3 'trim stale osdmaps' 2026-02-24T07:55:49.878 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:55:50.007 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:55:50.005+0000 7f2b37554640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:55:50.007 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:55:50.005+0000 7f2b37554640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:55:50.020 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 66 16 477412 477396 452.043 479.375 0.00259544 0.00221092 2026-02-24T07:55:50.030 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2026-02-24T07:55:50.032 INFO:teuthology.orchestra.run.trial035.stderr:no valid command found; 10 closest matches: 2026-02-24T07:55:50.032 INFO:teuthology.orchestra.run.trial035.stderr:0 2026-02-24T07:55:50.032 INFO:teuthology.orchestra.run.trial035.stderr:1 2026-02-24T07:55:50.032 INFO:teuthology.orchestra.run.trial035.stderr:2 2026-02-24T07:55:50.032 INFO:teuthology.orchestra.run.trial035.stderr:abort 2026-02-24T07:55:50.032 INFO:teuthology.orchestra.run.trial035.stderr:assert 2026-02-24T07:55:50.032 INFO:teuthology.orchestra.run.trial035.stderr:bench [] [] [] [] 2026-02-24T07:55:50.032 INFO:teuthology.orchestra.run.trial035.stderr:bluefs debug_inject_read_zeros 2026-02-24T07:55:50.032 INFO:teuthology.orchestra.run.trial035.stderr:bluefs files list 2026-02-24T07:55:50.032 INFO:teuthology.orchestra.run.trial035.stderr:bluefs stats 2026-02-24T07:55:50.032 INFO:teuthology.orchestra.run.trial035.stderr:bluestore allocator dump block 2026-02-24T07:55:50.032 INFO:teuthology.orchestra.run.trial035.stderr:Error EINVAL: invalid command 2026-02-24T07:55:50.077 DEBUG:teuthology.orchestra.run:got remote process result: 22 2026-02-24T07:55:50.078 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2026-02-24T07:55:50.170 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:55:50.196 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:55:49 trial035 ceph-mon[39284]: 6.9 scrub ok 2026-02-24T07:55:50.197 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:55:49 trial035 ceph-mon[39284]: 6.a scrub ok 2026-02-24T07:55:50.197 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:55:49 trial035 ceph-mon[39284]: pgmap v455: 17 pgs: 3 active+clean+scrubbing, 14 active+clean; 28 GiB data, 89 GiB used, 2.6 TiB / 2.7 TiB avail; 397 MiB/s wr, 6.35k op/s 2026-02-24T07:55:50.294 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:55:50.292+0000 7f1149b8b640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:55:50.294 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:55:50.292+0000 7f1149b8b640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:55:50.310 INFO:teuthology.orchestra.run.trial035.stdout:osd.0: {} 2026-02-24T07:55:50.311 INFO:teuthology.orchestra.run.trial035.stderr:osd.0: osd_enable_op_tracker = '' 2026-02-24T07:55:50.327 INFO:teuthology.orchestra.run.trial035.stdout:osd.1: {} 2026-02-24T07:55:50.327 INFO:teuthology.orchestra.run.trial035.stderr:osd.1: osd_enable_op_tracker = '' 2026-02-24T07:55:50.341 INFO:teuthology.orchestra.run.trial035.stdout:osd.2: {} 2026-02-24T07:55:50.341 INFO:teuthology.orchestra.run.trial035.stderr:osd.2: osd_enable_op_tracker = '' 2026-02-24T07:55:50.355 INFO:teuthology.orchestra.run.trial035.stdout:osd.3: {} 2026-02-24T07:55:50.355 INFO:teuthology.orchestra.run.trial035.stderr:osd.3: osd_enable_op_tracker = '' 2026-02-24T07:55:51.020 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 67 16 485015 484999 452.388 475.188 0.00260423 0.00220922 2026-02-24T07:55:51.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:55:50 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/1581934351' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2026-02-24T07:55:52.020 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 68 16 492634 492618 452.738 476.188 0.00147276 0.00220751 2026-02-24T07:55:52.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:55:51 trial035 ceph-mon[39284]: pgmap v456: 17 pgs: 17 active+clean; 30 GiB data, 95 GiB used, 2.6 TiB / 2.7 TiB avail; 556 MiB/s wr, 8.90k op/s 2026-02-24T07:55:52.401 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2026-02-24T07:55:52.548 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:55:52.690 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:55:52.688+0000 7ff5a88c5640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:55:52.691 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:55:52.688+0000 7ff5a88c5640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:55:52.706 INFO:teuthology.orchestra.run.trial035.stdout:osd.0: {} 2026-02-24T07:55:52.706 INFO:teuthology.orchestra.run.trial035.stderr:osd.0: osd_enable_op_tracker = '' 2026-02-24T07:55:52.722 INFO:teuthology.orchestra.run.trial035.stdout:osd.1: {} 2026-02-24T07:55:52.722 INFO:teuthology.orchestra.run.trial035.stderr:osd.1: osd_enable_op_tracker = '' 2026-02-24T07:55:52.734 INFO:teuthology.orchestra.run.trial035.stdout:osd.2: {} 2026-02-24T07:55:52.734 INFO:teuthology.orchestra.run.trial035.stderr:osd.2: osd_enable_op_tracker = '' 2026-02-24T07:55:52.747 INFO:teuthology.orchestra.run.trial035.stdout:osd.3: {} 2026-02-24T07:55:52.747 INFO:teuthology.orchestra.run.trial035.stderr:osd.3: osd_enable_op_tracker = '' 2026-02-24T07:55:53.020 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 69 16 500214 500198 453.042 473.75 0.00202543 0.00220604 2026-02-24T07:55:53.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:55:52 trial035 ceph-mon[39284]: purged_snaps scrub starts 2026-02-24T07:55:53.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:55:52 trial035 ceph-mon[39284]: purged_snaps scrub ok 2026-02-24T07:55:53.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:55:52 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/4247049580' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2026-02-24T07:55:54.020 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 70 15 507390 507375 452.977 448.562 0.00214364 0.00220636 2026-02-24T07:55:54.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:55:53 trial035 ceph-mon[39284]: purged_snaps scrub starts 2026-02-24T07:55:54.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:55:53 trial035 ceph-mon[39284]: purged_snaps scrub ok 2026-02-24T07:55:54.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:55:53 trial035 ceph-mon[39284]: pgmap v457: 17 pgs: 17 active+clean; 30 GiB data, 95 GiB used, 2.6 TiB / 2.7 TiB avail; 364 MiB/s wr, 5.83k op/s 2026-02-24T07:55:54.793 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2026-02-24T07:55:54.954 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:55:55.020 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 71 16 515108 515092 453.39 482.312 0.00244698 0.00220435 2026-02-24T07:55:55.079 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2026-02-24T07:55:55.079 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2026-02-24T07:55:55.079 INFO:tasks.thrashosds.thrasher:Growing pool .mgr 2026-02-24T07:55:55.080 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph pg dump --format=json 2026-02-24T07:55:55.199 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:55:55.197+0000 7f3be0424640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:55:55.199 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:55:55.197+0000 7f3be0424640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:55:55.215 INFO:teuthology.orchestra.run.trial035.stdout:osd.0: {} 2026-02-24T07:55:55.215 INFO:teuthology.orchestra.run.trial035.stderr:osd.0: osd_enable_op_tracker = '' 2026-02-24T07:55:55.229 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:55:55.236 INFO:teuthology.orchestra.run.trial035.stdout:osd.1: {} 2026-02-24T07:55:55.236 INFO:teuthology.orchestra.run.trial035.stderr:osd.1: osd_enable_op_tracker = '' 2026-02-24T07:55:55.252 INFO:teuthology.orchestra.run.trial035.stdout:osd.2: {} 2026-02-24T07:55:55.252 INFO:teuthology.orchestra.run.trial035.stderr:osd.2: osd_enable_op_tracker = '' 2026-02-24T07:55:55.264 INFO:teuthology.orchestra.run.trial035.stdout:osd.3: {} 2026-02-24T07:55:55.264 INFO:teuthology.orchestra.run.trial035.stderr:osd.3: osd_enable_op_tracker = '' 2026-02-24T07:55:55.350 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:55:55 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/3361989906' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2026-02-24T07:55:55.350 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:55:55.348+0000 7f0ff00b1640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:55:55.350 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:55:55.348+0000 7f0ff00b1640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:55:55.491 INFO:teuthology.orchestra.run.trial035.stdout: 2026-02-24T07:55:55.491 INFO:teuthology.orchestra.run.trial035.stderr:dumped all 2026-02-24T07:55:55.538 INFO:teuthology.orchestra.run.trial035.stdout:{"pg_ready":true,"pg_map":{"version":457,"stamp":"2026-02-24T07:55:53.762632+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":31685411360,"num_objects":483474,"num_object_clones":0,"num_object_copies":1450422,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":483474,"num_whiteouts":0,"num_read":136,"num_read_kb":240,"num_write":483607,"num_write_kb":30944358,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":56,"ondisk_log_size":56,"up":51,"acting":51,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":51,"num_osds":4,"num_per_pool_osds":4,"num_per_pool_omap_osds":4,"kb":2936012800,"kb_used":99276044,"kb_used_data":92288380,"kb_used_omap":75,"kb_used_meta":6987444,"kb_avail":2836736756,"statfs":{"total":3006477107200,"available":2904818438144,"internally_reserved":0,"allocated":94503301120,"data_stored":94501294200,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":76806,"internal_metadata":7155143674},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":4582146048,"num_objects":69918,"num_object_clones":0,"num_object_copies":209754,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":69918,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":69918,"num_write_kb":4474752,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001756"},"pg_stats":[{"pgid":"6.f","version":"96'30223","reported_seq":30245,"reported_epoch":96,"state":"active+clean","last_fresh":"2026-02-24T07:55:50.875759+0000","last_change":"2026-02-24T07:54:42.927271+0000","last_active":"2026-02-24T07:55:50.875759+0000","last_peered":"2026-02-24T07:55:50.875759+0000","last_clean":"2026-02-24T07:55:50.875759+0000","last_became_active":"2026-02-24T07:54:42.927080+0000","last_became_peered":"2026-02-24T07:54:42.927080+0000","last_unstale":"2026-02-24T07:55:50.875759+0000","last_undegraded":"2026-02-24T07:55:50.875759+0000","last_fullsized":"2026-02-24T07:55:50.875759+0000","mapping_epoch":90,"log_start":"96'30220","ondisk_log_start":"96'30220","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_clean_scrub_stamp":"2026-02-24T07:54:41.918128+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:56:00.801546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1980694528,"num_objects":30223,"num_object_clones":0,"num_object_copies":90669,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30223,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":30223,"num_write_kb":1934272,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.e","version":"96'30317","reported_seq":30339,"reported_epoch":96,"state":"active+clean","last_fresh":"2026-02-24T07:55:50.765848+0000","last_change":"2026-02-24T07:54:42.929104+0000","last_active":"2026-02-24T07:55:50.765848+0000","last_peered":"2026-02-24T07:55:50.765848+0000","last_clean":"2026-02-24T07:55:50.765848+0000","last_became_active":"2026-02-24T07:54:42.928748+0000","last_became_peered":"2026-02-24T07:54:42.928748+0000","last_unstale":"2026-02-24T07:55:50.765848+0000","last_undegraded":"2026-02-24T07:55:50.765848+0000","last_fullsized":"2026-02-24T07:55:50.765848+0000","mapping_epoch":90,"log_start":"96'30314","ondisk_log_start":"96'30314","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_clean_scrub_stamp":"2026-02-24T07:54:41.918128+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:55:53.670906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1986854912,"num_objects":30317,"num_object_clones":0,"num_object_copies":90951,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30317,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":30317,"num_write_kb":1940288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.d","version":"96'30143","reported_seq":30459,"reported_epoch":96,"state":"active+clean","last_fresh":"2026-02-24T07:55:50.770605+0000","last_change":"2026-02-24T07:55:48.151749+0000","last_active":"2026-02-24T07:55:50.770605+0000","last_peered":"2026-02-24T07:55:50.770605+0000","last_clean":"2026-02-24T07:55:50.770605+0000","last_became_active":"2026-02-24T07:54:42.926886+0000","last_became_peered":"2026-02-24T07:54:42.926886+0000","last_unstale":"2026-02-24T07:55:50.770605+0000","last_undegraded":"2026-02-24T07:55:50.770605+0000","last_fullsized":"2026-02-24T07:55:50.770605+0000","mapping_epoch":90,"log_start":"96'30139","ondisk_log_start":"96'30139","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"96'28924","last_scrub_stamp":"2026-02-24T07:55:48.151672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_clean_scrub_stamp":"2026-02-24T07:55:48.151672+0000","objects_scrubbed":28526,"log_size":4,"log_dups_size":3000,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:57:00.390677+0000","scrub_duration":2742,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1975451648,"num_objects":30143,"num_object_clones":0,"num_object_copies":90429,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30143,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":30143,"num_write_kb":1929152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.c","version":"96'30333","reported_seq":30355,"reported_epoch":96,"state":"active+clean","last_fresh":"2026-02-24T07:55:50.874622+0000","last_change":"2026-02-24T07:54:42.928482+0000","last_active":"2026-02-24T07:55:50.874622+0000","last_peered":"2026-02-24T07:55:50.874622+0000","last_clean":"2026-02-24T07:55:50.874622+0000","last_became_active":"2026-02-24T07:54:42.928238+0000","last_became_peered":"2026-02-24T07:54:42.928238+0000","last_unstale":"2026-02-24T07:55:50.874622+0000","last_undegraded":"2026-02-24T07:55:50.874622+0000","last_fullsized":"2026-02-24T07:55:50.874622+0000","mapping_epoch":90,"log_start":"96'30330","ondisk_log_start":"96'30330","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_clean_scrub_stamp":"2026-02-24T07:54:41.918128+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:56:09.181158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1987903488,"num_objects":30333,"num_object_clones":0,"num_object_copies":90999,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30333,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":30333,"num_write_kb":1941312,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.b","version":"96'30026","reported_seq":30048,"reported_epoch":96,"state":"active+clean","last_fresh":"2026-02-24T07:55:50.879590+0000","last_change":"2026-02-24T07:54:42.927223+0000","last_active":"2026-02-24T07:55:50.879590+0000","last_peered":"2026-02-24T07:55:50.879590+0000","last_clean":"2026-02-24T07:55:50.879590+0000","last_became_active":"2026-02-24T07:54:42.926965+0000","last_became_peered":"2026-02-24T07:54:42.926965+0000","last_unstale":"2026-02-24T07:55:50.879590+0000","last_undegraded":"2026-02-24T07:55:50.879590+0000","last_fullsized":"2026-02-24T07:55:50.879590+0000","mapping_epoch":90,"log_start":"96'30022","ondisk_log_start":"96'30022","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_clean_scrub_stamp":"2026-02-24T07:54:41.918128+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":3000,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:56:04.661233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1967783936,"num_objects":30026,"num_object_clones":0,"num_object_copies":90078,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30026,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":30026,"num_write_kb":1921664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.a","version":"96'30120","reported_seq":30439,"reported_epoch":96,"state":"active+clean","last_fresh":"2026-02-24T07:55:50.770461+0000","last_change":"2026-02-24T07:55:49.167491+0000","last_active":"2026-02-24T07:55:50.770461+0000","last_peered":"2026-02-24T07:55:50.770461+0000","last_clean":"2026-02-24T07:55:50.770461+0000","last_became_active":"2026-02-24T07:54:42.926477+0000","last_became_peered":"2026-02-24T07:54:42.926477+0000","last_unstale":"2026-02-24T07:55:50.770461+0000","last_undegraded":"2026-02-24T07:55:50.770461+0000","last_fullsized":"2026-02-24T07:55:50.770461+0000","mapping_epoch":90,"log_start":"96'30117","ondisk_log_start":"96'30117","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"96'29334","last_scrub_stamp":"2026-02-24T07:55:49.167461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_clean_scrub_stamp":"2026-02-24T07:55:49.167461+0000","objects_scrubbed":28822,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:56:59.424140+0000","scrub_duration":2726,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1973944320,"num_objects":30120,"num_object_clones":0,"num_object_copies":90360,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30120,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":30120,"num_write_kb":1927680,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"96'30227","reported_seq":30547,"reported_epoch":96,"state":"active+clean","last_fresh":"2026-02-24T07:55:50.804027+0000","last_change":"2026-02-24T07:55:48.535755+0000","last_active":"2026-02-24T07:55:50.804027+0000","last_peered":"2026-02-24T07:55:50.804027+0000","last_clean":"2026-02-24T07:55:50.804027+0000","last_became_active":"2026-02-24T07:54:42.926136+0000","last_became_peered":"2026-02-24T07:54:42.926136+0000","last_unstale":"2026-02-24T07:55:50.804027+0000","last_undegraded":"2026-02-24T07:55:50.804027+0000","last_fullsized":"2026-02-24T07:55:50.804027+0000","mapping_epoch":90,"log_start":"96'30224","ondisk_log_start":"96'30224","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"96'29148","last_scrub_stamp":"2026-02-24T07:55:48.535691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_clean_scrub_stamp":"2026-02-24T07:55:48.535691+0000","objects_scrubbed":28720,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:56:56.161155+0000","scrub_duration":2872,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1980956672,"num_objects":30227,"num_object_clones":0,"num_object_copies":90681,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30227,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":30227,"num_write_kb":1934528,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,1],"acting":[0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.8","version":"96'30320","reported_seq":30341,"reported_epoch":96,"state":"active+clean","last_fresh":"2026-02-24T07:55:50.783935+0000","last_change":"2026-02-24T07:54:42.929353+0000","last_active":"2026-02-24T07:55:50.783935+0000","last_peered":"2026-02-24T07:55:50.783935+0000","last_clean":"2026-02-24T07:55:50.783935+0000","last_became_active":"2026-02-24T07:54:42.929078+0000","last_became_peered":"2026-02-24T07:54:42.929078+0000","last_unstale":"2026-02-24T07:55:50.783935+0000","last_undegraded":"2026-02-24T07:55:50.783935+0000","last_fullsized":"2026-02-24T07:55:50.783935+0000","mapping_epoch":90,"log_start":"96'30316","ondisk_log_start":"96'30316","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_clean_scrub_stamp":"2026-02-24T07:54:41.918128+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":3000,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1987051520,"num_objects":30320,"num_object_clones":0,"num_object_copies":90960,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30320,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":30320,"num_write_kb":1940480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.6","version":"96'30183","reported_seq":30205,"reported_epoch":96,"state":"active+clean","last_fresh":"2026-02-24T07:55:50.873941+0000","last_change":"2026-02-24T07:54:42.926358+0000","last_active":"2026-02-24T07:55:50.873941+0000","last_peered":"2026-02-24T07:55:50.873941+0000","last_clean":"2026-02-24T07:55:50.873941+0000","last_became_active":"2026-02-24T07:54:42.926045+0000","last_became_peered":"2026-02-24T07:54:42.926045+0000","last_unstale":"2026-02-24T07:55:50.873941+0000","last_undegraded":"2026-02-24T07:55:50.873941+0000","last_fullsized":"2026-02-24T07:55:50.873941+0000","mapping_epoch":90,"log_start":"96'30180","ondisk_log_start":"96'30180","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_clean_scrub_stamp":"2026-02-24T07:54:41.918128+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:56:09.126690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1978073088,"num_objects":30183,"num_object_clones":0,"num_object_copies":90549,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30183,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":30183,"num_write_kb":1931712,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.5","version":"96'30162","reported_seq":30184,"reported_epoch":96,"state":"active+clean","last_fresh":"2026-02-24T07:55:50.768171+0000","last_change":"2026-02-24T07:54:42.928143+0000","last_active":"2026-02-24T07:55:50.768171+0000","last_peered":"2026-02-24T07:55:50.768171+0000","last_clean":"2026-02-24T07:55:50.768171+0000","last_became_active":"2026-02-24T07:54:42.927535+0000","last_became_peered":"2026-02-24T07:54:42.927535+0000","last_unstale":"2026-02-24T07:55:50.768171+0000","last_undegraded":"2026-02-24T07:55:50.768171+0000","last_fullsized":"2026-02-24T07:55:50.768171+0000","mapping_epoch":90,"log_start":"96'30159","ondisk_log_start":"96'30159","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_clean_scrub_stamp":"2026-02-24T07:54:41.918128+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:55:53.650972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1976696832,"num_objects":30162,"num_object_clones":0,"num_object_copies":90486,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30162,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":30162,"num_write_kb":1930368,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.4","version":"96'30369","reported_seq":30687,"reported_epoch":96,"state":"active+clean","last_fresh":"2026-02-24T07:55:50.876400+0000","last_change":"2026-02-24T07:55:47.544043+0000","last_active":"2026-02-24T07:55:50.876400+0000","last_peered":"2026-02-24T07:55:50.876400+0000","last_clean":"2026-02-24T07:55:50.876400+0000","last_became_active":"2026-02-24T07:54:42.928011+0000","last_became_peered":"2026-02-24T07:54:42.928011+0000","last_unstale":"2026-02-24T07:55:50.876400+0000","last_undegraded":"2026-02-24T07:55:50.876400+0000","last_fullsized":"2026-02-24T07:55:50.876400+0000","mapping_epoch":90,"log_start":"96'30366","ondisk_log_start":"96'30366","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"96'29055","last_scrub_stamp":"2026-02-24T07:55:47.543994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_clean_scrub_stamp":"2026-02-24T07:55:47.543994+0000","objects_scrubbed":28510,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:57:04.321401+0000","scrub_duration":3165,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1990262784,"num_objects":30369,"num_object_clones":0,"num_object_copies":91107,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30369,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":30369,"num_write_kb":1943616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.3","version":"96'30321","reported_seq":30343,"reported_epoch":96,"state":"active+clean","last_fresh":"2026-02-24T07:55:50.770518+0000","last_change":"2026-02-24T07:54:42.928536+0000","last_active":"2026-02-24T07:55:50.770518+0000","last_peered":"2026-02-24T07:55:50.770518+0000","last_clean":"2026-02-24T07:55:50.770518+0000","last_became_active":"2026-02-24T07:54:42.927730+0000","last_became_peered":"2026-02-24T07:54:42.927730+0000","last_unstale":"2026-02-24T07:55:50.770518+0000","last_undegraded":"2026-02-24T07:55:50.770518+0000","last_fullsized":"2026-02-24T07:55:50.770518+0000","mapping_epoch":90,"log_start":"96'30318","ondisk_log_start":"96'30318","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_clean_scrub_stamp":"2026-02-24T07:54:41.918128+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:55:58.987317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1987117056,"num_objects":30321,"num_object_clones":0,"num_object_copies":90963,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30321,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":30321,"num_write_kb":1940544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.2","version":"96'30098","reported_seq":30120,"reported_epoch":96,"state":"active+clean","last_fresh":"2026-02-24T07:55:50.783215+0000","last_change":"2026-02-24T07:54:42.930452+0000","last_active":"2026-02-24T07:55:50.783215+0000","last_peered":"2026-02-24T07:55:50.783215+0000","last_clean":"2026-02-24T07:55:50.783215+0000","last_became_active":"2026-02-24T07:54:42.930247+0000","last_became_peered":"2026-02-24T07:54:42.930247+0000","last_unstale":"2026-02-24T07:55:50.783215+0000","last_undegraded":"2026-02-24T07:55:50.783215+0000","last_fullsized":"2026-02-24T07:55:50.783215+0000","mapping_epoch":90,"log_start":"96'30095","ondisk_log_start":"96'30095","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_clean_scrub_stamp":"2026-02-24T07:54:41.918128+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1972502528,"num_objects":30098,"num_object_clones":0,"num_object_copies":90294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30098,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":30098,"num_write_kb":1926272,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1","version":"96'30379","reported_seq":30401,"reported_epoch":96,"state":"active+clean","last_fresh":"2026-02-24T07:55:50.876935+0000","last_change":"2026-02-24T07:54:42.926619+0000","last_active":"2026-02-24T07:55:50.876935+0000","last_peered":"2026-02-24T07:55:50.876935+0000","last_clean":"2026-02-24T07:55:50.876935+0000","last_became_active":"2026-02-24T07:54:42.926442+0000","last_became_peered":"2026-02-24T07:54:42.926442+0000","last_unstale":"2026-02-24T07:55:50.876935+0000","last_undegraded":"2026-02-24T07:55:50.876935+0000","last_fullsized":"2026-02-24T07:55:50.876935+0000","mapping_epoch":90,"log_start":"96'30376","ondisk_log_start":"96'30376","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_clean_scrub_stamp":"2026-02-24T07:54:41.918128+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:56:03.125413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1990918144,"num_objects":30379,"num_object_clones":0,"num_object_copies":91137,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30379,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":30379,"num_write_kb":1944256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.0","version":"96'29939","reported_seq":29961,"reported_epoch":96,"state":"active+clean","last_fresh":"2026-02-24T07:55:50.804086+0000","last_change":"2026-02-24T07:54:42.927178+0000","last_active":"2026-02-24T07:55:50.804086+0000","last_peered":"2026-02-24T07:55:50.804086+0000","last_clean":"2026-02-24T07:55:50.804086+0000","last_became_active":"2026-02-24T07:54:42.926910+0000","last_became_peered":"2026-02-24T07:54:42.926910+0000","last_unstale":"2026-02-24T07:55:50.804086+0000","last_undegraded":"2026-02-24T07:55:50.804086+0000","last_fullsized":"2026-02-24T07:55:50.804086+0000","mapping_epoch":90,"log_start":"96'29935","ondisk_log_start":"96'29935","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_clean_scrub_stamp":"2026-02-24T07:54:41.918128+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":3000,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:55:58.002045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1962082304,"num_objects":29939,"num_object_clones":0,"num_object_copies":89817,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29939,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":29939,"num_write_kb":1916096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.7","version":"96'30312","reported_seq":30334,"reported_epoch":96,"state":"active+clean","last_fresh":"2026-02-24T07:55:50.781386+0000","last_change":"2026-02-24T07:54:42.930348+0000","last_active":"2026-02-24T07:55:50.781386+0000","last_peered":"2026-02-24T07:55:50.781386+0000","last_clean":"2026-02-24T07:55:50.781386+0000","last_became_active":"2026-02-24T07:54:42.930142+0000","last_became_peered":"2026-02-24T07:54:42.930142+0000","last_unstale":"2026-02-24T07:55:50.781386+0000","last_undegraded":"2026-02-24T07:55:50.781386+0000","last_fullsized":"2026-02-24T07:55:50.781386+0000","mapping_epoch":90,"log_start":"96'30308","ondisk_log_start":"96'30308","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_clean_scrub_stamp":"2026-02-24T07:54:41.918128+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":3000,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:55:56.704334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1986527232,"num_objects":30312,"num_object_clones":0,"num_object_copies":90936,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30312,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":30312,"num_write_kb":1939968,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"29'99","reported_seq":398,"reported_epoch":96,"state":"active+clean","last_fresh":"2026-02-24T07:55:16.398160+0000","last_change":"2026-02-24T07:55:16.398160+0000","last_active":"2026-02-24T07:55:16.398160+0000","last_peered":"2026-02-24T07:55:16.398160+0000","last_clean":"2026-02-24T07:55:16.398160+0000","last_became_active":"2026-02-24T07:42:54.186719+0000","last_became_peered":"2026-02-24T07:42:54.186719+0000","last_unstale":"2026-02-24T07:55:16.398160+0000","last_undegraded":"2026-02-24T07:55:16.398160+0000","last_fullsized":"2026-02-24T07:55:16.398160+0000","mapping_epoch":27,"log_start":"29'96","ondisk_log_start":"29'96","created":20,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'99","last_scrub_stamp":"2026-02-24T07:55:16.398131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:42:43.640776+0000","last_clean_scrub_stamp":"2026-02-24T07:55:16.398131+0000","objects_scrubbed":2,"log_size":3,"log_dups_size":96,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:56:32.925996+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":240,"num_write":135,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":16,"stat_sum":{"num_bytes":31684820992,"num_objects":483472,"num_object_clones":0,"num_object_copies":1450416,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":483472,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":483472,"num_write_kb":30942208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":95045484544,"data_stored":95045484544,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":53,"ondisk_log_size":53,"up":48,"acting":48,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":240,"num_write":135,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1744,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":3,"up_from":25,"seq":107374182558,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":20847892,"kb_used_data":19145728,"kb_used_omap":17,"kb_used_meta":1702126,"kb_avail":713155308,"statfs":{"total":751619276800,"available":730271035392,"internally_reserved":0,"allocated":19605225472,"data_stored":19604722854,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":18320,"internal_metadata":1742977136},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":19,"seq":81604378784,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":24685332,"kb_used_data":23070656,"kb_used_omap":17,"kb_used_meta":1614638,"kb_avail":709317868,"statfs":{"total":751619276800,"available":726341496832,"internally_reserved":0,"allocated":23624351744,"data_stored":23623849126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":18000,"internal_metadata":1653389744},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542305,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":28868304,"kb_used_data":26912380,"kb_used_omap":24,"kb_used_meta":1955815,"kb_avail":705134896,"statfs":{"total":751619276800,"available":722058133504,"internally_reserved":0,"allocated":27558277120,"data_stored":27557778054,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":24784,"internal_metadata":2002755376},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705828,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":24874516,"kb_used_data":23159616,"kb_used_omap":15,"kb_used_meta":1714864,"kb_avail":709128684,"statfs":{"total":751619276800,"available":726147772416,"internally_reserved":0,"allocated":23715446784,"data_stored":23714944166,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":15702,"internal_metadata":1756021418},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1744,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":23759355904,"data_stored":23759355904,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":27759345664,"data_stored":27759345664,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":23731634176,"data_stored":23731634176,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":19795148800,"data_stored":19795148800,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-24T07:55:55.539 INFO:tasks.cephadm.ceph_manager.ceph:increase pool size by 10 2026-02-24T07:55:55.540 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph osd pool set .mgr pg_num 401 2026-02-24T07:55:55.734 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:55:55.865 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:55:55.863+0000 7fbae406a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:55:55.866 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:55:55.864+0000 7fbae406a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:55:56.015 INFO:teuthology.orchestra.run.trial035.stderr:Error EPERM: crimson_allow_pg_split is false; pg_num increase denied for crimson pool 2026-02-24T07:55:56.020 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 72 16 521940 521924 453.023 427 0.00451881 0.00220611 2026-02-24T07:55:56.056 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-24T07:55:56.704 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:55:56 trial035 ceph-mon[39284]: 6.8 scrub starts 2026-02-24T07:55:56.704 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:55:56 trial035 ceph-mon[39284]: from='client.15523 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2026-02-24T07:55:56.704 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:55:56 trial035 ceph-mon[39284]: pgmap v458: 17 pgs: 17 active+clean; 30 GiB data, 95 GiB used, 2.6 TiB / 2.7 TiB avail; 364 MiB/s wr, 5.83k op/s 2026-02-24T07:55:56.704 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:55:56 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/49304754' entity='client.admin' cmd={"prefix": "osd pool set", "pool": ".mgr", "var": "pg_num", "val": "401"} : dispatch 2026-02-24T07:55:57.021 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 73 16 528857 528841 452.739 432.312 0.00466599 0.0022075 2026-02-24T07:55:57.254 INFO:journalctl@ceph.mgr.x.trial035.stdout:Feb 24 07:55:56 trial035 ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-mgr-x[39621]: ::ffff:10.20.193.91 - - [24/Feb/2026:07:55:56] "GET /metrics HTTP/1.1" 200 4214 "" "Prometheus/3.6.0" 2026-02-24T07:55:57.312 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2026-02-24T07:55:57.356 INFO:journalctl@ceph.prometheus.a.trial091.stdout:Feb 24 07:55:56 trial091 ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-prometheus-a[66007]: time=2026-02-24T07:55:56.973Z level=ERROR source=scrape.go:1630 msg="Failed to determine correct type of scrape target." component="scrape manager" scrape_pool=ceph target=http://trial035:9283/metrics content_type="text/html;charset=utf-8" fallback_media_type="" err="received unsupported Content-Type \"text/html;charset=utf-8\" and no fallback_scrape_protocol specified for target" 2026-02-24T07:55:57.464 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:55:57.523 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:55:57 trial035 ceph-mon[39284]: 6.5 scrub starts 2026-02-24T07:55:57.594 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:55:57.591+0000 7f0568a88640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:55:57.594 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:55:57.592+0000 7f0568a88640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:55:57.607 INFO:teuthology.orchestra.run.trial035.stdout:osd.0: {} 2026-02-24T07:55:57.607 INFO:teuthology.orchestra.run.trial035.stderr:osd.0: osd_enable_op_tracker = '' 2026-02-24T07:55:57.619 INFO:teuthology.orchestra.run.trial035.stdout:osd.1: {} 2026-02-24T07:55:57.620 INFO:teuthology.orchestra.run.trial035.stderr:osd.1: osd_enable_op_tracker = '' 2026-02-24T07:55:57.631 INFO:teuthology.orchestra.run.trial035.stdout:osd.2: {} 2026-02-24T07:55:57.631 INFO:teuthology.orchestra.run.trial035.stderr:osd.2: osd_enable_op_tracker = '' 2026-02-24T07:55:57.643 INFO:teuthology.orchestra.run.trial035.stdout:osd.3: {} 2026-02-24T07:55:57.643 INFO:teuthology.orchestra.run.trial035.stderr:osd.3: osd_enable_op_tracker = '' 2026-02-24T07:55:58.021 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 74 16 535058 535042 451.857 387.562 0.00510665 0.00221181 2026-02-24T07:55:58.704 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:55:58 trial035 ceph-mon[39284]: 6.2 scrub starts 2026-02-24T07:55:58.704 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:55:58 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/2287805914' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2026-02-24T07:55:58.704 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:55:58 trial035 ceph-mon[39284]: pgmap v459: 17 pgs: 1 active+clean+scrubbing, 16 active+clean; 32 GiB data, 102 GiB used, 2.6 TiB / 2.7 TiB avail; 557 MiB/s wr, 8.91k op/s 2026-02-24T07:55:59.021 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 75 16 541006 540990 450.789 371.75 0.00132748 0.00221705 2026-02-24T07:55:59.687 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2026-02-24T07:55:59.704 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:55:59 trial035 ceph-mon[39284]: 6.7 scrub starts 2026-02-24T07:55:59.704 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:55:59 trial035 ceph-mon[39284]: 6.8 scrub ok 2026-02-24T07:55:59.832 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:55:59.956 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:55:59.953+0000 7fa74be94640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:55:59.956 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:55:59.953+0000 7fa74be94640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:55:59.972 INFO:teuthology.orchestra.run.trial035.stdout:osd.0: {} 2026-02-24T07:55:59.972 INFO:teuthology.orchestra.run.trial035.stderr:osd.0: osd_enable_op_tracker = '' 2026-02-24T07:55:59.988 INFO:teuthology.orchestra.run.trial035.stdout:osd.1: {} 2026-02-24T07:55:59.988 INFO:teuthology.orchestra.run.trial035.stderr:osd.1: osd_enable_op_tracker = '' 2026-02-24T07:56:00.002 INFO:teuthology.orchestra.run.trial035.stdout:osd.2: {} 2026-02-24T07:56:00.002 INFO:teuthology.orchestra.run.trial035.stderr:osd.2: osd_enable_op_tracker = '' 2026-02-24T07:56:00.017 INFO:teuthology.orchestra.run.trial035.stdout:osd.3: {} 2026-02-24T07:56:00.017 INFO:teuthology.orchestra.run.trial035.stderr:osd.3: osd_enable_op_tracker = '' 2026-02-24T07:56:00.021 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 76 15 546795 546780 449.619 361.875 0.00230013 0.00222282 2026-02-24T07:56:00.704 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:56:00 trial035 ceph-mon[39284]: pgmap v460: 17 pgs: 1 active+clean+scrubbing, 16 active+clean; 32 GiB data, 102 GiB used, 2.6 TiB / 2.7 TiB avail; 352 MiB/s wr, 5.63k op/s 2026-02-24T07:56:00.705 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:56:00 trial035 ceph-mon[39284]: 6.5 scrub ok 2026-02-24T07:56:00.705 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:56:00 trial035 ceph-mon[39284]: from='mgr.14284 10.20.193.35:0/3681373222' entity='mgr.x' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2026-02-24T07:56:00.705 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:56:00 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/1787017106' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2026-02-24T07:56:01.021 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 77 16 552999 552983 448.814 387.688 0.00142987 0.0022268 2026-02-24T07:56:01.057 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2026-02-24T07:56:01.058 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2026-02-24T07:56:01.058 INFO:tasks.thrashosds.thrasher:Growing pool .mgr 2026-02-24T07:56:01.058 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph pg dump --format=json 2026-02-24T07:56:01.200 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:56:01.433 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:56:01 trial035 ceph-mon[39284]: 6.0 scrub starts 2026-02-24T07:56:01.434 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:56:01 trial035 ceph-mon[39284]: 6.2 scrub ok 2026-02-24T07:56:01.569 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:56:01.558+0000 7f4df8712640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:56:01.569 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:56:01.559+0000 7f4df8712640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:56:01.696 INFO:teuthology.orchestra.run.trial035.stdout: 2026-02-24T07:56:01.697 INFO:teuthology.orchestra.run.trial035.stderr:dumped all 2026-02-24T07:56:01.748 INFO:teuthology.orchestra.run.trial035.stdout:{"pg_ready":true,"pg_map":{"version":460,"stamp":"2026-02-24T07:55:59.763487+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":34111095328,"num_objects":520487,"num_object_clones":0,"num_object_copies":1561461,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":520487,"num_whiteouts":0,"num_read":136,"num_read_kb":240,"num_write":520620,"num_write_kb":33313190,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":51,"ondisk_log_size":51,"up":51,"acting":51,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":51,"num_osds":4,"num_per_pool_osds":4,"num_per_pool_omap_osds":4,"kb":2936012800,"kb_used":106657740,"kb_used_data":99451068,"kb_used_omap":77,"kb_used_meta":7206514,"kb_avail":2829355060,"statfs":{"total":3006477107200,"available":2897259581440,"internally_reserved":0,"allocated":101837893632,"data_stored":101835886712,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":79199,"internal_metadata":7379471009},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[8,1],"upper_bound":4},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":4430299136,"num_objects":67601,"num_object_clones":0,"num_object_copies":202803,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":67601,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":67601,"num_write_kb":4326464,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001792"},"pg_stats":[{"pgid":"6.f","version":"96'32542","reported_seq":32564,"reported_epoch":96,"state":"active+clean","last_fresh":"2026-02-24T07:55:55.876033+0000","last_change":"2026-02-24T07:54:42.927271+0000","last_active":"2026-02-24T07:55:55.876033+0000","last_peered":"2026-02-24T07:55:55.876033+0000","last_clean":"2026-02-24T07:55:55.876033+0000","last_became_active":"2026-02-24T07:54:42.927080+0000","last_became_peered":"2026-02-24T07:54:42.927080+0000","last_unstale":"2026-02-24T07:55:55.876033+0000","last_undegraded":"2026-02-24T07:55:55.876033+0000","last_fullsized":"2026-02-24T07:55:55.876033+0000","mapping_epoch":90,"log_start":"96'32539","ondisk_log_start":"96'32539","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_clean_scrub_stamp":"2026-02-24T07:54:41.918128+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:56:00.801546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2132672512,"num_objects":32542,"num_object_clones":0,"num_object_copies":97626,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32542,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":32542,"num_write_kb":2082688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.e","version":"96'32614","reported_seq":32636,"reported_epoch":96,"state":"active+clean","last_fresh":"2026-02-24T07:55:55.769919+0000","last_change":"2026-02-24T07:54:42.929104+0000","last_active":"2026-02-24T07:55:55.769919+0000","last_peered":"2026-02-24T07:55:55.769919+0000","last_clean":"2026-02-24T07:55:55.769919+0000","last_became_active":"2026-02-24T07:54:42.928748+0000","last_became_peered":"2026-02-24T07:54:42.928748+0000","last_unstale":"2026-02-24T07:55:55.769919+0000","last_undegraded":"2026-02-24T07:55:55.769919+0000","last_fullsized":"2026-02-24T07:55:55.769919+0000","mapping_epoch":90,"log_start":"96'32611","ondisk_log_start":"96'32611","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_clean_scrub_stamp":"2026-02-24T07:54:41.918128+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2137391104,"num_objects":32614,"num_object_clones":0,"num_object_copies":97842,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32614,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":32614,"num_write_kb":2087296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.d","version":"96'32469","reported_seq":32786,"reported_epoch":96,"state":"active+clean","last_fresh":"2026-02-24T07:55:55.762880+0000","last_change":"2026-02-24T07:55:48.151749+0000","last_active":"2026-02-24T07:55:55.762880+0000","last_peered":"2026-02-24T07:55:55.762880+0000","last_clean":"2026-02-24T07:55:55.762880+0000","last_became_active":"2026-02-24T07:54:42.926886+0000","last_became_peered":"2026-02-24T07:54:42.926886+0000","last_unstale":"2026-02-24T07:55:55.762880+0000","last_undegraded":"2026-02-24T07:55:55.762880+0000","last_fullsized":"2026-02-24T07:55:55.762880+0000","mapping_epoch":90,"log_start":"96'32466","ondisk_log_start":"96'32466","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"96'28924","last_scrub_stamp":"2026-02-24T07:55:48.151672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_clean_scrub_stamp":"2026-02-24T07:55:48.151672+0000","objects_scrubbed":28526,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:57:00.390677+0000","scrub_duration":2742,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2127888384,"num_objects":32469,"num_object_clones":0,"num_object_copies":97407,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32469,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":32469,"num_write_kb":2078016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.c","version":"96'32625","reported_seq":32647,"reported_epoch":96,"state":"active+clean","last_fresh":"2026-02-24T07:55:55.874983+0000","last_change":"2026-02-24T07:54:42.928482+0000","last_active":"2026-02-24T07:55:55.874983+0000","last_peered":"2026-02-24T07:55:55.874983+0000","last_clean":"2026-02-24T07:55:55.874983+0000","last_became_active":"2026-02-24T07:54:42.928238+0000","last_became_peered":"2026-02-24T07:54:42.928238+0000","last_unstale":"2026-02-24T07:55:55.874983+0000","last_undegraded":"2026-02-24T07:55:55.874983+0000","last_fullsized":"2026-02-24T07:55:55.874983+0000","mapping_epoch":90,"log_start":"96'32622","ondisk_log_start":"96'32622","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_clean_scrub_stamp":"2026-02-24T07:54:41.918128+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:56:09.181158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2138112000,"num_objects":32625,"num_object_clones":0,"num_object_copies":97875,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32625,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":32625,"num_write_kb":2088000,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.b","version":"96'32248","reported_seq":32270,"reported_epoch":96,"state":"active+clean","last_fresh":"2026-02-24T07:55:55.879542+0000","last_change":"2026-02-24T07:54:42.927223+0000","last_active":"2026-02-24T07:55:55.879542+0000","last_peered":"2026-02-24T07:55:55.879542+0000","last_clean":"2026-02-24T07:55:55.879542+0000","last_became_active":"2026-02-24T07:54:42.926965+0000","last_became_peered":"2026-02-24T07:54:42.926965+0000","last_unstale":"2026-02-24T07:55:55.879542+0000","last_undegraded":"2026-02-24T07:55:55.879542+0000","last_fullsized":"2026-02-24T07:55:55.879542+0000","mapping_epoch":90,"log_start":"96'32245","ondisk_log_start":"96'32245","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_clean_scrub_stamp":"2026-02-24T07:54:41.918128+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:56:04.661233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2113404928,"num_objects":32248,"num_object_clones":0,"num_object_copies":96744,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32248,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":32248,"num_write_kb":2063872,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.a","version":"96'32402","reported_seq":32721,"reported_epoch":96,"state":"active+clean","last_fresh":"2026-02-24T07:55:55.766466+0000","last_change":"2026-02-24T07:55:49.167491+0000","last_active":"2026-02-24T07:55:55.766466+0000","last_peered":"2026-02-24T07:55:55.766466+0000","last_clean":"2026-02-24T07:55:55.766466+0000","last_became_active":"2026-02-24T07:54:42.926477+0000","last_became_peered":"2026-02-24T07:54:42.926477+0000","last_unstale":"2026-02-24T07:55:55.766466+0000","last_undegraded":"2026-02-24T07:55:55.766466+0000","last_fullsized":"2026-02-24T07:55:55.766466+0000","mapping_epoch":90,"log_start":"96'32399","ondisk_log_start":"96'32399","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"96'29334","last_scrub_stamp":"2026-02-24T07:55:49.167461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_clean_scrub_stamp":"2026-02-24T07:55:49.167461+0000","objects_scrubbed":28822,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:56:59.424140+0000","scrub_duration":2726,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2123497472,"num_objects":32402,"num_object_clones":0,"num_object_copies":97206,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32402,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":32402,"num_write_kb":2073728,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"96'32571","reported_seq":32891,"reported_epoch":96,"state":"active+clean","last_fresh":"2026-02-24T07:55:55.799805+0000","last_change":"2026-02-24T07:55:48.535755+0000","last_active":"2026-02-24T07:55:55.799805+0000","last_peered":"2026-02-24T07:55:55.799805+0000","last_clean":"2026-02-24T07:55:55.799805+0000","last_became_active":"2026-02-24T07:54:42.926136+0000","last_became_peered":"2026-02-24T07:54:42.926136+0000","last_unstale":"2026-02-24T07:55:55.799805+0000","last_undegraded":"2026-02-24T07:55:55.799805+0000","last_fullsized":"2026-02-24T07:55:55.799805+0000","mapping_epoch":90,"log_start":"96'32568","ondisk_log_start":"96'32568","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"96'29148","last_scrub_stamp":"2026-02-24T07:55:48.535691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_clean_scrub_stamp":"2026-02-24T07:55:48.535691+0000","objects_scrubbed":28720,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:56:56.161155+0000","scrub_duration":2872,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2134573056,"num_objects":32571,"num_object_clones":0,"num_object_copies":97713,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32571,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":32571,"num_write_kb":2084544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,1],"acting":[0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.8","version":"96'32587","reported_seq":32676,"reported_epoch":96,"state":"active+clean+scrubbing","last_fresh":"2026-02-24T07:55:55.789982+0000","last_change":"2026-02-24T07:55:55.197835+0000","last_active":"2026-02-24T07:55:55.789982+0000","last_peered":"2026-02-24T07:55:55.789982+0000","last_clean":"2026-02-24T07:55:55.789982+0000","last_became_active":"2026-02-24T07:54:42.929078+0000","last_became_peered":"2026-02-24T07:54:42.929078+0000","last_unstale":"2026-02-24T07:55:55.789982+0000","last_undegraded":"2026-02-24T07:55:55.789982+0000","last_fullsized":"2026-02-24T07:55:55.789982+0000","mapping_epoch":90,"log_start":"96'32584","ondisk_log_start":"96'32584","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_clean_scrub_stamp":"2026-02-24T07:54:41.918128+0000","objects_scrubbed":6499,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"scrubbing for 1s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2135621632,"num_objects":32587,"num_object_clones":0,"num_object_copies":97761,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32587,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":32587,"num_write_kb":2085568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.6","version":"96'32495","reported_seq":32517,"reported_epoch":96,"state":"active+clean","last_fresh":"2026-02-24T07:55:55.879822+0000","last_change":"2026-02-24T07:54:42.926358+0000","last_active":"2026-02-24T07:55:55.879822+0000","last_peered":"2026-02-24T07:55:55.879822+0000","last_clean":"2026-02-24T07:55:55.879822+0000","last_became_active":"2026-02-24T07:54:42.926045+0000","last_became_peered":"2026-02-24T07:54:42.926045+0000","last_unstale":"2026-02-24T07:55:55.879822+0000","last_undegraded":"2026-02-24T07:55:55.879822+0000","last_fullsized":"2026-02-24T07:55:55.879822+0000","mapping_epoch":90,"log_start":"96'32492","ondisk_log_start":"96'32492","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_clean_scrub_stamp":"2026-02-24T07:54:41.918128+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:56:09.126690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2129592320,"num_objects":32495,"num_object_clones":0,"num_object_copies":97485,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32495,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":32495,"num_write_kb":2079680,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.5","version":"96'32459","reported_seq":32481,"reported_epoch":96,"state":"active+clean","last_fresh":"2026-02-24T07:55:55.769702+0000","last_change":"2026-02-24T07:54:42.928143+0000","last_active":"2026-02-24T07:55:55.769702+0000","last_peered":"2026-02-24T07:55:55.769702+0000","last_clean":"2026-02-24T07:55:55.769702+0000","last_became_active":"2026-02-24T07:54:42.927535+0000","last_became_peered":"2026-02-24T07:54:42.927535+0000","last_unstale":"2026-02-24T07:55:55.769702+0000","last_undegraded":"2026-02-24T07:55:55.769702+0000","last_fullsized":"2026-02-24T07:55:55.769702+0000","mapping_epoch":90,"log_start":"96'32456","ondisk_log_start":"96'32456","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_clean_scrub_stamp":"2026-02-24T07:54:41.918128+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2127233024,"num_objects":32459,"num_object_clones":0,"num_object_copies":97377,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32459,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":32459,"num_write_kb":2077376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.4","version":"96'32644","reported_seq":32962,"reported_epoch":96,"state":"active+clean","last_fresh":"2026-02-24T07:55:55.877616+0000","last_change":"2026-02-24T07:55:47.544043+0000","last_active":"2026-02-24T07:55:55.877616+0000","last_peered":"2026-02-24T07:55:55.877616+0000","last_clean":"2026-02-24T07:55:55.877616+0000","last_became_active":"2026-02-24T07:54:42.928011+0000","last_became_peered":"2026-02-24T07:54:42.928011+0000","last_unstale":"2026-02-24T07:55:55.877616+0000","last_undegraded":"2026-02-24T07:55:55.877616+0000","last_fullsized":"2026-02-24T07:55:55.877616+0000","mapping_epoch":90,"log_start":"96'32641","ondisk_log_start":"96'32641","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"96'29055","last_scrub_stamp":"2026-02-24T07:55:47.543994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_clean_scrub_stamp":"2026-02-24T07:55:47.543994+0000","objects_scrubbed":28510,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:57:04.321401+0000","scrub_duration":3165,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2139357184,"num_objects":32644,"num_object_clones":0,"num_object_copies":97932,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32644,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":32644,"num_write_kb":2089216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.3","version":"96'32670","reported_seq":32692,"reported_epoch":96,"state":"active+clean","last_fresh":"2026-02-24T07:55:55.767516+0000","last_change":"2026-02-24T07:54:42.928536+0000","last_active":"2026-02-24T07:55:55.767516+0000","last_peered":"2026-02-24T07:55:55.767516+0000","last_clean":"2026-02-24T07:55:55.767516+0000","last_became_active":"2026-02-24T07:54:42.927730+0000","last_became_peered":"2026-02-24T07:54:42.927730+0000","last_unstale":"2026-02-24T07:55:55.767516+0000","last_undegraded":"2026-02-24T07:55:55.767516+0000","last_fullsized":"2026-02-24T07:55:55.767516+0000","mapping_epoch":90,"log_start":"96'32667","ondisk_log_start":"96'32667","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_clean_scrub_stamp":"2026-02-24T07:54:41.918128+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:55:58.987317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2141061120,"num_objects":32670,"num_object_clones":0,"num_object_copies":98010,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32670,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":32670,"num_write_kb":2090880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.2","version":"96'32506","reported_seq":32528,"reported_epoch":96,"state":"active+clean","last_fresh":"2026-02-24T07:55:55.782784+0000","last_change":"2026-02-24T07:54:42.930452+0000","last_active":"2026-02-24T07:55:55.782784+0000","last_peered":"2026-02-24T07:55:55.782784+0000","last_clean":"2026-02-24T07:55:55.782784+0000","last_became_active":"2026-02-24T07:54:42.930247+0000","last_became_peered":"2026-02-24T07:54:42.930247+0000","last_unstale":"2026-02-24T07:55:55.782784+0000","last_undegraded":"2026-02-24T07:55:55.782784+0000","last_fullsized":"2026-02-24T07:55:55.782784+0000","mapping_epoch":90,"log_start":"96'32503","ondisk_log_start":"96'32503","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_clean_scrub_stamp":"2026-02-24T07:54:41.918128+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2130313216,"num_objects":32506,"num_object_clones":0,"num_object_copies":97518,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32506,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":32506,"num_write_kb":2080384,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1","version":"96'32688","reported_seq":32710,"reported_epoch":96,"state":"active+clean","last_fresh":"2026-02-24T07:55:55.875921+0000","last_change":"2026-02-24T07:54:42.926619+0000","last_active":"2026-02-24T07:55:55.875921+0000","last_peered":"2026-02-24T07:55:55.875921+0000","last_clean":"2026-02-24T07:55:55.875921+0000","last_became_active":"2026-02-24T07:54:42.926442+0000","last_became_peered":"2026-02-24T07:54:42.926442+0000","last_unstale":"2026-02-24T07:55:55.875921+0000","last_undegraded":"2026-02-24T07:55:55.875921+0000","last_fullsized":"2026-02-24T07:55:55.875921+0000","mapping_epoch":90,"log_start":"96'32685","ondisk_log_start":"96'32685","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_clean_scrub_stamp":"2026-02-24T07:54:41.918128+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:56:03.125413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2142240768,"num_objects":32688,"num_object_clones":0,"num_object_copies":98064,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32688,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":32688,"num_write_kb":2092032,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.0","version":"96'32267","reported_seq":32289,"reported_epoch":96,"state":"active+clean","last_fresh":"2026-02-24T07:55:55.803102+0000","last_change":"2026-02-24T07:54:42.927178+0000","last_active":"2026-02-24T07:55:55.803102+0000","last_peered":"2026-02-24T07:55:55.803102+0000","last_clean":"2026-02-24T07:55:55.803102+0000","last_became_active":"2026-02-24T07:54:42.926910+0000","last_became_peered":"2026-02-24T07:54:42.926910+0000","last_unstale":"2026-02-24T07:55:55.803102+0000","last_undegraded":"2026-02-24T07:55:55.803102+0000","last_fullsized":"2026-02-24T07:55:55.803102+0000","mapping_epoch":90,"log_start":"96'32264","ondisk_log_start":"96'32264","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_clean_scrub_stamp":"2026-02-24T07:54:41.918128+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:55:58.002045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2114650112,"num_objects":32267,"num_object_clones":0,"num_object_copies":96801,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32267,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":32267,"num_write_kb":2065088,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.7","version":"96'32698","reported_seq":32720,"reported_epoch":96,"state":"active+clean","last_fresh":"2026-02-24T07:55:55.788224+0000","last_change":"2026-02-24T07:54:42.930348+0000","last_active":"2026-02-24T07:55:55.788224+0000","last_peered":"2026-02-24T07:55:55.788224+0000","last_clean":"2026-02-24T07:55:55.788224+0000","last_became_active":"2026-02-24T07:54:42.930142+0000","last_became_peered":"2026-02-24T07:54:42.930142+0000","last_unstale":"2026-02-24T07:55:55.788224+0000","last_undegraded":"2026-02-24T07:55:55.788224+0000","last_fullsized":"2026-02-24T07:55:55.788224+0000","mapping_epoch":90,"log_start":"96'32695","ondisk_log_start":"96'32695","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_clean_scrub_stamp":"2026-02-24T07:54:41.918128+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:55:56.704334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2142896128,"num_objects":32698,"num_object_clones":0,"num_object_copies":98094,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32698,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":32698,"num_write_kb":2092672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"29'99","reported_seq":398,"reported_epoch":96,"state":"active+clean","last_fresh":"2026-02-24T07:55:16.398160+0000","last_change":"2026-02-24T07:55:16.398160+0000","last_active":"2026-02-24T07:55:16.398160+0000","last_peered":"2026-02-24T07:55:16.398160+0000","last_clean":"2026-02-24T07:55:16.398160+0000","last_became_active":"2026-02-24T07:42:54.186719+0000","last_became_peered":"2026-02-24T07:42:54.186719+0000","last_unstale":"2026-02-24T07:55:16.398160+0000","last_undegraded":"2026-02-24T07:55:16.398160+0000","last_fullsized":"2026-02-24T07:55:16.398160+0000","mapping_epoch":27,"log_start":"29'96","ondisk_log_start":"29'96","created":20,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'99","last_scrub_stamp":"2026-02-24T07:55:16.398131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:42:43.640776+0000","last_clean_scrub_stamp":"2026-02-24T07:55:16.398131+0000","objects_scrubbed":2,"log_size":3,"log_dups_size":96,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:56:32.925996+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":240,"num_write":135,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":16,"stat_sum":{"num_bytes":34110504960,"num_objects":520485,"num_object_clones":0,"num_object_copies":1561455,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":520485,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":520485,"num_write_kb":33311040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":102331908096,"data_stored":102331908096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":48,"ondisk_log_size":48,"up":48,"acting":48,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":240,"num_write":135,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":2396,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":3,"up_from":25,"seq":107374182559,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":22433876,"kb_used_data":20659520,"kb_used_omap":18,"kb_used_meta":1774317,"kb_avail":711569324,"statfs":{"total":751619276800,"available":728646987776,"internally_reserved":0,"allocated":21155348480,"data_stored":21154845862,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":18972,"internal_metadata":1816901092},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":19,"seq":81604378785,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":26578836,"kb_used_data":24855872,"kb_used_omap":18,"kb_used_meta":1722925,"kb_avail":707424364,"statfs":{"total":751619276800,"available":724402548736,"internally_reserved":0,"allocated":25452412928,"data_stored":25451910310,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":18652,"internal_metadata":1764276004},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[6,1],"upper_bound":4},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542306,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":30904016,"kb_used_data":29017852,"kb_used_omap":25,"kb_used_meta":1886118,"kb_avail":703099184,"statfs":{"total":751619276800,"available":719973564416,"internally_reserved":0,"allocated":29714280448,"data_stored":29713781382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":25873,"internal_metadata":1931385583},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705829,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":26741012,"kb_used_data":24917824,"kb_used_omap":15,"kb_used_meta":1823152,"kb_avail":707262188,"statfs":{"total":751619276800,"available":724236480512,"internally_reserved":0,"allocated":25515851776,"data_stored":25515349158,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":15702,"internal_metadata":1866908330},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":2396,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":25578045440,"data_stored":25578045440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":29877927936,"data_stored":29877927936,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":25554911232,"data_stored":25554911232,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":21321023488,"data_stored":21321023488,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-24T07:56:01.750 INFO:tasks.cephadm.ceph_manager.ceph:increase pool size by 10 2026-02-24T07:56:01.750 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph osd pool set .mgr pg_num 411 2026-02-24T07:56:01.886 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:56:02.021 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 78 16 558592 558576 447.541 349.562 0.00282756 0.00223315 2026-02-24T07:56:02.022 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:56:02.020+0000 7f1139b54640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:56:02.022 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:56:02.020+0000 7f1139b54640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:56:02.062 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2026-02-24T07:56:02.166 INFO:teuthology.orchestra.run.trial035.stderr:Error EPERM: crimson_allow_pg_split is false; pg_num increase denied for crimson pool 2026-02-24T07:56:02.207 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-24T07:56:02.218 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:56:02.345 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:56:02.343+0000 7f210973b640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:56:02.345 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:56:02.343+0000 7f210973b640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:56:02.359 INFO:teuthology.orchestra.run.trial035.stdout:osd.0: {} 2026-02-24T07:56:02.359 INFO:teuthology.orchestra.run.trial035.stderr:osd.0: osd_enable_op_tracker = '' 2026-02-24T07:56:02.372 INFO:teuthology.orchestra.run.trial035.stdout:osd.1: {} 2026-02-24T07:56:02.372 INFO:teuthology.orchestra.run.trial035.stderr:osd.1: osd_enable_op_tracker = '' 2026-02-24T07:56:02.386 INFO:teuthology.orchestra.run.trial035.stdout:osd.2: {} 2026-02-24T07:56:02.386 INFO:teuthology.orchestra.run.trial035.stderr:osd.2: osd_enable_op_tracker = '' 2026-02-24T07:56:02.398 INFO:teuthology.orchestra.run.trial035.stdout:osd.3: {} 2026-02-24T07:56:02.399 INFO:teuthology.orchestra.run.trial035.stderr:osd.3: osd_enable_op_tracker = '' 2026-02-24T07:56:02.704 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:56:02 trial035 ceph-mon[39284]: 6.7 scrub ok 2026-02-24T07:56:02.704 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:56:02 trial035 ceph-mon[39284]: 6.e scrub starts 2026-02-24T07:56:02.704 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:56:02 trial035 ceph-mon[39284]: from='client.15531 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2026-02-24T07:56:02.704 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:56:02 trial035 ceph-mon[39284]: pgmap v461: 17 pgs: 2 active+clean+scrubbing, 15 active+clean; 34 GiB data, 107 GiB used, 2.6 TiB / 2.7 TiB avail; 515 MiB/s wr, 8.24k op/s 2026-02-24T07:56:02.704 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:56:02 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/2560445156' entity='client.admin' cmd={"prefix": "osd pool set", "pool": ".mgr", "var": "pg_num", "val": "411"} : dispatch 2026-02-24T07:56:03.021 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 79 16 564752 564736 446.749 385 0.00123882 0.0022371 2026-02-24T07:56:03.704 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:56:03 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/262032221' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2026-02-24T07:56:03.704 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:56:03 trial035 ceph-mon[39284]: 6.3 scrub starts 2026-02-24T07:56:04.021 INFO:tasks.radosbench.radosbench.0.trial035.stdout:2026-02-24T07:56:04.021056+0000 min lat: 0.000714289 max lat: 0.0462542 avg lat: 0.00223962 2026-02-24T07:56:04.022 INFO:tasks.radosbench.radosbench.0.trial035.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2026-02-24T07:56:04.022 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 80 16 571264 571248 446.252 407 0.0016379 0.00223962 2026-02-24T07:56:04.440 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2026-02-24T07:56:04.589 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:56:04.704 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:56:04 trial035 ceph-mon[39284]: 6.0 scrub ok 2026-02-24T07:56:04.704 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:56:04 trial035 ceph-mon[39284]: pgmap v462: 17 pgs: 2 active+clean+scrubbing, 15 active+clean; 34 GiB data, 107 GiB used, 2.6 TiB / 2.7 TiB avail; 356 MiB/s wr, 5.69k op/s 2026-02-24T07:56:04.956 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:56:04.925+0000 7f2868fb7640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:56:04.956 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:56:04.926+0000 7f2868fb7640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:56:04.956 INFO:teuthology.orchestra.run.trial035.stderr:osd.0: osd_enable_op_tracker = '' 2026-02-24T07:56:04.957 INFO:teuthology.orchestra.run.trial035.stdout:osd.0: {} 2026-02-24T07:56:04.959 INFO:teuthology.orchestra.run.trial035.stdout:osd.1: {} 2026-02-24T07:56:04.960 INFO:teuthology.orchestra.run.trial035.stderr:osd.1: osd_enable_op_tracker = '' 2026-02-24T07:56:04.975 INFO:teuthology.orchestra.run.trial035.stdout:osd.2: {} 2026-02-24T07:56:04.976 INFO:teuthology.orchestra.run.trial035.stderr:osd.2: osd_enable_op_tracker = '' 2026-02-24T07:56:04.990 INFO:teuthology.orchestra.run.trial035.stdout:osd.3: {} 2026-02-24T07:56:04.991 INFO:teuthology.orchestra.run.trial035.stderr:osd.3: osd_enable_op_tracker = '' 2026-02-24T07:56:05.021 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 81 16 577155 577139 445.288 368.188 0.00168375 0.00224446 2026-02-24T07:56:05.704 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:56:05 trial035 ceph-mon[39284]: purged_snaps scrub starts 2026-02-24T07:56:05.704 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:56:05 trial035 ceph-mon[39284]: purged_snaps scrub ok 2026-02-24T07:56:05.704 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:56:05 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/1699209058' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2026-02-24T07:56:06.021 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 82 16 583835 583819 444.948 417.5 0.00190204 0.00224618 2026-02-24T07:56:06.704 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:56:06 trial035 ceph-mon[39284]: 6.e scrub ok 2026-02-24T07:56:06.704 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:56:06 trial035 ceph-mon[39284]: pgmap v463: 17 pgs: 2 active+clean+scrubbing, 15 active+clean; 34 GiB data, 107 GiB used, 2.6 TiB / 2.7 TiB avail; 356 MiB/s wr, 5.69k op/s 2026-02-24T07:56:07.021 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 83 16 591716 591700 445.522 492.562 0.000865119 0.00224331 2026-02-24T07:56:07.033 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2026-02-24T07:56:07.185 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:56:07.209 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2026-02-24T07:56:07.209 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2026-02-24T07:56:07.209 INFO:tasks.thrashosds.thrasher:Growing pool unique_pool_4 2026-02-24T07:56:07.209 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph pg dump --format=json 2026-02-24T07:56:07.233 INFO:journalctl@ceph.mgr.x.trial035.stdout:Feb 24 07:56:06 trial035 ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-mgr-x[39621]: ::ffff:10.20.193.91 - - [24/Feb/2026:07:56:06] "GET /metrics HTTP/1.1" 200 4223 "" "Prometheus/3.6.0" 2026-02-24T07:56:07.348 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:56:07.356 INFO:journalctl@ceph.prometheus.a.trial091.stdout:Feb 24 07:56:06 trial091 ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-prometheus-a[66007]: time=2026-02-24T07:56:06.973Z level=ERROR source=scrape.go:1630 msg="Failed to determine correct type of scrape target." component="scrape manager" scrape_pool=ceph target=http://trial035:9283/metrics content_type="text/html;charset=utf-8" fallback_media_type="" err="received unsupported Content-Type \"text/html;charset=utf-8\" and no fallback_scrape_protocol specified for target" 2026-02-24T07:56:07.520 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:56:07 trial035 ceph-mon[39284]: 6.3 scrub ok 2026-02-24T07:56:07.574 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:56:07.518+0000 7fbd74e52640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:56:07.575 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:56:07.518+0000 7fbd74e52640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:56:07.575 INFO:teuthology.orchestra.run.trial035.stderr:osd.0: osd_enable_op_tracker = '' 2026-02-24T07:56:07.575 INFO:teuthology.orchestra.run.trial035.stderr:osd.1: osd_enable_op_tracker = '' 2026-02-24T07:56:07.575 INFO:teuthology.orchestra.run.trial035.stdout:osd.0: {} 2026-02-24T07:56:07.575 INFO:teuthology.orchestra.run.trial035.stdout:osd.1: {} 2026-02-24T07:56:07.583 INFO:teuthology.orchestra.run.trial035.stdout:osd.2: {} 2026-02-24T07:56:07.583 INFO:teuthology.orchestra.run.trial035.stderr:osd.2: osd_enable_op_tracker = '' 2026-02-24T07:56:07.597 INFO:teuthology.orchestra.run.trial035.stdout:osd.3: {} 2026-02-24T07:56:07.597 INFO:teuthology.orchestra.run.trial035.stderr:osd.3: osd_enable_op_tracker = '' 2026-02-24T07:56:08.021 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 84 16 597169 597153 444.275 340.812 0.00265672 0.00224959 2026-02-24T07:56:08.541 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:56:08.540+0000 7faf19f65640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:56:08.542 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:56:08.540+0000 7faf19f65640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:56:08.542 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:56:08 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/3687439881' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2026-02-24T07:56:08.542 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:56:08 trial035 ceph-mon[39284]: 6.f scrub starts 2026-02-24T07:56:08.542 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:56:08 trial035 ceph-mon[39284]: pgmap v464: 17 pgs: 1 active+clean+scrubbing, 16 active+clean; 36 GiB data, 113 GiB used, 2.6 TiB / 2.7 TiB avail; 515 MiB/s wr, 8.24k op/s 2026-02-24T07:56:08.698 INFO:teuthology.orchestra.run.trial035.stdout: 2026-02-24T07:56:08.698 INFO:teuthology.orchestra.run.trial035.stderr:dumped all 2026-02-24T07:56:08.742 INFO:teuthology.orchestra.run.trial035.stdout:{"pg_ready":true,"pg_map":{"version":464,"stamp":"2026-02-24T07:56:07.764570+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":38169412128,"num_objects":582412,"num_object_clones":0,"num_object_copies":1747236,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":582412,"num_whiteouts":0,"num_read":136,"num_read_kb":240,"num_write":582545,"num_write_kb":37276390,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":51,"ondisk_log_size":51,"up":51,"acting":51,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":51,"num_osds":4,"num_per_pool_osds":4,"num_per_pool_omap_osds":4,"kb":2936012800,"kb_used":118782028,"kb_used_data":111348092,"kb_used_omap":78,"kb_used_meta":7433777,"kb_avail":2817230772,"statfs":{"total":3006477107200,"available":2884844310528,"internally_reserved":0,"allocated":114020446208,"data_stored":114018439288,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":80839,"internal_metadata":7612187705},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":6484000768,"num_objects":98938,"num_object_clones":0,"num_object_copies":296814,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98938,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":98938,"num_write_kb":6332032,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001667"},"pg_stats":[{"pgid":"6.f","version":"96'36343","reported_seq":36365,"reported_epoch":96,"state":"active+clean","last_fresh":"2026-02-24T07:56:05.880311+0000","last_change":"2026-02-24T07:54:42.927271+0000","last_active":"2026-02-24T07:56:05.880311+0000","last_peered":"2026-02-24T07:56:05.880311+0000","last_clean":"2026-02-24T07:56:05.880311+0000","last_became_active":"2026-02-24T07:54:42.927080+0000","last_became_peered":"2026-02-24T07:54:42.927080+0000","last_unstale":"2026-02-24T07:56:05.880311+0000","last_undegraded":"2026-02-24T07:56:05.880311+0000","last_fullsized":"2026-02-24T07:56:05.880311+0000","mapping_epoch":90,"log_start":"96'36340","ondisk_log_start":"96'36340","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_clean_scrub_stamp":"2026-02-24T07:54:41.918128+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2381774848,"num_objects":36343,"num_object_clones":0,"num_object_copies":109029,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36343,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":36343,"num_write_kb":2325952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.e","version":"96'36491","reported_seq":36875,"reported_epoch":96,"state":"active+clean","last_fresh":"2026-02-24T07:56:05.775680+0000","last_change":"2026-02-24T07:56:05.374565+0000","last_active":"2026-02-24T07:56:05.775680+0000","last_peered":"2026-02-24T07:56:05.775680+0000","last_clean":"2026-02-24T07:56:05.775680+0000","last_became_active":"2026-02-24T07:54:42.928748+0000","last_became_peered":"2026-02-24T07:54:42.928748+0000","last_unstale":"2026-02-24T07:56:05.775680+0000","last_undegraded":"2026-02-24T07:56:05.775680+0000","last_fullsized":"2026-02-24T07:56:05.775680+0000","mapping_epoch":90,"log_start":"96'36488","ondisk_log_start":"96'36488","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"96'36307","last_scrub_stamp":"2026-02-24T07:56:05.374510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_clean_scrub_stamp":"2026-02-24T07:56:05.374510+0000","objects_scrubbed":35491,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:57:16.427813+0000","scrub_duration":4036,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2391474176,"num_objects":36491,"num_object_clones":0,"num_object_copies":109473,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36491,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":36491,"num_write_kb":2335424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.d","version":"96'36328","reported_seq":36644,"reported_epoch":96,"state":"active+clean","last_fresh":"2026-02-24T07:56:05.775578+0000","last_change":"2026-02-24T07:55:48.151749+0000","last_active":"2026-02-24T07:56:05.775578+0000","last_peered":"2026-02-24T07:56:05.775578+0000","last_clean":"2026-02-24T07:56:05.775578+0000","last_became_active":"2026-02-24T07:54:42.926886+0000","last_became_peered":"2026-02-24T07:54:42.926886+0000","last_unstale":"2026-02-24T07:56:05.775578+0000","last_undegraded":"2026-02-24T07:56:05.775578+0000","last_fullsized":"2026-02-24T07:56:05.775578+0000","mapping_epoch":90,"log_start":"96'36325","ondisk_log_start":"96'36325","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"96'28924","last_scrub_stamp":"2026-02-24T07:55:48.151672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_clean_scrub_stamp":"2026-02-24T07:55:48.151672+0000","objects_scrubbed":28526,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:57:00.390677+0000","scrub_duration":2742,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2380791808,"num_objects":36328,"num_object_clones":0,"num_object_copies":108984,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36328,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":36328,"num_write_kb":2324992,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.c","version":"96'36462","reported_seq":36484,"reported_epoch":96,"state":"active+clean","last_fresh":"2026-02-24T07:56:05.879753+0000","last_change":"2026-02-24T07:54:42.928482+0000","last_active":"2026-02-24T07:56:05.879753+0000","last_peered":"2026-02-24T07:56:05.879753+0000","last_clean":"2026-02-24T07:56:05.879753+0000","last_became_active":"2026-02-24T07:54:42.928238+0000","last_became_peered":"2026-02-24T07:54:42.928238+0000","last_unstale":"2026-02-24T07:56:05.879753+0000","last_undegraded":"2026-02-24T07:56:05.879753+0000","last_fullsized":"2026-02-24T07:56:05.879753+0000","mapping_epoch":90,"log_start":"96'36459","ondisk_log_start":"96'36459","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_clean_scrub_stamp":"2026-02-24T07:54:41.918128+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:56:09.181158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2389573632,"num_objects":36462,"num_object_clones":0,"num_object_copies":109386,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36462,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":36462,"num_write_kb":2333568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.b","version":"96'36084","reported_seq":36106,"reported_epoch":96,"state":"active+clean","last_fresh":"2026-02-24T07:56:05.866893+0000","last_change":"2026-02-24T07:54:42.927223+0000","last_active":"2026-02-24T07:56:05.866893+0000","last_peered":"2026-02-24T07:56:05.866893+0000","last_clean":"2026-02-24T07:56:05.866893+0000","last_became_active":"2026-02-24T07:54:42.926965+0000","last_became_peered":"2026-02-24T07:54:42.926965+0000","last_unstale":"2026-02-24T07:56:05.866893+0000","last_undegraded":"2026-02-24T07:56:05.866893+0000","last_fullsized":"2026-02-24T07:56:05.866893+0000","mapping_epoch":90,"log_start":"96'36081","ondisk_log_start":"96'36081","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_clean_scrub_stamp":"2026-02-24T07:54:41.918128+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2364801024,"num_objects":36084,"num_object_clones":0,"num_object_copies":108252,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36084,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":36084,"num_write_kb":2309376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.a","version":"96'36283","reported_seq":36602,"reported_epoch":96,"state":"active+clean","last_fresh":"2026-02-24T07:56:05.775626+0000","last_change":"2026-02-24T07:55:49.167491+0000","last_active":"2026-02-24T07:56:05.775626+0000","last_peered":"2026-02-24T07:56:05.775626+0000","last_clean":"2026-02-24T07:56:05.775626+0000","last_became_active":"2026-02-24T07:54:42.926477+0000","last_became_peered":"2026-02-24T07:54:42.926477+0000","last_unstale":"2026-02-24T07:56:05.775626+0000","last_undegraded":"2026-02-24T07:56:05.775626+0000","last_fullsized":"2026-02-24T07:56:05.775626+0000","mapping_epoch":90,"log_start":"96'36280","ondisk_log_start":"96'36280","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"96'29334","last_scrub_stamp":"2026-02-24T07:55:49.167461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_clean_scrub_stamp":"2026-02-24T07:55:49.167461+0000","objects_scrubbed":28822,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:56:59.424140+0000","scrub_duration":2726,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2377842688,"num_objects":36283,"num_object_clones":0,"num_object_copies":108849,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36283,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":36283,"num_write_kb":2322112,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"96'36522","reported_seq":36842,"reported_epoch":96,"state":"active+clean","last_fresh":"2026-02-24T07:56:05.807418+0000","last_change":"2026-02-24T07:55:48.535755+0000","last_active":"2026-02-24T07:56:05.807418+0000","last_peered":"2026-02-24T07:56:05.807418+0000","last_clean":"2026-02-24T07:56:05.807418+0000","last_became_active":"2026-02-24T07:54:42.926136+0000","last_became_peered":"2026-02-24T07:54:42.926136+0000","last_unstale":"2026-02-24T07:56:05.807418+0000","last_undegraded":"2026-02-24T07:56:05.807418+0000","last_fullsized":"2026-02-24T07:56:05.807418+0000","mapping_epoch":90,"log_start":"96'36519","ondisk_log_start":"96'36519","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"96'29148","last_scrub_stamp":"2026-02-24T07:55:48.535691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_clean_scrub_stamp":"2026-02-24T07:55:48.535691+0000","objects_scrubbed":28720,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:56:56.161155+0000","scrub_duration":2872,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2393505792,"num_objects":36522,"num_object_clones":0,"num_object_copies":109566,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36522,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":36522,"num_write_kb":2337408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,1],"acting":[0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.8","version":"96'36523","reported_seq":36884,"reported_epoch":96,"state":"active+clean","last_fresh":"2026-02-24T07:56:05.790198+0000","last_change":"2026-02-24T07:55:58.178460+0000","last_active":"2026-02-24T07:56:05.790198+0000","last_peered":"2026-02-24T07:56:05.790198+0000","last_clean":"2026-02-24T07:56:05.790198+0000","last_became_active":"2026-02-24T07:54:42.929078+0000","last_became_peered":"2026-02-24T07:54:42.929078+0000","last_unstale":"2026-02-24T07:56:05.790198+0000","last_undegraded":"2026-02-24T07:56:05.790198+0000","last_fullsized":"2026-02-24T07:56:05.790198+0000","mapping_epoch":90,"log_start":"96'36520","ondisk_log_start":"96'36520","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"96'33550","last_scrub_stamp":"2026-02-24T07:55:58.178395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_clean_scrub_stamp":"2026-02-24T07:55:58.178395+0000","objects_scrubbed":32948,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:57:14.607073+0000","scrub_duration":2981,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2393571328,"num_objects":36523,"num_object_clones":0,"num_object_copies":109569,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36523,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":36523,"num_write_kb":2337472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.6","version":"96'36311","reported_seq":36333,"reported_epoch":96,"state":"active+clean","last_fresh":"2026-02-24T07:56:05.882124+0000","last_change":"2026-02-24T07:54:42.926358+0000","last_active":"2026-02-24T07:56:05.882124+0000","last_peered":"2026-02-24T07:56:05.882124+0000","last_clean":"2026-02-24T07:56:05.882124+0000","last_became_active":"2026-02-24T07:54:42.926045+0000","last_became_peered":"2026-02-24T07:54:42.926045+0000","last_unstale":"2026-02-24T07:56:05.882124+0000","last_undegraded":"2026-02-24T07:56:05.882124+0000","last_fullsized":"2026-02-24T07:56:05.882124+0000","mapping_epoch":90,"log_start":"96'36308","ondisk_log_start":"96'36308","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_clean_scrub_stamp":"2026-02-24T07:54:41.918128+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:56:09.126690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2379677696,"num_objects":36311,"num_object_clones":0,"num_object_copies":108933,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36311,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":36311,"num_write_kb":2323904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.5","version":"96'36403","reported_seq":36768,"reported_epoch":96,"state":"active+clean","last_fresh":"2026-02-24T07:56:05.771783+0000","last_change":"2026-02-24T07:55:59.773616+0000","last_active":"2026-02-24T07:56:05.771783+0000","last_peered":"2026-02-24T07:56:05.771783+0000","last_clean":"2026-02-24T07:56:05.771783+0000","last_became_active":"2026-02-24T07:54:42.927535+0000","last_became_peered":"2026-02-24T07:54:42.927535+0000","last_unstale":"2026-02-24T07:56:05.771783+0000","last_undegraded":"2026-02-24T07:56:05.771783+0000","last_fullsized":"2026-02-24T07:56:05.771783+0000","mapping_epoch":90,"log_start":"96'36400","ondisk_log_start":"96'36400","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"96'34080","last_scrub_stamp":"2026-02-24T07:55:59.773556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_clean_scrub_stamp":"2026-02-24T07:55:59.773556+0000","objects_scrubbed":33431,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:57:04.302607+0000","scrub_duration":3324,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2385707008,"num_objects":36403,"num_object_clones":0,"num_object_copies":109209,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36403,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":36403,"num_write_kb":2329792,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.4","version":"96'36405","reported_seq":36723,"reported_epoch":96,"state":"active+clean","last_fresh":"2026-02-24T07:56:05.877564+0000","last_change":"2026-02-24T07:55:47.544043+0000","last_active":"2026-02-24T07:56:05.877564+0000","last_peered":"2026-02-24T07:56:05.877564+0000","last_clean":"2026-02-24T07:56:05.877564+0000","last_became_active":"2026-02-24T07:54:42.928011+0000","last_became_peered":"2026-02-24T07:54:42.928011+0000","last_unstale":"2026-02-24T07:56:05.877564+0000","last_undegraded":"2026-02-24T07:56:05.877564+0000","last_fullsized":"2026-02-24T07:56:05.877564+0000","mapping_epoch":90,"log_start":"96'36402","ondisk_log_start":"96'36402","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"96'29055","last_scrub_stamp":"2026-02-24T07:55:47.543994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_clean_scrub_stamp":"2026-02-24T07:55:47.543994+0000","objects_scrubbed":28510,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:57:04.321401+0000","scrub_duration":3165,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2385838080,"num_objects":36405,"num_object_clones":0,"num_object_copies":109215,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36405,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":36405,"num_write_kb":2329920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.3","version":"96'36482","reported_seq":36828,"reported_epoch":96,"state":"active+clean+scrubbing","last_fresh":"2026-02-24T07:56:05.769357+0000","last_change":"2026-02-24T07:56:02.374241+0000","last_active":"2026-02-24T07:56:05.769357+0000","last_peered":"2026-02-24T07:56:05.769357+0000","last_clean":"2026-02-24T07:56:05.769357+0000","last_became_active":"2026-02-24T07:54:42.927730+0000","last_became_peered":"2026-02-24T07:54:42.927730+0000","last_unstale":"2026-02-24T07:56:05.769357+0000","last_undegraded":"2026-02-24T07:56:05.769357+0000","last_fullsized":"2026-02-24T07:56:05.769357+0000","mapping_epoch":90,"log_start":"96'36479","ondisk_log_start":"96'36479","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_clean_scrub_stamp":"2026-02-24T07:54:41.918128+0000","objects_scrubbed":31799,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"scrubbing for 4s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2390884352,"num_objects":36482,"num_object_clones":0,"num_object_copies":109446,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36482,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":36482,"num_write_kb":2334848,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.2","version":"96'36450","reported_seq":36819,"reported_epoch":96,"state":"active+clean","last_fresh":"2026-02-24T07:56:05.790911+0000","last_change":"2026-02-24T07:56:00.120637+0000","last_active":"2026-02-24T07:56:05.790911+0000","last_peered":"2026-02-24T07:56:05.790911+0000","last_clean":"2026-02-24T07:56:05.790911+0000","last_became_active":"2026-02-24T07:54:42.930247+0000","last_became_peered":"2026-02-24T07:54:42.930247+0000","last_unstale":"2026-02-24T07:56:05.790911+0000","last_undegraded":"2026-02-24T07:56:05.790911+0000","last_fullsized":"2026-02-24T07:56:05.790911+0000","mapping_epoch":90,"log_start":"96'36447","ondisk_log_start":"96'36447","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"96'34210","last_scrub_stamp":"2026-02-24T07:56:00.120605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_clean_scrub_stamp":"2026-02-24T07:56:00.120605+0000","objects_scrubbed":33656,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:57:29.329949+0000","scrub_duration":2957,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2388787200,"num_objects":36450,"num_object_clones":0,"num_object_copies":109350,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36450,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":36450,"num_write_kb":2332800,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1","version":"96'36587","reported_seq":36609,"reported_epoch":96,"state":"active+clean","last_fresh":"2026-02-24T07:56:05.870344+0000","last_change":"2026-02-24T07:54:42.926619+0000","last_active":"2026-02-24T07:56:05.870344+0000","last_peered":"2026-02-24T07:56:05.870344+0000","last_clean":"2026-02-24T07:56:05.870344+0000","last_became_active":"2026-02-24T07:54:42.926442+0000","last_became_peered":"2026-02-24T07:54:42.926442+0000","last_unstale":"2026-02-24T07:56:05.870344+0000","last_undegraded":"2026-02-24T07:56:05.870344+0000","last_fullsized":"2026-02-24T07:56:05.870344+0000","mapping_epoch":90,"log_start":"96'36584","ondisk_log_start":"96'36584","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_clean_scrub_stamp":"2026-02-24T07:54:41.918128+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2397765632,"num_objects":36587,"num_object_clones":0,"num_object_copies":109761,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36587,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":36587,"num_write_kb":2341568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.0","version":"96'36056","reported_seq":36432,"reported_epoch":96,"state":"active+clean","last_fresh":"2026-02-24T07:56:05.807137+0000","last_change":"2026-02-24T07:56:02.992243+0000","last_active":"2026-02-24T07:56:05.807137+0000","last_peered":"2026-02-24T07:56:05.807137+0000","last_clean":"2026-02-24T07:56:05.807137+0000","last_became_active":"2026-02-24T07:54:42.926910+0000","last_became_peered":"2026-02-24T07:54:42.926910+0000","last_unstale":"2026-02-24T07:56:05.807137+0000","last_undegraded":"2026-02-24T07:56:05.807137+0000","last_fullsized":"2026-02-24T07:56:05.807137+0000","mapping_epoch":90,"log_start":"96'36053","ondisk_log_start":"96'36053","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"96'34952","last_scrub_stamp":"2026-02-24T07:56:02.992156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_clean_scrub_stamp":"2026-02-24T07:56:02.992156+0000","objects_scrubbed":34329,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:57:03.847719+0000","scrub_duration":3335,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2362966016,"num_objects":36056,"num_object_clones":0,"num_object_copies":108168,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36056,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":36056,"num_write_kb":2307584,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.7","version":"96'36680","reported_seq":37055,"reported_epoch":96,"state":"active+clean","last_fresh":"2026-02-24T07:56:05.789450+0000","last_change":"2026-02-24T07:56:01.088786+0000","last_active":"2026-02-24T07:56:05.789450+0000","last_peered":"2026-02-24T07:56:05.789450+0000","last_clean":"2026-02-24T07:56:05.789450+0000","last_became_active":"2026-02-24T07:54:42.930142+0000","last_became_peered":"2026-02-24T07:54:42.930142+0000","last_unstale":"2026-02-24T07:56:05.789450+0000","last_undegraded":"2026-02-24T07:56:05.789450+0000","last_fullsized":"2026-02-24T07:56:05.789450+0000","mapping_epoch":90,"log_start":"96'36677","ondisk_log_start":"96'36677","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"96'34838","last_scrub_stamp":"2026-02-24T07:56:01.088744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_clean_scrub_stamp":"2026-02-24T07:56:01.088744+0000","objects_scrubbed":34284,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:57:24.696242+0000","scrub_duration":2973,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2403860480,"num_objects":36680,"num_object_clones":0,"num_object_copies":110040,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36680,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":36680,"num_write_kb":2347520,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"29'99","reported_seq":398,"reported_epoch":96,"state":"active+clean","last_fresh":"2026-02-24T07:55:16.398160+0000","last_change":"2026-02-24T07:55:16.398160+0000","last_active":"2026-02-24T07:55:16.398160+0000","last_peered":"2026-02-24T07:55:16.398160+0000","last_clean":"2026-02-24T07:55:16.398160+0000","last_became_active":"2026-02-24T07:42:54.186719+0000","last_became_peered":"2026-02-24T07:42:54.186719+0000","last_unstale":"2026-02-24T07:55:16.398160+0000","last_undegraded":"2026-02-24T07:55:16.398160+0000","last_fullsized":"2026-02-24T07:55:16.398160+0000","mapping_epoch":27,"log_start":"29'96","ondisk_log_start":"29'96","created":20,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'99","last_scrub_stamp":"2026-02-24T07:55:16.398131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:42:43.640776+0000","last_clean_scrub_stamp":"2026-02-24T07:55:16.398131+0000","objects_scrubbed":2,"log_size":3,"log_dups_size":96,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:56:32.925996+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":240,"num_write":135,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":16,"stat_sum":{"num_bytes":38168821760,"num_objects":582410,"num_object_clones":0,"num_object_copies":1747230,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":582410,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":582410,"num_write_kb":37274240,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114504302592,"data_stored":114504302592,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":48,"ondisk_log_size":48,"up":48,"acting":48,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":240,"num_write":135,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":2396,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":3,"up_from":25,"seq":107374182561,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":24700052,"kb_used_data":23095744,"kb_used_omap":18,"kb_used_meta":1604269,"kb_avail":709303148,"statfs":{"total":751619276800,"available":726326423552,"internally_reserved":0,"allocated":23650041856,"data_stored":23649539238,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":18972,"internal_metadata":1642771940},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":19,"seq":81604378787,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":29709332,"kb_used_data":27805696,"kb_used_omap":18,"kb_used_meta":1903597,"kb_avail":704293868,"statfs":{"total":751619276800,"available":721196920832,"internally_reserved":0,"allocated":28473032704,"data_stored":28472530086,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":18652,"internal_metadata":1949284132},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542308,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":34587664,"kb_used_data":32521020,"kb_used_omap":25,"kb_used_meta":2066598,"kb_avail":699415536,"statfs":{"total":751619276800,"available":716201508864,"internally_reserved":0,"allocated":33301524480,"data_stored":33301025414,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":25873,"internal_metadata":2116197103},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705831,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":29784980,"kb_used_data":27925632,"kb_used_omap":16,"kb_used_meta":1859311,"kb_avail":704218220,"statfs":{"total":751619276800,"available":721119457280,"internally_reserved":0,"allocated":28595847168,"data_stored":28595344550,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":17342,"internal_metadata":1903934530},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":2396,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28624879616,"data_stored":28624879616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":33418051584,"data_stored":33418051584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":28589883392,"data_stored":28589883392,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":23871488000,"data_stored":23871488000,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-24T07:56:08.744 INFO:tasks.cephadm.ceph_manager.ceph:increase pool size by 10 2026-02-24T07:56:08.745 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph osd pool set unique_pool_4 pg_num 66 2026-02-24T07:56:08.894 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:56:09.022 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 85 16 602786 602770 443.178 351.062 0.00759929 0.00225518 2026-02-24T07:56:09.022 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:56:09.021+0000 7f1a36749640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:56:09.022 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:56:09.021+0000 7f1a36749640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:56:09.168 INFO:teuthology.orchestra.run.trial035.stderr:Error EPERM: crimson_allow_pg_split is false; pg_num increase denied for crimson pool 2026-02-24T07:56:09.210 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-24T07:56:09.454 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:56:09 trial035 ceph-mon[39284]: 6.1 scrub starts 2026-02-24T07:56:09.454 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:56:09 trial035 ceph-mon[39284]: from='client.15541 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2026-02-24T07:56:09.454 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:56:09 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/1116064345' entity='client.admin' cmd={"prefix": "osd pool set", "pool": "unique_pool_4", "var": "pg_num", "val": "66"} : dispatch 2026-02-24T07:56:10.022 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 86 16 608646 608630 442.283 366.25 0.00204787 0.00225971 2026-02-24T07:56:10.416 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2026-02-24T07:56:10.557 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:56:10.676 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:56:10 trial035 ceph-mon[39284]: pgmap v465: 17 pgs: 1 active+clean+scrubbing, 16 active+clean; 36 GiB data, 113 GiB used, 2.6 TiB / 2.7 TiB avail; 322 MiB/s wr, 5.16k op/s 2026-02-24T07:56:10.865 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:56:10.863+0000 7f17d76b8640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:56:10.865 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:56:10.863+0000 7f17d76b8640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:56:10.880 INFO:teuthology.orchestra.run.trial035.stdout:osd.0: {} 2026-02-24T07:56:10.880 INFO:teuthology.orchestra.run.trial035.stderr:osd.0: osd_enable_op_tracker = '' 2026-02-24T07:56:10.895 INFO:teuthology.orchestra.run.trial035.stdout:osd.1: {} 2026-02-24T07:56:10.895 INFO:teuthology.orchestra.run.trial035.stderr:osd.1: osd_enable_op_tracker = '' 2026-02-24T07:56:10.908 INFO:teuthology.orchestra.run.trial035.stdout:osd.2: {} 2026-02-24T07:56:10.908 INFO:teuthology.orchestra.run.trial035.stderr:osd.2: osd_enable_op_tracker = '' 2026-02-24T07:56:10.923 INFO:teuthology.orchestra.run.trial035.stdout:osd.3: {} 2026-02-24T07:56:10.924 INFO:teuthology.orchestra.run.trial035.stderr:osd.3: osd_enable_op_tracker = '' 2026-02-24T07:56:11.022 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 87 16 614023 614007 441.062 336.062 0.00148845 0.00226598 2026-02-24T07:56:11.704 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:56:11 trial035 ceph-mon[39284]: from='mgr.14284 10.20.193.35:0/3681373222' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2026-02-24T07:56:11.705 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:56:11 trial035 ceph-mon[39284]: from='mgr.14284 10.20.193.35:0/3681373222' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-24T07:56:11.705 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:56:11 trial035 ceph-mon[39284]: from='mgr.14284 10.20.193.35:0/3681373222' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2026-02-24T07:56:11.705 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:56:11 trial035 ceph-mon[39284]: from='mgr.14284 10.20.193.35:0/3681373222' entity='mgr.x' 2026-02-24T07:56:11.705 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:56:11 trial035 ceph-mon[39284]: from='mgr.14284 10.20.193.35:0/3681373222' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2026-02-24T07:56:11.705 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:56:11 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/1808774414' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2026-02-24T07:56:12.022 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 88 16 619588 619572 440.002 347.812 0.00236671 0.00227144 2026-02-24T07:56:12.704 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:56:12 trial035 ceph-mon[39284]: 6.b scrub starts 2026-02-24T07:56:12.704 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:56:12 trial035 ceph-mon[39284]: pgmap v466: 17 pgs: 2 active+clean+scrubbing, 15 active+clean; 37 GiB data, 118 GiB used, 2.6 TiB / 2.7 TiB avail; 481 MiB/s wr, 7.70k op/s 2026-02-24T07:56:12.971 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2026-02-24T07:56:13.022 INFO:tasks.radosbench.radosbench.0.trial035.stdout: 89 16 625772 625756 439.4 386.5 0.00174172 0.00227455 2026-02-24T07:56:13.129 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:56:13.257 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:56:13.255+0000 7f6b81428640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:56:13.257 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:56:13.255+0000 7f6b81428640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:56:13.275 INFO:teuthology.orchestra.run.trial035.stdout:osd.0: {} 2026-02-24T07:56:13.275 INFO:teuthology.orchestra.run.trial035.stderr:osd.0: osd_enable_op_tracker = '' 2026-02-24T07:56:13.291 INFO:teuthology.orchestra.run.trial035.stdout:osd.1: {} 2026-02-24T07:56:13.291 INFO:teuthology.orchestra.run.trial035.stderr:osd.1: osd_enable_op_tracker = '' 2026-02-24T07:56:13.304 INFO:teuthology.orchestra.run.trial035.stdout:osd.2: {} 2026-02-24T07:56:13.305 INFO:teuthology.orchestra.run.trial035.stderr:osd.2: osd_enable_op_tracker = '' 2026-02-24T07:56:13.318 INFO:teuthology.orchestra.run.trial035.stdout:osd.3: {} 2026-02-24T07:56:13.318 INFO:teuthology.orchestra.run.trial035.stderr:osd.3: osd_enable_op_tracker = '' 2026-02-24T07:56:13.704 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:56:13 trial035 ceph-mon[39284]: 6.f scrub ok 2026-02-24T07:56:13.704 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:56:13 trial035 ceph-mon[39284]: 6.1 scrub ok 2026-02-24T07:56:13.704 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:56:13 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/930317531' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2026-02-24T07:56:14.022 INFO:tasks.radosbench.radosbench.0.trial035.stdout:Total time run: 90.0036 2026-02-24T07:56:14.022 INFO:tasks.radosbench.radosbench.0.trial035.stdout:Total writes made: 632675 2026-02-24T07:56:14.022 INFO:tasks.radosbench.radosbench.0.trial035.stdout:Write size: 65536 2026-02-24T07:56:14.022 INFO:tasks.radosbench.radosbench.0.trial035.stdout:Object size: 65536 2026-02-24T07:56:14.022 INFO:tasks.radosbench.radosbench.0.trial035.stdout:Bandwidth (MB/sec): 439.34 2026-02-24T07:56:14.022 INFO:tasks.radosbench.radosbench.0.trial035.stdout:Stddev Bandwidth: 61.4571 2026-02-24T07:56:14.022 INFO:tasks.radosbench.radosbench.0.trial035.stdout:Max bandwidth (MB/sec): 534.875 2026-02-24T07:56:14.022 INFO:tasks.radosbench.radosbench.0.trial035.stdout:Min bandwidth (MB/sec): 229.125 2026-02-24T07:56:14.022 INFO:tasks.radosbench.radosbench.0.trial035.stdout:Average IOPS: 7029 2026-02-24T07:56:14.022 INFO:tasks.radosbench.radosbench.0.trial035.stdout:Stddev IOPS: 983.313 2026-02-24T07:56:14.022 INFO:tasks.radosbench.radosbench.0.trial035.stdout:Max IOPS: 8558 2026-02-24T07:56:14.022 INFO:tasks.radosbench.radosbench.0.trial035.stdout:Min IOPS: 3666 2026-02-24T07:56:14.022 INFO:tasks.radosbench.radosbench.0.trial035.stdout:Average Latency(s): 0.00227485 2026-02-24T07:56:14.022 INFO:tasks.radosbench.radosbench.0.trial035.stdout:Stddev Latency(s): 0.00211835 2026-02-24T07:56:14.022 INFO:tasks.radosbench.radosbench.0.trial035.stdout:Max latency(s): 0.0486567 2026-02-24T07:56:14.022 INFO:tasks.radosbench.radosbench.0.trial035.stdout:Min latency(s): 0.000714289 2026-02-24T07:56:14.023 INFO:tasks.radosbench.radosbench.0.trial035.stdout:Cleaning up (deleting benchmark objects) 2026-02-24T07:56:14.211 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2026-02-24T07:56:14.211 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2026-02-24T07:56:14.211 INFO:tasks.thrashosds.thrasher:Growing pool .mgr 2026-02-24T07:56:14.212 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph pg dump --format=json 2026-02-24T07:56:14.355 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:56:14.573 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:56:14.571+0000 7f75a337d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:56:14.573 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:56:14.572+0000 7f75a337d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:56:14.712 INFO:teuthology.orchestra.run.trial035.stdout: 2026-02-24T07:56:14.713 INFO:teuthology.orchestra.run.trial035.stderr:dumped all 2026-02-24T07:56:14.823 INFO:teuthology.orchestra.run.trial035.stdout:{"pg_ready":true,"pg_map":{"version":467,"stamp":"2026-02-24T07:56:13.765432+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":40167080480,"num_objects":612894,"num_object_clones":0,"num_object_copies":1838682,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":612894,"num_whiteouts":0,"num_read":136,"num_read_kb":240,"num_write":613027,"num_write_kb":39227238,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":52,"ondisk_log_size":52,"up":51,"acting":51,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":51,"num_osds":4,"num_per_pool_osds":4,"num_per_pool_omap_osds":4,"kb":2936012800,"kb_used":123688652,"kb_used_data":117084732,"kb_used_omap":74,"kb_used_meta":6603701,"kb_avail":2812324148,"statfs":{"total":3006477107200,"available":2879819927552,"internally_reserved":0,"allocated":119894765568,"data_stored":119892758648,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":76557,"internal_metadata":6762190067},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":4007329792,"num_objects":61147,"num_object_clones":0,"num_object_copies":183441,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":61147,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":61147,"num_write_kb":3913408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001682"},"pg_stats":[{"pgid":"6.f","version":"96'38299","reported_seq":38588,"reported_epoch":96,"state":"active+clean+scrubbing","last_fresh":"2026-02-24T07:56:10.890647+0000","last_change":"2026-02-24T07:56:07.577484+0000","last_active":"2026-02-24T07:56:10.890647+0000","last_peered":"2026-02-24T07:56:10.890647+0000","last_clean":"2026-02-24T07:56:10.890647+0000","last_became_active":"2026-02-24T07:54:42.927080+0000","last_became_peered":"2026-02-24T07:54:42.927080+0000","last_unstale":"2026-02-24T07:56:10.890647+0000","last_undegraded":"2026-02-24T07:56:10.890647+0000","last_fullsized":"2026-02-24T07:56:10.890647+0000","mapping_epoch":90,"log_start":"96'38296","ondisk_log_start":"96'38296","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_clean_scrub_stamp":"2026-02-24T07:54:41.918128+0000","objects_scrubbed":26299,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"scrubbing for 4s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2509963264,"num_objects":38299,"num_object_clones":0,"num_object_copies":114897,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38299,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":38299,"num_write_kb":2451136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.e","version":"96'38363","reported_seq":38748,"reported_epoch":96,"state":"active+clean","last_fresh":"2026-02-24T07:56:10.774757+0000","last_change":"2026-02-24T07:56:05.374565+0000","last_active":"2026-02-24T07:56:10.774757+0000","last_peered":"2026-02-24T07:56:10.774757+0000","last_clean":"2026-02-24T07:56:10.774757+0000","last_became_active":"2026-02-24T07:54:42.928748+0000","last_became_peered":"2026-02-24T07:54:42.928748+0000","last_unstale":"2026-02-24T07:56:10.774757+0000","last_undegraded":"2026-02-24T07:56:10.774757+0000","last_fullsized":"2026-02-24T07:56:10.774757+0000","mapping_epoch":90,"log_start":"96'38360","ondisk_log_start":"96'38360","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"96'36307","last_scrub_stamp":"2026-02-24T07:56:05.374510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_clean_scrub_stamp":"2026-02-24T07:56:05.374510+0000","objects_scrubbed":35491,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:57:16.427813+0000","scrub_duration":4036,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2514157568,"num_objects":38363,"num_object_clones":0,"num_object_copies":115089,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38363,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":38363,"num_write_kb":2455232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.d","version":"96'38258","reported_seq":38575,"reported_epoch":96,"state":"active+clean","last_fresh":"2026-02-24T07:56:10.776368+0000","last_change":"2026-02-24T07:55:48.151749+0000","last_active":"2026-02-24T07:56:10.776368+0000","last_peered":"2026-02-24T07:56:10.776368+0000","last_clean":"2026-02-24T07:56:10.776368+0000","last_became_active":"2026-02-24T07:54:42.926886+0000","last_became_peered":"2026-02-24T07:54:42.926886+0000","last_unstale":"2026-02-24T07:56:10.776368+0000","last_undegraded":"2026-02-24T07:56:10.776368+0000","last_fullsized":"2026-02-24T07:56:10.776368+0000","mapping_epoch":90,"log_start":"96'38255","ondisk_log_start":"96'38255","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"96'28924","last_scrub_stamp":"2026-02-24T07:55:48.151672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_clean_scrub_stamp":"2026-02-24T07:55:48.151672+0000","objects_scrubbed":28526,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:57:00.390677+0000","scrub_duration":2742,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2507276288,"num_objects":38258,"num_object_clones":0,"num_object_copies":114774,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38258,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":38258,"num_write_kb":2448512,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.c","version":"96'38404","reported_seq":38426,"reported_epoch":96,"state":"active+clean","last_fresh":"2026-02-24T07:56:10.883503+0000","last_change":"2026-02-24T07:54:42.928482+0000","last_active":"2026-02-24T07:56:10.883503+0000","last_peered":"2026-02-24T07:56:10.883503+0000","last_clean":"2026-02-24T07:56:10.883503+0000","last_became_active":"2026-02-24T07:54:42.928238+0000","last_became_peered":"2026-02-24T07:54:42.928238+0000","last_unstale":"2026-02-24T07:56:10.883503+0000","last_undegraded":"2026-02-24T07:56:10.883503+0000","last_fullsized":"2026-02-24T07:56:10.883503+0000","mapping_epoch":90,"log_start":"96'38401","ondisk_log_start":"96'38401","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_clean_scrub_stamp":"2026-02-24T07:54:41.918128+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2516844544,"num_objects":38404,"num_object_clones":0,"num_object_copies":115212,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38404,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":38404,"num_write_kb":2457856,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.b","version":"96'37998","reported_seq":38020,"reported_epoch":96,"state":"active+clean","last_fresh":"2026-02-24T07:56:10.887454+0000","last_change":"2026-02-24T07:54:42.927223+0000","last_active":"2026-02-24T07:56:10.887454+0000","last_peered":"2026-02-24T07:56:10.887454+0000","last_clean":"2026-02-24T07:56:10.887454+0000","last_became_active":"2026-02-24T07:54:42.926965+0000","last_became_peered":"2026-02-24T07:54:42.926965+0000","last_unstale":"2026-02-24T07:56:10.887454+0000","last_undegraded":"2026-02-24T07:56:10.887454+0000","last_fullsized":"2026-02-24T07:56:10.887454+0000","mapping_epoch":90,"log_start":"96'37995","ondisk_log_start":"96'37995","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_clean_scrub_stamp":"2026-02-24T07:54:41.918128+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2490236928,"num_objects":37998,"num_object_clones":0,"num_object_copies":113994,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37998,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":37998,"num_write_kb":2431872,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.a","version":"96'38200","reported_seq":38519,"reported_epoch":96,"state":"active+clean","last_fresh":"2026-02-24T07:56:10.772489+0000","last_change":"2026-02-24T07:55:49.167491+0000","last_active":"2026-02-24T07:56:10.772489+0000","last_peered":"2026-02-24T07:56:10.772489+0000","last_clean":"2026-02-24T07:56:10.772489+0000","last_became_active":"2026-02-24T07:54:42.926477+0000","last_became_peered":"2026-02-24T07:54:42.926477+0000","last_unstale":"2026-02-24T07:56:10.772489+0000","last_undegraded":"2026-02-24T07:56:10.772489+0000","last_fullsized":"2026-02-24T07:56:10.772489+0000","mapping_epoch":90,"log_start":"96'38197","ondisk_log_start":"96'38197","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"96'29334","last_scrub_stamp":"2026-02-24T07:55:49.167461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_clean_scrub_stamp":"2026-02-24T07:55:49.167461+0000","objects_scrubbed":28822,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:56:59.424140+0000","scrub_duration":2726,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2503475200,"num_objects":38200,"num_object_clones":0,"num_object_copies":114600,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38200,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":38200,"num_write_kb":2444800,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"96'38426","reported_seq":38744,"reported_epoch":96,"state":"active+clean","last_fresh":"2026-02-24T07:56:10.805525+0000","last_change":"2026-02-24T07:55:48.535755+0000","last_active":"2026-02-24T07:56:10.805525+0000","last_peered":"2026-02-24T07:56:10.805525+0000","last_clean":"2026-02-24T07:56:10.805525+0000","last_became_active":"2026-02-24T07:54:42.926136+0000","last_became_peered":"2026-02-24T07:54:42.926136+0000","last_unstale":"2026-02-24T07:56:10.805525+0000","last_undegraded":"2026-02-24T07:56:10.805525+0000","last_fullsized":"2026-02-24T07:56:10.805525+0000","mapping_epoch":90,"log_start":"96'38423","ondisk_log_start":"96'38423","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"96'29148","last_scrub_stamp":"2026-02-24T07:55:48.535691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_clean_scrub_stamp":"2026-02-24T07:55:48.535691+0000","objects_scrubbed":28720,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:56:56.161155+0000","scrub_duration":2872,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2518286336,"num_objects":38426,"num_object_clones":0,"num_object_copies":115278,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38426,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":38426,"num_write_kb":2459264,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,1],"acting":[0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.8","version":"96'38460","reported_seq":38821,"reported_epoch":96,"state":"active+clean","last_fresh":"2026-02-24T07:56:10.792059+0000","last_change":"2026-02-24T07:55:58.178460+0000","last_active":"2026-02-24T07:56:10.792059+0000","last_peered":"2026-02-24T07:56:10.792059+0000","last_clean":"2026-02-24T07:56:10.792059+0000","last_became_active":"2026-02-24T07:54:42.929078+0000","last_became_peered":"2026-02-24T07:54:42.929078+0000","last_unstale":"2026-02-24T07:56:10.792059+0000","last_undegraded":"2026-02-24T07:56:10.792059+0000","last_fullsized":"2026-02-24T07:56:10.792059+0000","mapping_epoch":90,"log_start":"96'38457","ondisk_log_start":"96'38457","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"96'33550","last_scrub_stamp":"2026-02-24T07:55:58.178395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_clean_scrub_stamp":"2026-02-24T07:55:58.178395+0000","objects_scrubbed":32948,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:57:14.607073+0000","scrub_duration":2981,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2520514560,"num_objects":38460,"num_object_clones":0,"num_object_copies":115380,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38460,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":38460,"num_write_kb":2461440,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.6","version":"96'38212","reported_seq":38234,"reported_epoch":96,"state":"active+clean","last_fresh":"2026-02-24T07:56:10.876401+0000","last_change":"2026-02-24T07:54:42.926358+0000","last_active":"2026-02-24T07:56:10.876401+0000","last_peered":"2026-02-24T07:56:10.876401+0000","last_clean":"2026-02-24T07:56:10.876401+0000","last_became_active":"2026-02-24T07:54:42.926045+0000","last_became_peered":"2026-02-24T07:54:42.926045+0000","last_unstale":"2026-02-24T07:56:10.876401+0000","last_undegraded":"2026-02-24T07:56:10.876401+0000","last_fullsized":"2026-02-24T07:56:10.876401+0000","mapping_epoch":90,"log_start":"96'38209","ondisk_log_start":"96'38209","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_clean_scrub_stamp":"2026-02-24T07:54:41.918128+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2504261632,"num_objects":38212,"num_object_clones":0,"num_object_copies":114636,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38212,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":38212,"num_write_kb":2445568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.5","version":"96'38246","reported_seq":38611,"reported_epoch":96,"state":"active+clean","last_fresh":"2026-02-24T07:56:10.776393+0000","last_change":"2026-02-24T07:55:59.773616+0000","last_active":"2026-02-24T07:56:10.776393+0000","last_peered":"2026-02-24T07:56:10.776393+0000","last_clean":"2026-02-24T07:56:10.776393+0000","last_became_active":"2026-02-24T07:54:42.927535+0000","last_became_peered":"2026-02-24T07:54:42.927535+0000","last_unstale":"2026-02-24T07:56:10.776393+0000","last_undegraded":"2026-02-24T07:56:10.776393+0000","last_fullsized":"2026-02-24T07:56:10.776393+0000","mapping_epoch":90,"log_start":"96'38243","ondisk_log_start":"96'38243","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"96'34080","last_scrub_stamp":"2026-02-24T07:55:59.773556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_clean_scrub_stamp":"2026-02-24T07:55:59.773556+0000","objects_scrubbed":33431,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:57:04.302607+0000","scrub_duration":3324,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2506489856,"num_objects":38246,"num_object_clones":0,"num_object_copies":114738,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38246,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":38246,"num_write_kb":2447744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.4","version":"96'38294","reported_seq":38612,"reported_epoch":96,"state":"active+clean","last_fresh":"2026-02-24T07:56:10.883353+0000","last_change":"2026-02-24T07:55:47.544043+0000","last_active":"2026-02-24T07:56:10.883353+0000","last_peered":"2026-02-24T07:56:10.883353+0000","last_clean":"2026-02-24T07:56:10.883353+0000","last_became_active":"2026-02-24T07:54:42.928011+0000","last_became_peered":"2026-02-24T07:54:42.928011+0000","last_unstale":"2026-02-24T07:56:10.883353+0000","last_undegraded":"2026-02-24T07:56:10.883353+0000","last_fullsized":"2026-02-24T07:56:10.883353+0000","mapping_epoch":90,"log_start":"96'38291","ondisk_log_start":"96'38291","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"96'29055","last_scrub_stamp":"2026-02-24T07:55:47.543994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_clean_scrub_stamp":"2026-02-24T07:55:47.543994+0000","objects_scrubbed":28510,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:57:04.321401+0000","scrub_duration":3165,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2509635584,"num_objects":38294,"num_object_clones":0,"num_object_copies":114882,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38294,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":38294,"num_write_kb":2450816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.3","version":"96'38434","reported_seq":38826,"reported_epoch":96,"state":"active+clean","last_fresh":"2026-02-24T07:56:10.773641+0000","last_change":"2026-02-24T07:56:06.184345+0000","last_active":"2026-02-24T07:56:10.773641+0000","last_peered":"2026-02-24T07:56:10.773641+0000","last_clean":"2026-02-24T07:56:10.773641+0000","last_became_active":"2026-02-24T07:54:42.927730+0000","last_became_peered":"2026-02-24T07:54:42.927730+0000","last_unstale":"2026-02-24T07:56:10.773641+0000","last_undegraded":"2026-02-24T07:56:10.773641+0000","last_fullsized":"2026-02-24T07:56:10.773641+0000","mapping_epoch":90,"log_start":"96'38431","ondisk_log_start":"96'38431","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"96'36664","last_scrub_stamp":"2026-02-24T07:56:06.184309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_clean_scrub_stamp":"2026-02-24T07:56:06.184309+0000","objects_scrubbed":35946,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:57:11.733048+0000","scrub_duration":3811,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2518810624,"num_objects":38434,"num_object_clones":0,"num_object_copies":115302,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38434,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":38434,"num_write_kb":2459776,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.2","version":"96'38335","reported_seq":38704,"reported_epoch":96,"state":"active+clean","last_fresh":"2026-02-24T07:56:10.786392+0000","last_change":"2026-02-24T07:56:00.120637+0000","last_active":"2026-02-24T07:56:10.786392+0000","last_peered":"2026-02-24T07:56:10.786392+0000","last_clean":"2026-02-24T07:56:10.786392+0000","last_became_active":"2026-02-24T07:54:42.930247+0000","last_became_peered":"2026-02-24T07:54:42.930247+0000","last_unstale":"2026-02-24T07:56:10.786392+0000","last_undegraded":"2026-02-24T07:56:10.786392+0000","last_fullsized":"2026-02-24T07:56:10.786392+0000","mapping_epoch":90,"log_start":"96'38332","ondisk_log_start":"96'38332","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"96'34210","last_scrub_stamp":"2026-02-24T07:56:00.120605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_clean_scrub_stamp":"2026-02-24T07:56:00.120605+0000","objects_scrubbed":33656,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:57:29.329949+0000","scrub_duration":2957,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2512322560,"num_objects":38335,"num_object_clones":0,"num_object_copies":115005,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38335,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":38335,"num_write_kb":2453440,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1","version":"96'38535","reported_seq":38753,"reported_epoch":96,"state":"active+clean+scrubbing","last_fresh":"2026-02-24T07:56:10.889443+0000","last_change":"2026-02-24T07:56:08.618530+0000","last_active":"2026-02-24T07:56:10.889443+0000","last_peered":"2026-02-24T07:56:10.889443+0000","last_clean":"2026-02-24T07:56:10.889443+0000","last_became_active":"2026-02-24T07:54:42.926442+0000","last_became_peered":"2026-02-24T07:54:42.926442+0000","last_unstale":"2026-02-24T07:56:10.889443+0000","last_undegraded":"2026-02-24T07:56:10.889443+0000","last_fullsized":"2026-02-24T07:56:10.889443+0000","mapping_epoch":90,"log_start":"96'38531","ondisk_log_start":"96'38531","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_clean_scrub_stamp":"2026-02-24T07:54:41.918128+0000","objects_scrubbed":19199,"log_size":4,"log_dups_size":3000,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"scrubbing for 3s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2525429760,"num_objects":38535,"num_object_clones":0,"num_object_copies":115605,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38535,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":38535,"num_write_kb":2466240,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.0","version":"96'37906","reported_seq":38282,"reported_epoch":96,"state":"active+clean","last_fresh":"2026-02-24T07:56:10.805996+0000","last_change":"2026-02-24T07:56:02.992243+0000","last_active":"2026-02-24T07:56:10.805996+0000","last_peered":"2026-02-24T07:56:10.805996+0000","last_clean":"2026-02-24T07:56:10.805996+0000","last_became_active":"2026-02-24T07:54:42.926910+0000","last_became_peered":"2026-02-24T07:54:42.926910+0000","last_unstale":"2026-02-24T07:56:10.805996+0000","last_undegraded":"2026-02-24T07:56:10.805996+0000","last_fullsized":"2026-02-24T07:56:10.805996+0000","mapping_epoch":90,"log_start":"96'37903","ondisk_log_start":"96'37903","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"96'34952","last_scrub_stamp":"2026-02-24T07:56:02.992156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_clean_scrub_stamp":"2026-02-24T07:56:02.992156+0000","objects_scrubbed":34329,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:57:03.847719+0000","scrub_duration":3335,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2484207616,"num_objects":37906,"num_object_clones":0,"num_object_copies":113718,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37906,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":37906,"num_write_kb":2425984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.7","version":"96'38522","reported_seq":38897,"reported_epoch":96,"state":"active+clean","last_fresh":"2026-02-24T07:56:10.787188+0000","last_change":"2026-02-24T07:56:01.088786+0000","last_active":"2026-02-24T07:56:10.787188+0000","last_peered":"2026-02-24T07:56:10.787188+0000","last_clean":"2026-02-24T07:56:10.787188+0000","last_became_active":"2026-02-24T07:54:42.930142+0000","last_became_peered":"2026-02-24T07:54:42.930142+0000","last_unstale":"2026-02-24T07:56:10.787188+0000","last_undegraded":"2026-02-24T07:56:10.787188+0000","last_fullsized":"2026-02-24T07:56:10.787188+0000","mapping_epoch":90,"log_start":"96'38519","ondisk_log_start":"96'38519","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"96'34838","last_scrub_stamp":"2026-02-24T07:56:01.088744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_clean_scrub_stamp":"2026-02-24T07:56:01.088744+0000","objects_scrubbed":34284,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:57:24.696242+0000","scrub_duration":2973,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2524577792,"num_objects":38522,"num_object_clones":0,"num_object_copies":115566,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38522,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":38522,"num_write_kb":2465408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"29'99","reported_seq":398,"reported_epoch":96,"state":"active+clean","last_fresh":"2026-02-24T07:55:16.398160+0000","last_change":"2026-02-24T07:55:16.398160+0000","last_active":"2026-02-24T07:55:16.398160+0000","last_peered":"2026-02-24T07:55:16.398160+0000","last_clean":"2026-02-24T07:55:16.398160+0000","last_became_active":"2026-02-24T07:42:54.186719+0000","last_became_peered":"2026-02-24T07:42:54.186719+0000","last_unstale":"2026-02-24T07:55:16.398160+0000","last_undegraded":"2026-02-24T07:55:16.398160+0000","last_fullsized":"2026-02-24T07:55:16.398160+0000","mapping_epoch":27,"log_start":"29'96","ondisk_log_start":"29'96","created":20,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'99","last_scrub_stamp":"2026-02-24T07:55:16.398131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:42:43.640776+0000","last_clean_scrub_stamp":"2026-02-24T07:55:16.398131+0000","objects_scrubbed":2,"log_size":3,"log_dups_size":96,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:56:32.925996+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":240,"num_write":135,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":16,"stat_sum":{"num_bytes":40166490112,"num_objects":612892,"num_object_clones":0,"num_object_copies":1838676,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":612892,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":612892,"num_write_kb":39225088,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":120495538176,"data_stored":120495538176,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":49,"ondisk_log_size":49,"up":48,"acting":48,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":240,"num_write":135,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":2396,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":3,"up_from":25,"seq":107374182562,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":26032148,"kb_used_data":24355648,"kb_used_omap":18,"kb_used_meta":1676461,"kb_avail":707971052,"statfs":{"total":751619276800,"available":724962357248,"internally_reserved":0,"allocated":24940183552,"data_stored":24939680934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":18972,"internal_metadata":1716696548},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":19,"seq":81604378788,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":31308500,"kb_used_data":29278464,"kb_used_omap":18,"kb_used_meta":2029933,"kb_avail":702694700,"statfs":{"total":751619276800,"available":719559372800,"internally_reserved":0,"allocated":29981147136,"data_stored":29980644518,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":18652,"internal_metadata":2078652196},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542309,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":35263888,"kb_used_data":34279996,"kb_used_omap":21,"kb_used_meta":983850,"kb_avail":698739312,"statfs":{"total":751619276800,"available":715509055488,"internally_reserved":0,"allocated":35102715904,"data_stored":35102216838,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":21591,"internal_metadata":1007463337},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705832,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":31084116,"kb_used_data":29170624,"kb_used_omap":16,"kb_used_meta":1913455,"kb_avail":702919084,"statfs":{"total":751619276800,"available":719789142016,"internally_reserved":0,"allocated":29870718976,"data_stored":29870216358,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":17342,"internal_metadata":1959377986},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":2396,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":30120280064,"data_stored":30120280064,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":35167207424,"data_stored":35167207424,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":30085939200,"data_stored":30085939200,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":25122111488,"data_stored":25122111488,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-24T07:56:14.824 INFO:tasks.cephadm.ceph_manager.ceph:increase pool size by 10 2026-02-24T07:56:14.824 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph osd pool set .mgr pg_num 421 2026-02-24T07:56:14.964 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:56:15.077 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:56:14 trial035 ceph-mon[39284]: pgmap v467: 17 pgs: 2 active+clean+scrubbing, 15 active+clean; 37 GiB data, 118 GiB used, 2.6 TiB / 2.7 TiB avail; 318 MiB/s wr, 5.09k op/s 2026-02-24T07:56:15.077 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:56:14 trial035 ceph-mon[39284]: from='mgr.14284 10.20.193.35:0/3681373222' entity='mgr.x' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2026-02-24T07:56:15.344 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:56:15.324+0000 7fb9b6d0e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:56:15.344 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:56:15.325+0000 7fb9b6d0e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:56:15.364 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2026-02-24T07:56:15.473 INFO:teuthology.orchestra.run.trial035.stderr:Error EPERM: crimson_allow_pg_split is false; pg_num increase denied for crimson pool 2026-02-24T07:56:15.638 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:56:15.658 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-24T07:56:15.769 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:56:15.768+0000 7fc0d8b54640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:56:15.770 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:56:15.768+0000 7fc0d8b54640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:56:15.783 INFO:teuthology.orchestra.run.trial035.stdout:osd.0: {} 2026-02-24T07:56:15.784 INFO:teuthology.orchestra.run.trial035.stderr:osd.0: osd_enable_op_tracker = '' 2026-02-24T07:56:15.796 INFO:teuthology.orchestra.run.trial035.stdout:osd.1: {} 2026-02-24T07:56:15.796 INFO:teuthology.orchestra.run.trial035.stderr:osd.1: osd_enable_op_tracker = '' 2026-02-24T07:56:15.809 INFO:teuthology.orchestra.run.trial035.stdout:osd.2: {} 2026-02-24T07:56:15.809 INFO:teuthology.orchestra.run.trial035.stderr:osd.2: osd_enable_op_tracker = '' 2026-02-24T07:56:15.821 INFO:teuthology.orchestra.run.trial035.stdout:osd.3: {} 2026-02-24T07:56:15.821 INFO:teuthology.orchestra.run.trial035.stderr:osd.3: osd_enable_op_tracker = '' 2026-02-24T07:56:16.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:56:15 trial035 ceph-mon[39284]: 6.6 scrub starts 2026-02-24T07:56:16.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:56:15 trial035 ceph-mon[39284]: from='client.15549 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2026-02-24T07:56:16.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:56:15 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/1255152109' entity='client.admin' cmd={"prefix": "osd pool set", "pool": ".mgr", "var": "pg_num", "val": "421"} : dispatch 2026-02-24T07:56:16.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:56:15 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/649818083' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2026-02-24T07:56:17.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:56:16 trial035 ceph-mon[39284]: pgmap v468: 17 pgs: 2 active+clean+scrubbing, 15 active+clean; 37 GiB data, 118 GiB used, 2.6 TiB / 2.7 TiB avail; 318 MiB/s wr, 5.09k op/s 2026-02-24T07:56:17.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:56:16 trial035 ceph-mon[39284]: 6.b scrub ok 2026-02-24T07:56:17.204 INFO:journalctl@ceph.mgr.x.trial035.stdout:Feb 24 07:56:16 trial035 ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-mgr-x[39621]: ::ffff:10.20.193.91 - - [24/Feb/2026:07:56:16] "GET /metrics HTTP/1.1" 200 4225 "" "Prometheus/3.6.0" 2026-02-24T07:56:17.265 INFO:journalctl@ceph.prometheus.a.trial091.stdout:Feb 24 07:56:16 trial091 ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-prometheus-a[66007]: time=2026-02-24T07:56:16.974Z level=ERROR source=scrape.go:1630 msg="Failed to determine correct type of scrape target." component="scrape manager" scrape_pool=ceph target=http://trial035:9283/metrics content_type="text/html;charset=utf-8" fallback_media_type="" err="received unsupported Content-Type \"text/html;charset=utf-8\" and no fallback_scrape_protocol specified for target" 2026-02-24T07:56:17.862 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2026-02-24T07:56:18.006 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:56:18.134 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:56:18.133+0000 7fbe96d9c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:56:18.135 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:56:18.134+0000 7fbe96d9c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:56:18.149 INFO:teuthology.orchestra.run.trial035.stdout:osd.0: {} 2026-02-24T07:56:18.149 INFO:teuthology.orchestra.run.trial035.stderr:osd.0: osd_enable_op_tracker = '' 2026-02-24T07:56:18.167 INFO:teuthology.orchestra.run.trial035.stdout:osd.1: {} 2026-02-24T07:56:18.168 INFO:teuthology.orchestra.run.trial035.stderr:osd.1: osd_enable_op_tracker = '' 2026-02-24T07:56:18.180 INFO:teuthology.orchestra.run.trial035.stdout:osd.2: {} 2026-02-24T07:56:18.180 INFO:teuthology.orchestra.run.trial035.stderr:osd.2: osd_enable_op_tracker = '' 2026-02-24T07:56:18.193 INFO:teuthology.orchestra.run.trial035.stdout:osd.3: {} 2026-02-24T07:56:18.193 INFO:teuthology.orchestra.run.trial035.stderr:osd.3: osd_enable_op_tracker = '' 2026-02-24T07:56:19.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:56:18 trial035 ceph-mon[39284]: pgmap v469: 17 pgs: 2 active+clean+scrubbing, 15 active+clean; 38 GiB data, 121 GiB used, 2.6 TiB / 2.7 TiB avail; 85 B/s rd, 421 MiB/s wr, 7.66k op/s 2026-02-24T07:56:19.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:56:18 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/712098290' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2026-02-24T07:56:20.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:56:19 trial035 ceph-mon[39284]: 6.6 scrub ok 2026-02-24T07:56:20.237 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2026-02-24T07:56:20.385 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:56:20.509 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:56:20.506+0000 7fc75e503640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:56:20.509 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:56:20.506+0000 7fc75e503640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:56:20.524 INFO:teuthology.orchestra.run.trial035.stdout:osd.0: {} 2026-02-24T07:56:20.524 INFO:teuthology.orchestra.run.trial035.stderr:osd.0: osd_enable_op_tracker = '' 2026-02-24T07:56:20.539 INFO:teuthology.orchestra.run.trial035.stdout:osd.1: {} 2026-02-24T07:56:20.539 INFO:teuthology.orchestra.run.trial035.stderr:osd.1: osd_enable_op_tracker = '' 2026-02-24T07:56:20.551 INFO:teuthology.orchestra.run.trial035.stdout:osd.2: {} 2026-02-24T07:56:20.552 INFO:teuthology.orchestra.run.trial035.stderr:osd.2: osd_enable_op_tracker = '' 2026-02-24T07:56:20.565 INFO:teuthology.orchestra.run.trial035.stdout:osd.3: {} 2026-02-24T07:56:20.565 INFO:teuthology.orchestra.run.trial035.stderr:osd.3: osd_enable_op_tracker = '' 2026-02-24T07:56:20.659 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2026-02-24T07:56:20.659 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2026-02-24T07:56:20.660 INFO:tasks.thrashosds.thrasher:Growing pool .mgr 2026-02-24T07:56:20.660 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph pg dump --format=json 2026-02-24T07:56:20.795 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:56:20.926 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:56:20.923+0000 7f8186ce5640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:56:20.926 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:56:20.923+0000 7f8186ce5640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:56:21.069 INFO:teuthology.orchestra.run.trial035.stdout: 2026-02-24T07:56:21.070 INFO:teuthology.orchestra.run.trial035.stderr:dumped all 2026-02-24T07:56:21.079 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:56:20 trial035 ceph-mon[39284]: pgmap v470: 17 pgs: 2 active+clean+scrubbing, 15 active+clean; 38 GiB data, 121 GiB used, 2.6 TiB / 2.7 TiB avail; 85 B/s rd, 262 MiB/s wr, 5.11k op/s 2026-02-24T07:56:21.079 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:56:20 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/3919300412' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2026-02-24T07:56:21.113 INFO:teuthology.orchestra.run.trial035.stdout:{"pg_ready":true,"pg_map":{"version":470,"stamp":"2026-02-24T07:56:19.766283+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":40739013176,"num_objects":621622,"num_object_clones":0,"num_object_copies":1864866,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":621622,"num_whiteouts":0,"num_read":137,"num_read_kb":241,"num_write":643867,"num_write_kb":40493351,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":51,"ondisk_log_size":51,"up":51,"acting":51,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":51,"num_osds":4,"num_per_pool_osds":4,"num_per_pool_omap_osds":4,"kb":2936012800,"kb_used":126784024,"kb_used_data":120036808,"kb_used_omap":76,"kb_used_meta":6746995,"kb_avail":2809228776,"statfs":{"total":3006477107200,"available":2876650266624,"internally_reserved":0,"allocated":122917691392,"data_stored":122915672256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":78735,"internal_metadata":6908922993},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,2],"upper_bound":4},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2569601048,"num_objects":39210,"num_object_clones":0,"num_object_copies":117630,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39210,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":61322,"num_write_kb":3216961,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001712"},"pg_stats":[{"pgid":"6.f","version":"96'40242","reported_seq":40653,"reported_epoch":96,"state":"active+clean","last_fresh":"2026-02-24T07:56:15.894194+0000","last_change":"2026-02-24T07:56:12.247365+0000","last_active":"2026-02-24T07:56:15.894194+0000","last_peered":"2026-02-24T07:56:15.894194+0000","last_clean":"2026-02-24T07:56:15.894194+0000","last_became_active":"2026-02-24T07:54:42.927080+0000","last_became_peered":"2026-02-24T07:54:42.927080+0000","last_unstale":"2026-02-24T07:56:15.894194+0000","last_undegraded":"2026-02-24T07:56:15.894194+0000","last_fullsized":"2026-02-24T07:56:15.894194+0000","mapping_epoch":90,"log_start":"96'40239","ondisk_log_start":"96'40239","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"96'38780","last_scrub_stamp":"2026-02-24T07:56:12.247318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_clean_scrub_stamp":"2026-02-24T07:56:12.247318+0000","objects_scrubbed":37979,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:57:38.474117+0000","scrub_duration":4670,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2540961792,"num_objects":38772,"num_object_clones":0,"num_object_copies":116316,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38772,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":40242,"num_write_kb":2528448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.e","version":"96'40211","reported_seq":40595,"reported_epoch":96,"state":"active+clean","last_fresh":"2026-02-24T07:56:15.775789+0000","last_change":"2026-02-24T07:56:05.374565+0000","last_active":"2026-02-24T07:56:15.775789+0000","last_peered":"2026-02-24T07:56:15.775789+0000","last_clean":"2026-02-24T07:56:15.775789+0000","last_became_active":"2026-02-24T07:54:42.928748+0000","last_became_peered":"2026-02-24T07:54:42.928748+0000","last_unstale":"2026-02-24T07:56:15.775789+0000","last_undegraded":"2026-02-24T07:56:15.775789+0000","last_fullsized":"2026-02-24T07:56:15.775789+0000","mapping_epoch":90,"log_start":"96'40208","ondisk_log_start":"96'40208","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"96'36307","last_scrub_stamp":"2026-02-24T07:56:05.374510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_clean_scrub_stamp":"2026-02-24T07:56:05.374510+0000","objects_scrubbed":35491,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:57:16.427813+0000","scrub_duration":4036,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2549809152,"num_objects":38907,"num_object_clones":0,"num_object_copies":116721,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38907,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":40211,"num_write_kb":2531776,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.d","version":"96'40229","reported_seq":40546,"reported_epoch":96,"state":"active+clean","last_fresh":"2026-02-24T07:56:15.777129+0000","last_change":"2026-02-24T07:55:48.151749+0000","last_active":"2026-02-24T07:56:15.777129+0000","last_peered":"2026-02-24T07:56:15.777129+0000","last_clean":"2026-02-24T07:56:15.777129+0000","last_became_active":"2026-02-24T07:54:42.926886+0000","last_became_peered":"2026-02-24T07:54:42.926886+0000","last_unstale":"2026-02-24T07:56:15.777129+0000","last_undegraded":"2026-02-24T07:56:15.777129+0000","last_fullsized":"2026-02-24T07:56:15.777129+0000","mapping_epoch":90,"log_start":"96'40226","ondisk_log_start":"96'40226","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"96'28924","last_scrub_stamp":"2026-02-24T07:55:48.151672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_clean_scrub_stamp":"2026-02-24T07:55:48.151672+0000","objects_scrubbed":28526,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:57:00.390677+0000","scrub_duration":2742,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2548432920,"num_objects":38887,"num_object_clones":0,"num_object_copies":116661,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38887,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":40229,"num_write_kb":2531649,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.c","version":"96'40281","reported_seq":40303,"reported_epoch":96,"state":"active+clean","last_fresh":"2026-02-24T07:56:15.891320+0000","last_change":"2026-02-24T07:54:42.928482+0000","last_active":"2026-02-24T07:56:15.891320+0000","last_peered":"2026-02-24T07:56:15.891320+0000","last_clean":"2026-02-24T07:56:15.891320+0000","last_became_active":"2026-02-24T07:54:42.928238+0000","last_became_peered":"2026-02-24T07:54:42.928238+0000","last_unstale":"2026-02-24T07:56:15.891320+0000","last_undegraded":"2026-02-24T07:56:15.891320+0000","last_fullsized":"2026-02-24T07:56:15.891320+0000","mapping_epoch":90,"log_start":"96'40278","ondisk_log_start":"96'40278","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_clean_scrub_stamp":"2026-02-24T07:54:41.918128+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2548498432,"num_objects":38887,"num_object_clones":0,"num_object_copies":116661,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38887,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":40281,"num_write_kb":2533376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.b","version":"96'39877","reported_seq":40271,"reported_epoch":96,"state":"active+clean+scrubbing","last_fresh":"2026-02-24T07:56:15.896291+0000","last_change":"2026-02-24T07:56:11.628661+0000","last_active":"2026-02-24T07:56:15.896291+0000","last_peered":"2026-02-24T07:56:15.896291+0000","last_clean":"2026-02-24T07:56:15.896291+0000","last_became_active":"2026-02-24T07:54:42.926965+0000","last_became_peered":"2026-02-24T07:54:42.926965+0000","last_unstale":"2026-02-24T07:56:15.896291+0000","last_undegraded":"2026-02-24T07:56:15.896291+0000","last_fullsized":"2026-02-24T07:56:15.896291+0000","mapping_epoch":90,"log_start":"96'39874","ondisk_log_start":"96'39874","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_clean_scrub_stamp":"2026-02-24T07:54:41.918128+0000","objects_scrubbed":36849,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"scrubbing for 5s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2522415104,"num_objects":38489,"num_object_clones":0,"num_object_copies":115467,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38489,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":39877,"num_write_kb":2507712,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.a","version":"96'40213","reported_seq":40532,"reported_epoch":96,"state":"active+clean","last_fresh":"2026-02-24T07:56:15.776988+0000","last_change":"2026-02-24T07:55:49.167491+0000","last_active":"2026-02-24T07:56:15.776988+0000","last_peered":"2026-02-24T07:56:15.776988+0000","last_clean":"2026-02-24T07:56:15.776988+0000","last_became_active":"2026-02-24T07:54:42.926477+0000","last_became_peered":"2026-02-24T07:54:42.926477+0000","last_unstale":"2026-02-24T07:56:15.776988+0000","last_undegraded":"2026-02-24T07:56:15.776988+0000","last_fullsized":"2026-02-24T07:56:15.776988+0000","mapping_epoch":90,"log_start":"96'40210","ondisk_log_start":"96'40210","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"96'29334","last_scrub_stamp":"2026-02-24T07:55:49.167461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_clean_scrub_stamp":"2026-02-24T07:55:49.167461+0000","objects_scrubbed":28822,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:56:59.424140+0000","scrub_duration":2726,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2544828416,"num_objects":38831,"num_object_clones":0,"num_object_copies":116493,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38831,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":40213,"num_write_kb":2529408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"96'40370","reported_seq":40690,"reported_epoch":96,"state":"active+clean","last_fresh":"2026-02-24T07:56:15.810162+0000","last_change":"2026-02-24T07:55:48.535755+0000","last_active":"2026-02-24T07:56:15.810162+0000","last_peered":"2026-02-24T07:56:15.810162+0000","last_clean":"2026-02-24T07:56:15.810162+0000","last_became_active":"2026-02-24T07:54:42.926136+0000","last_became_peered":"2026-02-24T07:54:42.926136+0000","last_unstale":"2026-02-24T07:56:15.810162+0000","last_undegraded":"2026-02-24T07:56:15.810162+0000","last_fullsized":"2026-02-24T07:56:15.810162+0000","mapping_epoch":90,"log_start":"96'40367","ondisk_log_start":"96'40367","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"96'29148","last_scrub_stamp":"2026-02-24T07:55:48.535691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_clean_scrub_stamp":"2026-02-24T07:55:48.535691+0000","objects_scrubbed":28720,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:56:56.161155+0000","scrub_duration":2872,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2555772928,"num_objects":38998,"num_object_clones":0,"num_object_copies":116994,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38998,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":40370,"num_write_kb":2539776,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,1],"acting":[0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.8","version":"96'40420","reported_seq":40781,"reported_epoch":96,"state":"active+clean","last_fresh":"2026-02-24T07:56:15.792527+0000","last_change":"2026-02-24T07:55:58.178460+0000","last_active":"2026-02-24T07:56:15.792527+0000","last_peered":"2026-02-24T07:56:15.792527+0000","last_clean":"2026-02-24T07:56:15.792527+0000","last_became_active":"2026-02-24T07:54:42.929078+0000","last_became_peered":"2026-02-24T07:54:42.929078+0000","last_unstale":"2026-02-24T07:56:15.792527+0000","last_undegraded":"2026-02-24T07:56:15.792527+0000","last_fullsized":"2026-02-24T07:56:15.792527+0000","mapping_epoch":90,"log_start":"96'40417","ondisk_log_start":"96'40417","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"96'33550","last_scrub_stamp":"2026-02-24T07:55:58.178395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_clean_scrub_stamp":"2026-02-24T07:55:58.178395+0000","objects_scrubbed":32948,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:57:14.607073+0000","scrub_duration":2981,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2559705088,"num_objects":39058,"num_object_clones":0,"num_object_copies":117174,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39058,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":40420,"num_write_kb":2543296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.6","version":"96'40197","reported_seq":40309,"reported_epoch":96,"state":"active+clean+scrubbing","last_fresh":"2026-02-24T07:56:15.896413+0000","last_change":"2026-02-24T07:56:14.673162+0000","last_active":"2026-02-24T07:56:15.896413+0000","last_peered":"2026-02-24T07:56:15.896413+0000","last_clean":"2026-02-24T07:56:15.896413+0000","last_became_active":"2026-02-24T07:54:42.926045+0000","last_became_peered":"2026-02-24T07:54:42.926045+0000","last_unstale":"2026-02-24T07:56:15.896413+0000","last_undegraded":"2026-02-24T07:56:15.896413+0000","last_fullsized":"2026-02-24T07:56:15.896413+0000","mapping_epoch":90,"log_start":"96'40194","ondisk_log_start":"96'40194","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_clean_scrub_stamp":"2026-02-24T07:54:41.918128+0000","objects_scrubbed":8599,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"scrubbing for 2s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2537095168,"num_objects":38713,"num_object_clones":0,"num_object_copies":116139,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38713,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":40197,"num_write_kb":2525120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.5","version":"96'40142","reported_seq":40507,"reported_epoch":96,"state":"active+clean","last_fresh":"2026-02-24T07:56:15.766194+0000","last_change":"2026-02-24T07:55:59.773616+0000","last_active":"2026-02-24T07:56:15.766194+0000","last_peered":"2026-02-24T07:56:15.766194+0000","last_clean":"2026-02-24T07:56:15.766194+0000","last_became_active":"2026-02-24T07:54:42.927535+0000","last_became_peered":"2026-02-24T07:54:42.927535+0000","last_unstale":"2026-02-24T07:56:15.766194+0000","last_undegraded":"2026-02-24T07:56:15.766194+0000","last_fullsized":"2026-02-24T07:56:15.766194+0000","mapping_epoch":90,"log_start":"96'40139","ondisk_log_start":"96'40139","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"96'34080","last_scrub_stamp":"2026-02-24T07:55:59.773556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_clean_scrub_stamp":"2026-02-24T07:55:59.773556+0000","objects_scrubbed":33431,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:57:04.302607+0000","scrub_duration":3324,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2546204672,"num_objects":38852,"num_object_clones":0,"num_object_copies":116556,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38852,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":40142,"num_write_kb":2527808,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.4","version":"96'40198","reported_seq":40516,"reported_epoch":96,"state":"active+clean","last_fresh":"2026-02-24T07:56:15.891798+0000","last_change":"2026-02-24T07:55:47.544043+0000","last_active":"2026-02-24T07:56:15.891798+0000","last_peered":"2026-02-24T07:56:15.891798+0000","last_clean":"2026-02-24T07:56:15.891798+0000","last_became_active":"2026-02-24T07:54:42.928011+0000","last_became_peered":"2026-02-24T07:54:42.928011+0000","last_unstale":"2026-02-24T07:56:15.891798+0000","last_undegraded":"2026-02-24T07:56:15.891798+0000","last_fullsized":"2026-02-24T07:56:15.891798+0000","mapping_epoch":90,"log_start":"96'40195","ondisk_log_start":"96'40195","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"96'29055","last_scrub_stamp":"2026-02-24T07:55:47.543994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_clean_scrub_stamp":"2026-02-24T07:55:47.543994+0000","objects_scrubbed":28510,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:57:04.321401+0000","scrub_duration":3165,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2535981056,"num_objects":38696,"num_object_clones":0,"num_object_copies":116088,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38696,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":40198,"num_write_kb":2524608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.3","version":"96'40395","reported_seq":40787,"reported_epoch":96,"state":"active+clean","last_fresh":"2026-02-24T07:56:15.775625+0000","last_change":"2026-02-24T07:56:06.184345+0000","last_active":"2026-02-24T07:56:15.775625+0000","last_peered":"2026-02-24T07:56:15.775625+0000","last_clean":"2026-02-24T07:56:15.775625+0000","last_became_active":"2026-02-24T07:54:42.927730+0000","last_became_peered":"2026-02-24T07:54:42.927730+0000","last_unstale":"2026-02-24T07:56:15.775625+0000","last_undegraded":"2026-02-24T07:56:15.775625+0000","last_fullsized":"2026-02-24T07:56:15.775625+0000","mapping_epoch":90,"log_start":"96'40392","ondisk_log_start":"96'40392","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"96'36664","last_scrub_stamp":"2026-02-24T07:56:06.184309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_clean_scrub_stamp":"2026-02-24T07:56:06.184309+0000","objects_scrubbed":35946,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:57:11.733048+0000","scrub_duration":3811,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2559639552,"num_objects":39057,"num_object_clones":0,"num_object_copies":117171,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39057,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":40395,"num_write_kb":2542464,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.2","version":"96'40178","reported_seq":40547,"reported_epoch":96,"state":"active+clean","last_fresh":"2026-02-24T07:56:15.793200+0000","last_change":"2026-02-24T07:56:00.120637+0000","last_active":"2026-02-24T07:56:15.793200+0000","last_peered":"2026-02-24T07:56:15.793200+0000","last_clean":"2026-02-24T07:56:15.793200+0000","last_became_active":"2026-02-24T07:54:42.930247+0000","last_became_peered":"2026-02-24T07:54:42.930247+0000","last_unstale":"2026-02-24T07:56:15.793200+0000","last_undegraded":"2026-02-24T07:56:15.793200+0000","last_fullsized":"2026-02-24T07:56:15.793200+0000","mapping_epoch":90,"log_start":"96'40175","ondisk_log_start":"96'40175","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"96'34210","last_scrub_stamp":"2026-02-24T07:56:00.120605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_clean_scrub_stamp":"2026-02-24T07:56:00.120605+0000","objects_scrubbed":33656,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:57:29.329949+0000","scrub_duration":2957,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2544762880,"num_objects":38830,"num_object_clones":0,"num_object_copies":116490,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38830,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":40178,"num_write_kb":2528256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1","version":"96'40474","reported_seq":40890,"reported_epoch":96,"state":"active+clean","last_fresh":"2026-02-24T07:56:15.892047+0000","last_change":"2026-02-24T07:56:12.897479+0000","last_active":"2026-02-24T07:56:15.892047+0000","last_peered":"2026-02-24T07:56:15.892047+0000","last_clean":"2026-02-24T07:56:15.892047+0000","last_became_active":"2026-02-24T07:54:42.926442+0000","last_became_peered":"2026-02-24T07:54:42.926442+0000","last_unstale":"2026-02-24T07:56:15.892047+0000","last_undegraded":"2026-02-24T07:56:15.892047+0000","last_fullsized":"2026-02-24T07:56:15.892047+0000","mapping_epoch":90,"log_start":"96'40471","ondisk_log_start":"96'40471","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"96'39245","last_scrub_stamp":"2026-02-24T07:56:12.897436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_clean_scrub_stamp":"2026-02-24T07:56:12.897436+0000","objects_scrubbed":38506,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:57:42.057233+0000","scrub_duration":4279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2557607936,"num_objects":39026,"num_object_clones":0,"num_object_copies":117078,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39026,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":40474,"num_write_kb":2544000,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.0","version":"96'39857","reported_seq":40233,"reported_epoch":96,"state":"active+clean","last_fresh":"2026-02-24T07:56:15.802685+0000","last_change":"2026-02-24T07:56:02.992243+0000","last_active":"2026-02-24T07:56:15.802685+0000","last_peered":"2026-02-24T07:56:15.802685+0000","last_clean":"2026-02-24T07:56:15.802685+0000","last_became_active":"2026-02-24T07:54:42.926910+0000","last_became_peered":"2026-02-24T07:54:42.926910+0000","last_unstale":"2026-02-24T07:56:15.802685+0000","last_undegraded":"2026-02-24T07:56:15.802685+0000","last_fullsized":"2026-02-24T07:56:15.802685+0000","mapping_epoch":90,"log_start":"96'39854","ondisk_log_start":"96'39854","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"96'34952","last_scrub_stamp":"2026-02-24T07:56:02.992156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_clean_scrub_stamp":"2026-02-24T07:56:02.992156+0000","objects_scrubbed":34329,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:57:03.847719+0000","scrub_duration":3335,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2519662592,"num_objects":38447,"num_object_clones":0,"num_object_copies":115341,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38447,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":39857,"num_write_kb":2505728,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.7","version":"96'40448","reported_seq":40823,"reported_epoch":96,"state":"active+clean","last_fresh":"2026-02-24T07:56:15.789592+0000","last_change":"2026-02-24T07:56:01.088786+0000","last_active":"2026-02-24T07:56:15.789592+0000","last_peered":"2026-02-24T07:56:15.789592+0000","last_clean":"2026-02-24T07:56:15.789592+0000","last_became_active":"2026-02-24T07:54:42.930142+0000","last_became_peered":"2026-02-24T07:54:42.930142+0000","last_unstale":"2026-02-24T07:56:15.789592+0000","last_undegraded":"2026-02-24T07:56:15.789592+0000","last_fullsized":"2026-02-24T07:56:15.789592+0000","mapping_epoch":90,"log_start":"96'40445","ondisk_log_start":"96'40445","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"96'34838","last_scrub_stamp":"2026-02-24T07:56:01.088744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_clean_scrub_stamp":"2026-02-24T07:56:01.088744+0000","objects_scrubbed":34284,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:57:24.696242+0000","scrub_duration":2973,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2567045120,"num_objects":39170,"num_object_clones":0,"num_object_copies":117510,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39170,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":40448,"num_write_kb":2547776,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"29'99","reported_seq":398,"reported_epoch":96,"state":"active+clean","last_fresh":"2026-02-24T07:55:16.398160+0000","last_change":"2026-02-24T07:55:16.398160+0000","last_active":"2026-02-24T07:55:16.398160+0000","last_peered":"2026-02-24T07:55:16.398160+0000","last_clean":"2026-02-24T07:55:16.398160+0000","last_became_active":"2026-02-24T07:42:54.186719+0000","last_became_peered":"2026-02-24T07:42:54.186719+0000","last_unstale":"2026-02-24T07:55:16.398160+0000","last_undegraded":"2026-02-24T07:55:16.398160+0000","last_fullsized":"2026-02-24T07:55:16.398160+0000","mapping_epoch":27,"log_start":"29'96","ondisk_log_start":"29'96","created":20,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'99","last_scrub_stamp":"2026-02-24T07:55:16.398131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:42:43.640776+0000","last_clean_scrub_stamp":"2026-02-24T07:55:16.398131+0000","objects_scrubbed":2,"log_size":3,"log_dups_size":96,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:56:32.925996+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":240,"num_write":135,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":16,"stat_sum":{"num_bytes":40738422808,"num_objects":621620,"num_object_clones":0,"num_object_copies":1864860,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":621620,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":643732,"num_write_kb":40491201,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":122220720128,"data_stored":122220707912,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":48,"ondisk_log_size":48,"up":48,"acting":48,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":240,"num_write":135,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":2396,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":3,"up_from":25,"seq":107374182563,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":26819028,"kb_used_data":25106432,"kb_used_omap":18,"kb_used_meta":1712557,"kb_avail":707184172,"statfs":{"total":751619276800,"available":724156592128,"internally_reserved":0,"allocated":25708986368,"data_stored":25708483750,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":18972,"internal_metadata":1753658852},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":19,"seq":81604378789,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":31972248,"kb_used_data":29961412,"kb_used_omap":20,"kb_used_meta":2010795,"kb_avail":702030952,"statfs":{"total":751619276800,"available":718879694848,"internally_reserved":0,"allocated":30680485888,"data_stored":30679979198,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":20830,"internal_metadata":2059054754},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,2],"upper_bound":4},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542310,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":35922452,"kb_used_data":34884352,"kb_used_omap":21,"kb_used_meta":1037994,"kb_avail":698080748,"statfs":{"total":751619276800,"available":714834685952,"internally_reserved":0,"allocated":35721576448,"data_stored":35721073310,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":21591,"internal_metadata":1062906793},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705833,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":32070296,"kb_used_data":30084612,"kb_used_omap":16,"kb_used_meta":1985647,"kb_avail":701932904,"statfs":{"total":751619276800,"available":718779293696,"internally_reserved":0,"allocated":30806642688,"data_stored":30806135998,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":17342,"internal_metadata":2033302594},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":2396,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":30552494080,"data_stored":30552490008,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":35625897984,"data_stored":35625893912,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":30540763136,"data_stored":30540759064,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":25501564928,"data_stored":25501564928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-24T07:56:21.114 INFO:tasks.cephadm.ceph_manager.ceph:increase pool size by 10 2026-02-24T07:56:21.114 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph osd pool set .mgr pg_num 431 2026-02-24T07:56:21.259 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:56:21.398 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:56:21.395+0000 7fb5d3d30640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:56:21.398 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:56:21.396+0000 7fb5d3d30640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:56:21.542 INFO:teuthology.orchestra.run.trial035.stderr:Error EPERM: crimson_allow_pg_split is false; pg_num increase denied for crimson pool 2026-02-24T07:56:21.583 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-24T07:56:22.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:56:21 trial035 ceph-mon[39284]: 6.c scrub starts 2026-02-24T07:56:22.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:56:21 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/2494182177' entity='client.admin' cmd={"prefix": "osd pool set", "pool": ".mgr", "var": "pg_num", "val": "431"} : dispatch 2026-02-24T07:56:22.606 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2026-02-24T07:56:22.763 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:56:23.141 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:56:22 trial035 ceph-mon[39284]: from='client.15559 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2026-02-24T07:56:23.141 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:56:22 trial035 ceph-mon[39284]: pgmap v471: 17 pgs: 1 active+clean+scrubbing, 16 active+clean; 35 GiB data, 114 GiB used, 2.6 TiB / 2.7 TiB avail; 85 B/s rd, 262 MiB/s wr, 8.70k op/s 2026-02-24T07:56:23.204 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:56:23.190+0000 7f96c3582640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:56:23.205 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:56:23.191+0000 7f96c3582640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:56:23.220 INFO:teuthology.orchestra.run.trial035.stdout:osd.0: {} 2026-02-24T07:56:23.221 INFO:teuthology.orchestra.run.trial035.stderr:osd.0: osd_enable_op_tracker = '' 2026-02-24T07:56:23.238 INFO:teuthology.orchestra.run.trial035.stdout:osd.1: {} 2026-02-24T07:56:23.238 INFO:teuthology.orchestra.run.trial035.stderr:osd.1: osd_enable_op_tracker = '' 2026-02-24T07:56:23.250 INFO:teuthology.orchestra.run.trial035.stdout:osd.2: {} 2026-02-24T07:56:23.250 INFO:teuthology.orchestra.run.trial035.stderr:osd.2: osd_enable_op_tracker = '' 2026-02-24T07:56:23.261 INFO:teuthology.orchestra.run.trial035.stdout:osd.3: {} 2026-02-24T07:56:23.261 INFO:teuthology.orchestra.run.trial035.stderr:osd.3: osd_enable_op_tracker = '' 2026-02-24T07:56:24.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:56:23 trial035 ceph-mon[39284]: purged_snaps scrub starts 2026-02-24T07:56:24.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:56:23 trial035 ceph-mon[39284]: purged_snaps scrub ok 2026-02-24T07:56:24.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:56:23 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/3496443022' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2026-02-24T07:56:24.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:56:23 trial035 ceph-mon[39284]: pgmap v472: 17 pgs: 1 active+clean+scrubbing, 16 active+clean; 35 GiB data, 114 GiB used, 2.6 TiB / 2.7 TiB avail; 85 B/s rd, 103 MiB/s wr, 6.16k op/s 2026-02-24T07:56:25.384 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2026-02-24T07:56:25.521 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:56:25.548 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:56:25 trial035 ceph-mon[39284]: 6.c scrub ok 2026-02-24T07:56:25.644 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:56:25.642+0000 7f81612ca640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:56:25.645 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:56:25.642+0000 7f81612ca640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:56:25.660 INFO:teuthology.orchestra.run.trial035.stdout:osd.0: {} 2026-02-24T07:56:25.660 INFO:teuthology.orchestra.run.trial035.stderr:osd.0: osd_enable_op_tracker = '' 2026-02-24T07:56:25.677 INFO:teuthology.orchestra.run.trial035.stdout:osd.1: {} 2026-02-24T07:56:25.678 INFO:teuthology.orchestra.run.trial035.stderr:osd.1: osd_enable_op_tracker = '' 2026-02-24T07:56:25.691 INFO:teuthology.orchestra.run.trial035.stdout:osd.2: {} 2026-02-24T07:56:25.692 INFO:teuthology.orchestra.run.trial035.stderr:osd.2: osd_enable_op_tracker = '' 2026-02-24T07:56:25.705 INFO:teuthology.orchestra.run.trial035.stdout:osd.3: {} 2026-02-24T07:56:25.705 INFO:teuthology.orchestra.run.trial035.stderr:osd.3: osd_enable_op_tracker = '' 2026-02-24T07:56:26.584 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2026-02-24T07:56:26.584 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2026-02-24T07:56:26.584 INFO:tasks.thrashosds.thrasher:Growing pool unique_pool_4 2026-02-24T07:56:26.584 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph pg dump --format=json 2026-02-24T07:56:26.607 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:56:26 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/784529600' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2026-02-24T07:56:26.607 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:56:26 trial035 ceph-mon[39284]: pgmap v473: 17 pgs: 1 active+clean+scrubbing, 16 active+clean; 35 GiB data, 114 GiB used, 2.6 TiB / 2.7 TiB avail; 85 B/s rd, 103 MiB/s wr, 6.16k op/s 2026-02-24T07:56:26.734 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:56:26.855 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:56:26.853+0000 7f134e038640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:56:26.855 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:56:26.853+0000 7f134e038640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:56:27.010 INFO:teuthology.orchestra.run.trial035.stdout: 2026-02-24T07:56:27.011 INFO:teuthology.orchestra.run.trial035.stderr:dumped all 2026-02-24T07:56:27.055 INFO:teuthology.orchestra.run.trial035.stdout:{"pg_ready":true,"pg_map":{"version":473,"stamp":"2026-02-24T07:56:25.767111+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":37912773176,"num_objects":578497,"num_object_clones":0,"num_object_copies":1735491,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":578497,"num_whiteouts":0,"num_read":137,"num_read_kb":241,"num_write":686992,"num_write_kb":40493351,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":54,"ondisk_log_size":54,"up":51,"acting":51,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":51,"num_osds":4,"num_per_pool_osds":4,"num_per_pool_omap_osds":4,"kb":2936012800,"kb_used":119250520,"kb_used_data":112232648,"kb_used_omap":76,"kb_used_meta":7017715,"kb_avail":2816762280,"statfs":{"total":3006477107200,"available":2884364574720,"internally_reserved":0,"allocated":114926231552,"data_stored":114924212416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":78735,"internal_metadata":7186140273},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-2254307304,"num_objects":-34397,"num_object_clones":0,"num_object_copies":-103191,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-34397,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":73965,"num_write_kb":1266113,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001679"},"pg_stats":[{"pgid":"6.f","version":"96'42930","reported_seq":43341,"reported_epoch":96,"state":"active+clean","last_fresh":"2026-02-24T07:56:20.891201+0000","last_change":"2026-02-24T07:56:12.247365+0000","last_active":"2026-02-24T07:56:20.891201+0000","last_peered":"2026-02-24T07:56:20.891201+0000","last_clean":"2026-02-24T07:56:20.891201+0000","last_became_active":"2026-02-24T07:54:42.927080+0000","last_became_peered":"2026-02-24T07:54:42.927080+0000","last_unstale":"2026-02-24T07:56:20.891201+0000","last_undegraded":"2026-02-24T07:56:20.891201+0000","last_fullsized":"2026-02-24T07:56:20.891201+0000","mapping_epoch":90,"log_start":"96'42927","ondisk_log_start":"96'42927","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"96'38780","last_scrub_stamp":"2026-02-24T07:56:12.247318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_clean_scrub_stamp":"2026-02-24T07:56:12.247318+0000","objects_scrubbed":37979,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:57:38.474117+0000","scrub_duration":4670,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2364801024,"num_objects":36084,"num_object_clones":0,"num_object_copies":108252,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36084,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":42930,"num_write_kb":2528448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.e","version":"96'42860","reported_seq":43245,"reported_epoch":96,"state":"active+clean","last_fresh":"2026-02-24T07:56:20.778246+0000","last_change":"2026-02-24T07:56:05.374565+0000","last_active":"2026-02-24T07:56:20.778246+0000","last_peered":"2026-02-24T07:56:20.778246+0000","last_clean":"2026-02-24T07:56:20.778246+0000","last_became_active":"2026-02-24T07:54:42.928748+0000","last_became_peered":"2026-02-24T07:54:42.928748+0000","last_unstale":"2026-02-24T07:56:20.778246+0000","last_undegraded":"2026-02-24T07:56:20.778246+0000","last_fullsized":"2026-02-24T07:56:20.778246+0000","mapping_epoch":90,"log_start":"96'42857","ondisk_log_start":"96'42857","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"96'36307","last_scrub_stamp":"2026-02-24T07:56:05.374510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_clean_scrub_stamp":"2026-02-24T07:56:05.374510+0000","objects_scrubbed":35491,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:57:16.427813+0000","scrub_duration":4036,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2376204288,"num_objects":36258,"num_object_clones":0,"num_object_copies":108774,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36258,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":42860,"num_write_kb":2531776,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.d","version":"96'42939","reported_seq":43257,"reported_epoch":96,"state":"active+clean","last_fresh":"2026-02-24T07:56:20.772125+0000","last_change":"2026-02-24T07:55:48.151749+0000","last_active":"2026-02-24T07:56:20.772125+0000","last_peered":"2026-02-24T07:56:20.772125+0000","last_clean":"2026-02-24T07:56:20.772125+0000","last_became_active":"2026-02-24T07:54:42.926886+0000","last_became_peered":"2026-02-24T07:54:42.926886+0000","last_unstale":"2026-02-24T07:56:20.772125+0000","last_undegraded":"2026-02-24T07:56:20.772125+0000","last_fullsized":"2026-02-24T07:56:20.772125+0000","mapping_epoch":90,"log_start":"96'42936","ondisk_log_start":"96'42936","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"96'28924","last_scrub_stamp":"2026-02-24T07:55:48.151672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_clean_scrub_stamp":"2026-02-24T07:55:48.151672+0000","objects_scrubbed":28526,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:57:00.390677+0000","scrub_duration":2742,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2370830360,"num_objects":36177,"num_object_clones":0,"num_object_copies":108531,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36177,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":42939,"num_write_kb":2531649,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.c","version":"96'43024","reported_seq":43075,"reported_epoch":96,"state":"active+clean+scrubbing","last_fresh":"2026-02-24T07:56:20.895323+0000","last_change":"2026-02-24T07:56:20.613422+0000","last_active":"2026-02-24T07:56:20.895323+0000","last_peered":"2026-02-24T07:56:20.895323+0000","last_clean":"2026-02-24T07:56:20.895323+0000","last_became_active":"2026-02-24T07:54:42.928238+0000","last_became_peered":"2026-02-24T07:54:42.928238+0000","last_unstale":"2026-02-24T07:56:20.895323+0000","last_undegraded":"2026-02-24T07:56:20.895323+0000","last_fullsized":"2026-02-24T07:56:20.895323+0000","mapping_epoch":90,"log_start":"96'43019","ondisk_log_start":"96'43019","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_clean_scrub_stamp":"2026-02-24T07:54:41.918128+0000","objects_scrubbed":2949,"log_size":5,"log_dups_size":3000,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"scrubbing for 1s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2368733184,"num_objects":36144,"num_object_clones":0,"num_object_copies":108432,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36144,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":43024,"num_write_kb":2533376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.b","version":"96'42571","reported_seq":42989,"reported_epoch":96,"state":"active+clean","last_fresh":"2026-02-24T07:56:20.894844+0000","last_change":"2026-02-24T07:56:16.120475+0000","last_active":"2026-02-24T07:56:20.894844+0000","last_peered":"2026-02-24T07:56:20.894844+0000","last_clean":"2026-02-24T07:56:20.894844+0000","last_became_active":"2026-02-24T07:54:42.926965+0000","last_became_peered":"2026-02-24T07:54:42.926965+0000","last_unstale":"2026-02-24T07:56:20.894844+0000","last_undegraded":"2026-02-24T07:56:20.894844+0000","last_fullsized":"2026-02-24T07:56:20.894844+0000","mapping_epoch":90,"log_start":"96'42568","ondisk_log_start":"96'42568","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"96'39969","last_scrub_stamp":"2026-02-24T07:56:16.120290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_clean_scrub_stamp":"2026-02-24T07:56:16.120290+0000","objects_scrubbed":38739,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:57:36.957012+0000","scrub_duration":4491,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2345861120,"num_objects":35795,"num_object_clones":0,"num_object_copies":107385,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35795,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":42571,"num_write_kb":2507712,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.a","version":"96'42848","reported_seq":43167,"reported_epoch":96,"state":"active+clean","last_fresh":"2026-02-24T07:56:20.775141+0000","last_change":"2026-02-24T07:55:49.167491+0000","last_active":"2026-02-24T07:56:20.775141+0000","last_peered":"2026-02-24T07:56:20.775141+0000","last_clean":"2026-02-24T07:56:20.775141+0000","last_became_active":"2026-02-24T07:54:42.926477+0000","last_became_peered":"2026-02-24T07:54:42.926477+0000","last_unstale":"2026-02-24T07:56:20.775141+0000","last_undegraded":"2026-02-24T07:56:20.775141+0000","last_fullsized":"2026-02-24T07:56:20.775141+0000","mapping_epoch":90,"log_start":"96'42845","ondisk_log_start":"96'42845","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"96'29334","last_scrub_stamp":"2026-02-24T07:55:49.167461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_clean_scrub_stamp":"2026-02-24T07:55:49.167461+0000","objects_scrubbed":28822,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:56:59.424140+0000","scrub_duration":2726,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2372141056,"num_objects":36196,"num_object_clones":0,"num_object_copies":108588,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36196,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":42848,"num_write_kb":2529408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"96'43068","reported_seq":43388,"reported_epoch":96,"state":"active+clean","last_fresh":"2026-02-24T07:56:20.812891+0000","last_change":"2026-02-24T07:55:48.535755+0000","last_active":"2026-02-24T07:56:20.812891+0000","last_peered":"2026-02-24T07:56:20.812891+0000","last_clean":"2026-02-24T07:56:20.812891+0000","last_became_active":"2026-02-24T07:54:42.926136+0000","last_became_peered":"2026-02-24T07:54:42.926136+0000","last_unstale":"2026-02-24T07:56:20.812891+0000","last_undegraded":"2026-02-24T07:56:20.812891+0000","last_fullsized":"2026-02-24T07:56:20.812891+0000","mapping_epoch":90,"log_start":"96'43065","ondisk_log_start":"96'43065","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"96'29148","last_scrub_stamp":"2026-02-24T07:55:48.535691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_clean_scrub_stamp":"2026-02-24T07:55:48.535691+0000","objects_scrubbed":28720,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:56:56.161155+0000","scrub_duration":2872,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2378956800,"num_objects":36300,"num_object_clones":0,"num_object_copies":108900,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36300,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":43068,"num_write_kb":2539776,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,1],"acting":[0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.8","version":"96'43098","reported_seq":43459,"reported_epoch":96,"state":"active+clean","last_fresh":"2026-02-24T07:56:20.790893+0000","last_change":"2026-02-24T07:55:58.178460+0000","last_active":"2026-02-24T07:56:20.790893+0000","last_peered":"2026-02-24T07:56:20.790893+0000","last_clean":"2026-02-24T07:56:20.790893+0000","last_became_active":"2026-02-24T07:54:42.929078+0000","last_became_peered":"2026-02-24T07:54:42.929078+0000","last_unstale":"2026-02-24T07:56:20.790893+0000","last_undegraded":"2026-02-24T07:56:20.790893+0000","last_fullsized":"2026-02-24T07:56:20.790893+0000","mapping_epoch":90,"log_start":"96'43095","ondisk_log_start":"96'43095","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"96'33550","last_scrub_stamp":"2026-02-24T07:55:58.178395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_clean_scrub_stamp":"2026-02-24T07:55:58.178395+0000","objects_scrubbed":32948,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:57:14.607073+0000","scrub_duration":2981,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2384199680,"num_objects":36380,"num_object_clones":0,"num_object_copies":109140,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36380,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":43098,"num_write_kb":2543296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.6","version":"96'42959","reported_seq":43372,"reported_epoch":96,"state":"active+clean","last_fresh":"2026-02-24T07:56:20.881802+0000","last_change":"2026-02-24T07:56:18.901177+0000","last_active":"2026-02-24T07:56:20.881802+0000","last_peered":"2026-02-24T07:56:20.881802+0000","last_clean":"2026-02-24T07:56:20.881802+0000","last_became_active":"2026-02-24T07:54:42.926045+0000","last_became_peered":"2026-02-24T07:54:42.926045+0000","last_unstale":"2026-02-24T07:56:20.881802+0000","last_undegraded":"2026-02-24T07:56:20.881802+0000","last_fullsized":"2026-02-24T07:56:20.881802+0000","mapping_epoch":90,"log_start":"96'42956","ondisk_log_start":"96'42956","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"96'41769","last_scrub_stamp":"2026-02-24T07:56:18.901111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_clean_scrub_stamp":"2026-02-24T07:56:18.901111+0000","objects_scrubbed":38122,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:57:27.205697+0000","scrub_duration":4228,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2356084736,"num_objects":35951,"num_object_clones":0,"num_object_copies":107853,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35951,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":42959,"num_write_kb":2525120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.5","version":"96'42865","reported_seq":43230,"reported_epoch":96,"state":"active+clean","last_fresh":"2026-02-24T07:56:20.769388+0000","last_change":"2026-02-24T07:55:59.773616+0000","last_active":"2026-02-24T07:56:20.769388+0000","last_peered":"2026-02-24T07:56:20.769388+0000","last_clean":"2026-02-24T07:56:20.769388+0000","last_became_active":"2026-02-24T07:54:42.927535+0000","last_became_peered":"2026-02-24T07:54:42.927535+0000","last_unstale":"2026-02-24T07:56:20.769388+0000","last_undegraded":"2026-02-24T07:56:20.769388+0000","last_fullsized":"2026-02-24T07:56:20.769388+0000","mapping_epoch":90,"log_start":"96'42862","ondisk_log_start":"96'42862","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"96'34080","last_scrub_stamp":"2026-02-24T07:55:59.773556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_clean_scrub_stamp":"2026-02-24T07:55:59.773556+0000","objects_scrubbed":33431,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:57:04.302607+0000","scrub_duration":3324,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2367750144,"num_objects":36129,"num_object_clones":0,"num_object_copies":108387,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36129,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":42865,"num_write_kb":2527808,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.4","version":"96'42935","reported_seq":43250,"reported_epoch":96,"state":"active+clean","last_fresh":"2026-02-24T07:56:20.891242+0000","last_change":"2026-02-24T07:55:47.544043+0000","last_active":"2026-02-24T07:56:20.891242+0000","last_peered":"2026-02-24T07:56:20.891242+0000","last_clean":"2026-02-24T07:56:20.891242+0000","last_became_active":"2026-02-24T07:54:42.928011+0000","last_became_peered":"2026-02-24T07:54:42.928011+0000","last_unstale":"2026-02-24T07:56:20.891242+0000","last_undegraded":"2026-02-24T07:56:20.891242+0000","last_fullsized":"2026-02-24T07:56:20.891242+0000","mapping_epoch":90,"log_start":"96'42931","ondisk_log_start":"96'42931","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"96'29055","last_scrub_stamp":"2026-02-24T07:55:47.543994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_clean_scrub_stamp":"2026-02-24T07:55:47.543994+0000","objects_scrubbed":28510,"log_size":4,"log_dups_size":3000,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:57:04.321401+0000","scrub_duration":3165,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2356609024,"num_objects":35959,"num_object_clones":0,"num_object_copies":107877,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35959,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":42935,"num_write_kb":2524608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.3","version":"96'43087","reported_seq":43479,"reported_epoch":96,"state":"active+clean","last_fresh":"2026-02-24T07:56:20.761684+0000","last_change":"2026-02-24T07:56:06.184345+0000","last_active":"2026-02-24T07:56:20.761684+0000","last_peered":"2026-02-24T07:56:20.761684+0000","last_clean":"2026-02-24T07:56:20.761684+0000","last_became_active":"2026-02-24T07:54:42.927730+0000","last_became_peered":"2026-02-24T07:54:42.927730+0000","last_unstale":"2026-02-24T07:56:20.761684+0000","last_undegraded":"2026-02-24T07:56:20.761684+0000","last_fullsized":"2026-02-24T07:56:20.761684+0000","mapping_epoch":90,"log_start":"96'43084","ondisk_log_start":"96'43084","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"96'36664","last_scrub_stamp":"2026-02-24T07:56:06.184309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_clean_scrub_stamp":"2026-02-24T07:56:06.184309+0000","objects_scrubbed":35946,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:57:11.733048+0000","scrub_duration":3811,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2383216640,"num_objects":36365,"num_object_clones":0,"num_object_copies":109095,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36365,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":43087,"num_write_kb":2542464,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.2","version":"96'42895","reported_seq":43264,"reported_epoch":96,"state":"active+clean","last_fresh":"2026-02-24T07:56:20.792844+0000","last_change":"2026-02-24T07:56:00.120637+0000","last_active":"2026-02-24T07:56:20.792844+0000","last_peered":"2026-02-24T07:56:20.792844+0000","last_clean":"2026-02-24T07:56:20.792844+0000","last_became_active":"2026-02-24T07:54:42.930247+0000","last_became_peered":"2026-02-24T07:54:42.930247+0000","last_unstale":"2026-02-24T07:56:20.792844+0000","last_undegraded":"2026-02-24T07:56:20.792844+0000","last_fullsized":"2026-02-24T07:56:20.792844+0000","mapping_epoch":90,"log_start":"96'42892","ondisk_log_start":"96'42892","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"96'34210","last_scrub_stamp":"2026-02-24T07:56:00.120605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_clean_scrub_stamp":"2026-02-24T07:56:00.120605+0000","objects_scrubbed":33656,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:57:29.329949+0000","scrub_duration":2957,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2366701568,"num_objects":36113,"num_object_clones":0,"num_object_copies":108339,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36113,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":42895,"num_write_kb":2528256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1","version":"96'43132","reported_seq":43548,"reported_epoch":96,"state":"active+clean","last_fresh":"2026-02-24T07:56:20.893011+0000","last_change":"2026-02-24T07:56:12.897479+0000","last_active":"2026-02-24T07:56:20.893011+0000","last_peered":"2026-02-24T07:56:20.893011+0000","last_clean":"2026-02-24T07:56:20.893011+0000","last_became_active":"2026-02-24T07:54:42.926442+0000","last_became_peered":"2026-02-24T07:54:42.926442+0000","last_unstale":"2026-02-24T07:56:20.893011+0000","last_undegraded":"2026-02-24T07:56:20.893011+0000","last_fullsized":"2026-02-24T07:56:20.893011+0000","mapping_epoch":90,"log_start":"96'43129","ondisk_log_start":"96'43129","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"96'39245","last_scrub_stamp":"2026-02-24T07:56:12.897436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_clean_scrub_stamp":"2026-02-24T07:56:12.897436+0000","objects_scrubbed":38506,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:57:42.057233+0000","scrub_duration":4279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2383413248,"num_objects":36368,"num_object_clones":0,"num_object_copies":109104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36368,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":43132,"num_write_kb":2544000,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.0","version":"96'42522","reported_seq":42898,"reported_epoch":96,"state":"active+clean","last_fresh":"2026-02-24T07:56:20.806176+0000","last_change":"2026-02-24T07:56:02.992243+0000","last_active":"2026-02-24T07:56:20.806176+0000","last_peered":"2026-02-24T07:56:20.806176+0000","last_clean":"2026-02-24T07:56:20.806176+0000","last_became_active":"2026-02-24T07:54:42.926910+0000","last_became_peered":"2026-02-24T07:54:42.926910+0000","last_unstale":"2026-02-24T07:56:20.806176+0000","last_undegraded":"2026-02-24T07:56:20.806176+0000","last_fullsized":"2026-02-24T07:56:20.806176+0000","mapping_epoch":90,"log_start":"96'42519","ondisk_log_start":"96'42519","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"96'34952","last_scrub_stamp":"2026-02-24T07:56:02.992156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_clean_scrub_stamp":"2026-02-24T07:56:02.992156+0000","objects_scrubbed":34329,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:57:03.847719+0000","scrub_duration":3335,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2345009152,"num_objects":35782,"num_object_clones":0,"num_object_copies":107346,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35782,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":42522,"num_write_kb":2505728,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.7","version":"96'43124","reported_seq":43499,"reported_epoch":96,"state":"active+clean","last_fresh":"2026-02-24T07:56:20.787018+0000","last_change":"2026-02-24T07:56:01.088786+0000","last_active":"2026-02-24T07:56:20.787018+0000","last_peered":"2026-02-24T07:56:20.787018+0000","last_clean":"2026-02-24T07:56:20.787018+0000","last_became_active":"2026-02-24T07:54:42.930142+0000","last_became_peered":"2026-02-24T07:54:42.930142+0000","last_unstale":"2026-02-24T07:56:20.787018+0000","last_undegraded":"2026-02-24T07:56:20.787018+0000","last_fullsized":"2026-02-24T07:56:20.787018+0000","mapping_epoch":90,"log_start":"96'43121","ondisk_log_start":"96'43121","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"96'34838","last_scrub_stamp":"2026-02-24T07:56:01.088744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_clean_scrub_stamp":"2026-02-24T07:56:01.088744+0000","objects_scrubbed":34284,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:57:24.696242+0000","scrub_duration":2973,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2391670784,"num_objects":36494,"num_object_clones":0,"num_object_copies":109482,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36494,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":43124,"num_write_kb":2547776,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"29'99","reported_seq":398,"reported_epoch":96,"state":"active+clean","last_fresh":"2026-02-24T07:55:16.398160+0000","last_change":"2026-02-24T07:55:16.398160+0000","last_active":"2026-02-24T07:55:16.398160+0000","last_peered":"2026-02-24T07:55:16.398160+0000","last_clean":"2026-02-24T07:55:16.398160+0000","last_became_active":"2026-02-24T07:42:54.186719+0000","last_became_peered":"2026-02-24T07:42:54.186719+0000","last_unstale":"2026-02-24T07:55:16.398160+0000","last_undegraded":"2026-02-24T07:55:16.398160+0000","last_fullsized":"2026-02-24T07:55:16.398160+0000","mapping_epoch":27,"log_start":"29'96","ondisk_log_start":"29'96","created":20,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'99","last_scrub_stamp":"2026-02-24T07:55:16.398131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:42:43.640776+0000","last_clean_scrub_stamp":"2026-02-24T07:55:16.398131+0000","objects_scrubbed":2,"log_size":3,"log_dups_size":96,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:56:32.925996+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":240,"num_write":135,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":16,"stat_sum":{"num_bytes":37912182808,"num_objects":578495,"num_object_clones":0,"num_object_copies":1735485,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":578495,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":686857,"num_write_kb":40491201,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":113744228352,"data_stored":113744216136,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":51,"ondisk_log_size":51,"up":48,"acting":48,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":240,"num_write":135,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":2396,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":3,"up_from":25,"seq":107374182564,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":25295892,"kb_used_data":23511104,"kb_used_omap":18,"kb_used_meta":1784749,"kb_avail":708707308,"statfs":{"total":751619276800,"available":725716283392,"internally_reserved":0,"allocated":24075370496,"data_stored":24074867878,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":18972,"internal_metadata":1827583460},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":19,"seq":81604378790,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":30090072,"kb_used_data":28025092,"kb_used_omap":20,"kb_used_meta":2064939,"kb_avail":703913128,"statfs":{"total":751619276800,"available":720807043072,"internally_reserved":0,"allocated":28697694208,"data_stored":28697187518,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":20830,"internal_metadata":2114498210},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542311,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":33616340,"kb_used_data":32506112,"kb_used_omap":21,"kb_used_meta":1110186,"kb_avail":700386860,"statfs":{"total":751619276800,"available":717196144640,"internally_reserved":0,"allocated":33286258688,"data_stored":33285755550,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":21591,"internal_metadata":1136831401},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705834,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":30248216,"kb_used_data":28190340,"kb_used_omap":16,"kb_used_meta":2057839,"kb_avail":703754984,"statfs":{"total":751619276800,"available":720645103616,"internally_reserved":0,"allocated":28866908160,"data_stored":28866401470,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":17342,"internal_metadata":2107227202},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":2396,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28432011264,"data_stored":28432007192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":33149030400,"data_stored":33149026328,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":28427882496,"data_stored":28427878424,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":23735304192,"data_stored":23735304192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-24T07:56:27.055 INFO:tasks.cephadm.ceph_manager.ceph:increase pool size by 10 2026-02-24T07:56:27.056 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph osd pool set unique_pool_4 pg_num 76 2026-02-24T07:56:27.244 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:56:27.271 INFO:journalctl@ceph.mgr.x.trial035.stdout:Feb 24 07:56:26 trial035 ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-mgr-x[39621]: ::ffff:10.20.193.91 - - [24/Feb/2026:07:56:26] "GET /metrics HTTP/1.1" 200 4225 "" "Prometheus/3.6.0" 2026-02-24T07:56:27.356 INFO:journalctl@ceph.prometheus.a.trial091.stdout:Feb 24 07:56:26 trial091 ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-prometheus-a[66007]: time=2026-02-24T07:56:26.972Z level=ERROR source=scrape.go:1630 msg="Failed to determine correct type of scrape target." component="scrape manager" scrape_pool=ceph target=http://trial035:9283/metrics content_type="text/html;charset=utf-8" fallback_media_type="" err="received unsupported Content-Type \"text/html;charset=utf-8\" and no fallback_scrape_protocol specified for target" 2026-02-24T07:56:27.376 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:56:27.374+0000 7fd71d48f640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:56:27.376 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:56:27.374+0000 7fd71d48f640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:56:27.537 INFO:teuthology.orchestra.run.trial035.stderr:Error EPERM: crimson_allow_pg_split is false; pg_num increase denied for crimson pool 2026-02-24T07:56:27.578 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-24T07:56:27.746 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2026-02-24T07:56:27.897 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:56:27.923 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:56:27 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/3497640650' entity='client.admin' cmd={"prefix": "osd pool set", "pool": "unique_pool_4", "var": "pg_num", "val": "76"} : dispatch 2026-02-24T07:56:28.020 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:56:28.017+0000 7f6d106ef640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:56:28.020 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:56:28.018+0000 7f6d106ef640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:56:28.035 INFO:teuthology.orchestra.run.trial035.stdout:osd.0: {} 2026-02-24T07:56:28.035 INFO:teuthology.orchestra.run.trial035.stderr:osd.0: osd_enable_op_tracker = '' 2026-02-24T07:56:28.047 INFO:teuthology.orchestra.run.trial035.stdout:osd.1: {} 2026-02-24T07:56:28.047 INFO:teuthology.orchestra.run.trial035.stderr:osd.1: osd_enable_op_tracker = '' 2026-02-24T07:56:28.058 INFO:teuthology.orchestra.run.trial035.stdout:osd.2: {} 2026-02-24T07:56:28.059 INFO:teuthology.orchestra.run.trial035.stderr:osd.2: osd_enable_op_tracker = '' 2026-02-24T07:56:28.071 INFO:teuthology.orchestra.run.trial035.stdout:osd.3: {} 2026-02-24T07:56:28.071 INFO:teuthology.orchestra.run.trial035.stderr:osd.3: osd_enable_op_tracker = '' 2026-02-24T07:56:28.954 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:56:28 trial035 ceph-mon[39284]: from='client.15567 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2026-02-24T07:56:28.954 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:56:28 trial035 ceph-mon[39284]: pgmap v474: 17 pgs: 17 active+clean; 33 GiB data, 107 GiB used, 2.6 TiB / 2.7 TiB avail; 85 B/s rd, 103 MiB/s wr, 9.34k op/s 2026-02-24T07:56:28.954 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:56:28 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/2531309733' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2026-02-24T07:56:30.112 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2026-02-24T07:56:30.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:56:29 trial035 ceph-mon[39284]: from='mgr.14284 10.20.193.35:0/3681373222' entity='mgr.x' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2026-02-24T07:56:30.267 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:56:30.391 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:56:30.389+0000 7f04ccc71640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:56:30.391 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:56:30.389+0000 7f04ccc71640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:56:30.406 INFO:teuthology.orchestra.run.trial035.stdout:osd.0: {} 2026-02-24T07:56:30.407 INFO:teuthology.orchestra.run.trial035.stderr:osd.0: osd_enable_op_tracker = '' 2026-02-24T07:56:30.422 INFO:teuthology.orchestra.run.trial035.stdout:osd.1: {} 2026-02-24T07:56:30.423 INFO:teuthology.orchestra.run.trial035.stderr:osd.1: osd_enable_op_tracker = '' 2026-02-24T07:56:30.435 INFO:teuthology.orchestra.run.trial035.stdout:osd.2: {} 2026-02-24T07:56:30.436 INFO:teuthology.orchestra.run.trial035.stderr:osd.2: osd_enable_op_tracker = '' 2026-02-24T07:56:30.448 INFO:teuthology.orchestra.run.trial035.stdout:osd.3: {} 2026-02-24T07:56:30.448 INFO:teuthology.orchestra.run.trial035.stderr:osd.3: osd_enable_op_tracker = '' 2026-02-24T07:56:31.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:56:30 trial035 ceph-mon[39284]: pgmap v475: 17 pgs: 17 active+clean; 33 GiB data, 107 GiB used, 2.6 TiB / 2.7 TiB avail; 0 B/s wr, 6.77k op/s 2026-02-24T07:56:31.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:56:30 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/4081704182' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2026-02-24T07:56:32.493 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2026-02-24T07:56:32.579 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2026-02-24T07:56:32.579 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2026-02-24T07:56:32.580 INFO:tasks.thrashosds.thrasher:Growing pool .mgr 2026-02-24T07:56:32.580 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph pg dump --format=json 2026-02-24T07:56:32.644 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:56:33.067 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:56:32 trial035 ceph-mon[39284]: pgmap v476: 17 pgs: 17 active+clean; 30 GiB data, 96 GiB used, 2.6 TiB / 2.7 TiB avail; 0 B/s wr, 10.67k op/s 2026-02-24T07:56:33.097 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:56:33.028+0000 7f36065d6640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:56:33.097 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:56:33.028+0000 7f36065d6640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:56:33.097 INFO:teuthology.orchestra.run.trial035.stderr:osd.0: osd_enable_op_tracker = '' 2026-02-24T07:56:33.097 INFO:teuthology.orchestra.run.trial035.stderr:osd.1: osd_enable_op_tracker = '' 2026-02-24T07:56:33.098 INFO:teuthology.orchestra.run.trial035.stdout:osd.0: {} 2026-02-24T07:56:33.098 INFO:teuthology.orchestra.run.trial035.stdout:osd.1: {} 2026-02-24T07:56:33.100 INFO:teuthology.orchestra.run.trial035.stdout:osd.2: {} 2026-02-24T07:56:33.100 INFO:teuthology.orchestra.run.trial035.stderr:osd.2: osd_enable_op_tracker = '' 2026-02-24T07:56:33.112 INFO:teuthology.orchestra.run.trial035.stdout:osd.3: {} 2026-02-24T07:56:33.112 INFO:teuthology.orchestra.run.trial035.stderr:osd.3: osd_enable_op_tracker = '' 2026-02-24T07:56:33.748 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:56:34.066 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:56:34.064+0000 7fa4a913f640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:56:34.066 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:56:34.064+0000 7fa4a913f640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:56:34.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:56:33 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/1552131737' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2026-02-24T07:56:34.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:56:33 trial035 ceph-mon[39284]: 1.0 scrub starts 2026-02-24T07:56:34.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:56:33 trial035 ceph-mon[39284]: 1.0 scrub ok 2026-02-24T07:56:34.207 INFO:teuthology.orchestra.run.trial035.stdout: 2026-02-24T07:56:34.208 INFO:teuthology.orchestra.run.trial035.stderr:dumped all 2026-02-24T07:56:34.821 INFO:teuthology.orchestra.run.trial035.stdout:{"pg_ready":true,"pg_map":{"version":477,"stamp":"2026-02-24T07:56:33.768227+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":32349684280,"num_objects":493611,"num_object_clones":0,"num_object_copies":1480833,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":493611,"num_whiteouts":0,"num_read":137,"num_read_kb":241,"num_write":771878,"num_write_kb":40493351,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":52,"ondisk_log_size":52,"up":51,"acting":51,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":51,"num_osds":4,"num_per_pool_osds":4,"num_per_pool_omap_osds":4,"kb":2936012800,"kb_used":101062104,"kb_used_data":95577992,"kb_used_omap":78,"kb_used_meta":5483697,"kb_avail":2834950696,"statfs":{"total":3006477107200,"available":2902989512704,"internally_reserved":0,"allocated":97871863808,"data_stored":97869844672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":80476,"internal_metadata":5615306148},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[10,3],"upper_bound":4},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-5563088896,"num_objects":-84886,"num_object_clones":0,"num_object_copies":-254658,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-84886,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":84886,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001662"},"pg_stats":[{"pgid":"6.f","version":"96'48233","reported_seq":48644,"reported_epoch":96,"state":"active+clean","last_fresh":"2026-02-24T07:56:30.897040+0000","last_change":"2026-02-24T07:56:12.247365+0000","last_active":"2026-02-24T07:56:30.897040+0000","last_peered":"2026-02-24T07:56:30.897040+0000","last_clean":"2026-02-24T07:56:30.897040+0000","last_became_active":"2026-02-24T07:54:42.927080+0000","last_became_peered":"2026-02-24T07:54:42.927080+0000","last_unstale":"2026-02-24T07:56:30.897040+0000","last_undegraded":"2026-02-24T07:56:30.897040+0000","last_fullsized":"2026-02-24T07:56:30.897040+0000","mapping_epoch":90,"log_start":"96'48230","ondisk_log_start":"96'48230","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"96'38780","last_scrub_stamp":"2026-02-24T07:56:12.247318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_clean_scrub_stamp":"2026-02-24T07:56:12.247318+0000","objects_scrubbed":37979,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:57:38.474117+0000","scrub_duration":4670,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2017263616,"num_objects":30781,"num_object_clones":0,"num_object_copies":92343,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30781,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":48233,"num_write_kb":2528448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.e","version":"96'48171","reported_seq":48556,"reported_epoch":96,"state":"active+clean","last_fresh":"2026-02-24T07:56:30.778111+0000","last_change":"2026-02-24T07:56:05.374565+0000","last_active":"2026-02-24T07:56:30.778111+0000","last_peered":"2026-02-24T07:56:30.778111+0000","last_clean":"2026-02-24T07:56:30.778111+0000","last_became_active":"2026-02-24T07:54:42.928748+0000","last_became_peered":"2026-02-24T07:54:42.928748+0000","last_unstale":"2026-02-24T07:56:30.778111+0000","last_undegraded":"2026-02-24T07:56:30.778111+0000","last_fullsized":"2026-02-24T07:56:30.778111+0000","mapping_epoch":90,"log_start":"96'48168","ondisk_log_start":"96'48168","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"96'36307","last_scrub_stamp":"2026-02-24T07:56:05.374510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_clean_scrub_stamp":"2026-02-24T07:56:05.374510+0000","objects_scrubbed":35491,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:57:16.427813+0000","scrub_duration":4036,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2028142592,"num_objects":30947,"num_object_clones":0,"num_object_copies":92841,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30947,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":48171,"num_write_kb":2531776,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.d","version":"96'48238","reported_seq":48556,"reported_epoch":96,"state":"active+clean","last_fresh":"2026-02-24T07:56:30.766449+0000","last_change":"2026-02-24T07:55:48.151749+0000","last_active":"2026-02-24T07:56:30.766449+0000","last_peered":"2026-02-24T07:56:30.766449+0000","last_clean":"2026-02-24T07:56:30.766449+0000","last_became_active":"2026-02-24T07:54:42.926886+0000","last_became_peered":"2026-02-24T07:54:42.926886+0000","last_unstale":"2026-02-24T07:56:30.766449+0000","last_undegraded":"2026-02-24T07:56:30.766449+0000","last_fullsized":"2026-02-24T07:56:30.766449+0000","mapping_epoch":90,"log_start":"96'48235","ondisk_log_start":"96'48235","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"96'28924","last_scrub_stamp":"2026-02-24T07:55:48.151672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_clean_scrub_stamp":"2026-02-24T07:55:48.151672+0000","objects_scrubbed":28526,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:57:00.390677+0000","scrub_duration":2742,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2023555096,"num_objects":30878,"num_object_clones":0,"num_object_copies":92634,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30878,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":48238,"num_write_kb":2531649,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.c","version":"96'48318","reported_seq":48704,"reported_epoch":96,"state":"active+clean","last_fresh":"2026-02-24T07:56:30.894882+0000","last_change":"2026-02-24T07:56:25.008652+0000","last_active":"2026-02-24T07:56:30.894882+0000","last_peered":"2026-02-24T07:56:30.894882+0000","last_clean":"2026-02-24T07:56:30.894882+0000","last_became_active":"2026-02-24T07:54:42.928238+0000","last_became_peered":"2026-02-24T07:54:42.928238+0000","last_unstale":"2026-02-24T07:56:30.894882+0000","last_undegraded":"2026-02-24T07:56:30.894882+0000","last_fullsized":"2026-02-24T07:56:30.894882+0000","mapping_epoch":90,"log_start":"96'48315","ondisk_log_start":"96'48315","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"96'44918","last_scrub_stamp":"2026-02-24T07:56:25.008567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_clean_scrub_stamp":"2026-02-24T07:56:25.008567+0000","objects_scrubbed":35276,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:57:40.096151+0000","scrub_duration":4395,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2021785600,"num_objects":30850,"num_object_clones":0,"num_object_copies":92550,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30850,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":48318,"num_write_kb":2533376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.b","version":"96'47844","reported_seq":48263,"reported_epoch":96,"state":"active+clean","last_fresh":"2026-02-24T07:56:30.892542+0000","last_change":"2026-02-24T07:56:16.120475+0000","last_active":"2026-02-24T07:56:30.892542+0000","last_peered":"2026-02-24T07:56:30.892542+0000","last_clean":"2026-02-24T07:56:30.892542+0000","last_became_active":"2026-02-24T07:54:42.926965+0000","last_became_peered":"2026-02-24T07:54:42.926965+0000","last_unstale":"2026-02-24T07:56:30.892542+0000","last_undegraded":"2026-02-24T07:56:30.892542+0000","last_fullsized":"2026-02-24T07:56:30.892542+0000","mapping_epoch":90,"log_start":"96'47841","ondisk_log_start":"96'47841","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"96'39969","last_scrub_stamp":"2026-02-24T07:56:16.120290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_clean_scrub_stamp":"2026-02-24T07:56:16.120290+0000","objects_scrubbed":38739,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:57:36.957012+0000","scrub_duration":4491,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2000289792,"num_objects":30522,"num_object_clones":0,"num_object_copies":91566,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30522,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":47844,"num_write_kb":2507712,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.a","version":"96'48087","reported_seq":48405,"reported_epoch":96,"state":"active+clean","last_fresh":"2026-02-24T07:56:30.779232+0000","last_change":"2026-02-24T07:55:49.167491+0000","last_active":"2026-02-24T07:56:30.779232+0000","last_peered":"2026-02-24T07:56:30.779232+0000","last_clean":"2026-02-24T07:56:30.779232+0000","last_became_active":"2026-02-24T07:54:42.926477+0000","last_became_peered":"2026-02-24T07:54:42.926477+0000","last_unstale":"2026-02-24T07:56:30.779232+0000","last_undegraded":"2026-02-24T07:56:30.779232+0000","last_fullsized":"2026-02-24T07:56:30.779232+0000","mapping_epoch":90,"log_start":"96'48084","ondisk_log_start":"96'48084","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"96'29334","last_scrub_stamp":"2026-02-24T07:55:49.167461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_clean_scrub_stamp":"2026-02-24T07:55:49.167461+0000","objects_scrubbed":28822,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:56:59.424140+0000","scrub_duration":2726,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2028797952,"num_objects":30957,"num_object_clones":0,"num_object_copies":92871,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30957,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":48087,"num_write_kb":2529408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"96'48314","reported_seq":48634,"reported_epoch":96,"state":"active+clean","last_fresh":"2026-02-24T07:56:30.813311+0000","last_change":"2026-02-24T07:55:48.535755+0000","last_active":"2026-02-24T07:56:30.813311+0000","last_peered":"2026-02-24T07:56:30.813311+0000","last_clean":"2026-02-24T07:56:30.813311+0000","last_became_active":"2026-02-24T07:54:42.926136+0000","last_became_peered":"2026-02-24T07:54:42.926136+0000","last_unstale":"2026-02-24T07:56:30.813311+0000","last_undegraded":"2026-02-24T07:56:30.813311+0000","last_fullsized":"2026-02-24T07:56:30.813311+0000","mapping_epoch":90,"log_start":"96'48310","ondisk_log_start":"96'48310","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"96'29148","last_scrub_stamp":"2026-02-24T07:55:48.535691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_clean_scrub_stamp":"2026-02-24T07:55:48.535691+0000","objects_scrubbed":28720,"log_size":4,"log_dups_size":3000,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:56:56.161155+0000","scrub_duration":2872,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2035154944,"num_objects":31054,"num_object_clones":0,"num_object_copies":93162,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31054,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":48314,"num_write_kb":2539776,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,1],"acting":[0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.8","version":"96'48366","reported_seq":48727,"reported_epoch":96,"state":"active+clean","last_fresh":"2026-02-24T07:56:30.793004+0000","last_change":"2026-02-24T07:55:58.178460+0000","last_active":"2026-02-24T07:56:30.793004+0000","last_peered":"2026-02-24T07:56:30.793004+0000","last_clean":"2026-02-24T07:56:30.793004+0000","last_became_active":"2026-02-24T07:54:42.929078+0000","last_became_peered":"2026-02-24T07:54:42.929078+0000","last_unstale":"2026-02-24T07:56:30.793004+0000","last_undegraded":"2026-02-24T07:56:30.793004+0000","last_fullsized":"2026-02-24T07:56:30.793004+0000","mapping_epoch":90,"log_start":"96'48363","ondisk_log_start":"96'48363","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"96'33550","last_scrub_stamp":"2026-02-24T07:55:58.178395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_clean_scrub_stamp":"2026-02-24T07:55:58.178395+0000","objects_scrubbed":32948,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:57:14.607073+0000","scrub_duration":2981,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2038956032,"num_objects":31112,"num_object_clones":0,"num_object_copies":93336,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31112,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":48366,"num_write_kb":2543296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.6","version":"96'48251","reported_seq":48664,"reported_epoch":96,"state":"active+clean","last_fresh":"2026-02-24T07:56:30.897352+0000","last_change":"2026-02-24T07:56:18.901177+0000","last_active":"2026-02-24T07:56:30.897352+0000","last_peered":"2026-02-24T07:56:30.897352+0000","last_clean":"2026-02-24T07:56:30.897352+0000","last_became_active":"2026-02-24T07:54:42.926045+0000","last_became_peered":"2026-02-24T07:54:42.926045+0000","last_unstale":"2026-02-24T07:56:30.897352+0000","last_undegraded":"2026-02-24T07:56:30.897352+0000","last_fullsized":"2026-02-24T07:56:30.897352+0000","mapping_epoch":90,"log_start":"96'48248","ondisk_log_start":"96'48248","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"96'41769","last_scrub_stamp":"2026-02-24T07:56:18.901111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_clean_scrub_stamp":"2026-02-24T07:56:18.901111+0000","objects_scrubbed":38122,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:57:27.205697+0000","scrub_duration":4228,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2009268224,"num_objects":30659,"num_object_clones":0,"num_object_copies":91977,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30659,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":48251,"num_write_kb":2525120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.5","version":"96'48168","reported_seq":48533,"reported_epoch":96,"state":"active+clean","last_fresh":"2026-02-24T07:56:30.779582+0000","last_change":"2026-02-24T07:55:59.773616+0000","last_active":"2026-02-24T07:56:30.779582+0000","last_peered":"2026-02-24T07:56:30.779582+0000","last_clean":"2026-02-24T07:56:30.779582+0000","last_became_active":"2026-02-24T07:54:42.927535+0000","last_became_peered":"2026-02-24T07:54:42.927535+0000","last_unstale":"2026-02-24T07:56:30.779582+0000","last_undegraded":"2026-02-24T07:56:30.779582+0000","last_fullsized":"2026-02-24T07:56:30.779582+0000","mapping_epoch":90,"log_start":"96'48165","ondisk_log_start":"96'48165","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"96'34080","last_scrub_stamp":"2026-02-24T07:55:59.773556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_clean_scrub_stamp":"2026-02-24T07:55:59.773556+0000","objects_scrubbed":33431,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:57:04.302607+0000","scrub_duration":3324,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2020212736,"num_objects":30826,"num_object_clones":0,"num_object_copies":92478,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30826,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":48168,"num_write_kb":2527808,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.4","version":"96'48231","reported_seq":48549,"reported_epoch":96,"state":"active+clean","last_fresh":"2026-02-24T07:56:30.898215+0000","last_change":"2026-02-24T07:55:47.544043+0000","last_active":"2026-02-24T07:56:30.898215+0000","last_peered":"2026-02-24T07:56:30.898215+0000","last_clean":"2026-02-24T07:56:30.898215+0000","last_became_active":"2026-02-24T07:54:42.928011+0000","last_became_peered":"2026-02-24T07:54:42.928011+0000","last_unstale":"2026-02-24T07:56:30.898215+0000","last_undegraded":"2026-02-24T07:56:30.898215+0000","last_fullsized":"2026-02-24T07:56:30.898215+0000","mapping_epoch":90,"log_start":"96'48228","ondisk_log_start":"96'48228","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"96'29055","last_scrub_stamp":"2026-02-24T07:55:47.543994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_clean_scrub_stamp":"2026-02-24T07:55:47.543994+0000","objects_scrubbed":28510,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:57:04.321401+0000","scrub_duration":3165,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2009530368,"num_objects":30663,"num_object_clones":0,"num_object_copies":91989,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30663,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":48231,"num_write_kb":2524608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.3","version":"96'48511","reported_seq":48903,"reported_epoch":96,"state":"active+clean","last_fresh":"2026-02-24T07:56:30.779086+0000","last_change":"2026-02-24T07:56:06.184345+0000","last_active":"2026-02-24T07:56:30.779086+0000","last_peered":"2026-02-24T07:56:30.779086+0000","last_clean":"2026-02-24T07:56:30.779086+0000","last_became_active":"2026-02-24T07:54:42.927730+0000","last_became_peered":"2026-02-24T07:54:42.927730+0000","last_unstale":"2026-02-24T07:56:30.779086+0000","last_undegraded":"2026-02-24T07:56:30.779086+0000","last_fullsized":"2026-02-24T07:56:30.779086+0000","mapping_epoch":90,"log_start":"96'48508","ondisk_log_start":"96'48508","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"96'36664","last_scrub_stamp":"2026-02-24T07:56:06.184309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_clean_scrub_stamp":"2026-02-24T07:56:06.184309+0000","objects_scrubbed":35946,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:57:11.733048+0000","scrub_duration":3811,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2027749376,"num_objects":30941,"num_object_clones":0,"num_object_copies":92823,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30941,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":48511,"num_write_kb":2542464,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.2","version":"96'48214","reported_seq":48582,"reported_epoch":96,"state":"active+clean","last_fresh":"2026-02-24T07:56:30.795661+0000","last_change":"2026-02-24T07:56:00.120637+0000","last_active":"2026-02-24T07:56:30.795661+0000","last_peered":"2026-02-24T07:56:30.795661+0000","last_clean":"2026-02-24T07:56:30.795661+0000","last_became_active":"2026-02-24T07:54:42.930247+0000","last_became_peered":"2026-02-24T07:54:42.930247+0000","last_unstale":"2026-02-24T07:56:30.795661+0000","last_undegraded":"2026-02-24T07:56:30.795661+0000","last_fullsized":"2026-02-24T07:56:30.795661+0000","mapping_epoch":90,"log_start":"96'48211","ondisk_log_start":"96'48211","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"96'34210","last_scrub_stamp":"2026-02-24T07:56:00.120605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_clean_scrub_stamp":"2026-02-24T07:56:00.120605+0000","objects_scrubbed":33656,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:57:29.329949+0000","scrub_duration":2957,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2018115584,"num_objects":30794,"num_object_clones":0,"num_object_copies":92382,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30794,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":48214,"num_write_kb":2528256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1","version":"96'48550","reported_seq":48964,"reported_epoch":96,"state":"active+clean","last_fresh":"2026-02-24T07:56:30.896117+0000","last_change":"2026-02-24T07:56:12.897479+0000","last_active":"2026-02-24T07:56:30.896117+0000","last_peered":"2026-02-24T07:56:30.896117+0000","last_clean":"2026-02-24T07:56:30.896117+0000","last_became_active":"2026-02-24T07:54:42.926442+0000","last_became_peered":"2026-02-24T07:54:42.926442+0000","last_unstale":"2026-02-24T07:56:30.896117+0000","last_undegraded":"2026-02-24T07:56:30.896117+0000","last_fullsized":"2026-02-24T07:56:30.896117+0000","mapping_epoch":90,"log_start":"96'48547","ondisk_log_start":"96'48547","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"96'39245","last_scrub_stamp":"2026-02-24T07:56:12.897436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_clean_scrub_stamp":"2026-02-24T07:56:12.897436+0000","objects_scrubbed":38506,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:57:42.057233+0000","scrub_duration":4279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2028339200,"num_objects":30950,"num_object_clones":0,"num_object_copies":92850,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30950,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":48550,"num_write_kb":2544000,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.0","version":"96'47794","reported_seq":48170,"reported_epoch":96,"state":"active+clean","last_fresh":"2026-02-24T07:56:30.810873+0000","last_change":"2026-02-24T07:56:02.992243+0000","last_active":"2026-02-24T07:56:30.810873+0000","last_peered":"2026-02-24T07:56:30.810873+0000","last_clean":"2026-02-24T07:56:30.810873+0000","last_became_active":"2026-02-24T07:54:42.926910+0000","last_became_peered":"2026-02-24T07:54:42.926910+0000","last_unstale":"2026-02-24T07:56:30.810873+0000","last_undegraded":"2026-02-24T07:56:30.810873+0000","last_fullsized":"2026-02-24T07:56:30.810873+0000","mapping_epoch":90,"log_start":"96'47791","ondisk_log_start":"96'47791","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"96'34952","last_scrub_stamp":"2026-02-24T07:56:02.992156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_clean_scrub_stamp":"2026-02-24T07:56:02.992156+0000","objects_scrubbed":34329,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:57:03.847719+0000","scrub_duration":3335,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1999503360,"num_objects":30510,"num_object_clones":0,"num_object_copies":91530,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30510,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":47794,"num_write_kb":2505728,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.7","version":"96'48453","reported_seq":48828,"reported_epoch":96,"state":"active+clean","last_fresh":"2026-02-24T07:56:30.794619+0000","last_change":"2026-02-24T07:56:01.088786+0000","last_active":"2026-02-24T07:56:30.794619+0000","last_peered":"2026-02-24T07:56:30.794619+0000","last_clean":"2026-02-24T07:56:30.794619+0000","last_became_active":"2026-02-24T07:54:42.930142+0000","last_became_peered":"2026-02-24T07:54:42.930142+0000","last_unstale":"2026-02-24T07:56:30.794619+0000","last_undegraded":"2026-02-24T07:56:30.794619+0000","last_fullsized":"2026-02-24T07:56:30.794619+0000","mapping_epoch":90,"log_start":"96'48450","ondisk_log_start":"96'48450","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"96'34838","last_scrub_stamp":"2026-02-24T07:56:01.088744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_clean_scrub_stamp":"2026-02-24T07:56:01.088744+0000","objects_scrubbed":34284,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:57:24.696242+0000","scrub_duration":2973,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2042429440,"num_objects":31165,"num_object_clones":0,"num_object_copies":93495,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31165,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":48453,"num_write_kb":2547776,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"29'99","reported_seq":398,"reported_epoch":96,"state":"active+clean","last_fresh":"2026-02-24T07:55:16.398160+0000","last_change":"2026-02-24T07:55:16.398160+0000","last_active":"2026-02-24T07:55:16.398160+0000","last_peered":"2026-02-24T07:55:16.398160+0000","last_clean":"2026-02-24T07:55:16.398160+0000","last_became_active":"2026-02-24T07:42:54.186719+0000","last_became_peered":"2026-02-24T07:42:54.186719+0000","last_unstale":"2026-02-24T07:55:16.398160+0000","last_undegraded":"2026-02-24T07:55:16.398160+0000","last_fullsized":"2026-02-24T07:55:16.398160+0000","mapping_epoch":27,"log_start":"29'96","ondisk_log_start":"29'96","created":20,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'99","last_scrub_stamp":"2026-02-24T07:55:16.398131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:42:43.640776+0000","last_clean_scrub_stamp":"2026-02-24T07:55:16.398131+0000","objects_scrubbed":2,"log_size":3,"log_dups_size":96,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:56:32.925996+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":240,"num_write":135,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":16,"stat_sum":{"num_bytes":32349093912,"num_objects":493609,"num_object_clones":0,"num_object_copies":1480827,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":493609,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":771743,"num_write_kb":40491201,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":97056862208,"data_stored":97056849992,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":49,"ondisk_log_size":49,"up":48,"acting":48,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":240,"num_write":135,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":2396,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":3,"up_from":25,"seq":107374182566,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":21949012,"kb_used_data":20073920,"kb_used_omap":18,"kb_used_meta":1874989,"kb_avail":712054188,"statfs":{"total":751619276800,"available":729143488512,"internally_reserved":0,"allocated":20555694080,"data_stored":20555191462,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":18972,"internal_metadata":1919989220},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":19,"seq":81604378792,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":24646296,"kb_used_data":23916740,"kb_used_omap":20,"kb_used_meta":729451,"kb_avail":709356904,"statfs":{"total":751619276800,"available":726381469696,"internally_reserved":0,"allocated":24490741760,"data_stored":24490235070,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":20830,"internal_metadata":746958498},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[4],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542313,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":28606612,"kb_used_data":27802624,"kb_used_omap":21,"kb_used_meta":803818,"kb_avail":705396588,"statfs":{"total":751619276800,"available":722326106112,"internally_reserved":0,"allocated":28469886976,"data_stored":28469383838,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":22243,"internal_metadata":823109917},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[4,3],"upper_bound":4},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705836,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":25860184,"kb_used_data":23784708,"kb_used_omap":17,"kb_used_meta":2075438,"kb_avail":708143016,"statfs":{"total":751619276800,"available":725138448384,"internally_reserved":0,"allocated":24355540992,"data_stored":24355034302,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":18431,"internal_metadata":2125248513},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":2396,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":24267853824,"data_stored":24267849752,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28275511296,"data_stored":28275507224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":24257892352,"data_stored":24257888280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":20255604736,"data_stored":20255604736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-24T07:56:34.823 INFO:tasks.cephadm.ceph_manager.ceph:increase pool size by 10 2026-02-24T07:56:34.823 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph osd pool set .mgr pg_num 441 2026-02-24T07:56:34.959 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:56:35.077 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:56:34 trial035 ceph-mon[39284]: pgmap v477: 17 pgs: 17 active+clean; 30 GiB data, 96 GiB used, 2.6 TiB / 2.7 TiB avail; 0 B/s wr, 7.07k op/s 2026-02-24T07:56:35.488 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:56:35.428+0000 7f58ff638640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:56:35.488 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:56:35.428+0000 7f58ff638640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:56:35.580 INFO:teuthology.orchestra.run.trial035.stderr:Error EPERM: crimson_allow_pg_split is false; pg_num increase denied for crimson pool 2026-02-24T07:56:35.679 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2026-02-24T07:56:35.954 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:56:35 trial035 ceph-mon[39284]: from='client.15577 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2026-02-24T07:56:35.954 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:56:35 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/2134629018' entity='client.admin' cmd={"prefix": "osd pool set", "pool": ".mgr", "var": "pg_num", "val": "441"} : dispatch 2026-02-24T07:56:35.954 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:56:35 trial035 ceph-mon[39284]: pgmap v478: 17 pgs: 17 active+clean; 30 GiB data, 96 GiB used, 2.6 TiB / 2.7 TiB avail; 0 B/s wr, 7.07k op/s 2026-02-24T07:56:36.293 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-24T07:56:36.347 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:56:36.874 INFO:teuthology.orchestra.run.trial035.stdout:osd.0: {} 2026-02-24T07:56:36.874 INFO:teuthology.orchestra.run.trial035.stdout:osd.1: {} 2026-02-24T07:56:36.874 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:56:36.839+0000 7f1fcd894640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:56:36.874 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:56:36.839+0000 7f1fcd894640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:56:36.874 INFO:teuthology.orchestra.run.trial035.stderr:osd.0: osd_enable_op_tracker = '' 2026-02-24T07:56:36.874 INFO:teuthology.orchestra.run.trial035.stderr:osd.1: osd_enable_op_tracker = '' 2026-02-24T07:56:36.882 INFO:teuthology.orchestra.run.trial035.stdout:osd.2: {} 2026-02-24T07:56:36.882 INFO:teuthology.orchestra.run.trial035.stderr:osd.2: osd_enable_op_tracker = '' 2026-02-24T07:56:36.894 INFO:teuthology.orchestra.run.trial035.stdout:osd.3: {} 2026-02-24T07:56:36.894 INFO:teuthology.orchestra.run.trial035.stderr:osd.3: osd_enable_op_tracker = '' 2026-02-24T07:56:37.196 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:56:36 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/3139262572' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2026-02-24T07:56:37.196 INFO:journalctl@ceph.mgr.x.trial035.stdout:Feb 24 07:56:36 trial035 ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-mgr-x[39621]: ::ffff:10.20.193.91 - - [24/Feb/2026:07:56:36] "GET /metrics HTTP/1.1" 200 4235 "" "Prometheus/3.6.0" 2026-02-24T07:56:37.356 INFO:journalctl@ceph.prometheus.a.trial091.stdout:Feb 24 07:56:36 trial091 ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-prometheus-a[66007]: time=2026-02-24T07:56:36.973Z level=ERROR source=scrape.go:1630 msg="Failed to determine correct type of scrape target." component="scrape manager" scrape_pool=ceph target=http://trial035:9283/metrics content_type="text/html;charset=utf-8" fallback_media_type="" err="received unsupported Content-Type \"text/html;charset=utf-8\" and no fallback_scrape_protocol specified for target" 2026-02-24T07:56:38.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:56:37 trial035 ceph-mon[39284]: pgmap v479: 17 pgs: 17 active+clean; 28 GiB data, 88 GiB used, 2.6 TiB / 2.7 TiB avail; 0 B/s wr, 10.10k op/s 2026-02-24T07:56:39.198 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2026-02-24T07:56:39.342 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:56:39.476 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:56:39.474+0000 7f94fedb8640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:56:39.476 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:56:39.474+0000 7f94fedb8640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:56:39.491 INFO:teuthology.orchestra.run.trial035.stdout:osd.0: {} 2026-02-24T07:56:39.492 INFO:teuthology.orchestra.run.trial035.stderr:osd.0: osd_enable_op_tracker = '' 2026-02-24T07:56:39.507 INFO:teuthology.orchestra.run.trial035.stdout:osd.1: {} 2026-02-24T07:56:39.507 INFO:teuthology.orchestra.run.trial035.stderr:osd.1: osd_enable_op_tracker = '' 2026-02-24T07:56:39.521 INFO:teuthology.orchestra.run.trial035.stdout:osd.2: {} 2026-02-24T07:56:39.521 INFO:teuthology.orchestra.run.trial035.stderr:osd.2: osd_enable_op_tracker = '' 2026-02-24T07:56:39.533 INFO:teuthology.orchestra.run.trial035.stdout:osd.3: {} 2026-02-24T07:56:39.534 INFO:teuthology.orchestra.run.trial035.stderr:osd.3: osd_enable_op_tracker = '' 2026-02-24T07:56:40.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:56:39 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/4189587610' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2026-02-24T07:56:41.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:56:40 trial035 ceph-mon[39284]: pgmap v480: 17 pgs: 17 active+clean; 28 GiB data, 88 GiB used, 2.6 TiB / 2.7 TiB avail; 0 B/s wr, 6.92k op/s 2026-02-24T07:56:41.294 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2026-02-24T07:56:41.295 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2026-02-24T07:56:41.295 INFO:tasks.thrashosds.thrasher:Growing pool unique_pool_4 2026-02-24T07:56:41.296 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph pg dump --format=json 2026-02-24T07:56:41.437 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:56:41.575 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:56:41.573+0000 7fa1c8857640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:56:41.575 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:56:41.574+0000 7fa1c8857640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:56:41.575 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2026-02-24T07:56:41.724 INFO:teuthology.orchestra.run.trial035.stdout: 2026-02-24T07:56:41.724 INFO:teuthology.orchestra.run.trial035.stderr:dumped all 2026-02-24T07:56:41.725 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:56:41.770 INFO:teuthology.orchestra.run.trial035.stdout:{"pg_ready":true,"pg_map":{"version":480,"stamp":"2026-02-24T07:56:39.769022+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":29969220152,"num_objects":457288,"num_object_clones":0,"num_object_copies":1371864,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":457288,"num_whiteouts":0,"num_read":137,"num_read_kb":241,"num_write":808201,"num_write_kb":40493351,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":51,"ondisk_log_size":51,"up":51,"acting":51,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":51,"num_osds":4,"num_per_pool_osds":4,"num_per_pool_omap_osds":4,"kb":2936012800,"kb_used":92578264,"kb_used_data":88374664,"kb_used_omap":78,"kb_used_meta":4203249,"kb_avail":2843434536,"statfs":{"total":3006477107200,"available":2911676964864,"internally_reserved":0,"allocated":90495655936,"data_stored":90493636800,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":80382,"internal_metadata":4304127490},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-5445844992,"num_objects":-83097,"num_object_clones":0,"num_object_copies":-249291,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-83097,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":83097,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001659"},"pg_stats":[{"pgid":"6.f","version":"96'50410","reported_seq":50821,"reported_epoch":96,"state":"active+clean","last_fresh":"2026-02-24T07:56:35.880617+0000","last_change":"2026-02-24T07:56:12.247365+0000","last_active":"2026-02-24T07:56:35.880617+0000","last_peered":"2026-02-24T07:56:35.880617+0000","last_clean":"2026-02-24T07:56:35.880617+0000","last_became_active":"2026-02-24T07:54:42.927080+0000","last_became_peered":"2026-02-24T07:54:42.927080+0000","last_unstale":"2026-02-24T07:56:35.880617+0000","last_undegraded":"2026-02-24T07:56:35.880617+0000","last_fullsized":"2026-02-24T07:56:35.880617+0000","mapping_epoch":90,"log_start":"96'50407","ondisk_log_start":"96'50407","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"96'38780","last_scrub_stamp":"2026-02-24T07:56:12.247318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_clean_scrub_stamp":"2026-02-24T07:56:12.247318+0000","objects_scrubbed":37979,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:57:38.474117+0000","scrub_duration":4670,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1874591744,"num_objects":28604,"num_object_clones":0,"num_object_copies":85812,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28604,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":50410,"num_write_kb":2528448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.e","version":"96'50595","reported_seq":50979,"reported_epoch":96,"state":"active+clean","last_fresh":"2026-02-24T07:56:35.780234+0000","last_change":"2026-02-24T07:56:05.374565+0000","last_active":"2026-02-24T07:56:35.780234+0000","last_peered":"2026-02-24T07:56:35.780234+0000","last_clean":"2026-02-24T07:56:35.780234+0000","last_became_active":"2026-02-24T07:54:42.928748+0000","last_became_peered":"2026-02-24T07:54:42.928748+0000","last_unstale":"2026-02-24T07:56:35.780234+0000","last_undegraded":"2026-02-24T07:56:35.780234+0000","last_fullsized":"2026-02-24T07:56:35.780234+0000","mapping_epoch":90,"log_start":"96'50592","ondisk_log_start":"96'50592","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"96'36307","last_scrub_stamp":"2026-02-24T07:56:05.374510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_clean_scrub_stamp":"2026-02-24T07:56:05.374510+0000","objects_scrubbed":35491,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:57:16.427813+0000","scrub_duration":4036,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1869283328,"num_objects":28523,"num_object_clones":0,"num_object_copies":85569,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28523,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":50595,"num_write_kb":2531776,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.d","version":"96'50537","reported_seq":50855,"reported_epoch":96,"state":"active+clean","last_fresh":"2026-02-24T07:56:35.776076+0000","last_change":"2026-02-24T07:55:48.151749+0000","last_active":"2026-02-24T07:56:35.776076+0000","last_peered":"2026-02-24T07:56:35.776076+0000","last_clean":"2026-02-24T07:56:35.776076+0000","last_became_active":"2026-02-24T07:54:42.926886+0000","last_became_peered":"2026-02-24T07:54:42.926886+0000","last_unstale":"2026-02-24T07:56:35.776076+0000","last_undegraded":"2026-02-24T07:56:35.776076+0000","last_fullsized":"2026-02-24T07:56:35.776076+0000","mapping_epoch":90,"log_start":"96'50534","ondisk_log_start":"96'50534","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"96'28924","last_scrub_stamp":"2026-02-24T07:55:48.151672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_clean_scrub_stamp":"2026-02-24T07:55:48.151672+0000","objects_scrubbed":28526,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:57:00.390677+0000","scrub_duration":2742,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1872887832,"num_objects":28579,"num_object_clones":0,"num_object_copies":85737,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28579,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":50537,"num_write_kb":2531649,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.c","version":"96'50637","reported_seq":51023,"reported_epoch":96,"state":"active+clean","last_fresh":"2026-02-24T07:56:35.879298+0000","last_change":"2026-02-24T07:56:25.008652+0000","last_active":"2026-02-24T07:56:35.879298+0000","last_peered":"2026-02-24T07:56:35.879298+0000","last_clean":"2026-02-24T07:56:35.879298+0000","last_became_active":"2026-02-24T07:54:42.928238+0000","last_became_peered":"2026-02-24T07:54:42.928238+0000","last_unstale":"2026-02-24T07:56:35.879298+0000","last_undegraded":"2026-02-24T07:56:35.879298+0000","last_fullsized":"2026-02-24T07:56:35.879298+0000","mapping_epoch":90,"log_start":"96'50634","ondisk_log_start":"96'50634","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"96'44918","last_scrub_stamp":"2026-02-24T07:56:25.008567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_clean_scrub_stamp":"2026-02-24T07:56:25.008567+0000","objects_scrubbed":35276,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:57:40.096151+0000","scrub_duration":4395,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1869807616,"num_objects":28531,"num_object_clones":0,"num_object_copies":85593,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28531,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":50637,"num_write_kb":2533376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.b","version":"96'50121","reported_seq":50538,"reported_epoch":96,"state":"active+clean","last_fresh":"2026-02-24T07:56:35.883117+0000","last_change":"2026-02-24T07:56:16.120475+0000","last_active":"2026-02-24T07:56:35.883117+0000","last_peered":"2026-02-24T07:56:35.883117+0000","last_clean":"2026-02-24T07:56:35.883117+0000","last_became_active":"2026-02-24T07:54:42.926965+0000","last_became_peered":"2026-02-24T07:54:42.926965+0000","last_unstale":"2026-02-24T07:56:35.883117+0000","last_undegraded":"2026-02-24T07:56:35.883117+0000","last_fullsized":"2026-02-24T07:56:35.883117+0000","mapping_epoch":90,"log_start":"96'50118","ondisk_log_start":"96'50118","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"96'39969","last_scrub_stamp":"2026-02-24T07:56:16.120290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_clean_scrub_stamp":"2026-02-24T07:56:16.120290+0000","objects_scrubbed":38739,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:57:36.957012+0000","scrub_duration":4491,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1851064320,"num_objects":28245,"num_object_clones":0,"num_object_copies":84735,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28245,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":50121,"num_write_kb":2507712,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.a","version":"96'50296","reported_seq":50615,"reported_epoch":96,"state":"active+clean","last_fresh":"2026-02-24T07:56:35.776118+0000","last_change":"2026-02-24T07:55:49.167491+0000","last_active":"2026-02-24T07:56:35.776118+0000","last_peered":"2026-02-24T07:56:35.776118+0000","last_clean":"2026-02-24T07:56:35.776118+0000","last_became_active":"2026-02-24T07:54:42.926477+0000","last_became_peered":"2026-02-24T07:54:42.926477+0000","last_unstale":"2026-02-24T07:56:35.776118+0000","last_undegraded":"2026-02-24T07:56:35.776118+0000","last_fullsized":"2026-02-24T07:56:35.776118+0000","mapping_epoch":90,"log_start":"96'50293","ondisk_log_start":"96'50293","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"96'29334","last_scrub_stamp":"2026-02-24T07:55:49.167461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_clean_scrub_stamp":"2026-02-24T07:55:49.167461+0000","objects_scrubbed":28822,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:56:59.424140+0000","scrub_duration":2726,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1884028928,"num_objects":28748,"num_object_clones":0,"num_object_copies":86244,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28748,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":50296,"num_write_kb":2529408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"96'50597","reported_seq":50917,"reported_epoch":96,"state":"active+clean","last_fresh":"2026-02-24T07:56:35.777827+0000","last_change":"2026-02-24T07:55:48.535755+0000","last_active":"2026-02-24T07:56:35.777827+0000","last_peered":"2026-02-24T07:56:35.777827+0000","last_clean":"2026-02-24T07:56:35.777827+0000","last_became_active":"2026-02-24T07:54:42.926136+0000","last_became_peered":"2026-02-24T07:54:42.926136+0000","last_unstale":"2026-02-24T07:56:35.777827+0000","last_undegraded":"2026-02-24T07:56:35.777827+0000","last_fullsized":"2026-02-24T07:56:35.777827+0000","mapping_epoch":90,"log_start":"96'50594","ondisk_log_start":"96'50594","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"96'29148","last_scrub_stamp":"2026-02-24T07:55:48.535691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_clean_scrub_stamp":"2026-02-24T07:55:48.535691+0000","objects_scrubbed":28720,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:56:56.161155+0000","scrub_duration":2872,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1885536256,"num_objects":28771,"num_object_clones":0,"num_object_copies":86313,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28771,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":50597,"num_write_kb":2539776,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,1],"acting":[0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.8","version":"96'50643","reported_seq":51004,"reported_epoch":96,"state":"active+clean","last_fresh":"2026-02-24T07:56:35.775134+0000","last_change":"2026-02-24T07:55:58.178460+0000","last_active":"2026-02-24T07:56:35.775134+0000","last_peered":"2026-02-24T07:56:35.775134+0000","last_clean":"2026-02-24T07:56:35.775134+0000","last_became_active":"2026-02-24T07:54:42.929078+0000","last_became_peered":"2026-02-24T07:54:42.929078+0000","last_unstale":"2026-02-24T07:56:35.775134+0000","last_undegraded":"2026-02-24T07:56:35.775134+0000","last_fullsized":"2026-02-24T07:56:35.775134+0000","mapping_epoch":90,"log_start":"96'50640","ondisk_log_start":"96'50640","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"96'33550","last_scrub_stamp":"2026-02-24T07:55:58.178395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_clean_scrub_stamp":"2026-02-24T07:55:58.178395+0000","objects_scrubbed":32948,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:57:14.607073+0000","scrub_duration":2981,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1889730560,"num_objects":28835,"num_object_clones":0,"num_object_copies":86505,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28835,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":50643,"num_write_kb":2543296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.6","version":"96'50485","reported_seq":50898,"reported_epoch":96,"state":"active+clean","last_fresh":"2026-02-24T07:56:35.880310+0000","last_change":"2026-02-24T07:56:18.901177+0000","last_active":"2026-02-24T07:56:35.880310+0000","last_peered":"2026-02-24T07:56:35.880310+0000","last_clean":"2026-02-24T07:56:35.880310+0000","last_became_active":"2026-02-24T07:54:42.926045+0000","last_became_peered":"2026-02-24T07:54:42.926045+0000","last_unstale":"2026-02-24T07:56:35.880310+0000","last_undegraded":"2026-02-24T07:56:35.880310+0000","last_fullsized":"2026-02-24T07:56:35.880310+0000","mapping_epoch":90,"log_start":"96'50482","ondisk_log_start":"96'50482","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"96'41769","last_scrub_stamp":"2026-02-24T07:56:18.901111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_clean_scrub_stamp":"2026-02-24T07:56:18.901111+0000","objects_scrubbed":38122,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:57:27.205697+0000","scrub_duration":4228,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1862860800,"num_objects":28425,"num_object_clones":0,"num_object_copies":85275,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28425,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":50485,"num_write_kb":2525120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.5","version":"96'50487","reported_seq":50852,"reported_epoch":96,"state":"active+clean","last_fresh":"2026-02-24T07:56:35.775575+0000","last_change":"2026-02-24T07:55:59.773616+0000","last_active":"2026-02-24T07:56:35.775575+0000","last_peered":"2026-02-24T07:56:35.775575+0000","last_clean":"2026-02-24T07:56:35.775575+0000","last_became_active":"2026-02-24T07:54:42.927535+0000","last_became_peered":"2026-02-24T07:54:42.927535+0000","last_unstale":"2026-02-24T07:56:35.775575+0000","last_undegraded":"2026-02-24T07:56:35.775575+0000","last_fullsized":"2026-02-24T07:56:35.775575+0000","mapping_epoch":90,"log_start":"96'50484","ondisk_log_start":"96'50484","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"96'34080","last_scrub_stamp":"2026-02-24T07:55:59.773556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_clean_scrub_stamp":"2026-02-24T07:55:59.773556+0000","objects_scrubbed":33431,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:57:04.302607+0000","scrub_duration":3324,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1868234752,"num_objects":28507,"num_object_clones":0,"num_object_copies":85521,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28507,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":50487,"num_write_kb":2527808,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.4","version":"96'50494","reported_seq":50812,"reported_epoch":96,"state":"active+clean","last_fresh":"2026-02-24T07:56:35.877397+0000","last_change":"2026-02-24T07:55:47.544043+0000","last_active":"2026-02-24T07:56:35.877397+0000","last_peered":"2026-02-24T07:56:35.877397+0000","last_clean":"2026-02-24T07:56:35.877397+0000","last_became_active":"2026-02-24T07:54:42.928011+0000","last_became_peered":"2026-02-24T07:54:42.928011+0000","last_unstale":"2026-02-24T07:56:35.877397+0000","last_undegraded":"2026-02-24T07:56:35.877397+0000","last_fullsized":"2026-02-24T07:56:35.877397+0000","mapping_epoch":90,"log_start":"96'50491","ondisk_log_start":"96'50491","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"96'29055","last_scrub_stamp":"2026-02-24T07:55:47.543994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_clean_scrub_stamp":"2026-02-24T07:55:47.543994+0000","objects_scrubbed":28510,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:57:04.321401+0000","scrub_duration":3165,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1861222400,"num_objects":28400,"num_object_clones":0,"num_object_copies":85200,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28400,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":50494,"num_write_kb":2524608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.3","version":"96'50836","reported_seq":51228,"reported_epoch":96,"state":"active+clean","last_fresh":"2026-02-24T07:56:35.773173+0000","last_change":"2026-02-24T07:56:06.184345+0000","last_active":"2026-02-24T07:56:35.773173+0000","last_peered":"2026-02-24T07:56:35.773173+0000","last_clean":"2026-02-24T07:56:35.773173+0000","last_became_active":"2026-02-24T07:54:42.927730+0000","last_became_peered":"2026-02-24T07:54:42.927730+0000","last_unstale":"2026-02-24T07:56:35.773173+0000","last_undegraded":"2026-02-24T07:56:35.773173+0000","last_fullsized":"2026-02-24T07:56:35.773173+0000","mapping_epoch":90,"log_start":"96'50833","ondisk_log_start":"96'50833","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"96'36664","last_scrub_stamp":"2026-02-24T07:56:06.184309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_clean_scrub_stamp":"2026-02-24T07:56:06.184309+0000","objects_scrubbed":35946,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:57:11.733048+0000","scrub_duration":3811,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1875378176,"num_objects":28616,"num_object_clones":0,"num_object_copies":85848,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28616,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":50836,"num_write_kb":2542464,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.2","version":"96'50425","reported_seq":50794,"reported_epoch":96,"state":"active+clean","last_fresh":"2026-02-24T07:56:35.771661+0000","last_change":"2026-02-24T07:56:00.120637+0000","last_active":"2026-02-24T07:56:35.771661+0000","last_peered":"2026-02-24T07:56:35.771661+0000","last_clean":"2026-02-24T07:56:35.771661+0000","last_became_active":"2026-02-24T07:54:42.930247+0000","last_became_peered":"2026-02-24T07:54:42.930247+0000","last_unstale":"2026-02-24T07:56:35.771661+0000","last_undegraded":"2026-02-24T07:56:35.771661+0000","last_fullsized":"2026-02-24T07:56:35.771661+0000","mapping_epoch":90,"log_start":"96'50422","ondisk_log_start":"96'50422","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"96'34210","last_scrub_stamp":"2026-02-24T07:56:00.120605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_clean_scrub_stamp":"2026-02-24T07:56:00.120605+0000","objects_scrubbed":33656,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:57:29.329949+0000","scrub_duration":2957,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1873215488,"num_objects":28583,"num_object_clones":0,"num_object_copies":85749,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28583,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":50425,"num_write_kb":2528256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1","version":"96'50789","reported_seq":51203,"reported_epoch":96,"state":"active+clean","last_fresh":"2026-02-24T07:56:35.883647+0000","last_change":"2026-02-24T07:56:12.897479+0000","last_active":"2026-02-24T07:56:35.883647+0000","last_peered":"2026-02-24T07:56:35.883647+0000","last_clean":"2026-02-24T07:56:35.883647+0000","last_became_active":"2026-02-24T07:54:42.926442+0000","last_became_peered":"2026-02-24T07:54:42.926442+0000","last_unstale":"2026-02-24T07:56:35.883647+0000","last_undegraded":"2026-02-24T07:56:35.883647+0000","last_fullsized":"2026-02-24T07:56:35.883647+0000","mapping_epoch":90,"log_start":"96'50786","ondisk_log_start":"96'50786","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"96'39245","last_scrub_stamp":"2026-02-24T07:56:12.897436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_clean_scrub_stamp":"2026-02-24T07:56:12.897436+0000","objects_scrubbed":38506,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:57:42.057233+0000","scrub_duration":4279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1881604096,"num_objects":28711,"num_object_clones":0,"num_object_copies":86133,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28711,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":50789,"num_write_kb":2544000,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.0","version":"96'49974","reported_seq":50350,"reported_epoch":96,"state":"active+clean","last_fresh":"2026-02-24T07:56:35.811689+0000","last_change":"2026-02-24T07:56:02.992243+0000","last_active":"2026-02-24T07:56:35.811689+0000","last_peered":"2026-02-24T07:56:35.811689+0000","last_clean":"2026-02-24T07:56:35.811689+0000","last_became_active":"2026-02-24T07:54:42.926910+0000","last_became_peered":"2026-02-24T07:54:42.926910+0000","last_unstale":"2026-02-24T07:56:35.811689+0000","last_undegraded":"2026-02-24T07:56:35.811689+0000","last_fullsized":"2026-02-24T07:56:35.811689+0000","mapping_epoch":90,"log_start":"96'49971","ondisk_log_start":"96'49971","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"96'34952","last_scrub_stamp":"2026-02-24T07:56:02.992156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_clean_scrub_stamp":"2026-02-24T07:56:02.992156+0000","objects_scrubbed":34329,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:57:03.847719+0000","scrub_duration":3335,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1856634880,"num_objects":28330,"num_object_clones":0,"num_object_copies":84990,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28330,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":49974,"num_write_kb":2505728,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.7","version":"96'50740","reported_seq":51115,"reported_epoch":96,"state":"active+clean","last_fresh":"2026-02-24T07:56:35.774730+0000","last_change":"2026-02-24T07:56:01.088786+0000","last_active":"2026-02-24T07:56:35.774730+0000","last_peered":"2026-02-24T07:56:35.774730+0000","last_clean":"2026-02-24T07:56:35.774730+0000","last_became_active":"2026-02-24T07:54:42.930142+0000","last_became_peered":"2026-02-24T07:54:42.930142+0000","last_unstale":"2026-02-24T07:56:35.774730+0000","last_undegraded":"2026-02-24T07:56:35.774730+0000","last_fullsized":"2026-02-24T07:56:35.774730+0000","mapping_epoch":90,"log_start":"96'50737","ondisk_log_start":"96'50737","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"96'34838","last_scrub_stamp":"2026-02-24T07:56:01.088744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_clean_scrub_stamp":"2026-02-24T07:56:01.088744+0000","objects_scrubbed":34284,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:57:24.696242+0000","scrub_duration":2973,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1892548608,"num_objects":28878,"num_object_clones":0,"num_object_copies":86634,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28878,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":50740,"num_write_kb":2547776,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"29'99","reported_seq":406,"reported_epoch":96,"state":"active+clean","last_fresh":"2026-02-24T07:56:33.056711+0000","last_change":"2026-02-24T07:56:33.056711+0000","last_active":"2026-02-24T07:56:33.056711+0000","last_peered":"2026-02-24T07:56:33.056711+0000","last_clean":"2026-02-24T07:56:33.056711+0000","last_became_active":"2026-02-24T07:42:54.186719+0000","last_became_peered":"2026-02-24T07:42:54.186719+0000","last_unstale":"2026-02-24T07:56:33.056711+0000","last_undegraded":"2026-02-24T07:56:33.056711+0000","last_fullsized":"2026-02-24T07:56:33.056711+0000","mapping_epoch":27,"log_start":"29'96","ondisk_log_start":"29'96","created":20,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'99","last_scrub_stamp":"2026-02-24T07:56:33.056678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:42:43.640776+0000","last_clean_scrub_stamp":"2026-02-24T07:56:33.056678+0000","objects_scrubbed":2,"log_size":3,"log_dups_size":96,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:57:56.381501+0000","scrub_duration":21,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":240,"num_write":135,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":16,"stat_sum":{"num_bytes":29968629784,"num_objects":457286,"num_object_clones":0,"num_object_copies":1371858,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":457286,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":808066,"num_write_kb":40491201,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":89913044992,"data_stored":89913032776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":48,"ondisk_log_size":48,"up":48,"acting":48,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":240,"num_write":135,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":2147,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":3,"up_from":25,"seq":107374182567,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":20435732,"kb_used_data":18524992,"kb_used_omap":18,"kb_used_meta":1910637,"kb_avail":713567468,"statfs":{"total":751619276800,"available":730693087232,"internally_reserved":0,"allocated":18969591808,"data_stored":18969089190,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":18878,"internal_metadata":1956492866},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":19,"seq":81604378793,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":22877080,"kb_used_data":22093444,"kb_used_omap":20,"kb_used_meta":783595,"kb_avail":711126120,"statfs":{"total":751619276800,"available":728193146880,"internally_reserved":0,"allocated":22623686656,"data_stored":22623179966,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":20830,"internal_metadata":802401954},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542314,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":26580820,"kb_used_data":25722816,"kb_used_omap":21,"kb_used_meta":857962,"kb_avail":707422380,"statfs":{"total":751619276800,"available":724400517120,"internally_reserved":0,"allocated":26340163584,"data_stored":26339660446,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":22243,"internal_metadata":878553373},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705837,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":22684632,"kb_used_data":22033412,"kb_used_omap":17,"kb_used_meta":651054,"kb_avail":711318568,"statfs":{"total":751619276800,"available":728390213632,"internally_reserved":0,"allocated":22562213888,"data_stored":22561707198,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":18431,"internal_metadata":666679297},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":2147,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":22483308544,"data_stored":22483304472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":26205949952,"data_stored":26205945880,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":22458929152,"data_stored":22458925080,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":18764857344,"data_stored":18764857344,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-24T07:56:41.773 INFO:tasks.cephadm.ceph_manager.ceph:increase pool size by 10 2026-02-24T07:56:41.773 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph osd pool set unique_pool_4 pg_num 86 2026-02-24T07:56:41.858 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:56:41.856+0000 7fe7d726a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:56:41.858 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:56:41.856+0000 7fe7d726a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:56:41.873 INFO:teuthology.orchestra.run.trial035.stdout:osd.0: {} 2026-02-24T07:56:41.873 INFO:teuthology.orchestra.run.trial035.stderr:osd.0: osd_enable_op_tracker = '' 2026-02-24T07:56:41.888 INFO:teuthology.orchestra.run.trial035.stdout:osd.1: {} 2026-02-24T07:56:41.888 INFO:teuthology.orchestra.run.trial035.stderr:osd.1: osd_enable_op_tracker = '' 2026-02-24T07:56:41.902 INFO:teuthology.orchestra.run.trial035.stdout:osd.2: {} 2026-02-24T07:56:41.902 INFO:teuthology.orchestra.run.trial035.stderr:osd.2: osd_enable_op_tracker = '' 2026-02-24T07:56:41.916 INFO:teuthology.orchestra.run.trial035.stdout:osd.3: {} 2026-02-24T07:56:41.916 INFO:teuthology.orchestra.run.trial035.stderr:osd.3: osd_enable_op_tracker = '' 2026-02-24T07:56:41.924 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:56:42.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:56:41 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/3427887729' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2026-02-24T07:56:42.988 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:56:42.954+0000 7fb3ab179640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:56:42.988 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:56:42.954+0000 7fb3ab179640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:56:43.108 INFO:teuthology.orchestra.run.trial035.stderr:Error EPERM: crimson_allow_pg_split is false; pg_num increase denied for crimson pool 2026-02-24T07:56:43.108 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:56:42 trial035 ceph-mon[39284]: from='client.15585 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2026-02-24T07:56:43.108 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:56:42 trial035 ceph-mon[39284]: pgmap v481: 17 pgs: 17 active+clean; 25 GiB data, 81 GiB used, 2.7 TiB / 2.7 TiB avail; 0 B/s wr, 10.53k op/s 2026-02-24T07:56:43.667 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-24T07:56:44.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:56:43 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/1299720416' entity='client.admin' cmd={"prefix": "osd pool set", "pool": "unique_pool_4", "var": "pg_num", "val": "86"} : dispatch 2026-02-24T07:56:44.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:56:43 trial035 ceph-mon[39284]: pgmap v482: 17 pgs: 17 active+clean; 25 GiB data, 81 GiB used, 2.7 TiB / 2.7 TiB avail; 0 B/s wr, 6.64k op/s 2026-02-24T07:56:44.515 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2026-02-24T07:56:44.658 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:56:45.002 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:56:45.001+0000 7fbf4d7ae640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:56:45.003 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:56:45.001+0000 7fbf4d7ae640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:56:45.017 INFO:teuthology.orchestra.run.trial035.stdout:osd.0: {} 2026-02-24T07:56:45.018 INFO:teuthology.orchestra.run.trial035.stderr:osd.0: osd_enable_op_tracker = '' 2026-02-24T07:56:45.033 INFO:teuthology.orchestra.run.trial035.stdout:osd.1: {} 2026-02-24T07:56:45.033 INFO:teuthology.orchestra.run.trial035.stderr:osd.1: osd_enable_op_tracker = '' 2026-02-24T07:56:45.047 INFO:teuthology.orchestra.run.trial035.stdout:osd.2: {} 2026-02-24T07:56:45.047 INFO:teuthology.orchestra.run.trial035.stderr:osd.2: osd_enable_op_tracker = '' 2026-02-24T07:56:45.061 INFO:teuthology.orchestra.run.trial035.stdout:osd.3: {} 2026-02-24T07:56:45.061 INFO:teuthology.orchestra.run.trial035.stderr:osd.3: osd_enable_op_tracker = '' 2026-02-24T07:56:45.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:56:44 trial035 ceph-mon[39284]: from='mgr.14284 10.20.193.35:0/3681373222' entity='mgr.x' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2026-02-24T07:56:46.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:56:45 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/3167286994' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2026-02-24T07:56:46.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:56:45 trial035 ceph-mon[39284]: pgmap v483: 17 pgs: 17 active+clean; 25 GiB data, 81 GiB used, 2.7 TiB / 2.7 TiB avail; 0 B/s wr, 6.64k op/s 2026-02-24T07:56:47.104 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2026-02-24T07:56:47.258 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:56:47.266 INFO:journalctl@ceph.prometheus.a.trial091.stdout:Feb 24 07:56:46 trial091 ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-prometheus-a[66007]: time=2026-02-24T07:56:46.973Z level=ERROR source=scrape.go:1630 msg="Failed to determine correct type of scrape target." component="scrape manager" scrape_pool=ceph target=http://trial035:9283/metrics content_type="text/html;charset=utf-8" fallback_media_type="" err="received unsupported Content-Type \"text/html;charset=utf-8\" and no fallback_scrape_protocol specified for target" 2026-02-24T07:56:47.282 INFO:journalctl@ceph.mgr.x.trial035.stdout:Feb 24 07:56:46 trial035 ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-mgr-x[39621]: ::ffff:10.20.193.91 - - [24/Feb/2026:07:56:46] "GET /metrics HTTP/1.1" 200 4230 "" "Prometheus/3.6.0" 2026-02-24T07:56:47.379 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:56:47.378+0000 7f041bb6e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:56:47.380 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:56:47.378+0000 7f041bb6e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:56:47.394 INFO:teuthology.orchestra.run.trial035.stdout:osd.0: {} 2026-02-24T07:56:47.395 INFO:teuthology.orchestra.run.trial035.stderr:osd.0: osd_enable_op_tracker = '' 2026-02-24T07:56:47.409 INFO:teuthology.orchestra.run.trial035.stdout:osd.1: {} 2026-02-24T07:56:47.409 INFO:teuthology.orchestra.run.trial035.stderr:osd.1: osd_enable_op_tracker = '' 2026-02-24T07:56:47.421 INFO:teuthology.orchestra.run.trial035.stdout:osd.2: {} 2026-02-24T07:56:47.421 INFO:teuthology.orchestra.run.trial035.stderr:osd.2: osd_enable_op_tracker = '' 2026-02-24T07:56:47.434 INFO:teuthology.orchestra.run.trial035.stdout:osd.3: {} 2026-02-24T07:56:47.434 INFO:teuthology.orchestra.run.trial035.stderr:osd.3: osd_enable_op_tracker = '' 2026-02-24T07:56:47.704 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:56:47 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/2477176780' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2026-02-24T07:56:48.668 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2026-02-24T07:56:48.668 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph osd reweight 0 1 2026-02-24T07:56:48.695 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:56:48 trial035 ceph-mon[39284]: pgmap v484: 17 pgs: 17 active+clean; 23 GiB data, 74 GiB used, 2.7 TiB / 2.7 TiB avail; 0 B/s wr, 9.95k op/s 2026-02-24T07:56:48.829 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:56:49.206 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:56:49.197+0000 7f1b1baad640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:56:49.207 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:56:49.197+0000 7f1b1baad640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:56:49.436 INFO:teuthology.orchestra.run.trial035.stderr:reweighted osd.0 to 1 (10000) 2026-02-24T07:56:49.448 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:56:49 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/4064569117' entity='client.admin' cmd={"prefix": "osd reweight", "id": 0, "weight": 1.0} : dispatch 2026-02-24T07:56:49.479 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2026-02-24T07:56:49.540 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph osd reweight 1 1 2026-02-24T07:56:49.624 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:56:49.691 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:56:49.746 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:56:49.745+0000 7ffad0db2640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:56:49.746 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:56:49.746+0000 7ffad0db2640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:56:49.761 INFO:teuthology.orchestra.run.trial035.stdout:osd.0: {} 2026-02-24T07:56:49.762 INFO:teuthology.orchestra.run.trial035.stderr:osd.0: osd_enable_op_tracker = '' 2026-02-24T07:56:49.780 INFO:teuthology.orchestra.run.trial035.stdout:osd.1: {} 2026-02-24T07:56:49.781 INFO:teuthology.orchestra.run.trial035.stderr:osd.1: osd_enable_op_tracker = '' 2026-02-24T07:56:49.794 INFO:teuthology.orchestra.run.trial035.stdout:osd.2: {} 2026-02-24T07:56:49.795 INFO:teuthology.orchestra.run.trial035.stderr:osd.2: osd_enable_op_tracker = '' 2026-02-24T07:56:49.809 INFO:teuthology.orchestra.run.trial035.stdout:osd.3: {} 2026-02-24T07:56:49.809 INFO:teuthology.orchestra.run.trial035.stderr:osd.3: osd_enable_op_tracker = '' 2026-02-24T07:56:49.818 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:56:49.817+0000 7f1bf2e82640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:56:49.818 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:56:49.817+0000 7f1bf2e82640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:56:50.440 INFO:teuthology.orchestra.run.trial035.stderr:reweighted osd.1 to 1 (10000) 2026-02-24T07:56:50.704 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:56:50 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/4064569117' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 0, "weight": 1.0}]': finished 2026-02-24T07:56:50.704 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:56:50 trial035 ceph-mon[39284]: osdmap e97: 4 total, 4 up, 4 in 2026-02-24T07:56:50.704 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:56:50 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/638016401' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2026-02-24T07:56:50.705 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:56:50 trial035 ceph-mon[39284]: pgmap v486: 17 pgs: 17 active+clean; 23 GiB data, 74 GiB used, 2.7 TiB / 2.7 TiB avail; 0 B/s wr, 8.31k op/s 2026-02-24T07:56:50.705 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:56:50 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/559887538' entity='client.admin' cmd={"prefix": "osd reweight", "id": 1, "weight": 1.0} : dispatch 2026-02-24T07:56:51.033 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph osd reweight 2 1 2026-02-24T07:56:51.175 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:56:51.551 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:56:51 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/559887538' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 1, "weight": 1.0}]': finished 2026-02-24T07:56:51.551 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:56:51 trial035 ceph-mon[39284]: osdmap e98: 4 total, 4 up, 4 in 2026-02-24T07:56:51.567 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:56:51.549+0000 7fcd2fb4d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:56:51.567 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:56:51.550+0000 7fcd2fb4d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:56:51.851 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2026-02-24T07:56:51.989 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:56:52.114 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:56:52.112+0000 7fa380c70640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:56:52.114 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:56:52.113+0000 7fa380c70640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:56:52.130 INFO:teuthology.orchestra.run.trial035.stdout:osd.0: {} 2026-02-24T07:56:52.131 INFO:teuthology.orchestra.run.trial035.stderr:osd.0: osd_enable_op_tracker = '' 2026-02-24T07:56:52.145 INFO:teuthology.orchestra.run.trial035.stdout:osd.1: {} 2026-02-24T07:56:52.145 INFO:teuthology.orchestra.run.trial035.stderr:osd.1: osd_enable_op_tracker = '' 2026-02-24T07:56:52.160 INFO:teuthology.orchestra.run.trial035.stdout:osd.2: {} 2026-02-24T07:56:52.160 INFO:teuthology.orchestra.run.trial035.stderr:osd.2: osd_enable_op_tracker = '' 2026-02-24T07:56:52.174 INFO:teuthology.orchestra.run.trial035.stdout:osd.3: {} 2026-02-24T07:56:52.174 INFO:teuthology.orchestra.run.trial035.stderr:osd.3: osd_enable_op_tracker = '' 2026-02-24T07:56:52.443 INFO:teuthology.orchestra.run.trial035.stderr:reweighted osd.2 to 1 (10000) 2026-02-24T07:56:52.490 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph osd reweight 3 1 2026-02-24T07:56:52.632 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:56:52.704 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:56:52 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/520049312' entity='client.admin' cmd={"prefix": "osd reweight", "id": 2, "weight": 1.0} : dispatch 2026-02-24T07:56:52.704 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:56:52 trial035 ceph-mon[39284]: pgmap v488: 17 pgs: 17 active+clean; 20 GiB data, 65 GiB used, 2.7 TiB / 2.7 TiB avail; 0 B/s wr, 10.56k op/s 2026-02-24T07:56:52.704 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:56:52 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/1778940284' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2026-02-24T07:56:52.957 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:56:52.945+0000 7fea3db17640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:56:52.957 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:56:52.945+0000 7fea3db17640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:56:53.444 INFO:teuthology.orchestra.run.trial035.stderr:reweighted osd.3 to 1 (10000) 2026-02-24T07:56:53.487 INFO:tasks.cephadm.ceph_manager.ceph:waiting for recovery to complete 2026-02-24T07:56:53.487 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph pg dump --format=json 2026-02-24T07:56:53.632 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:56:53.698 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:56:53 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/520049312' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 2, "weight": 1.0}]': finished 2026-02-24T07:56:53.698 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:56:53 trial035 ceph-mon[39284]: osdmap e99: 4 total, 4 up, 4 in 2026-02-24T07:56:53.698 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:56:53 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/2027204627' entity='client.admin' cmd={"prefix": "osd reweight", "id": 3, "weight": 1.0} : dispatch 2026-02-24T07:56:53.758 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:56:53.756+0000 7f5c75cfd640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:56:53.759 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:56:53.757+0000 7f5c75cfd640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:56:53.896 INFO:teuthology.orchestra.run.trial035.stdout: 2026-02-24T07:56:53.897 INFO:teuthology.orchestra.run.trial035.stderr:dumped all 2026-02-24T07:56:53.943 INFO:teuthology.orchestra.run.trial035.stdout:{"pg_ready":true,"pg_map":{"version":491,"stamp":"2026-02-24T07:56:53.771000+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":21593522744,"num_objects":329485,"num_object_clones":0,"num_object_copies":988455,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":329485,"num_whiteouts":0,"num_read":137,"num_read_kb":241,"num_write":936004,"num_write_kb":40493351,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":54,"ondisk_log_size":54,"up":51,"acting":51,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":51,"num_osds":4,"num_per_pool_osds":4,"num_per_pool_omap_osds":4,"kb":2936012800,"kb_used":68291592,"kb_used_data":64015736,"kb_used_omap":82,"kb_used_meta":4275629,"kb_avail":2867721208,"statfs":{"total":3006477107200,"available":2936546516992,"internally_reserved":0,"allocated":65552113664,"data_stored":65550063012,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":84840,"internal_metadata":4378244248},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[12,0,2],"upper_bound":8},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-2928672768,"num_objects":-44688,"num_object_clones":0,"num_object_copies":-134064,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-44688,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44688,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.334367"},"pg_stats":[{"pgid":"6.f","version":"98'58465","reported_seq":58879,"reported_epoch":98,"state":"active+clean","last_fresh":"2026-02-24T07:56:50.917307+0000","last_change":"2026-02-24T07:56:12.247365+0000","last_active":"2026-02-24T07:56:50.917307+0000","last_peered":"2026-02-24T07:56:50.917307+0000","last_clean":"2026-02-24T07:56:50.917307+0000","last_became_active":"2026-02-24T07:54:42.927080+0000","last_became_peered":"2026-02-24T07:54:42.927080+0000","last_unstale":"2026-02-24T07:56:50.917307+0000","last_undegraded":"2026-02-24T07:56:50.917307+0000","last_fullsized":"2026-02-24T07:56:50.917307+0000","mapping_epoch":90,"log_start":"98'58462","ondisk_log_start":"98'58462","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"96'38780","last_scrub_stamp":"2026-02-24T07:56:12.247318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_clean_scrub_stamp":"2026-02-24T07:56:12.247318+0000","objects_scrubbed":37979,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:57:38.474117+0000","scrub_duration":4670,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1346699264,"num_objects":20549,"num_object_clones":0,"num_object_copies":61647,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20549,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":58465,"num_write_kb":2528448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.e","version":"98'58494","reported_seq":58883,"reported_epoch":98,"state":"active+clean","last_fresh":"2026-02-24T07:56:50.780609+0000","last_change":"2026-02-24T07:56:05.374565+0000","last_active":"2026-02-24T07:56:50.780609+0000","last_peered":"2026-02-24T07:56:50.780609+0000","last_clean":"2026-02-24T07:56:50.780609+0000","last_became_active":"2026-02-24T07:54:42.928748+0000","last_became_peered":"2026-02-24T07:54:42.928748+0000","last_unstale":"2026-02-24T07:56:50.780609+0000","last_undegraded":"2026-02-24T07:56:50.780609+0000","last_fullsized":"2026-02-24T07:56:50.780609+0000","mapping_epoch":90,"log_start":"98'58488","ondisk_log_start":"98'58488","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"96'36307","last_scrub_stamp":"2026-02-24T07:56:05.374510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_clean_scrub_stamp":"2026-02-24T07:56:05.374510+0000","objects_scrubbed":35491,"log_size":6,"log_dups_size":3000,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:57:16.427813+0000","scrub_duration":4036,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1351614464,"num_objects":20624,"num_object_clones":0,"num_object_copies":61872,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20624,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":58494,"num_write_kb":2531776,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.d","version":"98'58506","reported_seq":58828,"reported_epoch":98,"state":"active+clean","last_fresh":"2026-02-24T07:56:50.780908+0000","last_change":"2026-02-24T07:55:48.151749+0000","last_active":"2026-02-24T07:56:50.780908+0000","last_peered":"2026-02-24T07:56:50.780908+0000","last_clean":"2026-02-24T07:56:50.780908+0000","last_became_active":"2026-02-24T07:54:42.926886+0000","last_became_peered":"2026-02-24T07:54:42.926886+0000","last_unstale":"2026-02-24T07:56:50.780908+0000","last_undegraded":"2026-02-24T07:56:50.780908+0000","last_fullsized":"2026-02-24T07:56:50.780908+0000","mapping_epoch":90,"log_start":"98'58503","ondisk_log_start":"98'58503","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"96'28924","last_scrub_stamp":"2026-02-24T07:55:48.151672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_clean_scrub_stamp":"2026-02-24T07:55:48.151672+0000","objects_scrubbed":28526,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:57:00.390677+0000","scrub_duration":2742,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1350631448,"num_objects":20610,"num_object_clones":0,"num_object_copies":61830,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20610,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":58506,"num_write_kb":2531649,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.c","version":"98'58623","reported_seq":59013,"reported_epoch":98,"state":"active+clean","last_fresh":"2026-02-24T07:56:50.916228+0000","last_change":"2026-02-24T07:56:25.008652+0000","last_active":"2026-02-24T07:56:50.916228+0000","last_peered":"2026-02-24T07:56:50.916228+0000","last_clean":"2026-02-24T07:56:50.916228+0000","last_became_active":"2026-02-24T07:54:42.928238+0000","last_became_peered":"2026-02-24T07:54:42.928238+0000","last_unstale":"2026-02-24T07:56:50.916228+0000","last_undegraded":"2026-02-24T07:56:50.916228+0000","last_fullsized":"2026-02-24T07:56:50.916228+0000","mapping_epoch":90,"log_start":"98'58620","ondisk_log_start":"98'58620","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"96'44918","last_scrub_stamp":"2026-02-24T07:56:25.008567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_clean_scrub_stamp":"2026-02-24T07:56:25.008567+0000","objects_scrubbed":35276,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:57:40.096151+0000","scrub_duration":4395,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1346437120,"num_objects":20545,"num_object_clones":0,"num_object_copies":61635,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20545,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":58623,"num_write_kb":2533376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.b","version":"98'58132","reported_seq":58555,"reported_epoch":98,"state":"active+clean","last_fresh":"2026-02-24T07:56:50.913534+0000","last_change":"2026-02-24T07:56:16.120475+0000","last_active":"2026-02-24T07:56:50.913534+0000","last_peered":"2026-02-24T07:56:50.913534+0000","last_clean":"2026-02-24T07:56:50.913534+0000","last_became_active":"2026-02-24T07:54:42.926965+0000","last_became_peered":"2026-02-24T07:54:42.926965+0000","last_unstale":"2026-02-24T07:56:50.913534+0000","last_undegraded":"2026-02-24T07:56:50.913534+0000","last_fullsized":"2026-02-24T07:56:50.913534+0000","mapping_epoch":90,"log_start":"98'58129","ondisk_log_start":"98'58129","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"96'39969","last_scrub_stamp":"2026-02-24T07:56:16.120290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_clean_scrub_stamp":"2026-02-24T07:56:16.120290+0000","objects_scrubbed":38739,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:57:36.957012+0000","scrub_duration":4491,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1326055424,"num_objects":20234,"num_object_clones":0,"num_object_copies":60702,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20234,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":58132,"num_write_kb":2507712,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.a","version":"98'58254","reported_seq":58577,"reported_epoch":98,"state":"active+clean","last_fresh":"2026-02-24T07:56:50.776275+0000","last_change":"2026-02-24T07:55:49.167491+0000","last_active":"2026-02-24T07:56:50.776275+0000","last_peered":"2026-02-24T07:56:50.776275+0000","last_clean":"2026-02-24T07:56:50.776275+0000","last_became_active":"2026-02-24T07:54:42.926477+0000","last_became_peered":"2026-02-24T07:54:42.926477+0000","last_unstale":"2026-02-24T07:56:50.776275+0000","last_undegraded":"2026-02-24T07:56:50.776275+0000","last_fullsized":"2026-02-24T07:56:50.776275+0000","mapping_epoch":90,"log_start":"98'58251","ondisk_log_start":"98'58251","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"96'29334","last_scrub_stamp":"2026-02-24T07:55:49.167461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_clean_scrub_stamp":"2026-02-24T07:55:49.167461+0000","objects_scrubbed":28822,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:56:59.424140+0000","scrub_duration":2726,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1362493440,"num_objects":20790,"num_object_clones":0,"num_object_copies":62370,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20790,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":58254,"num_write_kb":2529408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"98'58639","reported_seq":58963,"reported_epoch":98,"state":"active+clean","last_fresh":"2026-02-24T07:56:50.790628+0000","last_change":"2026-02-24T07:55:48.535755+0000","last_active":"2026-02-24T07:56:50.790628+0000","last_peered":"2026-02-24T07:56:50.790628+0000","last_clean":"2026-02-24T07:56:50.790628+0000","last_became_active":"2026-02-24T07:54:42.926136+0000","last_became_peered":"2026-02-24T07:54:42.926136+0000","last_unstale":"2026-02-24T07:56:50.790628+0000","last_undegraded":"2026-02-24T07:56:50.790628+0000","last_fullsized":"2026-02-24T07:56:50.790628+0000","mapping_epoch":90,"log_start":"98'58636","ondisk_log_start":"98'58636","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"96'29148","last_scrub_stamp":"2026-02-24T07:55:48.535691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_clean_scrub_stamp":"2026-02-24T07:55:48.535691+0000","objects_scrubbed":28720,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:56:56.161155+0000","scrub_duration":2872,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1358495744,"num_objects":20729,"num_object_clones":0,"num_object_copies":62187,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20729,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":58639,"num_write_kb":2539776,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,1],"acting":[0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.8","version":"98'58649","reported_seq":59014,"reported_epoch":98,"state":"active+clean","last_fresh":"2026-02-24T07:56:50.790858+0000","last_change":"2026-02-24T07:55:58.178460+0000","last_active":"2026-02-24T07:56:50.790858+0000","last_peered":"2026-02-24T07:56:50.790858+0000","last_clean":"2026-02-24T07:56:50.790858+0000","last_became_active":"2026-02-24T07:54:42.929078+0000","last_became_peered":"2026-02-24T07:54:42.929078+0000","last_unstale":"2026-02-24T07:56:50.790858+0000","last_undegraded":"2026-02-24T07:56:50.790858+0000","last_fullsized":"2026-02-24T07:56:50.790858+0000","mapping_epoch":90,"log_start":"98'58646","ondisk_log_start":"98'58646","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"96'33550","last_scrub_stamp":"2026-02-24T07:55:58.178395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_clean_scrub_stamp":"2026-02-24T07:55:58.178395+0000","objects_scrubbed":32948,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:57:14.607073+0000","scrub_duration":2981,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1365049344,"num_objects":20829,"num_object_clones":0,"num_object_copies":62487,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20829,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":58649,"num_write_kb":2543296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.6","version":"98'58389","reported_seq":58806,"reported_epoch":98,"state":"active+clean","last_fresh":"2026-02-24T07:56:50.916832+0000","last_change":"2026-02-24T07:56:18.901177+0000","last_active":"2026-02-24T07:56:50.916832+0000","last_peered":"2026-02-24T07:56:50.916832+0000","last_clean":"2026-02-24T07:56:50.916832+0000","last_became_active":"2026-02-24T07:54:42.926045+0000","last_became_peered":"2026-02-24T07:54:42.926045+0000","last_unstale":"2026-02-24T07:56:50.916832+0000","last_undegraded":"2026-02-24T07:56:50.916832+0000","last_fullsized":"2026-02-24T07:56:50.916832+0000","mapping_epoch":90,"log_start":"98'58386","ondisk_log_start":"98'58386","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"96'41769","last_scrub_stamp":"2026-02-24T07:56:18.901111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_clean_scrub_stamp":"2026-02-24T07:56:18.901111+0000","objects_scrubbed":38122,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:57:27.205697+0000","scrub_duration":4228,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1344864256,"num_objects":20521,"num_object_clones":0,"num_object_copies":61563,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20521,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":58389,"num_write_kb":2525120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.5","version":"98'58457","reported_seq":58827,"reported_epoch":98,"state":"active+clean","last_fresh":"2026-02-24T07:56:50.775949+0000","last_change":"2026-02-24T07:55:59.773616+0000","last_active":"2026-02-24T07:56:50.775949+0000","last_peered":"2026-02-24T07:56:50.775949+0000","last_clean":"2026-02-24T07:56:50.775949+0000","last_became_active":"2026-02-24T07:54:42.927535+0000","last_became_peered":"2026-02-24T07:54:42.927535+0000","last_unstale":"2026-02-24T07:56:50.775949+0000","last_undegraded":"2026-02-24T07:56:50.775949+0000","last_fullsized":"2026-02-24T07:56:50.775949+0000","mapping_epoch":90,"log_start":"98'58454","ondisk_log_start":"98'58454","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"96'34080","last_scrub_stamp":"2026-02-24T07:55:59.773556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_clean_scrub_stamp":"2026-02-24T07:55:59.773556+0000","objects_scrubbed":33431,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:57:04.302607+0000","scrub_duration":3324,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1345912832,"num_objects":20537,"num_object_clones":0,"num_object_copies":61611,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20537,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":58457,"num_write_kb":2527808,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.4","version":"98'58572","reported_seq":58894,"reported_epoch":98,"state":"active+clean","last_fresh":"2026-02-24T07:56:50.916029+0000","last_change":"2026-02-24T07:55:47.544043+0000","last_active":"2026-02-24T07:56:50.916029+0000","last_peered":"2026-02-24T07:56:50.916029+0000","last_clean":"2026-02-24T07:56:50.916029+0000","last_became_active":"2026-02-24T07:54:42.928011+0000","last_became_peered":"2026-02-24T07:54:42.928011+0000","last_unstale":"2026-02-24T07:56:50.916029+0000","last_undegraded":"2026-02-24T07:56:50.916029+0000","last_fullsized":"2026-02-24T07:56:50.916029+0000","mapping_epoch":90,"log_start":"98'58569","ondisk_log_start":"98'58569","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"96'29055","last_scrub_stamp":"2026-02-24T07:55:47.543994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_clean_scrub_stamp":"2026-02-24T07:55:47.543994+0000","objects_scrubbed":28510,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:57:04.321401+0000","scrub_duration":3165,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1331822592,"num_objects":20322,"num_object_clones":0,"num_object_copies":60966,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20322,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":58572,"num_write_kb":2524608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.3","version":"98'58812","reported_seq":59207,"reported_epoch":98,"state":"active+clean","last_fresh":"2026-02-24T07:56:50.781373+0000","last_change":"2026-02-24T07:56:06.184345+0000","last_active":"2026-02-24T07:56:50.781373+0000","last_peered":"2026-02-24T07:56:50.781373+0000","last_clean":"2026-02-24T07:56:50.781373+0000","last_became_active":"2026-02-24T07:54:42.927730+0000","last_became_peered":"2026-02-24T07:54:42.927730+0000","last_unstale":"2026-02-24T07:56:50.781373+0000","last_undegraded":"2026-02-24T07:56:50.781373+0000","last_fullsized":"2026-02-24T07:56:50.781373+0000","mapping_epoch":90,"log_start":"98'58809","ondisk_log_start":"98'58809","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"96'36664","last_scrub_stamp":"2026-02-24T07:56:06.184309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_clean_scrub_stamp":"2026-02-24T07:56:06.184309+0000","objects_scrubbed":35946,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:57:11.733048+0000","scrub_duration":3811,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1352663040,"num_objects":20640,"num_object_clones":0,"num_object_copies":61920,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20640,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":58812,"num_write_kb":2542464,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.2","version":"98'58466","reported_seq":58840,"reported_epoch":98,"state":"active+clean","last_fresh":"2026-02-24T07:56:50.797297+0000","last_change":"2026-02-24T07:56:00.120637+0000","last_active":"2026-02-24T07:56:50.797297+0000","last_peered":"2026-02-24T07:56:50.797297+0000","last_clean":"2026-02-24T07:56:50.797297+0000","last_became_active":"2026-02-24T07:54:42.930247+0000","last_became_peered":"2026-02-24T07:54:42.930247+0000","last_unstale":"2026-02-24T07:56:50.797297+0000","last_undegraded":"2026-02-24T07:56:50.797297+0000","last_fullsized":"2026-02-24T07:56:50.797297+0000","mapping_epoch":90,"log_start":"98'58463","ondisk_log_start":"98'58463","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"96'34210","last_scrub_stamp":"2026-02-24T07:56:00.120605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_clean_scrub_stamp":"2026-02-24T07:56:00.120605+0000","objects_scrubbed":33656,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:57:29.329949+0000","scrub_duration":2957,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1346240512,"num_objects":20542,"num_object_clones":0,"num_object_copies":61626,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20542,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":58466,"num_write_kb":2528256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1","version":"98'58774","reported_seq":59194,"reported_epoch":98,"state":"active+clean","last_fresh":"2026-02-24T07:56:50.913641+0000","last_change":"2026-02-24T07:56:12.897479+0000","last_active":"2026-02-24T07:56:50.913641+0000","last_peered":"2026-02-24T07:56:50.913641+0000","last_clean":"2026-02-24T07:56:50.913641+0000","last_became_active":"2026-02-24T07:54:42.926442+0000","last_became_peered":"2026-02-24T07:54:42.926442+0000","last_unstale":"2026-02-24T07:56:50.913641+0000","last_undegraded":"2026-02-24T07:56:50.913641+0000","last_fullsized":"2026-02-24T07:56:50.913641+0000","mapping_epoch":90,"log_start":"98'58771","ondisk_log_start":"98'58771","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"96'39245","last_scrub_stamp":"2026-02-24T07:56:12.897436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_clean_scrub_stamp":"2026-02-24T07:56:12.897436+0000","objects_scrubbed":38506,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:57:42.057233+0000","scrub_duration":4279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1358299136,"num_objects":20726,"num_object_clones":0,"num_object_copies":62178,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20726,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":58774,"num_write_kb":2544000,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.0","version":"98'57825","reported_seq":58205,"reported_epoch":98,"state":"active+clean","last_fresh":"2026-02-24T07:56:50.784436+0000","last_change":"2026-02-24T07:56:02.992243+0000","last_active":"2026-02-24T07:56:50.784436+0000","last_peered":"2026-02-24T07:56:50.784436+0000","last_clean":"2026-02-24T07:56:50.784436+0000","last_became_active":"2026-02-24T07:54:42.926910+0000","last_became_peered":"2026-02-24T07:54:42.926910+0000","last_unstale":"2026-02-24T07:56:50.784436+0000","last_undegraded":"2026-02-24T07:56:50.784436+0000","last_fullsized":"2026-02-24T07:56:50.784436+0000","mapping_epoch":90,"log_start":"98'57822","ondisk_log_start":"98'57822","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"96'34952","last_scrub_stamp":"2026-02-24T07:56:02.992156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_clean_scrub_stamp":"2026-02-24T07:56:02.992156+0000","objects_scrubbed":34329,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:57:03.847719+0000","scrub_duration":3335,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1342111744,"num_objects":20479,"num_object_clones":0,"num_object_copies":61437,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20479,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":57825,"num_write_kb":2505728,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.7","version":"98'58812","reported_seq":59191,"reported_epoch":98,"state":"active+clean","last_fresh":"2026-02-24T07:56:50.790030+0000","last_change":"2026-02-24T07:56:01.088786+0000","last_active":"2026-02-24T07:56:50.790030+0000","last_peered":"2026-02-24T07:56:50.790030+0000","last_clean":"2026-02-24T07:56:50.790030+0000","last_became_active":"2026-02-24T07:54:42.930142+0000","last_became_peered":"2026-02-24T07:54:42.930142+0000","last_unstale":"2026-02-24T07:56:50.790030+0000","last_undegraded":"2026-02-24T07:56:50.790030+0000","last_fullsized":"2026-02-24T07:56:50.790030+0000","mapping_epoch":90,"log_start":"98'58809","ondisk_log_start":"98'58809","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"96'34838","last_scrub_stamp":"2026-02-24T07:56:01.088744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_clean_scrub_stamp":"2026-02-24T07:56:01.088744+0000","objects_scrubbed":34284,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:57:24.696242+0000","scrub_duration":2973,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1363542016,"num_objects":20806,"num_object_clones":0,"num_object_copies":62418,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20806,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":58812,"num_write_kb":2547776,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"29'99","reported_seq":410,"reported_epoch":98,"state":"active+clean","last_fresh":"2026-02-24T07:56:50.439759+0000","last_change":"2026-02-24T07:56:33.056711+0000","last_active":"2026-02-24T07:56:50.439759+0000","last_peered":"2026-02-24T07:56:50.439759+0000","last_clean":"2026-02-24T07:56:50.439759+0000","last_became_active":"2026-02-24T07:42:54.186719+0000","last_became_peered":"2026-02-24T07:42:54.186719+0000","last_unstale":"2026-02-24T07:56:50.439759+0000","last_undegraded":"2026-02-24T07:56:50.439759+0000","last_fullsized":"2026-02-24T07:56:50.439759+0000","mapping_epoch":27,"log_start":"29'96","ondisk_log_start":"29'96","created":20,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'99","last_scrub_stamp":"2026-02-24T07:56:33.056678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:42:43.640776+0000","last_clean_scrub_stamp":"2026-02-24T07:56:33.056678+0000","objects_scrubbed":2,"log_size":3,"log_dups_size":96,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:57:56.381501+0000","scrub_duration":21,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":240,"num_write":135,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":16,"stat_sum":{"num_bytes":21592932376,"num_objects":329483,"num_object_clones":0,"num_object_copies":988449,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":329483,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":935869,"num_write_kb":40491201,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":64786542592,"data_stored":64786530376,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":51,"ondisk_log_size":51,"up":48,"acting":48,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":240,"num_write":135,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1979,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":3,"up_from":25,"seq":107374182570,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":14820128,"kb_used_data":13360460,"kb_used_omap":20,"kb_used_meta":1459627,"kb_avail":719183072,"statfs":{"total":751619276800,"available":736443465728,"internally_reserved":0,"allocated":13681111040,"data_stored":13680600543,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":21380,"internal_metadata":1494658172},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":19,"seq":81604378796,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":16922084,"kb_used_data":15976016,"kb_used_omap":20,"kb_used_meta":946027,"kb_avail":717081116,"statfs":{"total":751619276800,"available":734291062784,"internally_reserved":0,"allocated":16359440384,"data_stored":16358925815,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":21482,"internal_metadata":968731670},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542317,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":19718240,"kb_used_data":18661644,"kb_used_omap":22,"kb_used_meta":1056489,"kb_avail":714284960,"statfs":{"total":751619276800,"available":731427799040,"internally_reserved":0,"allocated":19109523456,"data_stored":19109012439,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":22895,"internal_metadata":1081845393},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[9,0,2],"upper_bound":8},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705840,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":16831140,"kb_used_data":16017616,"kb_used_omap":18,"kb_used_meta":813485,"kb_avail":717172060,"statfs":{"total":751619276800,"available":734384189440,"internally_reserved":0,"allocated":16402038784,"data_stored":16401524215,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19083,"internal_metadata":833009013},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1979,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16212103168,"data_stored":16212099096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":18879221760,"data_stored":18879217688,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16177958912,"data_stored":16177954840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":13517258752,"data_stored":13517258752,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-24T07:56:53.945 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph pg dump --format=json 2026-02-24T07:56:54.088 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:56:54.213 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:56:54.211+0000 7f2e205c7640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:56:54.214 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:56:54.211+0000 7f2e205c7640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:56:54.220 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2026-02-24T07:56:54.363 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:56:54.366 INFO:teuthology.orchestra.run.trial035.stdout: 2026-02-24T07:56:54.367 INFO:teuthology.orchestra.run.trial035.stderr:dumped all 2026-02-24T07:56:54.414 INFO:teuthology.orchestra.run.trial035.stdout:{"pg_ready":true,"pg_map":{"version":492,"stamp":"2026-02-24T07:56:53.926294+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":21593522744,"num_objects":329485,"num_object_clones":0,"num_object_copies":988455,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":329485,"num_whiteouts":0,"num_read":137,"num_read_kb":241,"num_write":936004,"num_write_kb":40493351,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":54,"ondisk_log_size":54,"up":51,"acting":51,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":51,"num_osds":4,"num_per_pool_osds":4,"num_per_pool_omap_osds":4,"kb":2936012800,"kb_used":68291592,"kb_used_data":64015736,"kb_used_omap":82,"kb_used_meta":4275629,"kb_avail":2867721208,"statfs":{"total":3006477107200,"available":2936546516992,"internally_reserved":0,"allocated":65552113664,"data_stored":65550063012,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":84840,"internal_metadata":4378244248},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[12,0,2],"upper_bound":8},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-2928672768,"num_objects":-44688,"num_object_clones":0,"num_object_copies":-134064,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-44688,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44688,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.155853"},"pg_stats":[{"pgid":"6.f","version":"98'58465","reported_seq":58879,"reported_epoch":98,"state":"active+clean","last_fresh":"2026-02-24T07:56:50.917307+0000","last_change":"2026-02-24T07:56:12.247365+0000","last_active":"2026-02-24T07:56:50.917307+0000","last_peered":"2026-02-24T07:56:50.917307+0000","last_clean":"2026-02-24T07:56:50.917307+0000","last_became_active":"2026-02-24T07:54:42.927080+0000","last_became_peered":"2026-02-24T07:54:42.927080+0000","last_unstale":"2026-02-24T07:56:50.917307+0000","last_undegraded":"2026-02-24T07:56:50.917307+0000","last_fullsized":"2026-02-24T07:56:50.917307+0000","mapping_epoch":90,"log_start":"98'58462","ondisk_log_start":"98'58462","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"96'38780","last_scrub_stamp":"2026-02-24T07:56:12.247318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_clean_scrub_stamp":"2026-02-24T07:56:12.247318+0000","objects_scrubbed":37979,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:57:38.474117+0000","scrub_duration":4670,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1346699264,"num_objects":20549,"num_object_clones":0,"num_object_copies":61647,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20549,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":58465,"num_write_kb":2528448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.e","version":"98'58494","reported_seq":58883,"reported_epoch":98,"state":"active+clean","last_fresh":"2026-02-24T07:56:50.780609+0000","last_change":"2026-02-24T07:56:05.374565+0000","last_active":"2026-02-24T07:56:50.780609+0000","last_peered":"2026-02-24T07:56:50.780609+0000","last_clean":"2026-02-24T07:56:50.780609+0000","last_became_active":"2026-02-24T07:54:42.928748+0000","last_became_peered":"2026-02-24T07:54:42.928748+0000","last_unstale":"2026-02-24T07:56:50.780609+0000","last_undegraded":"2026-02-24T07:56:50.780609+0000","last_fullsized":"2026-02-24T07:56:50.780609+0000","mapping_epoch":90,"log_start":"98'58488","ondisk_log_start":"98'58488","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"96'36307","last_scrub_stamp":"2026-02-24T07:56:05.374510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_clean_scrub_stamp":"2026-02-24T07:56:05.374510+0000","objects_scrubbed":35491,"log_size":6,"log_dups_size":3000,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:57:16.427813+0000","scrub_duration":4036,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1351614464,"num_objects":20624,"num_object_clones":0,"num_object_copies":61872,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20624,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":58494,"num_write_kb":2531776,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.d","version":"98'58506","reported_seq":58828,"reported_epoch":98,"state":"active+clean","last_fresh":"2026-02-24T07:56:50.780908+0000","last_change":"2026-02-24T07:55:48.151749+0000","last_active":"2026-02-24T07:56:50.780908+0000","last_peered":"2026-02-24T07:56:50.780908+0000","last_clean":"2026-02-24T07:56:50.780908+0000","last_became_active":"2026-02-24T07:54:42.926886+0000","last_became_peered":"2026-02-24T07:54:42.926886+0000","last_unstale":"2026-02-24T07:56:50.780908+0000","last_undegraded":"2026-02-24T07:56:50.780908+0000","last_fullsized":"2026-02-24T07:56:50.780908+0000","mapping_epoch":90,"log_start":"98'58503","ondisk_log_start":"98'58503","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"96'28924","last_scrub_stamp":"2026-02-24T07:55:48.151672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_clean_scrub_stamp":"2026-02-24T07:55:48.151672+0000","objects_scrubbed":28526,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:57:00.390677+0000","scrub_duration":2742,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1350631448,"num_objects":20610,"num_object_clones":0,"num_object_copies":61830,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20610,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":58506,"num_write_kb":2531649,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.c","version":"98'58623","reported_seq":59013,"reported_epoch":98,"state":"active+clean","last_fresh":"2026-02-24T07:56:50.916228+0000","last_change":"2026-02-24T07:56:25.008652+0000","last_active":"2026-02-24T07:56:50.916228+0000","last_peered":"2026-02-24T07:56:50.916228+0000","last_clean":"2026-02-24T07:56:50.916228+0000","last_became_active":"2026-02-24T07:54:42.928238+0000","last_became_peered":"2026-02-24T07:54:42.928238+0000","last_unstale":"2026-02-24T07:56:50.916228+0000","last_undegraded":"2026-02-24T07:56:50.916228+0000","last_fullsized":"2026-02-24T07:56:50.916228+0000","mapping_epoch":90,"log_start":"98'58620","ondisk_log_start":"98'58620","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"96'44918","last_scrub_stamp":"2026-02-24T07:56:25.008567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_clean_scrub_stamp":"2026-02-24T07:56:25.008567+0000","objects_scrubbed":35276,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:57:40.096151+0000","scrub_duration":4395,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1346437120,"num_objects":20545,"num_object_clones":0,"num_object_copies":61635,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20545,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":58623,"num_write_kb":2533376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.b","version":"98'58132","reported_seq":58555,"reported_epoch":98,"state":"active+clean","last_fresh":"2026-02-24T07:56:50.913534+0000","last_change":"2026-02-24T07:56:16.120475+0000","last_active":"2026-02-24T07:56:50.913534+0000","last_peered":"2026-02-24T07:56:50.913534+0000","last_clean":"2026-02-24T07:56:50.913534+0000","last_became_active":"2026-02-24T07:54:42.926965+0000","last_became_peered":"2026-02-24T07:54:42.926965+0000","last_unstale":"2026-02-24T07:56:50.913534+0000","last_undegraded":"2026-02-24T07:56:50.913534+0000","last_fullsized":"2026-02-24T07:56:50.913534+0000","mapping_epoch":90,"log_start":"98'58129","ondisk_log_start":"98'58129","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"96'39969","last_scrub_stamp":"2026-02-24T07:56:16.120290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_clean_scrub_stamp":"2026-02-24T07:56:16.120290+0000","objects_scrubbed":38739,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:57:36.957012+0000","scrub_duration":4491,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1326055424,"num_objects":20234,"num_object_clones":0,"num_object_copies":60702,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20234,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":58132,"num_write_kb":2507712,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.a","version":"98'58254","reported_seq":58577,"reported_epoch":98,"state":"active+clean","last_fresh":"2026-02-24T07:56:50.776275+0000","last_change":"2026-02-24T07:55:49.167491+0000","last_active":"2026-02-24T07:56:50.776275+0000","last_peered":"2026-02-24T07:56:50.776275+0000","last_clean":"2026-02-24T07:56:50.776275+0000","last_became_active":"2026-02-24T07:54:42.926477+0000","last_became_peered":"2026-02-24T07:54:42.926477+0000","last_unstale":"2026-02-24T07:56:50.776275+0000","last_undegraded":"2026-02-24T07:56:50.776275+0000","last_fullsized":"2026-02-24T07:56:50.776275+0000","mapping_epoch":90,"log_start":"98'58251","ondisk_log_start":"98'58251","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"96'29334","last_scrub_stamp":"2026-02-24T07:55:49.167461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_clean_scrub_stamp":"2026-02-24T07:55:49.167461+0000","objects_scrubbed":28822,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:56:59.424140+0000","scrub_duration":2726,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1362493440,"num_objects":20790,"num_object_clones":0,"num_object_copies":62370,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20790,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":58254,"num_write_kb":2529408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"98'58639","reported_seq":58963,"reported_epoch":98,"state":"active+clean","last_fresh":"2026-02-24T07:56:50.790628+0000","last_change":"2026-02-24T07:55:48.535755+0000","last_active":"2026-02-24T07:56:50.790628+0000","last_peered":"2026-02-24T07:56:50.790628+0000","last_clean":"2026-02-24T07:56:50.790628+0000","last_became_active":"2026-02-24T07:54:42.926136+0000","last_became_peered":"2026-02-24T07:54:42.926136+0000","last_unstale":"2026-02-24T07:56:50.790628+0000","last_undegraded":"2026-02-24T07:56:50.790628+0000","last_fullsized":"2026-02-24T07:56:50.790628+0000","mapping_epoch":90,"log_start":"98'58636","ondisk_log_start":"98'58636","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"96'29148","last_scrub_stamp":"2026-02-24T07:55:48.535691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_clean_scrub_stamp":"2026-02-24T07:55:48.535691+0000","objects_scrubbed":28720,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:56:56.161155+0000","scrub_duration":2872,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1358495744,"num_objects":20729,"num_object_clones":0,"num_object_copies":62187,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20729,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":58639,"num_write_kb":2539776,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,1],"acting":[0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.8","version":"98'58649","reported_seq":59014,"reported_epoch":98,"state":"active+clean","last_fresh":"2026-02-24T07:56:50.790858+0000","last_change":"2026-02-24T07:55:58.178460+0000","last_active":"2026-02-24T07:56:50.790858+0000","last_peered":"2026-02-24T07:56:50.790858+0000","last_clean":"2026-02-24T07:56:50.790858+0000","last_became_active":"2026-02-24T07:54:42.929078+0000","last_became_peered":"2026-02-24T07:54:42.929078+0000","last_unstale":"2026-02-24T07:56:50.790858+0000","last_undegraded":"2026-02-24T07:56:50.790858+0000","last_fullsized":"2026-02-24T07:56:50.790858+0000","mapping_epoch":90,"log_start":"98'58646","ondisk_log_start":"98'58646","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"96'33550","last_scrub_stamp":"2026-02-24T07:55:58.178395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_clean_scrub_stamp":"2026-02-24T07:55:58.178395+0000","objects_scrubbed":32948,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:57:14.607073+0000","scrub_duration":2981,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1365049344,"num_objects":20829,"num_object_clones":0,"num_object_copies":62487,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20829,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":58649,"num_write_kb":2543296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.6","version":"98'58389","reported_seq":58806,"reported_epoch":98,"state":"active+clean","last_fresh":"2026-02-24T07:56:50.916832+0000","last_change":"2026-02-24T07:56:18.901177+0000","last_active":"2026-02-24T07:56:50.916832+0000","last_peered":"2026-02-24T07:56:50.916832+0000","last_clean":"2026-02-24T07:56:50.916832+0000","last_became_active":"2026-02-24T07:54:42.926045+0000","last_became_peered":"2026-02-24T07:54:42.926045+0000","last_unstale":"2026-02-24T07:56:50.916832+0000","last_undegraded":"2026-02-24T07:56:50.916832+0000","last_fullsized":"2026-02-24T07:56:50.916832+0000","mapping_epoch":90,"log_start":"98'58386","ondisk_log_start":"98'58386","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"96'41769","last_scrub_stamp":"2026-02-24T07:56:18.901111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_clean_scrub_stamp":"2026-02-24T07:56:18.901111+0000","objects_scrubbed":38122,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:57:27.205697+0000","scrub_duration":4228,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1344864256,"num_objects":20521,"num_object_clones":0,"num_object_copies":61563,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20521,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":58389,"num_write_kb":2525120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.5","version":"98'58457","reported_seq":58827,"reported_epoch":98,"state":"active+clean","last_fresh":"2026-02-24T07:56:50.775949+0000","last_change":"2026-02-24T07:55:59.773616+0000","last_active":"2026-02-24T07:56:50.775949+0000","last_peered":"2026-02-24T07:56:50.775949+0000","last_clean":"2026-02-24T07:56:50.775949+0000","last_became_active":"2026-02-24T07:54:42.927535+0000","last_became_peered":"2026-02-24T07:54:42.927535+0000","last_unstale":"2026-02-24T07:56:50.775949+0000","last_undegraded":"2026-02-24T07:56:50.775949+0000","last_fullsized":"2026-02-24T07:56:50.775949+0000","mapping_epoch":90,"log_start":"98'58454","ondisk_log_start":"98'58454","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"96'34080","last_scrub_stamp":"2026-02-24T07:55:59.773556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_clean_scrub_stamp":"2026-02-24T07:55:59.773556+0000","objects_scrubbed":33431,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:57:04.302607+0000","scrub_duration":3324,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1345912832,"num_objects":20537,"num_object_clones":0,"num_object_copies":61611,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20537,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":58457,"num_write_kb":2527808,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.4","version":"98'58572","reported_seq":58894,"reported_epoch":98,"state":"active+clean","last_fresh":"2026-02-24T07:56:50.916029+0000","last_change":"2026-02-24T07:55:47.544043+0000","last_active":"2026-02-24T07:56:50.916029+0000","last_peered":"2026-02-24T07:56:50.916029+0000","last_clean":"2026-02-24T07:56:50.916029+0000","last_became_active":"2026-02-24T07:54:42.928011+0000","last_became_peered":"2026-02-24T07:54:42.928011+0000","last_unstale":"2026-02-24T07:56:50.916029+0000","last_undegraded":"2026-02-24T07:56:50.916029+0000","last_fullsized":"2026-02-24T07:56:50.916029+0000","mapping_epoch":90,"log_start":"98'58569","ondisk_log_start":"98'58569","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"96'29055","last_scrub_stamp":"2026-02-24T07:55:47.543994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_clean_scrub_stamp":"2026-02-24T07:55:47.543994+0000","objects_scrubbed":28510,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:57:04.321401+0000","scrub_duration":3165,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1331822592,"num_objects":20322,"num_object_clones":0,"num_object_copies":60966,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20322,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":58572,"num_write_kb":2524608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.3","version":"98'58812","reported_seq":59207,"reported_epoch":98,"state":"active+clean","last_fresh":"2026-02-24T07:56:50.781373+0000","last_change":"2026-02-24T07:56:06.184345+0000","last_active":"2026-02-24T07:56:50.781373+0000","last_peered":"2026-02-24T07:56:50.781373+0000","last_clean":"2026-02-24T07:56:50.781373+0000","last_became_active":"2026-02-24T07:54:42.927730+0000","last_became_peered":"2026-02-24T07:54:42.927730+0000","last_unstale":"2026-02-24T07:56:50.781373+0000","last_undegraded":"2026-02-24T07:56:50.781373+0000","last_fullsized":"2026-02-24T07:56:50.781373+0000","mapping_epoch":90,"log_start":"98'58809","ondisk_log_start":"98'58809","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"96'36664","last_scrub_stamp":"2026-02-24T07:56:06.184309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_clean_scrub_stamp":"2026-02-24T07:56:06.184309+0000","objects_scrubbed":35946,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:57:11.733048+0000","scrub_duration":3811,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1352663040,"num_objects":20640,"num_object_clones":0,"num_object_copies":61920,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20640,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":58812,"num_write_kb":2542464,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.2","version":"98'58466","reported_seq":58840,"reported_epoch":98,"state":"active+clean","last_fresh":"2026-02-24T07:56:50.797297+0000","last_change":"2026-02-24T07:56:00.120637+0000","last_active":"2026-02-24T07:56:50.797297+0000","last_peered":"2026-02-24T07:56:50.797297+0000","last_clean":"2026-02-24T07:56:50.797297+0000","last_became_active":"2026-02-24T07:54:42.930247+0000","last_became_peered":"2026-02-24T07:54:42.930247+0000","last_unstale":"2026-02-24T07:56:50.797297+0000","last_undegraded":"2026-02-24T07:56:50.797297+0000","last_fullsized":"2026-02-24T07:56:50.797297+0000","mapping_epoch":90,"log_start":"98'58463","ondisk_log_start":"98'58463","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"96'34210","last_scrub_stamp":"2026-02-24T07:56:00.120605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_clean_scrub_stamp":"2026-02-24T07:56:00.120605+0000","objects_scrubbed":33656,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:57:29.329949+0000","scrub_duration":2957,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1346240512,"num_objects":20542,"num_object_clones":0,"num_object_copies":61626,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20542,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":58466,"num_write_kb":2528256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1","version":"98'58774","reported_seq":59194,"reported_epoch":98,"state":"active+clean","last_fresh":"2026-02-24T07:56:50.913641+0000","last_change":"2026-02-24T07:56:12.897479+0000","last_active":"2026-02-24T07:56:50.913641+0000","last_peered":"2026-02-24T07:56:50.913641+0000","last_clean":"2026-02-24T07:56:50.913641+0000","last_became_active":"2026-02-24T07:54:42.926442+0000","last_became_peered":"2026-02-24T07:54:42.926442+0000","last_unstale":"2026-02-24T07:56:50.913641+0000","last_undegraded":"2026-02-24T07:56:50.913641+0000","last_fullsized":"2026-02-24T07:56:50.913641+0000","mapping_epoch":90,"log_start":"98'58771","ondisk_log_start":"98'58771","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"96'39245","last_scrub_stamp":"2026-02-24T07:56:12.897436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_clean_scrub_stamp":"2026-02-24T07:56:12.897436+0000","objects_scrubbed":38506,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:57:42.057233+0000","scrub_duration":4279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1358299136,"num_objects":20726,"num_object_clones":0,"num_object_copies":62178,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20726,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":58774,"num_write_kb":2544000,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.0","version":"98'57825","reported_seq":58205,"reported_epoch":98,"state":"active+clean","last_fresh":"2026-02-24T07:56:50.784436+0000","last_change":"2026-02-24T07:56:02.992243+0000","last_active":"2026-02-24T07:56:50.784436+0000","last_peered":"2026-02-24T07:56:50.784436+0000","last_clean":"2026-02-24T07:56:50.784436+0000","last_became_active":"2026-02-24T07:54:42.926910+0000","last_became_peered":"2026-02-24T07:54:42.926910+0000","last_unstale":"2026-02-24T07:56:50.784436+0000","last_undegraded":"2026-02-24T07:56:50.784436+0000","last_fullsized":"2026-02-24T07:56:50.784436+0000","mapping_epoch":90,"log_start":"98'57822","ondisk_log_start":"98'57822","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"96'34952","last_scrub_stamp":"2026-02-24T07:56:02.992156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_clean_scrub_stamp":"2026-02-24T07:56:02.992156+0000","objects_scrubbed":34329,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:57:03.847719+0000","scrub_duration":3335,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1342111744,"num_objects":20479,"num_object_clones":0,"num_object_copies":61437,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20479,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":57825,"num_write_kb":2505728,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.7","version":"98'58812","reported_seq":59191,"reported_epoch":98,"state":"active+clean","last_fresh":"2026-02-24T07:56:50.790030+0000","last_change":"2026-02-24T07:56:01.088786+0000","last_active":"2026-02-24T07:56:50.790030+0000","last_peered":"2026-02-24T07:56:50.790030+0000","last_clean":"2026-02-24T07:56:50.790030+0000","last_became_active":"2026-02-24T07:54:42.930142+0000","last_became_peered":"2026-02-24T07:54:42.930142+0000","last_unstale":"2026-02-24T07:56:50.790030+0000","last_undegraded":"2026-02-24T07:56:50.790030+0000","last_fullsized":"2026-02-24T07:56:50.790030+0000","mapping_epoch":90,"log_start":"98'58809","ondisk_log_start":"98'58809","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"96'34838","last_scrub_stamp":"2026-02-24T07:56:01.088744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_clean_scrub_stamp":"2026-02-24T07:56:01.088744+0000","objects_scrubbed":34284,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:57:24.696242+0000","scrub_duration":2973,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1363542016,"num_objects":20806,"num_object_clones":0,"num_object_copies":62418,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20806,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":58812,"num_write_kb":2547776,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"29'99","reported_seq":410,"reported_epoch":98,"state":"active+clean","last_fresh":"2026-02-24T07:56:50.439759+0000","last_change":"2026-02-24T07:56:33.056711+0000","last_active":"2026-02-24T07:56:50.439759+0000","last_peered":"2026-02-24T07:56:50.439759+0000","last_clean":"2026-02-24T07:56:50.439759+0000","last_became_active":"2026-02-24T07:42:54.186719+0000","last_became_peered":"2026-02-24T07:42:54.186719+0000","last_unstale":"2026-02-24T07:56:50.439759+0000","last_undegraded":"2026-02-24T07:56:50.439759+0000","last_fullsized":"2026-02-24T07:56:50.439759+0000","mapping_epoch":27,"log_start":"29'96","ondisk_log_start":"29'96","created":20,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'99","last_scrub_stamp":"2026-02-24T07:56:33.056678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:42:43.640776+0000","last_clean_scrub_stamp":"2026-02-24T07:56:33.056678+0000","objects_scrubbed":2,"log_size":3,"log_dups_size":96,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:57:56.381501+0000","scrub_duration":21,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":240,"num_write":135,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":16,"stat_sum":{"num_bytes":21592932376,"num_objects":329483,"num_object_clones":0,"num_object_copies":988449,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":329483,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":935869,"num_write_kb":40491201,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":64786542592,"data_stored":64786530376,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":51,"ondisk_log_size":51,"up":48,"acting":48,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":240,"num_write":135,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1979,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":3,"up_from":25,"seq":107374182570,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":14820128,"kb_used_data":13360460,"kb_used_omap":20,"kb_used_meta":1459627,"kb_avail":719183072,"statfs":{"total":751619276800,"available":736443465728,"internally_reserved":0,"allocated":13681111040,"data_stored":13680600543,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":21380,"internal_metadata":1494658172},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":19,"seq":81604378796,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":16922084,"kb_used_data":15976016,"kb_used_omap":20,"kb_used_meta":946027,"kb_avail":717081116,"statfs":{"total":751619276800,"available":734291062784,"internally_reserved":0,"allocated":16359440384,"data_stored":16358925815,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":21482,"internal_metadata":968731670},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542317,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":19718240,"kb_used_data":18661644,"kb_used_omap":22,"kb_used_meta":1056489,"kb_avail":714284960,"statfs":{"total":751619276800,"available":731427799040,"internally_reserved":0,"allocated":19109523456,"data_stored":19109012439,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":22895,"internal_metadata":1081845393},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[9,0,2],"upper_bound":8},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705840,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":16831140,"kb_used_data":16017616,"kb_used_omap":18,"kb_used_meta":813485,"kb_avail":717172060,"statfs":{"total":751619276800,"available":734384189440,"internally_reserved":0,"allocated":16402038784,"data_stored":16401524215,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19083,"internal_metadata":833009013},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1979,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16212103168,"data_stored":16212099096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":18879221760,"data_stored":18879217688,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16177958912,"data_stored":16177954840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":13517258752,"data_stored":13517258752,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-24T07:56:54.416 INFO:tasks.cephadm.ceph_manager.ceph:recovered! 2026-02-24T07:56:54.416 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2026-02-24T07:56:54.416 INFO:tasks.thrashosds.thrasher:Growing pool .mgr 2026-02-24T07:56:54.417 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph pg dump --format=json 2026-02-24T07:56:54.481 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:56:54.479+0000 7f13c822d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:56:54.482 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:56:54.479+0000 7f13c822d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:56:54.482 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:56:54 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/2027204627' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 3, "weight": 1.0}]': finished 2026-02-24T07:56:54.482 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:56:54 trial035 ceph-mon[39284]: osdmap e100: 4 total, 4 up, 4 in 2026-02-24T07:56:54.482 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:56:54 trial035 ceph-mon[39284]: pgmap v491: 17 pgs: 17 active+clean; 20 GiB data, 65 GiB used, 2.7 TiB / 2.7 TiB avail; 0 B/s wr, 10.31k op/s 2026-02-24T07:56:54.482 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:56:54 trial035 ceph-mon[39284]: from='client.15607 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2026-02-24T07:56:54.482 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:56:54 trial035 ceph-mon[39284]: osdmap e101: 4 total, 4 up, 4 in 2026-02-24T07:56:54.499 INFO:teuthology.orchestra.run.trial035.stdout:osd.0: {} 2026-02-24T07:56:54.499 INFO:teuthology.orchestra.run.trial035.stderr:osd.0: osd_enable_op_tracker = '' 2026-02-24T07:56:54.517 INFO:teuthology.orchestra.run.trial035.stdout:osd.1: {} 2026-02-24T07:56:54.518 INFO:teuthology.orchestra.run.trial035.stderr:osd.1: osd_enable_op_tracker = '' 2026-02-24T07:56:54.532 INFO:teuthology.orchestra.run.trial035.stdout:osd.2: {} 2026-02-24T07:56:54.532 INFO:teuthology.orchestra.run.trial035.stderr:osd.2: osd_enable_op_tracker = '' 2026-02-24T07:56:54.545 INFO:teuthology.orchestra.run.trial035.stdout:osd.3: {} 2026-02-24T07:56:54.545 INFO:teuthology.orchestra.run.trial035.stderr:osd.3: osd_enable_op_tracker = '' 2026-02-24T07:56:54.557 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:56:54.695 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:56:54.693+0000 7f8815790640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:56:54.695 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:56:54.693+0000 7f8815790640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:56:54.842 INFO:teuthology.orchestra.run.trial035.stdout: 2026-02-24T07:56:54.843 INFO:teuthology.orchestra.run.trial035.stderr:dumped all 2026-02-24T07:56:54.888 INFO:teuthology.orchestra.run.trial035.stdout:{"pg_ready":true,"pg_map":{"version":492,"stamp":"2026-02-24T07:56:53.926294+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":21593522744,"num_objects":329485,"num_object_clones":0,"num_object_copies":988455,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":329485,"num_whiteouts":0,"num_read":137,"num_read_kb":241,"num_write":936004,"num_write_kb":40493351,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":54,"ondisk_log_size":54,"up":51,"acting":51,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":51,"num_osds":4,"num_per_pool_osds":4,"num_per_pool_omap_osds":4,"kb":2936012800,"kb_used":68291592,"kb_used_data":64015736,"kb_used_omap":82,"kb_used_meta":4275629,"kb_avail":2867721208,"statfs":{"total":3006477107200,"available":2936546516992,"internally_reserved":0,"allocated":65552113664,"data_stored":65550063012,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":84840,"internal_metadata":4378244248},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[12,0,2],"upper_bound":8},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-2928672768,"num_objects":-44688,"num_object_clones":0,"num_object_copies":-134064,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-44688,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44688,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.155853"},"pg_stats":[{"pgid":"6.f","version":"98'58465","reported_seq":58879,"reported_epoch":98,"state":"active+clean","last_fresh":"2026-02-24T07:56:50.917307+0000","last_change":"2026-02-24T07:56:12.247365+0000","last_active":"2026-02-24T07:56:50.917307+0000","last_peered":"2026-02-24T07:56:50.917307+0000","last_clean":"2026-02-24T07:56:50.917307+0000","last_became_active":"2026-02-24T07:54:42.927080+0000","last_became_peered":"2026-02-24T07:54:42.927080+0000","last_unstale":"2026-02-24T07:56:50.917307+0000","last_undegraded":"2026-02-24T07:56:50.917307+0000","last_fullsized":"2026-02-24T07:56:50.917307+0000","mapping_epoch":90,"log_start":"98'58462","ondisk_log_start":"98'58462","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"96'38780","last_scrub_stamp":"2026-02-24T07:56:12.247318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_clean_scrub_stamp":"2026-02-24T07:56:12.247318+0000","objects_scrubbed":37979,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:57:38.474117+0000","scrub_duration":4670,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1346699264,"num_objects":20549,"num_object_clones":0,"num_object_copies":61647,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20549,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":58465,"num_write_kb":2528448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.e","version":"98'58494","reported_seq":58883,"reported_epoch":98,"state":"active+clean","last_fresh":"2026-02-24T07:56:50.780609+0000","last_change":"2026-02-24T07:56:05.374565+0000","last_active":"2026-02-24T07:56:50.780609+0000","last_peered":"2026-02-24T07:56:50.780609+0000","last_clean":"2026-02-24T07:56:50.780609+0000","last_became_active":"2026-02-24T07:54:42.928748+0000","last_became_peered":"2026-02-24T07:54:42.928748+0000","last_unstale":"2026-02-24T07:56:50.780609+0000","last_undegraded":"2026-02-24T07:56:50.780609+0000","last_fullsized":"2026-02-24T07:56:50.780609+0000","mapping_epoch":90,"log_start":"98'58488","ondisk_log_start":"98'58488","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"96'36307","last_scrub_stamp":"2026-02-24T07:56:05.374510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_clean_scrub_stamp":"2026-02-24T07:56:05.374510+0000","objects_scrubbed":35491,"log_size":6,"log_dups_size":3000,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:57:16.427813+0000","scrub_duration":4036,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1351614464,"num_objects":20624,"num_object_clones":0,"num_object_copies":61872,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20624,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":58494,"num_write_kb":2531776,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.d","version":"98'58506","reported_seq":58828,"reported_epoch":98,"state":"active+clean","last_fresh":"2026-02-24T07:56:50.780908+0000","last_change":"2026-02-24T07:55:48.151749+0000","last_active":"2026-02-24T07:56:50.780908+0000","last_peered":"2026-02-24T07:56:50.780908+0000","last_clean":"2026-02-24T07:56:50.780908+0000","last_became_active":"2026-02-24T07:54:42.926886+0000","last_became_peered":"2026-02-24T07:54:42.926886+0000","last_unstale":"2026-02-24T07:56:50.780908+0000","last_undegraded":"2026-02-24T07:56:50.780908+0000","last_fullsized":"2026-02-24T07:56:50.780908+0000","mapping_epoch":90,"log_start":"98'58503","ondisk_log_start":"98'58503","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"96'28924","last_scrub_stamp":"2026-02-24T07:55:48.151672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_clean_scrub_stamp":"2026-02-24T07:55:48.151672+0000","objects_scrubbed":28526,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:57:00.390677+0000","scrub_duration":2742,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1350631448,"num_objects":20610,"num_object_clones":0,"num_object_copies":61830,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20610,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":58506,"num_write_kb":2531649,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.c","version":"98'58623","reported_seq":59013,"reported_epoch":98,"state":"active+clean","last_fresh":"2026-02-24T07:56:50.916228+0000","last_change":"2026-02-24T07:56:25.008652+0000","last_active":"2026-02-24T07:56:50.916228+0000","last_peered":"2026-02-24T07:56:50.916228+0000","last_clean":"2026-02-24T07:56:50.916228+0000","last_became_active":"2026-02-24T07:54:42.928238+0000","last_became_peered":"2026-02-24T07:54:42.928238+0000","last_unstale":"2026-02-24T07:56:50.916228+0000","last_undegraded":"2026-02-24T07:56:50.916228+0000","last_fullsized":"2026-02-24T07:56:50.916228+0000","mapping_epoch":90,"log_start":"98'58620","ondisk_log_start":"98'58620","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"96'44918","last_scrub_stamp":"2026-02-24T07:56:25.008567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_clean_scrub_stamp":"2026-02-24T07:56:25.008567+0000","objects_scrubbed":35276,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:57:40.096151+0000","scrub_duration":4395,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1346437120,"num_objects":20545,"num_object_clones":0,"num_object_copies":61635,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20545,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":58623,"num_write_kb":2533376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.b","version":"98'58132","reported_seq":58555,"reported_epoch":98,"state":"active+clean","last_fresh":"2026-02-24T07:56:50.913534+0000","last_change":"2026-02-24T07:56:16.120475+0000","last_active":"2026-02-24T07:56:50.913534+0000","last_peered":"2026-02-24T07:56:50.913534+0000","last_clean":"2026-02-24T07:56:50.913534+0000","last_became_active":"2026-02-24T07:54:42.926965+0000","last_became_peered":"2026-02-24T07:54:42.926965+0000","last_unstale":"2026-02-24T07:56:50.913534+0000","last_undegraded":"2026-02-24T07:56:50.913534+0000","last_fullsized":"2026-02-24T07:56:50.913534+0000","mapping_epoch":90,"log_start":"98'58129","ondisk_log_start":"98'58129","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"96'39969","last_scrub_stamp":"2026-02-24T07:56:16.120290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_clean_scrub_stamp":"2026-02-24T07:56:16.120290+0000","objects_scrubbed":38739,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:57:36.957012+0000","scrub_duration":4491,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1326055424,"num_objects":20234,"num_object_clones":0,"num_object_copies":60702,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20234,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":58132,"num_write_kb":2507712,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.a","version":"98'58254","reported_seq":58577,"reported_epoch":98,"state":"active+clean","last_fresh":"2026-02-24T07:56:50.776275+0000","last_change":"2026-02-24T07:55:49.167491+0000","last_active":"2026-02-24T07:56:50.776275+0000","last_peered":"2026-02-24T07:56:50.776275+0000","last_clean":"2026-02-24T07:56:50.776275+0000","last_became_active":"2026-02-24T07:54:42.926477+0000","last_became_peered":"2026-02-24T07:54:42.926477+0000","last_unstale":"2026-02-24T07:56:50.776275+0000","last_undegraded":"2026-02-24T07:56:50.776275+0000","last_fullsized":"2026-02-24T07:56:50.776275+0000","mapping_epoch":90,"log_start":"98'58251","ondisk_log_start":"98'58251","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"96'29334","last_scrub_stamp":"2026-02-24T07:55:49.167461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_clean_scrub_stamp":"2026-02-24T07:55:49.167461+0000","objects_scrubbed":28822,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:56:59.424140+0000","scrub_duration":2726,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1362493440,"num_objects":20790,"num_object_clones":0,"num_object_copies":62370,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20790,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":58254,"num_write_kb":2529408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"98'58639","reported_seq":58963,"reported_epoch":98,"state":"active+clean","last_fresh":"2026-02-24T07:56:50.790628+0000","last_change":"2026-02-24T07:55:48.535755+0000","last_active":"2026-02-24T07:56:50.790628+0000","last_peered":"2026-02-24T07:56:50.790628+0000","last_clean":"2026-02-24T07:56:50.790628+0000","last_became_active":"2026-02-24T07:54:42.926136+0000","last_became_peered":"2026-02-24T07:54:42.926136+0000","last_unstale":"2026-02-24T07:56:50.790628+0000","last_undegraded":"2026-02-24T07:56:50.790628+0000","last_fullsized":"2026-02-24T07:56:50.790628+0000","mapping_epoch":90,"log_start":"98'58636","ondisk_log_start":"98'58636","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"96'29148","last_scrub_stamp":"2026-02-24T07:55:48.535691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_clean_scrub_stamp":"2026-02-24T07:55:48.535691+0000","objects_scrubbed":28720,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:56:56.161155+0000","scrub_duration":2872,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1358495744,"num_objects":20729,"num_object_clones":0,"num_object_copies":62187,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20729,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":58639,"num_write_kb":2539776,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,1],"acting":[0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.8","version":"98'58649","reported_seq":59014,"reported_epoch":98,"state":"active+clean","last_fresh":"2026-02-24T07:56:50.790858+0000","last_change":"2026-02-24T07:55:58.178460+0000","last_active":"2026-02-24T07:56:50.790858+0000","last_peered":"2026-02-24T07:56:50.790858+0000","last_clean":"2026-02-24T07:56:50.790858+0000","last_became_active":"2026-02-24T07:54:42.929078+0000","last_became_peered":"2026-02-24T07:54:42.929078+0000","last_unstale":"2026-02-24T07:56:50.790858+0000","last_undegraded":"2026-02-24T07:56:50.790858+0000","last_fullsized":"2026-02-24T07:56:50.790858+0000","mapping_epoch":90,"log_start":"98'58646","ondisk_log_start":"98'58646","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"96'33550","last_scrub_stamp":"2026-02-24T07:55:58.178395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_clean_scrub_stamp":"2026-02-24T07:55:58.178395+0000","objects_scrubbed":32948,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:57:14.607073+0000","scrub_duration":2981,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1365049344,"num_objects":20829,"num_object_clones":0,"num_object_copies":62487,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20829,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":58649,"num_write_kb":2543296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.6","version":"98'58389","reported_seq":58806,"reported_epoch":98,"state":"active+clean","last_fresh":"2026-02-24T07:56:50.916832+0000","last_change":"2026-02-24T07:56:18.901177+0000","last_active":"2026-02-24T07:56:50.916832+0000","last_peered":"2026-02-24T07:56:50.916832+0000","last_clean":"2026-02-24T07:56:50.916832+0000","last_became_active":"2026-02-24T07:54:42.926045+0000","last_became_peered":"2026-02-24T07:54:42.926045+0000","last_unstale":"2026-02-24T07:56:50.916832+0000","last_undegraded":"2026-02-24T07:56:50.916832+0000","last_fullsized":"2026-02-24T07:56:50.916832+0000","mapping_epoch":90,"log_start":"98'58386","ondisk_log_start":"98'58386","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"96'41769","last_scrub_stamp":"2026-02-24T07:56:18.901111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_clean_scrub_stamp":"2026-02-24T07:56:18.901111+0000","objects_scrubbed":38122,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:57:27.205697+0000","scrub_duration":4228,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1344864256,"num_objects":20521,"num_object_clones":0,"num_object_copies":61563,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20521,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":58389,"num_write_kb":2525120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.5","version":"98'58457","reported_seq":58827,"reported_epoch":98,"state":"active+clean","last_fresh":"2026-02-24T07:56:50.775949+0000","last_change":"2026-02-24T07:55:59.773616+0000","last_active":"2026-02-24T07:56:50.775949+0000","last_peered":"2026-02-24T07:56:50.775949+0000","last_clean":"2026-02-24T07:56:50.775949+0000","last_became_active":"2026-02-24T07:54:42.927535+0000","last_became_peered":"2026-02-24T07:54:42.927535+0000","last_unstale":"2026-02-24T07:56:50.775949+0000","last_undegraded":"2026-02-24T07:56:50.775949+0000","last_fullsized":"2026-02-24T07:56:50.775949+0000","mapping_epoch":90,"log_start":"98'58454","ondisk_log_start":"98'58454","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"96'34080","last_scrub_stamp":"2026-02-24T07:55:59.773556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_clean_scrub_stamp":"2026-02-24T07:55:59.773556+0000","objects_scrubbed":33431,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:57:04.302607+0000","scrub_duration":3324,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1345912832,"num_objects":20537,"num_object_clones":0,"num_object_copies":61611,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20537,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":58457,"num_write_kb":2527808,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.4","version":"98'58572","reported_seq":58894,"reported_epoch":98,"state":"active+clean","last_fresh":"2026-02-24T07:56:50.916029+0000","last_change":"2026-02-24T07:55:47.544043+0000","last_active":"2026-02-24T07:56:50.916029+0000","last_peered":"2026-02-24T07:56:50.916029+0000","last_clean":"2026-02-24T07:56:50.916029+0000","last_became_active":"2026-02-24T07:54:42.928011+0000","last_became_peered":"2026-02-24T07:54:42.928011+0000","last_unstale":"2026-02-24T07:56:50.916029+0000","last_undegraded":"2026-02-24T07:56:50.916029+0000","last_fullsized":"2026-02-24T07:56:50.916029+0000","mapping_epoch":90,"log_start":"98'58569","ondisk_log_start":"98'58569","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"96'29055","last_scrub_stamp":"2026-02-24T07:55:47.543994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_clean_scrub_stamp":"2026-02-24T07:55:47.543994+0000","objects_scrubbed":28510,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:57:04.321401+0000","scrub_duration":3165,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1331822592,"num_objects":20322,"num_object_clones":0,"num_object_copies":60966,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20322,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":58572,"num_write_kb":2524608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.3","version":"98'58812","reported_seq":59207,"reported_epoch":98,"state":"active+clean","last_fresh":"2026-02-24T07:56:50.781373+0000","last_change":"2026-02-24T07:56:06.184345+0000","last_active":"2026-02-24T07:56:50.781373+0000","last_peered":"2026-02-24T07:56:50.781373+0000","last_clean":"2026-02-24T07:56:50.781373+0000","last_became_active":"2026-02-24T07:54:42.927730+0000","last_became_peered":"2026-02-24T07:54:42.927730+0000","last_unstale":"2026-02-24T07:56:50.781373+0000","last_undegraded":"2026-02-24T07:56:50.781373+0000","last_fullsized":"2026-02-24T07:56:50.781373+0000","mapping_epoch":90,"log_start":"98'58809","ondisk_log_start":"98'58809","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"96'36664","last_scrub_stamp":"2026-02-24T07:56:06.184309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_clean_scrub_stamp":"2026-02-24T07:56:06.184309+0000","objects_scrubbed":35946,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:57:11.733048+0000","scrub_duration":3811,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1352663040,"num_objects":20640,"num_object_clones":0,"num_object_copies":61920,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20640,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":58812,"num_write_kb":2542464,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.2","version":"98'58466","reported_seq":58840,"reported_epoch":98,"state":"active+clean","last_fresh":"2026-02-24T07:56:50.797297+0000","last_change":"2026-02-24T07:56:00.120637+0000","last_active":"2026-02-24T07:56:50.797297+0000","last_peered":"2026-02-24T07:56:50.797297+0000","last_clean":"2026-02-24T07:56:50.797297+0000","last_became_active":"2026-02-24T07:54:42.930247+0000","last_became_peered":"2026-02-24T07:54:42.930247+0000","last_unstale":"2026-02-24T07:56:50.797297+0000","last_undegraded":"2026-02-24T07:56:50.797297+0000","last_fullsized":"2026-02-24T07:56:50.797297+0000","mapping_epoch":90,"log_start":"98'58463","ondisk_log_start":"98'58463","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"96'34210","last_scrub_stamp":"2026-02-24T07:56:00.120605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_clean_scrub_stamp":"2026-02-24T07:56:00.120605+0000","objects_scrubbed":33656,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:57:29.329949+0000","scrub_duration":2957,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1346240512,"num_objects":20542,"num_object_clones":0,"num_object_copies":61626,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20542,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":58466,"num_write_kb":2528256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1","version":"98'58774","reported_seq":59194,"reported_epoch":98,"state":"active+clean","last_fresh":"2026-02-24T07:56:50.913641+0000","last_change":"2026-02-24T07:56:12.897479+0000","last_active":"2026-02-24T07:56:50.913641+0000","last_peered":"2026-02-24T07:56:50.913641+0000","last_clean":"2026-02-24T07:56:50.913641+0000","last_became_active":"2026-02-24T07:54:42.926442+0000","last_became_peered":"2026-02-24T07:54:42.926442+0000","last_unstale":"2026-02-24T07:56:50.913641+0000","last_undegraded":"2026-02-24T07:56:50.913641+0000","last_fullsized":"2026-02-24T07:56:50.913641+0000","mapping_epoch":90,"log_start":"98'58771","ondisk_log_start":"98'58771","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"96'39245","last_scrub_stamp":"2026-02-24T07:56:12.897436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_clean_scrub_stamp":"2026-02-24T07:56:12.897436+0000","objects_scrubbed":38506,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:57:42.057233+0000","scrub_duration":4279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1358299136,"num_objects":20726,"num_object_clones":0,"num_object_copies":62178,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20726,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":58774,"num_write_kb":2544000,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.0","version":"98'57825","reported_seq":58205,"reported_epoch":98,"state":"active+clean","last_fresh":"2026-02-24T07:56:50.784436+0000","last_change":"2026-02-24T07:56:02.992243+0000","last_active":"2026-02-24T07:56:50.784436+0000","last_peered":"2026-02-24T07:56:50.784436+0000","last_clean":"2026-02-24T07:56:50.784436+0000","last_became_active":"2026-02-24T07:54:42.926910+0000","last_became_peered":"2026-02-24T07:54:42.926910+0000","last_unstale":"2026-02-24T07:56:50.784436+0000","last_undegraded":"2026-02-24T07:56:50.784436+0000","last_fullsized":"2026-02-24T07:56:50.784436+0000","mapping_epoch":90,"log_start":"98'57822","ondisk_log_start":"98'57822","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"96'34952","last_scrub_stamp":"2026-02-24T07:56:02.992156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_clean_scrub_stamp":"2026-02-24T07:56:02.992156+0000","objects_scrubbed":34329,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:57:03.847719+0000","scrub_duration":3335,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1342111744,"num_objects":20479,"num_object_clones":0,"num_object_copies":61437,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20479,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":57825,"num_write_kb":2505728,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.7","version":"98'58812","reported_seq":59191,"reported_epoch":98,"state":"active+clean","last_fresh":"2026-02-24T07:56:50.790030+0000","last_change":"2026-02-24T07:56:01.088786+0000","last_active":"2026-02-24T07:56:50.790030+0000","last_peered":"2026-02-24T07:56:50.790030+0000","last_clean":"2026-02-24T07:56:50.790030+0000","last_became_active":"2026-02-24T07:54:42.930142+0000","last_became_peered":"2026-02-24T07:54:42.930142+0000","last_unstale":"2026-02-24T07:56:50.790030+0000","last_undegraded":"2026-02-24T07:56:50.790030+0000","last_fullsized":"2026-02-24T07:56:50.790030+0000","mapping_epoch":90,"log_start":"98'58809","ondisk_log_start":"98'58809","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"96'34838","last_scrub_stamp":"2026-02-24T07:56:01.088744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_clean_scrub_stamp":"2026-02-24T07:56:01.088744+0000","objects_scrubbed":34284,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:57:24.696242+0000","scrub_duration":2973,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1363542016,"num_objects":20806,"num_object_clones":0,"num_object_copies":62418,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20806,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":58812,"num_write_kb":2547776,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"29'99","reported_seq":410,"reported_epoch":98,"state":"active+clean","last_fresh":"2026-02-24T07:56:50.439759+0000","last_change":"2026-02-24T07:56:33.056711+0000","last_active":"2026-02-24T07:56:50.439759+0000","last_peered":"2026-02-24T07:56:50.439759+0000","last_clean":"2026-02-24T07:56:50.439759+0000","last_became_active":"2026-02-24T07:42:54.186719+0000","last_became_peered":"2026-02-24T07:42:54.186719+0000","last_unstale":"2026-02-24T07:56:50.439759+0000","last_undegraded":"2026-02-24T07:56:50.439759+0000","last_fullsized":"2026-02-24T07:56:50.439759+0000","mapping_epoch":27,"log_start":"29'96","ondisk_log_start":"29'96","created":20,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'99","last_scrub_stamp":"2026-02-24T07:56:33.056678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:42:43.640776+0000","last_clean_scrub_stamp":"2026-02-24T07:56:33.056678+0000","objects_scrubbed":2,"log_size":3,"log_dups_size":96,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:57:56.381501+0000","scrub_duration":21,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":240,"num_write":135,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":16,"stat_sum":{"num_bytes":21592932376,"num_objects":329483,"num_object_clones":0,"num_object_copies":988449,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":329483,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":935869,"num_write_kb":40491201,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":64786542592,"data_stored":64786530376,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":51,"ondisk_log_size":51,"up":48,"acting":48,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":240,"num_write":135,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1979,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":3,"up_from":25,"seq":107374182570,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":14820128,"kb_used_data":13360460,"kb_used_omap":20,"kb_used_meta":1459627,"kb_avail":719183072,"statfs":{"total":751619276800,"available":736443465728,"internally_reserved":0,"allocated":13681111040,"data_stored":13680600543,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":21380,"internal_metadata":1494658172},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":19,"seq":81604378796,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":16922084,"kb_used_data":15976016,"kb_used_omap":20,"kb_used_meta":946027,"kb_avail":717081116,"statfs":{"total":751619276800,"available":734291062784,"internally_reserved":0,"allocated":16359440384,"data_stored":16358925815,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":21482,"internal_metadata":968731670},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542317,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":19718240,"kb_used_data":18661644,"kb_used_omap":22,"kb_used_meta":1056489,"kb_avail":714284960,"statfs":{"total":751619276800,"available":731427799040,"internally_reserved":0,"allocated":19109523456,"data_stored":19109012439,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":22895,"internal_metadata":1081845393},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[9,0,2],"upper_bound":8},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705840,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":16831140,"kb_used_data":16017616,"kb_used_omap":18,"kb_used_meta":813485,"kb_avail":717172060,"statfs":{"total":751619276800,"available":734384189440,"internally_reserved":0,"allocated":16402038784,"data_stored":16401524215,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19083,"internal_metadata":833009013},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1979,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16212103168,"data_stored":16212099096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":18879221760,"data_stored":18879217688,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16177958912,"data_stored":16177954840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":13517258752,"data_stored":13517258752,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-24T07:56:54.890 INFO:tasks.cephadm.ceph_manager.ceph:increase pool size by 10 2026-02-24T07:56:54.891 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph osd pool set .mgr pg_num 451 2026-02-24T07:56:55.036 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:56:55.155 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:56:55.153+0000 7ff05d5b7640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:56:55.155 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:56:55.153+0000 7ff05d5b7640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:56:55.319 INFO:teuthology.orchestra.run.trial035.stderr:Error EPERM: crimson_allow_pg_split is false; pg_num increase denied for crimson pool 2026-02-24T07:56:55.360 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-24T07:56:55.704 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:56:55 trial035 ceph-mon[39284]: from='client.15609 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2026-02-24T07:56:55.704 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:56:55 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/2618192238' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2026-02-24T07:56:55.704 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:56:55 trial035 ceph-mon[39284]: from='client.15613 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2026-02-24T07:56:55.704 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:56:55 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/139845224' entity='client.admin' cmd={"prefix": "osd pool set", "pool": ".mgr", "var": "pg_num", "val": "451"} : dispatch 2026-02-24T07:56:56.588 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2026-02-24T07:56:56.704 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:56:56 trial035 ceph-mon[39284]: pgmap v493: 17 pgs: 17 active+clean; 20 GiB data, 65 GiB used, 2.7 TiB / 2.7 TiB avail; 0 B/s wr, 8.38k op/s 2026-02-24T07:56:56.704 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:56:56 trial035 ceph-mon[39284]: 6.9 scrub starts 2026-02-24T07:56:56.735 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:56:56.866 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:56:56.863+0000 7fe358525640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:56:56.866 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:56:56.863+0000 7fe358525640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:56:56.881 INFO:teuthology.orchestra.run.trial035.stdout:osd.0: {} 2026-02-24T07:56:56.882 INFO:teuthology.orchestra.run.trial035.stderr:osd.0: osd_enable_op_tracker = '' 2026-02-24T07:56:56.896 INFO:teuthology.orchestra.run.trial035.stdout:osd.1: {} 2026-02-24T07:56:56.896 INFO:teuthology.orchestra.run.trial035.stderr:osd.1: osd_enable_op_tracker = '' 2026-02-24T07:56:56.909 INFO:teuthology.orchestra.run.trial035.stdout:osd.2: {} 2026-02-24T07:56:56.909 INFO:teuthology.orchestra.run.trial035.stderr:osd.2: osd_enable_op_tracker = '' 2026-02-24T07:56:56.921 INFO:teuthology.orchestra.run.trial035.stdout:osd.3: {} 2026-02-24T07:56:56.922 INFO:teuthology.orchestra.run.trial035.stderr:osd.3: osd_enable_op_tracker = '' 2026-02-24T07:56:57.204 INFO:journalctl@ceph.mgr.x.trial035.stdout:Feb 24 07:56:56 trial035 ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-mgr-x[39621]: ::ffff:10.20.193.91 - - [24/Feb/2026:07:56:56] "GET /metrics HTTP/1.1" 200 4230 "" "Prometheus/3.6.0" 2026-02-24T07:56:57.356 INFO:journalctl@ceph.prometheus.a.trial091.stdout:Feb 24 07:56:56 trial091 ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-prometheus-a[66007]: time=2026-02-24T07:56:56.973Z level=ERROR source=scrape.go:1630 msg="Failed to determine correct type of scrape target." component="scrape manager" scrape_pool=ceph target=http://trial035:9283/metrics content_type="text/html;charset=utf-8" fallback_media_type="" err="received unsupported Content-Type \"text/html;charset=utf-8\" and no fallback_scrape_protocol specified for target" 2026-02-24T07:56:57.704 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:56:57 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/274809129' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2026-02-24T07:56:58.704 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:56:58 trial035 ceph-mon[39284]: purged_snaps scrub starts 2026-02-24T07:56:58.704 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:56:58 trial035 ceph-mon[39284]: purged_snaps scrub ok 2026-02-24T07:56:58.704 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:56:58 trial035 ceph-mon[39284]: 6.9 scrub ok 2026-02-24T07:56:58.705 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:56:58 trial035 ceph-mon[39284]: pgmap v494: 17 pgs: 17 active+clean; 17 GiB data, 58 GiB used, 2.7 TiB / 2.7 TiB avail; 0 B/s wr, 7.15k op/s 2026-02-24T07:56:58.968 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2026-02-24T07:56:59.096 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:56:59.540 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:56:59.485+0000 7f0a43e17640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:56:59.541 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:56:59.485+0000 7f0a43e17640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:56:59.541 INFO:teuthology.orchestra.run.trial035.stderr:osd.0: osd_enable_op_tracker = '' 2026-02-24T07:56:59.541 INFO:teuthology.orchestra.run.trial035.stderr:osd.1: osd_enable_op_tracker = '' 2026-02-24T07:56:59.541 INFO:teuthology.orchestra.run.trial035.stdout:osd.0: {} 2026-02-24T07:56:59.541 INFO:teuthology.orchestra.run.trial035.stdout:osd.1: {} 2026-02-24T07:56:59.549 INFO:teuthology.orchestra.run.trial035.stdout:osd.2: {} 2026-02-24T07:56:59.549 INFO:teuthology.orchestra.run.trial035.stderr:osd.2: osd_enable_op_tracker = '' 2026-02-24T07:56:59.560 INFO:teuthology.orchestra.run.trial035.stdout:osd.3: {} 2026-02-24T07:56:59.560 INFO:teuthology.orchestra.run.trial035.stderr:osd.3: osd_enable_op_tracker = '' 2026-02-24T07:57:00.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:56:59 trial035 ceph-mon[39284]: osdmap e102: 4 total, 4 up, 4 in 2026-02-24T07:57:00.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:56:59 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/1267861163' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2026-02-24T07:57:00.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:56:59 trial035 ceph-mon[39284]: from='mgr.14284 10.20.193.35:0/3681373222' entity='mgr.x' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2026-02-24T07:57:00.361 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2026-02-24T07:57:00.361 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2026-02-24T07:57:00.361 INFO:tasks.thrashosds.thrasher:Growing pool .mgr 2026-02-24T07:57:00.361 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph pg dump --format=json 2026-02-24T07:57:00.494 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:57:00.960 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:57:00.887+0000 7fc8ccd64640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:57:00.961 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:57:00.887+0000 7fc8ccd64640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:57:01.030 INFO:teuthology.orchestra.run.trial035.stdout: 2026-02-24T07:57:01.031 INFO:teuthology.orchestra.run.trial035.stderr:dumped all 2026-02-24T07:57:01.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:57:00 trial035 ceph-mon[39284]: pgmap v496: 17 pgs: 17 active+clean; 17 GiB data, 58 GiB used, 2.7 TiB / 2.7 TiB avail; 0 B/s wr, 6.78k op/s 2026-02-24T07:57:01.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:57:00 trial035 ceph-mon[39284]: 6.a scrub starts 2026-02-24T07:57:01.739 INFO:teuthology.orchestra.run.trial035.stdout:{"pg_ready":true,"pg_map":{"version":496,"stamp":"2026-02-24T07:56:59.771821+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":18779865656,"num_objects":286552,"num_object_clones":0,"num_object_copies":859656,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":286552,"num_whiteouts":0,"num_read":137,"num_read_kb":241,"num_write":978937,"num_write_kb":40493351,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":53,"ondisk_log_size":53,"up":51,"acting":51,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":51,"num_osds":4,"num_per_pool_osds":4,"num_per_pool_omap_osds":4,"kb":2936012800,"kb_used":60645640,"kb_used_data":56081016,"kb_used_omap":91,"kb_used_meta":4564388,"kb_avail":2875367160,"statfs":{"total":3006477107200,"available":2944375971840,"internally_reserved":0,"allocated":57426960384,"data_stored":57424783636,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":93324,"internal_metadata":4673934196},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-2813657088,"num_objects":-42933,"num_object_clones":0,"num_object_copies":-128799,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-42933,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":42933,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.328182"},"pg_stats":[{"pgid":"6.f","version":"101'61240","reported_seq":61660,"reported_epoch":101,"state":"active+clean","last_fresh":"2026-02-24T07:56:55.919028+0000","last_change":"2026-02-24T07:56:12.247365+0000","last_active":"2026-02-24T07:56:55.919028+0000","last_peered":"2026-02-24T07:56:55.919028+0000","last_clean":"2026-02-24T07:56:55.919028+0000","last_became_active":"2026-02-24T07:54:42.927080+0000","last_became_peered":"2026-02-24T07:54:42.927080+0000","last_unstale":"2026-02-24T07:56:55.919028+0000","last_undegraded":"2026-02-24T07:56:55.919028+0000","last_fullsized":"2026-02-24T07:56:55.919028+0000","mapping_epoch":90,"log_start":"101'61237","ondisk_log_start":"101'61237","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"96'38780","last_scrub_stamp":"2026-02-24T07:56:12.247318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_clean_scrub_stamp":"2026-02-24T07:56:12.247318+0000","objects_scrubbed":37979,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:57:38.474117+0000","scrub_duration":4670,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1164836864,"num_objects":17774,"num_object_clones":0,"num_object_copies":53322,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17774,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":61240,"num_write_kb":2528448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.e","version":"101'61224","reported_seq":61619,"reported_epoch":101,"state":"active+clean","last_fresh":"2026-02-24T07:56:55.781288+0000","last_change":"2026-02-24T07:56:05.374565+0000","last_active":"2026-02-24T07:56:55.781288+0000","last_peered":"2026-02-24T07:56:55.781288+0000","last_clean":"2026-02-24T07:56:55.781288+0000","last_became_active":"2026-02-24T07:54:42.928748+0000","last_became_peered":"2026-02-24T07:54:42.928748+0000","last_unstale":"2026-02-24T07:56:55.781288+0000","last_undegraded":"2026-02-24T07:56:55.781288+0000","last_fullsized":"2026-02-24T07:56:55.781288+0000","mapping_epoch":90,"log_start":"101'61221","ondisk_log_start":"101'61221","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"96'36307","last_scrub_stamp":"2026-02-24T07:56:05.374510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_clean_scrub_stamp":"2026-02-24T07:56:05.374510+0000","objects_scrubbed":35491,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:57:16.427813+0000","scrub_duration":4036,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1172701184,"num_objects":17894,"num_object_clones":0,"num_object_copies":53682,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17894,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":61224,"num_write_kb":2531776,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.d","version":"101'61136","reported_seq":61464,"reported_epoch":101,"state":"active+clean","last_fresh":"2026-02-24T07:56:55.778753+0000","last_change":"2026-02-24T07:55:48.151749+0000","last_active":"2026-02-24T07:56:55.778753+0000","last_peered":"2026-02-24T07:56:55.778753+0000","last_clean":"2026-02-24T07:56:55.778753+0000","last_became_active":"2026-02-24T07:54:42.926886+0000","last_became_peered":"2026-02-24T07:54:42.926886+0000","last_unstale":"2026-02-24T07:56:55.778753+0000","last_undegraded":"2026-02-24T07:56:55.778753+0000","last_fullsized":"2026-02-24T07:56:55.778753+0000","mapping_epoch":90,"log_start":"101'61133","ondisk_log_start":"101'61133","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"96'28924","last_scrub_stamp":"2026-02-24T07:55:48.151672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_clean_scrub_stamp":"2026-02-24T07:55:48.151672+0000","objects_scrubbed":28526,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:57:00.390677+0000","scrub_duration":2742,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1178271768,"num_objects":17980,"num_object_clones":0,"num_object_copies":53940,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17980,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":61136,"num_write_kb":2531649,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.c","version":"101'61291","reported_seq":61687,"reported_epoch":101,"state":"active+clean","last_fresh":"2026-02-24T07:56:55.918546+0000","last_change":"2026-02-24T07:56:25.008652+0000","last_active":"2026-02-24T07:56:55.918546+0000","last_peered":"2026-02-24T07:56:55.918546+0000","last_clean":"2026-02-24T07:56:55.918546+0000","last_became_active":"2026-02-24T07:54:42.928238+0000","last_became_peered":"2026-02-24T07:54:42.928238+0000","last_unstale":"2026-02-24T07:56:55.918546+0000","last_undegraded":"2026-02-24T07:56:55.918546+0000","last_fullsized":"2026-02-24T07:56:55.918546+0000","mapping_epoch":90,"log_start":"101'61288","ondisk_log_start":"101'61288","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"96'44918","last_scrub_stamp":"2026-02-24T07:56:25.008567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_clean_scrub_stamp":"2026-02-24T07:56:25.008567+0000","objects_scrubbed":35276,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:57:40.096151+0000","scrub_duration":4395,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1171587072,"num_objects":17877,"num_object_clones":0,"num_object_copies":53631,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17877,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":61291,"num_write_kb":2533376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.b","version":"101'60717","reported_seq":61146,"reported_epoch":101,"state":"active+clean","last_fresh":"2026-02-24T07:56:55.915511+0000","last_change":"2026-02-24T07:56:16.120475+0000","last_active":"2026-02-24T07:56:55.915511+0000","last_peered":"2026-02-24T07:56:55.915511+0000","last_clean":"2026-02-24T07:56:55.915511+0000","last_became_active":"2026-02-24T07:54:42.926965+0000","last_became_peered":"2026-02-24T07:54:42.926965+0000","last_unstale":"2026-02-24T07:56:55.915511+0000","last_undegraded":"2026-02-24T07:56:55.915511+0000","last_fullsized":"2026-02-24T07:56:55.915511+0000","mapping_epoch":90,"log_start":"101'60714","ondisk_log_start":"101'60714","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"96'39969","last_scrub_stamp":"2026-02-24T07:56:16.120290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_clean_scrub_stamp":"2026-02-24T07:56:16.120290+0000","objects_scrubbed":38739,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:57:36.957012+0000","scrub_duration":4491,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1156644864,"num_objects":17649,"num_object_clones":0,"num_object_copies":52947,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17649,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":60717,"num_write_kb":2507712,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.a","version":"101'60969","reported_seq":61298,"reported_epoch":101,"state":"active+clean","last_fresh":"2026-02-24T07:56:55.778854+0000","last_change":"2026-02-24T07:55:49.167491+0000","last_active":"2026-02-24T07:56:55.778854+0000","last_peered":"2026-02-24T07:56:55.778854+0000","last_clean":"2026-02-24T07:56:55.778854+0000","last_became_active":"2026-02-24T07:54:42.926477+0000","last_became_peered":"2026-02-24T07:54:42.926477+0000","last_unstale":"2026-02-24T07:56:55.778854+0000","last_undegraded":"2026-02-24T07:56:55.778854+0000","last_fullsized":"2026-02-24T07:56:55.778854+0000","mapping_epoch":90,"log_start":"101'60966","ondisk_log_start":"101'60966","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"96'29334","last_scrub_stamp":"2026-02-24T07:55:49.167461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_clean_scrub_stamp":"2026-02-24T07:55:49.167461+0000","objects_scrubbed":28822,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:56:59.424140+0000","scrub_duration":2726,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1184563200,"num_objects":18075,"num_object_clones":0,"num_object_copies":54225,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18075,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":60969,"num_write_kb":2529408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"101'61329","reported_seq":61659,"reported_epoch":101,"state":"active+clean","last_fresh":"2026-02-24T07:56:55.814329+0000","last_change":"2026-02-24T07:55:48.535755+0000","last_active":"2026-02-24T07:56:55.814329+0000","last_peered":"2026-02-24T07:56:55.814329+0000","last_clean":"2026-02-24T07:56:55.814329+0000","last_became_active":"2026-02-24T07:54:42.926136+0000","last_became_peered":"2026-02-24T07:54:42.926136+0000","last_unstale":"2026-02-24T07:56:55.814329+0000","last_undegraded":"2026-02-24T07:56:55.814329+0000","last_fullsized":"2026-02-24T07:56:55.814329+0000","mapping_epoch":90,"log_start":"101'61326","ondisk_log_start":"101'61326","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"96'29148","last_scrub_stamp":"2026-02-24T07:55:48.535691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_clean_scrub_stamp":"2026-02-24T07:55:48.535691+0000","objects_scrubbed":28720,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:56:56.161155+0000","scrub_duration":2872,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1182203904,"num_objects":18039,"num_object_clones":0,"num_object_copies":54117,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18039,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":61329,"num_write_kb":2539776,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,1],"acting":[0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.8","version":"101'61326","reported_seq":61697,"reported_epoch":101,"state":"active+clean","last_fresh":"2026-02-24T07:56:55.794195+0000","last_change":"2026-02-24T07:55:58.178460+0000","last_active":"2026-02-24T07:56:55.794195+0000","last_peered":"2026-02-24T07:56:55.794195+0000","last_clean":"2026-02-24T07:56:55.794195+0000","last_became_active":"2026-02-24T07:54:42.929078+0000","last_became_peered":"2026-02-24T07:54:42.929078+0000","last_unstale":"2026-02-24T07:56:55.794195+0000","last_undegraded":"2026-02-24T07:56:55.794195+0000","last_fullsized":"2026-02-24T07:56:55.794195+0000","mapping_epoch":90,"log_start":"101'61323","ondisk_log_start":"101'61323","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"96'33550","last_scrub_stamp":"2026-02-24T07:55:58.178395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_clean_scrub_stamp":"2026-02-24T07:55:58.178395+0000","objects_scrubbed":32948,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:57:14.607073+0000","scrub_duration":2981,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1189609472,"num_objects":18152,"num_object_clones":0,"num_object_copies":54456,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18152,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":61326,"num_write_kb":2543296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.6","version":"101'61107","reported_seq":61528,"reported_epoch":101,"state":"active+clean","last_fresh":"2026-02-24T07:56:55.920532+0000","last_change":"2026-02-24T07:56:18.901177+0000","last_active":"2026-02-24T07:56:55.920532+0000","last_peered":"2026-02-24T07:56:55.920532+0000","last_clean":"2026-02-24T07:56:55.920532+0000","last_became_active":"2026-02-24T07:54:42.926045+0000","last_became_peered":"2026-02-24T07:54:42.926045+0000","last_unstale":"2026-02-24T07:56:55.920532+0000","last_undegraded":"2026-02-24T07:56:55.920532+0000","last_fullsized":"2026-02-24T07:56:55.920532+0000","mapping_epoch":90,"log_start":"101'61103","ondisk_log_start":"101'61103","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"96'41769","last_scrub_stamp":"2026-02-24T07:56:18.901111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_clean_scrub_stamp":"2026-02-24T07:56:18.901111+0000","objects_scrubbed":38122,"log_size":4,"log_dups_size":3000,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:57:27.205697+0000","scrub_duration":4228,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1166737408,"num_objects":17803,"num_object_clones":0,"num_object_copies":53409,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17803,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":61107,"num_write_kb":2525120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.5","version":"101'61070","reported_seq":61446,"reported_epoch":101,"state":"active+clean","last_fresh":"2026-02-24T07:56:55.769265+0000","last_change":"2026-02-24T07:55:59.773616+0000","last_active":"2026-02-24T07:56:55.769265+0000","last_peered":"2026-02-24T07:56:55.769265+0000","last_clean":"2026-02-24T07:56:55.769265+0000","last_became_active":"2026-02-24T07:54:42.927535+0000","last_became_peered":"2026-02-24T07:54:42.927535+0000","last_unstale":"2026-02-24T07:56:55.769265+0000","last_undegraded":"2026-02-24T07:56:55.769265+0000","last_fullsized":"2026-02-24T07:56:55.769265+0000","mapping_epoch":90,"log_start":"101'61067","ondisk_log_start":"101'61067","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"96'34080","last_scrub_stamp":"2026-02-24T07:55:59.773556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_clean_scrub_stamp":"2026-02-24T07:55:59.773556+0000","objects_scrubbed":33431,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:57:04.302607+0000","scrub_duration":3324,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1174667264,"num_objects":17924,"num_object_clones":0,"num_object_copies":53772,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17924,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":61070,"num_write_kb":2527808,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.4","version":"101'61284","reported_seq":61612,"reported_epoch":101,"state":"active+clean","last_fresh":"2026-02-24T07:56:55.913423+0000","last_change":"2026-02-24T07:55:47.544043+0000","last_active":"2026-02-24T07:56:55.913423+0000","last_peered":"2026-02-24T07:56:55.913423+0000","last_clean":"2026-02-24T07:56:55.913423+0000","last_became_active":"2026-02-24T07:54:42.928011+0000","last_became_peered":"2026-02-24T07:54:42.928011+0000","last_unstale":"2026-02-24T07:56:55.913423+0000","last_undegraded":"2026-02-24T07:56:55.913423+0000","last_fullsized":"2026-02-24T07:56:55.913423+0000","mapping_epoch":90,"log_start":"101'61281","ondisk_log_start":"101'61281","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"96'29055","last_scrub_stamp":"2026-02-24T07:55:47.543994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_clean_scrub_stamp":"2026-02-24T07:55:47.543994+0000","objects_scrubbed":28510,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:57:04.321401+0000","scrub_duration":3165,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1154088960,"num_objects":17610,"num_object_clones":0,"num_object_copies":52830,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17610,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":61284,"num_write_kb":2524608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.3","version":"101'61454","reported_seq":61856,"reported_epoch":101,"state":"active+clean","last_fresh":"2026-02-24T07:56:55.779374+0000","last_change":"2026-02-24T07:56:06.184345+0000","last_active":"2026-02-24T07:56:55.779374+0000","last_peered":"2026-02-24T07:56:55.779374+0000","last_clean":"2026-02-24T07:56:55.779374+0000","last_became_active":"2026-02-24T07:54:42.927730+0000","last_became_peered":"2026-02-24T07:54:42.927730+0000","last_unstale":"2026-02-24T07:56:55.779374+0000","last_undegraded":"2026-02-24T07:56:55.779374+0000","last_fullsized":"2026-02-24T07:56:55.779374+0000","mapping_epoch":90,"log_start":"101'61451","ondisk_log_start":"101'61451","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"96'36664","last_scrub_stamp":"2026-02-24T07:56:06.184309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_clean_scrub_stamp":"2026-02-24T07:56:06.184309+0000","objects_scrubbed":35946,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:57:11.733048+0000","scrub_duration":3811,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1179516928,"num_objects":17998,"num_object_clones":0,"num_object_copies":53994,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17998,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":61454,"num_write_kb":2542464,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.2","version":"101'61087","reported_seq":61467,"reported_epoch":101,"state":"active+clean","last_fresh":"2026-02-24T07:56:55.796491+0000","last_change":"2026-02-24T07:56:00.120637+0000","last_active":"2026-02-24T07:56:55.796491+0000","last_peered":"2026-02-24T07:56:55.796491+0000","last_clean":"2026-02-24T07:56:55.796491+0000","last_became_active":"2026-02-24T07:54:42.930247+0000","last_became_peered":"2026-02-24T07:54:42.930247+0000","last_unstale":"2026-02-24T07:56:55.796491+0000","last_undegraded":"2026-02-24T07:56:55.796491+0000","last_fullsized":"2026-02-24T07:56:55.796491+0000","mapping_epoch":90,"log_start":"101'61083","ondisk_log_start":"101'61083","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"96'34210","last_scrub_stamp":"2026-02-24T07:56:00.120605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_clean_scrub_stamp":"2026-02-24T07:56:00.120605+0000","objects_scrubbed":33656,"log_size":4,"log_dups_size":3000,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:57:29.329949+0000","scrub_duration":2957,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1174470656,"num_objects":17921,"num_object_clones":0,"num_object_copies":53763,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17921,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":61087,"num_write_kb":2528256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1","version":"101'61495","reported_seq":61921,"reported_epoch":101,"state":"active+clean","last_fresh":"2026-02-24T07:56:55.918368+0000","last_change":"2026-02-24T07:56:12.897479+0000","last_active":"2026-02-24T07:56:55.918368+0000","last_peered":"2026-02-24T07:56:55.918368+0000","last_clean":"2026-02-24T07:56:55.918368+0000","last_became_active":"2026-02-24T07:54:42.926442+0000","last_became_peered":"2026-02-24T07:54:42.926442+0000","last_unstale":"2026-02-24T07:56:55.918368+0000","last_undegraded":"2026-02-24T07:56:55.918368+0000","last_fullsized":"2026-02-24T07:56:55.918368+0000","mapping_epoch":90,"log_start":"101'61492","ondisk_log_start":"101'61492","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"96'39245","last_scrub_stamp":"2026-02-24T07:56:12.897436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_clean_scrub_stamp":"2026-02-24T07:56:12.897436+0000","objects_scrubbed":38506,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:57:42.057233+0000","scrub_duration":4279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1179975680,"num_objects":18005,"num_object_clones":0,"num_object_copies":54015,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18005,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":61495,"num_write_kb":2544000,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.0","version":"101'60562","reported_seq":60948,"reported_epoch":101,"state":"active+clean","last_fresh":"2026-02-24T07:56:55.808944+0000","last_change":"2026-02-24T07:56:02.992243+0000","last_active":"2026-02-24T07:56:55.808944+0000","last_peered":"2026-02-24T07:56:55.808944+0000","last_clean":"2026-02-24T07:56:55.808944+0000","last_became_active":"2026-02-24T07:54:42.926910+0000","last_became_peered":"2026-02-24T07:54:42.926910+0000","last_unstale":"2026-02-24T07:56:55.808944+0000","last_undegraded":"2026-02-24T07:56:55.808944+0000","last_fullsized":"2026-02-24T07:56:55.808944+0000","mapping_epoch":90,"log_start":"101'60559","ondisk_log_start":"101'60559","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"96'34952","last_scrub_stamp":"2026-02-24T07:56:02.992156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_clean_scrub_stamp":"2026-02-24T07:56:02.992156+0000","objects_scrubbed":34329,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:57:03.847719+0000","scrub_duration":3335,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1162739712,"num_objects":17742,"num_object_clones":0,"num_object_copies":53226,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17742,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":60562,"num_write_kb":2505728,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.7","version":"101'61511","reported_seq":61896,"reported_epoch":101,"state":"active+clean","last_fresh":"2026-02-24T07:56:55.797406+0000","last_change":"2026-02-24T07:56:01.088786+0000","last_active":"2026-02-24T07:56:55.797406+0000","last_peered":"2026-02-24T07:56:55.797406+0000","last_clean":"2026-02-24T07:56:55.797406+0000","last_became_active":"2026-02-24T07:54:42.930142+0000","last_became_peered":"2026-02-24T07:54:42.930142+0000","last_unstale":"2026-02-24T07:56:55.797406+0000","last_undegraded":"2026-02-24T07:56:55.797406+0000","last_fullsized":"2026-02-24T07:56:55.797406+0000","mapping_epoch":90,"log_start":"101'61508","ondisk_log_start":"101'61508","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"96'34838","last_scrub_stamp":"2026-02-24T07:56:01.088744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_clean_scrub_stamp":"2026-02-24T07:56:01.088744+0000","objects_scrubbed":34284,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:57:24.696242+0000","scrub_duration":2973,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1186660352,"num_objects":18107,"num_object_clones":0,"num_object_copies":54321,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18107,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":61511,"num_write_kb":2547776,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"29'99","reported_seq":417,"reported_epoch":101,"state":"active+clean","last_fresh":"2026-02-24T07:56:53.928077+0000","last_change":"2026-02-24T07:56:33.056711+0000","last_active":"2026-02-24T07:56:53.928077+0000","last_peered":"2026-02-24T07:56:53.928077+0000","last_clean":"2026-02-24T07:56:53.928077+0000","last_became_active":"2026-02-24T07:42:54.186719+0000","last_became_peered":"2026-02-24T07:42:54.186719+0000","last_unstale":"2026-02-24T07:56:53.928077+0000","last_undegraded":"2026-02-24T07:56:53.928077+0000","last_fullsized":"2026-02-24T07:56:53.928077+0000","mapping_epoch":27,"log_start":"29'96","ondisk_log_start":"29'96","created":20,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'99","last_scrub_stamp":"2026-02-24T07:56:33.056678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:42:43.640776+0000","last_clean_scrub_stamp":"2026-02-24T07:56:33.056678+0000","objects_scrubbed":2,"log_size":3,"log_dups_size":96,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:57:56.381501+0000","scrub_duration":21,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":240,"num_write":135,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":16,"stat_sum":{"num_bytes":18779275288,"num_objects":286550,"num_object_clones":0,"num_object_copies":859650,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":286550,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":978802,"num_write_kb":40491201,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":56347930624,"data_stored":56347918408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":50,"ondisk_log_size":50,"up":48,"acting":48,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":240,"num_write":135,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1682,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":3,"up_from":25,"seq":107374182571,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":13270224,"kb_used_data":11702268,"kb_used_omap":21,"kb_used_meta":1567914,"kb_avail":720732976,"statfs":{"total":751619276800,"available":738030567424,"internally_reserved":0,"allocated":11983122432,"data_stored":11982580411,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":22040,"internal_metadata":1605544424},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":19,"seq":81604378797,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":15015892,"kb_used_data":14015616,"kb_used_omap":23,"kb_used_meta":1000168,"kb_avail":718987308,"statfs":{"total":751619276800,"available":736243003392,"internally_reserved":0,"allocated":14351990784,"data_stored":14351444691,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":24090,"internal_metadata":1024172518},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542318,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":17457104,"kb_used_data":16328380,"kb_used_omap":24,"kb_used_meta":1128679,"kb_avail":716546096,"statfs":{"total":751619276800,"available":733743202304,"internally_reserved":0,"allocated":16720261120,"data_stored":16719718579,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":25503,"internal_metadata":1155767393},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705841,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":14902420,"kb_used_data":14034752,"kb_used_omap":21,"kb_used_meta":867626,"kb_avail":719100780,"statfs":{"total":751619276800,"available":736359198720,"internally_reserved":0,"allocated":14371586048,"data_stored":14371039955,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":21691,"internal_metadata":888449861},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1682,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":14099222528,"data_stored":14099218456,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16390033408,"data_stored":16390029336,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":14084083712,"data_stored":14084079640,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":11774590976,"data_stored":11774590976,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-24T07:57:01.739 INFO:tasks.cephadm.ceph_manager.ceph:increase pool size by 10 2026-02-24T07:57:01.740 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph osd pool set .mgr pg_num 461 2026-02-24T07:57:01.891 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:57:02.020 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:57:02.018+0000 7f7fbcd85640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:57:02.021 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:57:02.018+0000 7f7fbcd85640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:57:02.182 INFO:teuthology.orchestra.run.trial035.stderr:Error EPERM: crimson_allow_pg_split is false; pg_num increase denied for crimson pool 2026-02-24T07:57:02.222 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-24T07:57:02.277 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2026-02-24T07:57:02.302 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:57:01 trial035 ceph-mon[39284]: from='client.15621 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2026-02-24T07:57:02.303 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:57:01 trial035 ceph-mon[39284]: pgmap v497: 17 pgs: 1 active+clean+scrubbing, 16 active+clean; 15 GiB data, 51 GiB used, 2.7 TiB / 2.7 TiB avail; 0 B/s wr, 10.09k op/s 2026-02-24T07:57:02.421 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:57:02.536 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:57:02.534+0000 7ff983f1b640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:57:02.536 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:57:02.534+0000 7ff983f1b640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:57:02.551 INFO:teuthology.orchestra.run.trial035.stdout:osd.0: {} 2026-02-24T07:57:02.551 INFO:teuthology.orchestra.run.trial035.stderr:osd.0: osd_enable_op_tracker = '' 2026-02-24T07:57:02.564 INFO:teuthology.orchestra.run.trial035.stdout:osd.1: {} 2026-02-24T07:57:02.564 INFO:teuthology.orchestra.run.trial035.stderr:osd.1: osd_enable_op_tracker = '' 2026-02-24T07:57:02.577 INFO:teuthology.orchestra.run.trial035.stdout:osd.2: {} 2026-02-24T07:57:02.577 INFO:teuthology.orchestra.run.trial035.stderr:osd.2: osd_enable_op_tracker = '' 2026-02-24T07:57:02.589 INFO:teuthology.orchestra.run.trial035.stdout:osd.3: {} 2026-02-24T07:57:02.589 INFO:teuthology.orchestra.run.trial035.stderr:osd.3: osd_enable_op_tracker = '' 2026-02-24T07:57:03.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:57:02 trial035 ceph-mon[39284]: 6.a scrub ok 2026-02-24T07:57:03.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:57:02 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/1778409399' entity='client.admin' cmd={"prefix": "osd pool set", "pool": ".mgr", "var": "pg_num", "val": "461"} : dispatch 2026-02-24T07:57:03.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:57:02 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/737231157' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2026-02-24T07:57:04.938 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2026-02-24T07:57:05.081 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:57:05.198 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:57:04 trial035 ceph-mon[39284]: pgmap v498: 17 pgs: 1 active+clean+scrubbing, 16 active+clean; 15 GiB data, 51 GiB used, 2.7 TiB / 2.7 TiB avail; 0 B/s wr, 8.20k op/s 2026-02-24T07:57:05.199 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:57:04 trial035 ceph-mon[39284]: osdmap e103: 4 total, 4 up, 4 in 2026-02-24T07:57:05.578 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:57:05.524+0000 7f48324a7640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:57:05.579 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:57:05.524+0000 7f48324a7640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:57:05.579 INFO:teuthology.orchestra.run.trial035.stdout:osd.0: {} 2026-02-24T07:57:05.579 INFO:teuthology.orchestra.run.trial035.stdout:osd.1: {} 2026-02-24T07:57:05.580 INFO:teuthology.orchestra.run.trial035.stdout:osd.2: {} 2026-02-24T07:57:05.580 INFO:teuthology.orchestra.run.trial035.stderr:osd.0: osd_enable_op_tracker = '' 2026-02-24T07:57:05.580 INFO:teuthology.orchestra.run.trial035.stderr:osd.1: osd_enable_op_tracker = '' 2026-02-24T07:57:05.580 INFO:teuthology.orchestra.run.trial035.stderr:osd.2: osd_enable_op_tracker = '' 2026-02-24T07:57:05.586 INFO:teuthology.orchestra.run.trial035.stdout:osd.3: {} 2026-02-24T07:57:05.586 INFO:teuthology.orchestra.run.trial035.stderr:osd.3: osd_enable_op_tracker = '' 2026-02-24T07:57:06.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:57:05 trial035 ceph-mon[39284]: 6.d scrub starts 2026-02-24T07:57:06.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:57:05 trial035 ceph-mon[39284]: 6.4 scrub starts 2026-02-24T07:57:06.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:57:05 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/3953220196' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2026-02-24T07:57:07.204 INFO:journalctl@ceph.mgr.x.trial035.stdout:Feb 24 07:57:06 trial035 ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-mgr-x[39621]: ::ffff:10.20.193.91 - - [24/Feb/2026:07:57:06] "GET /metrics HTTP/1.1" 200 4226 "" "Prometheus/3.6.0" 2026-02-24T07:57:07.205 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:57:06 trial035 ceph-mon[39284]: pgmap v500: 17 pgs: 1 active+clean+scrubbing, 16 active+clean; 15 GiB data, 51 GiB used, 2.7 TiB / 2.7 TiB avail; 0 B/s wr, 4.72k op/s 2026-02-24T07:57:07.205 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:57:06 trial035 ceph-mon[39284]: 6.0 scrub starts 2026-02-24T07:57:07.205 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:57:06 trial035 ceph-mon[39284]: 6.d scrub ok 2026-02-24T07:57:07.223 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2026-02-24T07:57:07.223 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2026-02-24T07:57:07.224 INFO:tasks.thrashosds.thrasher:fixing pg num pool .mgr 2026-02-24T07:57:07.224 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph pg dump --format=json 2026-02-24T07:57:07.356 INFO:journalctl@ceph.prometheus.a.trial091.stdout:Feb 24 07:57:06 trial091 ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-prometheus-a[66007]: time=2026-02-24T07:57:06.973Z level=ERROR source=scrape.go:1630 msg="Failed to determine correct type of scrape target." component="scrape manager" scrape_pool=ceph target=http://trial035:9283/metrics content_type="text/html;charset=utf-8" fallback_media_type="" err="received unsupported Content-Type \"text/html;charset=utf-8\" and no fallback_scrape_protocol specified for target" 2026-02-24T07:57:07.366 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:57:07.500 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:57:07.497+0000 7efc65de5640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:57:07.500 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:57:07.498+0000 7efc65de5640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:57:07.631 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2026-02-24T07:57:07.644 INFO:teuthology.orchestra.run.trial035.stdout: 2026-02-24T07:57:07.645 INFO:teuthology.orchestra.run.trial035.stderr:dumped all 2026-02-24T07:57:07.689 INFO:teuthology.orchestra.run.trial035.stdout:{"pg_ready":true,"pg_map":{"version":500,"stamp":"2026-02-24T07:57:05.772621+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":16301949496,"num_objects":248742,"num_object_clones":0,"num_object_copies":746226,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":248742,"num_whiteouts":0,"num_read":137,"num_read_kb":241,"num_write":1016747,"num_write_kb":40493351,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":51,"ondisk_log_size":51,"up":51,"acting":51,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":51,"num_osds":4,"num_per_pool_osds":4,"num_per_pool_omap_osds":4,"kb":2936012800,"kb_used":53294648,"kb_used_data":48531496,"kb_used_omap":96,"kb_used_meta":4762911,"kb_avail":2882718152,"statfs":{"total":3006477107200,"available":2951903387648,"internally_reserved":0,"allocated":49696251904,"data_stored":49694043608,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":99099,"internal_metadata":4877221093},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[13,3,2],"upper_bound":8},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-2477916160,"num_objects":-37810,"num_object_clones":0,"num_object_copies":-113430,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-37810,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":37810,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001100"},"pg_stats":[{"pgid":"6.f","version":"102'63504","reported_seq":63926,"reported_epoch":102,"state":"active+clean","last_fresh":"2026-02-24T07:57:00.921032+0000","last_change":"2026-02-24T07:56:12.247365+0000","last_active":"2026-02-24T07:57:00.921032+0000","last_peered":"2026-02-24T07:57:00.921032+0000","last_clean":"2026-02-24T07:57:00.921032+0000","last_became_active":"2026-02-24T07:54:42.927080+0000","last_became_peered":"2026-02-24T07:54:42.927080+0000","last_unstale":"2026-02-24T07:57:00.921032+0000","last_undegraded":"2026-02-24T07:57:00.921032+0000","last_fullsized":"2026-02-24T07:57:00.921032+0000","mapping_epoch":90,"log_start":"102'63501","ondisk_log_start":"102'63501","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"96'38780","last_scrub_stamp":"2026-02-24T07:56:12.247318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_clean_scrub_stamp":"2026-02-24T07:56:12.247318+0000","objects_scrubbed":37979,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:57:38.474117+0000","scrub_duration":4670,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1016463360,"num_objects":15510,"num_object_clones":0,"num_object_copies":46530,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15510,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":63504,"num_write_kb":2528448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.e","version":"102'63546","reported_seq":63943,"reported_epoch":102,"state":"active+clean","last_fresh":"2026-02-24T07:57:00.783055+0000","last_change":"2026-02-24T07:56:05.374565+0000","last_active":"2026-02-24T07:57:00.783055+0000","last_peered":"2026-02-24T07:57:00.783055+0000","last_clean":"2026-02-24T07:57:00.783055+0000","last_became_active":"2026-02-24T07:54:42.928748+0000","last_became_peered":"2026-02-24T07:54:42.928748+0000","last_unstale":"2026-02-24T07:57:00.783055+0000","last_undegraded":"2026-02-24T07:57:00.783055+0000","last_fullsized":"2026-02-24T07:57:00.783055+0000","mapping_epoch":90,"log_start":"102'63543","ondisk_log_start":"102'63543","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"96'36307","last_scrub_stamp":"2026-02-24T07:56:05.374510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_clean_scrub_stamp":"2026-02-24T07:56:05.374510+0000","objects_scrubbed":35491,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:57:16.427813+0000","scrub_duration":4036,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1020526592,"num_objects":15572,"num_object_clones":0,"num_object_copies":46716,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15572,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":63546,"num_write_kb":2531776,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.d","version":"102'63474","reported_seq":63804,"reported_epoch":102,"state":"active+clean","last_fresh":"2026-02-24T07:57:00.778493+0000","last_change":"2026-02-24T07:55:48.151749+0000","last_active":"2026-02-24T07:57:00.778493+0000","last_peered":"2026-02-24T07:57:00.778493+0000","last_clean":"2026-02-24T07:57:00.778493+0000","last_became_active":"2026-02-24T07:54:42.926886+0000","last_became_peered":"2026-02-24T07:54:42.926886+0000","last_unstale":"2026-02-24T07:57:00.778493+0000","last_undegraded":"2026-02-24T07:57:00.778493+0000","last_fullsized":"2026-02-24T07:57:00.778493+0000","mapping_epoch":90,"log_start":"102'63471","ondisk_log_start":"102'63471","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"96'28924","last_scrub_stamp":"2026-02-24T07:55:48.151672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_clean_scrub_stamp":"2026-02-24T07:55:48.151672+0000","objects_scrubbed":28526,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"queued for scrub","scrub_duration":2742,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1025048600,"num_objects":15642,"num_object_clones":0,"num_object_copies":46926,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15642,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":63474,"num_write_kb":2531649,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.c","version":"102'63673","reported_seq":64071,"reported_epoch":102,"state":"active+clean","last_fresh":"2026-02-24T07:57:00.915102+0000","last_change":"2026-02-24T07:56:25.008652+0000","last_active":"2026-02-24T07:57:00.915102+0000","last_peered":"2026-02-24T07:57:00.915102+0000","last_clean":"2026-02-24T07:57:00.915102+0000","last_became_active":"2026-02-24T07:54:42.928238+0000","last_became_peered":"2026-02-24T07:54:42.928238+0000","last_unstale":"2026-02-24T07:57:00.915102+0000","last_undegraded":"2026-02-24T07:57:00.915102+0000","last_fullsized":"2026-02-24T07:57:00.915102+0000","mapping_epoch":90,"log_start":"102'63670","ondisk_log_start":"102'63670","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"96'44918","last_scrub_stamp":"2026-02-24T07:56:25.008567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_clean_scrub_stamp":"2026-02-24T07:56:25.008567+0000","objects_scrubbed":35276,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:57:40.096151+0000","scrub_duration":4395,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1015480320,"num_objects":15495,"num_object_clones":0,"num_object_copies":46485,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15495,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":63673,"num_write_kb":2533376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.b","version":"102'63052","reported_seq":63483,"reported_epoch":102,"state":"active+clean","last_fresh":"2026-02-24T07:57:00.917098+0000","last_change":"2026-02-24T07:56:16.120475+0000","last_active":"2026-02-24T07:57:00.917098+0000","last_peered":"2026-02-24T07:57:00.917098+0000","last_clean":"2026-02-24T07:57:00.917098+0000","last_became_active":"2026-02-24T07:54:42.926965+0000","last_became_peered":"2026-02-24T07:54:42.926965+0000","last_unstale":"2026-02-24T07:57:00.917098+0000","last_undegraded":"2026-02-24T07:57:00.917098+0000","last_fullsized":"2026-02-24T07:57:00.917098+0000","mapping_epoch":90,"log_start":"102'63049","ondisk_log_start":"102'63049","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"96'39969","last_scrub_stamp":"2026-02-24T07:56:16.120290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_clean_scrub_stamp":"2026-02-24T07:56:16.120290+0000","objects_scrubbed":38739,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:57:36.957012+0000","scrub_duration":4491,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1003618304,"num_objects":15314,"num_object_clones":0,"num_object_copies":45942,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15314,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":63052,"num_write_kb":2507712,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.a","version":"102'63324","reported_seq":63710,"reported_epoch":102,"state":"active+clean+scrubbing","last_fresh":"2026-02-24T07:57:00.782229+0000","last_change":"2026-02-24T07:57:00.123234+0000","last_active":"2026-02-24T07:57:00.782229+0000","last_peered":"2026-02-24T07:57:00.782229+0000","last_clean":"2026-02-24T07:57:00.782229+0000","last_became_active":"2026-02-24T07:54:42.926477+0000","last_became_peered":"2026-02-24T07:54:42.926477+0000","last_unstale":"2026-02-24T07:57:00.782229+0000","last_undegraded":"2026-02-24T07:57:00.782229+0000","last_fullsized":"2026-02-24T07:57:00.782229+0000","mapping_epoch":90,"log_start":"102'63321","ondisk_log_start":"102'63321","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"96'29334","last_scrub_stamp":"2026-02-24T07:55:49.167461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_clean_scrub_stamp":"2026-02-24T07:55:49.167461+0000","objects_scrubbed":5049,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"scrubbing for 1s","scrub_duration":2726,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1030225920,"num_objects":15720,"num_object_clones":0,"num_object_copies":47160,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15720,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":63324,"num_write_kb":2529408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"102'63745","reported_seq":64261,"reported_epoch":102,"state":"active+clean","last_fresh":"2026-02-24T07:57:00.812435+0000","last_change":"2026-02-24T07:56:57.764926+0000","last_active":"2026-02-24T07:57:00.812435+0000","last_peered":"2026-02-24T07:57:00.812435+0000","last_clean":"2026-02-24T07:57:00.812435+0000","last_became_active":"2026-02-24T07:54:42.926136+0000","last_became_peered":"2026-02-24T07:54:42.926136+0000","last_unstale":"2026-02-24T07:57:00.812435+0000","last_undegraded":"2026-02-24T07:57:00.812435+0000","last_fullsized":"2026-02-24T07:57:00.812435+0000","mapping_epoch":90,"log_start":"102'63742","ondisk_log_start":"102'63742","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"101'62265","last_scrub_stamp":"2026-02-24T07:56:57.764859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_clean_scrub_stamp":"2026-02-24T07:56:57.764859+0000","objects_scrubbed":17443,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:58:20.952507+0000","scrub_duration":1544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1023868928,"num_objects":15623,"num_object_clones":0,"num_object_copies":46869,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15623,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":63745,"num_write_kb":2539776,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,1],"acting":[0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.8","version":"102'63734","reported_seq":64107,"reported_epoch":102,"state":"active+clean","last_fresh":"2026-02-24T07:57:00.796397+0000","last_change":"2026-02-24T07:55:58.178460+0000","last_active":"2026-02-24T07:57:00.796397+0000","last_peered":"2026-02-24T07:57:00.796397+0000","last_clean":"2026-02-24T07:57:00.796397+0000","last_became_active":"2026-02-24T07:54:42.929078+0000","last_became_peered":"2026-02-24T07:54:42.929078+0000","last_unstale":"2026-02-24T07:57:00.796397+0000","last_undegraded":"2026-02-24T07:57:00.796397+0000","last_fullsized":"2026-02-24T07:57:00.796397+0000","mapping_epoch":90,"log_start":"102'63731","ondisk_log_start":"102'63731","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"96'33550","last_scrub_stamp":"2026-02-24T07:55:58.178395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_clean_scrub_stamp":"2026-02-24T07:55:58.178395+0000","objects_scrubbed":32948,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:57:14.607073+0000","scrub_duration":2981,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1031798784,"num_objects":15744,"num_object_clones":0,"num_object_copies":47232,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15744,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":63734,"num_write_kb":2543296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.6","version":"102'63480","reported_seq":63905,"reported_epoch":102,"state":"active+clean","last_fresh":"2026-02-24T07:57:00.915397+0000","last_change":"2026-02-24T07:56:18.901177+0000","last_active":"2026-02-24T07:57:00.915397+0000","last_peered":"2026-02-24T07:57:00.915397+0000","last_clean":"2026-02-24T07:57:00.915397+0000","last_became_active":"2026-02-24T07:54:42.926045+0000","last_became_peered":"2026-02-24T07:54:42.926045+0000","last_unstale":"2026-02-24T07:57:00.915397+0000","last_undegraded":"2026-02-24T07:57:00.915397+0000","last_fullsized":"2026-02-24T07:57:00.915397+0000","mapping_epoch":90,"log_start":"102'63477","ondisk_log_start":"102'63477","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"96'41769","last_scrub_stamp":"2026-02-24T07:56:18.901111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_clean_scrub_stamp":"2026-02-24T07:56:18.901111+0000","objects_scrubbed":38122,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:57:27.205697+0000","scrub_duration":4228,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1011220480,"num_objects":15430,"num_object_clones":0,"num_object_copies":46290,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15430,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":63480,"num_write_kb":2525120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.5","version":"102'63446","reported_seq":63823,"reported_epoch":102,"state":"active+clean","last_fresh":"2026-02-24T07:57:00.782385+0000","last_change":"2026-02-24T07:55:59.773616+0000","last_active":"2026-02-24T07:57:00.782385+0000","last_peered":"2026-02-24T07:57:00.782385+0000","last_clean":"2026-02-24T07:57:00.782385+0000","last_became_active":"2026-02-24T07:54:42.927535+0000","last_became_peered":"2026-02-24T07:54:42.927535+0000","last_unstale":"2026-02-24T07:57:00.782385+0000","last_undegraded":"2026-02-24T07:57:00.782385+0000","last_fullsized":"2026-02-24T07:57:00.782385+0000","mapping_epoch":90,"log_start":"102'63443","ondisk_log_start":"102'63443","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"96'34080","last_scrub_stamp":"2026-02-24T07:55:59.773556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_clean_scrub_stamp":"2026-02-24T07:55:59.773556+0000","objects_scrubbed":33431,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:57:04.302607+0000","scrub_duration":3324,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1018953728,"num_objects":15548,"num_object_clones":0,"num_object_copies":46644,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15548,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":63446,"num_write_kb":2527808,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.4","version":"102'63579","reported_seq":63909,"reported_epoch":102,"state":"active+clean","last_fresh":"2026-02-24T07:57:00.920321+0000","last_change":"2026-02-24T07:55:47.544043+0000","last_active":"2026-02-24T07:57:00.920321+0000","last_peered":"2026-02-24T07:57:00.920321+0000","last_clean":"2026-02-24T07:57:00.920321+0000","last_became_active":"2026-02-24T07:54:42.928011+0000","last_became_peered":"2026-02-24T07:54:42.928011+0000","last_unstale":"2026-02-24T07:57:00.920321+0000","last_undegraded":"2026-02-24T07:57:00.920321+0000","last_fullsized":"2026-02-24T07:57:00.920321+0000","mapping_epoch":90,"log_start":"102'63576","ondisk_log_start":"102'63576","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"96'29055","last_scrub_stamp":"2026-02-24T07:55:47.543994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_clean_scrub_stamp":"2026-02-24T07:55:47.543994+0000","objects_scrubbed":28510,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:57:04.321401+0000","scrub_duration":3165,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1003683840,"num_objects":15315,"num_object_clones":0,"num_object_copies":45945,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15315,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":63579,"num_write_kb":2524608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.3","version":"102'63926","reported_seq":64329,"reported_epoch":102,"state":"active+clean","last_fresh":"2026-02-24T07:57:00.780026+0000","last_change":"2026-02-24T07:56:06.184345+0000","last_active":"2026-02-24T07:57:00.780026+0000","last_peered":"2026-02-24T07:57:00.780026+0000","last_clean":"2026-02-24T07:57:00.780026+0000","last_became_active":"2026-02-24T07:54:42.927730+0000","last_became_peered":"2026-02-24T07:54:42.927730+0000","last_unstale":"2026-02-24T07:57:00.780026+0000","last_undegraded":"2026-02-24T07:57:00.780026+0000","last_fullsized":"2026-02-24T07:57:00.780026+0000","mapping_epoch":90,"log_start":"102'63923","ondisk_log_start":"102'63923","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"96'36664","last_scrub_stamp":"2026-02-24T07:56:06.184309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_clean_scrub_stamp":"2026-02-24T07:56:06.184309+0000","objects_scrubbed":35946,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:57:11.733048+0000","scrub_duration":3811,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1017511936,"num_objects":15526,"num_object_clones":0,"num_object_copies":46578,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15526,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":63926,"num_write_kb":2542464,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.2","version":"102'63381","reported_seq":63763,"reported_epoch":102,"state":"active+clean","last_fresh":"2026-02-24T07:57:00.785358+0000","last_change":"2026-02-24T07:56:00.120637+0000","last_active":"2026-02-24T07:57:00.785358+0000","last_peered":"2026-02-24T07:57:00.785358+0000","last_clean":"2026-02-24T07:57:00.785358+0000","last_became_active":"2026-02-24T07:54:42.930247+0000","last_became_peered":"2026-02-24T07:54:42.930247+0000","last_unstale":"2026-02-24T07:57:00.785358+0000","last_undegraded":"2026-02-24T07:57:00.785358+0000","last_fullsized":"2026-02-24T07:57:00.785358+0000","mapping_epoch":90,"log_start":"102'63378","ondisk_log_start":"102'63378","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"96'34210","last_scrub_stamp":"2026-02-24T07:56:00.120605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_clean_scrub_stamp":"2026-02-24T07:56:00.120605+0000","objects_scrubbed":33656,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:57:29.329949+0000","scrub_duration":2957,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1024131072,"num_objects":15627,"num_object_clones":0,"num_object_copies":46881,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15627,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":63381,"num_write_kb":2528256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1","version":"102'63896","reported_seq":64324,"reported_epoch":102,"state":"active+clean","last_fresh":"2026-02-24T07:57:00.919655+0000","last_change":"2026-02-24T07:56:12.897479+0000","last_active":"2026-02-24T07:57:00.919655+0000","last_peered":"2026-02-24T07:57:00.919655+0000","last_clean":"2026-02-24T07:57:00.919655+0000","last_became_active":"2026-02-24T07:54:42.926442+0000","last_became_peered":"2026-02-24T07:54:42.926442+0000","last_unstale":"2026-02-24T07:57:00.919655+0000","last_undegraded":"2026-02-24T07:57:00.919655+0000","last_fullsized":"2026-02-24T07:57:00.919655+0000","mapping_epoch":90,"log_start":"102'63893","ondisk_log_start":"102'63893","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"96'39245","last_scrub_stamp":"2026-02-24T07:56:12.897436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_clean_scrub_stamp":"2026-02-24T07:56:12.897436+0000","objects_scrubbed":38506,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:57:42.057233+0000","scrub_duration":4279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1022623744,"num_objects":15604,"num_object_clones":0,"num_object_copies":46812,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15604,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":63896,"num_write_kb":2544000,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.0","version":"102'62949","reported_seq":63336,"reported_epoch":102,"state":"active+clean","last_fresh":"2026-02-24T07:57:00.812140+0000","last_change":"2026-02-24T07:56:02.992243+0000","last_active":"2026-02-24T07:57:00.812140+0000","last_peered":"2026-02-24T07:57:00.812140+0000","last_clean":"2026-02-24T07:57:00.812140+0000","last_became_active":"2026-02-24T07:54:42.926910+0000","last_became_peered":"2026-02-24T07:54:42.926910+0000","last_unstale":"2026-02-24T07:57:00.812140+0000","last_undegraded":"2026-02-24T07:57:00.812140+0000","last_fullsized":"2026-02-24T07:57:00.812140+0000","mapping_epoch":90,"log_start":"102'62946","ondisk_log_start":"102'62946","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"96'34952","last_scrub_stamp":"2026-02-24T07:56:02.992156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_clean_scrub_stamp":"2026-02-24T07:56:02.992156+0000","objects_scrubbed":34329,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:57:03.847719+0000","scrub_duration":3335,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1006305280,"num_objects":15355,"num_object_clones":0,"num_object_copies":46065,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15355,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":62949,"num_write_kb":2505728,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.7","version":"102'63903","reported_seq":64290,"reported_epoch":102,"state":"active+clean","last_fresh":"2026-02-24T07:57:00.791250+0000","last_change":"2026-02-24T07:56:01.088786+0000","last_active":"2026-02-24T07:57:00.791250+0000","last_peered":"2026-02-24T07:57:00.791250+0000","last_clean":"2026-02-24T07:57:00.791250+0000","last_became_active":"2026-02-24T07:54:42.930142+0000","last_became_peered":"2026-02-24T07:54:42.930142+0000","last_unstale":"2026-02-24T07:57:00.791250+0000","last_undegraded":"2026-02-24T07:57:00.791250+0000","last_fullsized":"2026-02-24T07:57:00.791250+0000","mapping_epoch":90,"log_start":"102'63900","ondisk_log_start":"102'63900","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"96'34838","last_scrub_stamp":"2026-02-24T07:56:01.088744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_clean_scrub_stamp":"2026-02-24T07:56:01.088744+0000","objects_scrubbed":34284,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:57:24.696242+0000","scrub_duration":2973,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1029898240,"num_objects":15715,"num_object_clones":0,"num_object_copies":47145,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15715,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":63903,"num_write_kb":2547776,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"29'99","reported_seq":419,"reported_epoch":102,"state":"active+clean","last_fresh":"2026-02-24T07:56:58.929008+0000","last_change":"2026-02-24T07:56:33.056711+0000","last_active":"2026-02-24T07:56:58.929008+0000","last_peered":"2026-02-24T07:56:58.929008+0000","last_clean":"2026-02-24T07:56:58.929008+0000","last_became_active":"2026-02-24T07:42:54.186719+0000","last_became_peered":"2026-02-24T07:42:54.186719+0000","last_unstale":"2026-02-24T07:56:58.929008+0000","last_undegraded":"2026-02-24T07:56:58.929008+0000","last_fullsized":"2026-02-24T07:56:58.929008+0000","mapping_epoch":27,"log_start":"29'96","ondisk_log_start":"29'96","created":20,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'99","last_scrub_stamp":"2026-02-24T07:56:33.056678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:42:43.640776+0000","last_clean_scrub_stamp":"2026-02-24T07:56:33.056678+0000","objects_scrubbed":2,"log_size":3,"log_dups_size":96,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:57:56.381501+0000","scrub_duration":21,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":240,"num_write":135,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":16,"stat_sum":{"num_bytes":16301359128,"num_objects":248740,"num_object_clones":0,"num_object_copies":746220,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":248740,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1016612,"num_write_kb":40491201,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":48911495168,"data_stored":48911482952,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":186,"internal_metadata":0},"log_size":48,"ondisk_log_size":48,"up":48,"acting":48,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":240,"num_write":135,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1682,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":3,"up_from":25,"seq":107374182572,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":11720284,"kb_used_data":10116232,"kb_used_omap":24,"kb_used_meta":1604007,"kb_avail":722282916,"statfs":{"total":751619276800,"available":739617705984,"internally_reserved":0,"allocated":10359021568,"data_stored":10358471660,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":25114,"internal_metadata":1642503654},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":19,"seq":81604378798,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":13197664,"kb_used_data":12143308,"kb_used_omap":24,"kb_used_meta":1054311,"kb_avail":720805536,"statfs":{"total":751619276800,"available":738104868864,"internally_reserved":0,"allocated":12434747392,"data_stored":12434193412,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":25394,"internal_metadata":1079614670},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,1],"upper_bound":4},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542319,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":15319900,"kb_used_data":14136968,"kb_used_omap":25,"kb_used_meta":1182822,"kb_avail":718683300,"statfs":{"total":751619276800,"available":735931699200,"internally_reserved":0,"allocated":14476255232,"data_stored":14475704804,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":26155,"internal_metadata":1211210197},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[8,2],"upper_bound":4},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705842,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":13056800,"kb_used_data":12134988,"kb_used_omap":21,"kb_used_meta":921770,"kb_avail":720946400,"statfs":{"total":751619276800,"available":738249113600,"internally_reserved":0,"allocated":12426227712,"data_stored":12425673732,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":22436,"internal_metadata":943892572},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,2],"upper_bound":8},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1682,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12237803520,"data_stored":12237799448,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":93,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":14233178112,"data_stored":14233174040,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12228366336,"data_stored":12228362264,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":93,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":10212147200,"data_stored":10212147200,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-24T07:57:07.690 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph osd pool set .mgr pgp_num 461 2026-02-24T07:57:07.776 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:57:07.874 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:57:07.902 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:57:07.900+0000 7f91b3073640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:57:07.903 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:57:07.901+0000 7f91b3073640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:57:07.918 INFO:teuthology.orchestra.run.trial035.stdout:osd.0: {} 2026-02-24T07:57:07.919 INFO:teuthology.orchestra.run.trial035.stderr:osd.0: osd_enable_op_tracker = '' 2026-02-24T07:57:07.932 INFO:teuthology.orchestra.run.trial035.stdout:osd.1: {} 2026-02-24T07:57:07.933 INFO:teuthology.orchestra.run.trial035.stderr:osd.1: osd_enable_op_tracker = '' 2026-02-24T07:57:07.945 INFO:teuthology.orchestra.run.trial035.stdout:osd.2: {} 2026-02-24T07:57:07.946 INFO:teuthology.orchestra.run.trial035.stderr:osd.2: osd_enable_op_tracker = '' 2026-02-24T07:57:07.961 INFO:teuthology.orchestra.run.trial035.stdout:osd.3: {} 2026-02-24T07:57:07.961 INFO:teuthology.orchestra.run.trial035.stderr:osd.3: osd_enable_op_tracker = '' 2026-02-24T07:57:08.004 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:57:08.002+0000 7f95ed38a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:57:08.005 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:57:08.003+0000 7f95ed38a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:57:08.162 INFO:teuthology.orchestra.run.trial035.stderr:Error EPERM: crimson_allow_pg_split is false; pgp_num increase denied 2026-02-24T07:57:08.162 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:57:07 trial035 ceph-mon[39284]: 6.4 scrub ok 2026-02-24T07:57:08.162 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:57:07 trial035 ceph-mon[39284]: 6.0 scrub ok 2026-02-24T07:57:08.163 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:57:07 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/1761511323' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2026-02-24T07:57:08.202 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-24T07:57:09.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:57:08 trial035 ceph-mon[39284]: from='client.15629 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2026-02-24T07:57:09.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:57:08 trial035 ceph-mon[39284]: pgmap v501: 17 pgs: 2 active+clean+scrubbing, 15 active+clean; 13 GiB data, 44 GiB used, 2.7 TiB / 2.7 TiB avail; 0 B/s wr, 8.67k op/s 2026-02-24T07:57:09.205 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:57:08 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/1796205286' entity='client.admin' cmd={"prefix": "osd pool set", "pool": ".mgr", "var": "pgp_num", "val": "461"} : dispatch 2026-02-24T07:57:09.205 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:57:08 trial035 ceph-mon[39284]: osdmap e104: 4 total, 4 up, 4 in 2026-02-24T07:57:10.005 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2026-02-24T07:57:10.141 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:57:10.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:57:09 trial035 ceph-mon[39284]: pgmap v503: 17 pgs: 2 active+clean+scrubbing, 15 active+clean; 13 GiB data, 44 GiB used, 2.7 TiB / 2.7 TiB avail; 0 B/s wr, 4.86k op/s 2026-02-24T07:57:10.738 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:57:10.704+0000 7fdfbd203640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:57:10.739 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:57:10.704+0000 7fdfbd203640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:57:10.739 INFO:teuthology.orchestra.run.trial035.stderr:osd.0: osd_enable_op_tracker = '' 2026-02-24T07:57:10.739 INFO:teuthology.orchestra.run.trial035.stdout:osd.0: {} 2026-02-24T07:57:10.747 INFO:teuthology.orchestra.run.trial035.stdout:osd.1: {} 2026-02-24T07:57:10.747 INFO:teuthology.orchestra.run.trial035.stderr:osd.1: osd_enable_op_tracker = '' 2026-02-24T07:57:10.759 INFO:teuthology.orchestra.run.trial035.stdout:osd.2: {} 2026-02-24T07:57:10.759 INFO:teuthology.orchestra.run.trial035.stderr:osd.2: osd_enable_op_tracker = '' 2026-02-24T07:57:10.771 INFO:teuthology.orchestra.run.trial035.stdout:osd.3: {} 2026-02-24T07:57:10.771 INFO:teuthology.orchestra.run.trial035.stderr:osd.3: osd_enable_op_tracker = '' 2026-02-24T07:57:11.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:57:10 trial035 ceph-mon[39284]: 6.5 scrub starts 2026-02-24T07:57:11.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:57:10 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/1268413477' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2026-02-24T07:57:11.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:57:10 trial035 ceph-mon[39284]: from='mgr.14284 10.20.193.35:0/3681373222' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2026-02-24T07:57:12.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:57:11 trial035 ceph-mon[39284]: from='mgr.14284 10.20.193.35:0/3681373222' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-24T07:57:12.205 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:57:11 trial035 ceph-mon[39284]: from='mgr.14284 10.20.193.35:0/3681373222' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2026-02-24T07:57:12.205 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:57:11 trial035 ceph-mon[39284]: from='mgr.14284 10.20.193.35:0/3681373222' entity='mgr.x' 2026-02-24T07:57:12.205 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:57:11 trial035 ceph-mon[39284]: from='mgr.14284 10.20.193.35:0/3681373222' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2026-02-24T07:57:12.205 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:57:11 trial035 ceph-mon[39284]: 6.5 scrub ok 2026-02-24T07:57:12.205 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:57:11 trial035 ceph-mon[39284]: pgmap v504: 17 pgs: 1 active+clean+scrubbing, 16 active+clean; 10 GiB data, 37 GiB used, 2.7 TiB / 2.7 TiB avail; 0 B/s wr, 9.74k op/s 2026-02-24T07:57:13.203 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2026-02-24T07:57:13.203 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2026-02-24T07:57:13.203 INFO:tasks.thrashosds.thrasher:Growing pool unique_pool_4 2026-02-24T07:57:13.204 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph pg dump --format=json 2026-02-24T07:57:13.352 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:57:13.470 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:57:13.467+0000 7f65473d4640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:57:13.470 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:57:13.468+0000 7f65473d4640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:57:13.536 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2026-02-24T07:57:13.617 INFO:teuthology.orchestra.run.trial035.stdout: 2026-02-24T07:57:13.618 INFO:teuthology.orchestra.run.trial035.stderr:dumped all 2026-02-24T07:57:13.665 INFO:teuthology.orchestra.run.trial035.stdout:{"pg_ready":true,"pg_map":{"version":504,"stamp":"2026-02-24T07:57:11.773408+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":11192566328,"num_objects":170779,"num_object_clones":0,"num_object_copies":512337,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":170779,"num_whiteouts":0,"num_read":137,"num_read_kb":241,"num_write":1094710,"num_write_kb":40493351,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":51,"ondisk_log_size":51,"up":51,"acting":51,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":51,"num_osds":4,"num_per_pool_osds":4,"num_per_pool_omap_osds":4,"kb":2936012800,"kb_used":38785944,"kb_used_data":33535496,"kb_used_omap":101,"kb_used_meta":5250202,"kb_avail":2897226856,"statfs":{"total":3006477107200,"available":2966760300544,"internally_reserved":0,"allocated":34340347904,"data_stored":34338076512,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":104025,"internal_metadata":5376207271},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[15,2],"upper_bound":4},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-5109383168,"num_objects":-77963,"num_object_clones":0,"num_object_copies":-233889,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-77963,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":77963,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001036"},"pg_stats":[{"pgid":"6.f","version":"104'68324","reported_seq":68751,"reported_epoch":104,"state":"active+clean","last_fresh":"2026-02-24T07:57:10.922806+0000","last_change":"2026-02-24T07:56:12.247365+0000","last_active":"2026-02-24T07:57:10.922806+0000","last_peered":"2026-02-24T07:57:10.922806+0000","last_clean":"2026-02-24T07:57:10.922806+0000","last_became_active":"2026-02-24T07:54:42.927080+0000","last_became_peered":"2026-02-24T07:54:42.927080+0000","last_unstale":"2026-02-24T07:57:10.922806+0000","last_undegraded":"2026-02-24T07:57:10.922806+0000","last_fullsized":"2026-02-24T07:57:10.922806+0000","mapping_epoch":90,"log_start":"104'68321","ondisk_log_start":"104'68321","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"96'38780","last_scrub_stamp":"2026-02-24T07:56:12.247318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_clean_scrub_stamp":"2026-02-24T07:56:12.247318+0000","objects_scrubbed":37979,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:57:38.474117+0000","scrub_duration":4670,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":700579840,"num_objects":10690,"num_object_clones":0,"num_object_copies":32070,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10690,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":68324,"num_write_kb":2528448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.e","version":"104'68504","reported_seq":68906,"reported_epoch":104,"state":"active+clean","last_fresh":"2026-02-24T07:57:10.780912+0000","last_change":"2026-02-24T07:56:05.374565+0000","last_active":"2026-02-24T07:57:10.780912+0000","last_peered":"2026-02-24T07:57:10.780912+0000","last_clean":"2026-02-24T07:57:10.780912+0000","last_became_active":"2026-02-24T07:54:42.928748+0000","last_became_peered":"2026-02-24T07:54:42.928748+0000","last_unstale":"2026-02-24T07:57:10.780912+0000","last_undegraded":"2026-02-24T07:57:10.780912+0000","last_fullsized":"2026-02-24T07:57:10.780912+0000","mapping_epoch":90,"log_start":"104'68501","ondisk_log_start":"104'68501","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"96'36307","last_scrub_stamp":"2026-02-24T07:56:05.374510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_clean_scrub_stamp":"2026-02-24T07:56:05.374510+0000","objects_scrubbed":35491,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:57:16.427813+0000","scrub_duration":4036,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":695599104,"num_objects":10614,"num_object_clones":0,"num_object_copies":31842,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10614,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":68504,"num_write_kb":2531776,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.d","version":"104'68331","reported_seq":68804,"reported_epoch":104,"state":"active+clean","last_fresh":"2026-02-24T07:57:10.782467+0000","last_change":"2026-02-24T07:57:06.430234+0000","last_active":"2026-02-24T07:57:10.782467+0000","last_peered":"2026-02-24T07:57:10.782467+0000","last_clean":"2026-02-24T07:57:10.782467+0000","last_became_active":"2026-02-24T07:54:42.926886+0000","last_became_peered":"2026-02-24T07:54:42.926886+0000","last_unstale":"2026-02-24T07:57:10.782467+0000","last_undegraded":"2026-02-24T07:57:10.782467+0000","last_fullsized":"2026-02-24T07:57:10.782467+0000","mapping_epoch":90,"log_start":"104'68328","ondisk_log_start":"104'68328","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"103'66256","last_scrub_stamp":"2026-02-24T07:57:06.430163+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_clean_scrub_stamp":"2026-02-24T07:57:06.430163+0000","objects_scrubbed":13086,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:58:07.786827+0000","scrub_duration":1340,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":706740248,"num_objects":10785,"num_object_clones":0,"num_object_copies":32355,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10785,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":68331,"num_write_kb":2531649,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.c","version":"104'68532","reported_seq":68934,"reported_epoch":104,"state":"active+clean","last_fresh":"2026-02-24T07:57:10.922684+0000","last_change":"2026-02-24T07:56:25.008652+0000","last_active":"2026-02-24T07:57:10.922684+0000","last_peered":"2026-02-24T07:57:10.922684+0000","last_clean":"2026-02-24T07:57:10.922684+0000","last_became_active":"2026-02-24T07:54:42.928238+0000","last_became_peered":"2026-02-24T07:54:42.928238+0000","last_unstale":"2026-02-24T07:57:10.922684+0000","last_undegraded":"2026-02-24T07:57:10.922684+0000","last_fullsized":"2026-02-24T07:57:10.922684+0000","mapping_epoch":90,"log_start":"104'68529","ondisk_log_start":"104'68529","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"96'44918","last_scrub_stamp":"2026-02-24T07:56:25.008567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_clean_scrub_stamp":"2026-02-24T07:56:25.008567+0000","objects_scrubbed":35276,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:57:40.096151+0000","scrub_duration":4395,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":697040896,"num_objects":10636,"num_object_clones":0,"num_object_copies":31908,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10636,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":68532,"num_write_kb":2533376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.b","version":"104'67864","reported_seq":68299,"reported_epoch":104,"state":"active+clean","last_fresh":"2026-02-24T07:57:10.923100+0000","last_change":"2026-02-24T07:56:16.120475+0000","last_active":"2026-02-24T07:57:10.923100+0000","last_peered":"2026-02-24T07:57:10.923100+0000","last_clean":"2026-02-24T07:57:10.923100+0000","last_became_active":"2026-02-24T07:54:42.926965+0000","last_became_peered":"2026-02-24T07:54:42.926965+0000","last_unstale":"2026-02-24T07:57:10.923100+0000","last_undegraded":"2026-02-24T07:57:10.923100+0000","last_fullsized":"2026-02-24T07:57:10.923100+0000","mapping_epoch":90,"log_start":"104'67861","ondisk_log_start":"104'67861","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"96'39969","last_scrub_stamp":"2026-02-24T07:56:16.120290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_clean_scrub_stamp":"2026-02-24T07:56:16.120290+0000","objects_scrubbed":38739,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:57:36.957012+0000","scrub_duration":4491,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":688259072,"num_objects":10502,"num_object_clones":0,"num_object_copies":31506,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10502,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":67864,"num_write_kb":2507712,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.a","version":"104'68263","reported_seq":68764,"reported_epoch":104,"state":"active+clean","last_fresh":"2026-02-24T07:57:10.782182+0000","last_change":"2026-02-24T07:57:02.035437+0000","last_active":"2026-02-24T07:57:10.782182+0000","last_peered":"2026-02-24T07:57:10.782182+0000","last_clean":"2026-02-24T07:57:10.782182+0000","last_became_active":"2026-02-24T07:54:42.926477+0000","last_became_peered":"2026-02-24T07:54:42.926477+0000","last_unstale":"2026-02-24T07:57:10.782182+0000","last_undegraded":"2026-02-24T07:57:10.782182+0000","last_fullsized":"2026-02-24T07:57:10.782182+0000","mapping_epoch":90,"log_start":"104'68260","ondisk_log_start":"104'68260","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"102'63835","last_scrub_stamp":"2026-02-24T07:57:02.035399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_clean_scrub_stamp":"2026-02-24T07:57:02.035399+0000","objects_scrubbed":15563,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:58:02.340758+0000","scrub_duration":1912,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":706543616,"num_objects":10781,"num_object_clones":0,"num_object_copies":32343,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10781,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":68263,"num_write_kb":2529408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"104'68567","reported_seq":69087,"reported_epoch":104,"state":"active+clean","last_fresh":"2026-02-24T07:57:10.807773+0000","last_change":"2026-02-24T07:56:57.764926+0000","last_active":"2026-02-24T07:57:10.807773+0000","last_peered":"2026-02-24T07:57:10.807773+0000","last_clean":"2026-02-24T07:57:10.807773+0000","last_became_active":"2026-02-24T07:54:42.926136+0000","last_became_peered":"2026-02-24T07:54:42.926136+0000","last_unstale":"2026-02-24T07:57:10.807773+0000","last_undegraded":"2026-02-24T07:57:10.807773+0000","last_fullsized":"2026-02-24T07:57:10.807773+0000","mapping_epoch":90,"log_start":"104'68564","ondisk_log_start":"104'68564","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"101'62265","last_scrub_stamp":"2026-02-24T07:56:57.764859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_clean_scrub_stamp":"2026-02-24T07:56:57.764859+0000","objects_scrubbed":17443,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:58:20.952507+0000","scrub_duration":1544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":707854336,"num_objects":10801,"num_object_clones":0,"num_object_copies":32403,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10801,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":68567,"num_write_kb":2539776,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,1],"acting":[0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.8","version":"104'68675","reported_seq":69052,"reported_epoch":104,"state":"active+clean","last_fresh":"2026-02-24T07:57:10.799024+0000","last_change":"2026-02-24T07:55:58.178460+0000","last_active":"2026-02-24T07:57:10.799024+0000","last_peered":"2026-02-24T07:57:10.799024+0000","last_clean":"2026-02-24T07:57:10.799024+0000","last_became_active":"2026-02-24T07:54:42.929078+0000","last_became_peered":"2026-02-24T07:54:42.929078+0000","last_unstale":"2026-02-24T07:57:10.799024+0000","last_undegraded":"2026-02-24T07:57:10.799024+0000","last_fullsized":"2026-02-24T07:57:10.799024+0000","mapping_epoch":90,"log_start":"104'68672","ondisk_log_start":"104'68672","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"96'33550","last_scrub_stamp":"2026-02-24T07:55:58.178395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_clean_scrub_stamp":"2026-02-24T07:55:58.178395+0000","objects_scrubbed":32948,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:57:14.607073+0000","scrub_duration":2981,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":707985408,"num_objects":10803,"num_object_clones":0,"num_object_copies":32409,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10803,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":68675,"num_write_kb":2543296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.6","version":"104'68342","reported_seq":68771,"reported_epoch":104,"state":"active+clean","last_fresh":"2026-02-24T07:57:10.922038+0000","last_change":"2026-02-24T07:56:18.901177+0000","last_active":"2026-02-24T07:57:10.922038+0000","last_peered":"2026-02-24T07:57:10.922038+0000","last_clean":"2026-02-24T07:57:10.922038+0000","last_became_active":"2026-02-24T07:54:42.926045+0000","last_became_peered":"2026-02-24T07:54:42.926045+0000","last_unstale":"2026-02-24T07:57:10.922038+0000","last_undegraded":"2026-02-24T07:57:10.922038+0000","last_fullsized":"2026-02-24T07:57:10.922038+0000","mapping_epoch":90,"log_start":"104'68339","ondisk_log_start":"104'68339","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"96'41769","last_scrub_stamp":"2026-02-24T07:56:18.901111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_clean_scrub_stamp":"2026-02-24T07:56:18.901111+0000","objects_scrubbed":38122,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:57:27.205697+0000","scrub_duration":4228,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":692584448,"num_objects":10568,"num_object_clones":0,"num_object_copies":31704,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10568,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":68342,"num_write_kb":2525120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.5","version":"104'68313","reported_seq":68743,"reported_epoch":104,"state":"active+clean+scrubbing","last_fresh":"2026-02-24T07:57:10.784429+0000","last_change":"2026-02-24T07:57:10.111179+0000","last_active":"2026-02-24T07:57:10.784429+0000","last_peered":"2026-02-24T07:57:10.784429+0000","last_clean":"2026-02-24T07:57:10.784429+0000","last_became_active":"2026-02-24T07:54:42.927535+0000","last_became_peered":"2026-02-24T07:54:42.927535+0000","last_unstale":"2026-02-24T07:57:10.784429+0000","last_undegraded":"2026-02-24T07:57:10.784429+0000","last_fullsized":"2026-02-24T07:57:10.784429+0000","mapping_epoch":90,"log_start":"104'68310","ondisk_log_start":"104'68310","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"96'34080","last_scrub_stamp":"2026-02-24T07:55:59.773556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_clean_scrub_stamp":"2026-02-24T07:55:59.773556+0000","objects_scrubbed":4549,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"scrubbing for 1s","scrub_duration":3324,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":699990016,"num_objects":10681,"num_object_clones":0,"num_object_copies":32043,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10681,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":68313,"num_write_kb":2527808,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.4","version":"104'68539","reported_seq":69011,"reported_epoch":104,"state":"active+clean","last_fresh":"2026-02-24T07:57:10.922729+0000","last_change":"2026-02-24T07:57:06.901398+0000","last_active":"2026-02-24T07:57:10.922729+0000","last_peered":"2026-02-24T07:57:10.922729+0000","last_clean":"2026-02-24T07:57:10.922729+0000","last_became_active":"2026-02-24T07:54:42.928011+0000","last_became_peered":"2026-02-24T07:54:42.928011+0000","last_unstale":"2026-02-24T07:57:10.922729+0000","last_undegraded":"2026-02-24T07:57:10.922729+0000","last_fullsized":"2026-02-24T07:57:10.922729+0000","mapping_epoch":90,"log_start":"104'68536","ondisk_log_start":"104'68536","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"103'66457","last_scrub_stamp":"2026-02-24T07:57:06.901316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_clean_scrub_stamp":"2026-02-24T07:57:06.901316+0000","objects_scrubbed":12783,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:58:16.895385+0000","scrub_duration":1738,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":678625280,"num_objects":10355,"num_object_clones":0,"num_object_copies":31065,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10355,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":68539,"num_write_kb":2524608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.3","version":"104'68696","reported_seq":69104,"reported_epoch":104,"state":"active+clean","last_fresh":"2026-02-24T07:57:10.773432+0000","last_change":"2026-02-24T07:56:06.184345+0000","last_active":"2026-02-24T07:57:10.773432+0000","last_peered":"2026-02-24T07:57:10.773432+0000","last_clean":"2026-02-24T07:57:10.773432+0000","last_became_active":"2026-02-24T07:54:42.927730+0000","last_became_peered":"2026-02-24T07:54:42.927730+0000","last_unstale":"2026-02-24T07:57:10.773432+0000","last_undegraded":"2026-02-24T07:57:10.773432+0000","last_fullsized":"2026-02-24T07:57:10.773432+0000","mapping_epoch":90,"log_start":"104'68693","ondisk_log_start":"104'68693","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"96'36664","last_scrub_stamp":"2026-02-24T07:56:06.184309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_clean_scrub_stamp":"2026-02-24T07:56:06.184309+0000","objects_scrubbed":35946,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:57:11.733048+0000","scrub_duration":3811,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":704905216,"num_objects":10756,"num_object_clones":0,"num_object_copies":32268,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10756,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":68696,"num_write_kb":2542464,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.2","version":"104'68272","reported_seq":68658,"reported_epoch":104,"state":"active+clean","last_fresh":"2026-02-24T07:57:10.795670+0000","last_change":"2026-02-24T07:56:00.120637+0000","last_active":"2026-02-24T07:57:10.795670+0000","last_peered":"2026-02-24T07:57:10.795670+0000","last_clean":"2026-02-24T07:57:10.795670+0000","last_became_active":"2026-02-24T07:54:42.930247+0000","last_became_peered":"2026-02-24T07:54:42.930247+0000","last_unstale":"2026-02-24T07:57:10.795670+0000","last_undegraded":"2026-02-24T07:57:10.795670+0000","last_fullsized":"2026-02-24T07:57:10.795670+0000","mapping_epoch":90,"log_start":"104'68269","ondisk_log_start":"104'68269","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"96'34210","last_scrub_stamp":"2026-02-24T07:56:00.120605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_clean_scrub_stamp":"2026-02-24T07:56:00.120605+0000","objects_scrubbed":33656,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:57:29.329949+0000","scrub_duration":2957,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":703594496,"num_objects":10736,"num_object_clones":0,"num_object_copies":32208,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10736,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":68272,"num_write_kb":2528256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1","version":"104'68768","reported_seq":69200,"reported_epoch":104,"state":"active+clean","last_fresh":"2026-02-24T07:57:10.919721+0000","last_change":"2026-02-24T07:56:12.897479+0000","last_active":"2026-02-24T07:57:10.919721+0000","last_peered":"2026-02-24T07:57:10.919721+0000","last_clean":"2026-02-24T07:57:10.919721+0000","last_became_active":"2026-02-24T07:54:42.926442+0000","last_became_peered":"2026-02-24T07:54:42.926442+0000","last_unstale":"2026-02-24T07:57:10.919721+0000","last_undegraded":"2026-02-24T07:57:10.919721+0000","last_fullsized":"2026-02-24T07:57:10.919721+0000","mapping_epoch":90,"log_start":"104'68765","ondisk_log_start":"104'68765","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"96'39245","last_scrub_stamp":"2026-02-24T07:56:12.897436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_clean_scrub_stamp":"2026-02-24T07:56:12.897436+0000","objects_scrubbed":38506,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:57:42.057233+0000","scrub_duration":4279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":703332352,"num_objects":10732,"num_object_clones":0,"num_object_copies":32196,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10732,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":68768,"num_write_kb":2544000,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.0","version":"104'67808","reported_seq":68334,"reported_epoch":104,"state":"active+clean","last_fresh":"2026-02-24T07:57:10.809056+0000","last_change":"2026-02-24T07:57:07.321353+0000","last_active":"2026-02-24T07:57:10.809056+0000","last_peered":"2026-02-24T07:57:10.809056+0000","last_clean":"2026-02-24T07:57:10.809056+0000","last_became_active":"2026-02-24T07:54:42.926910+0000","last_became_peered":"2026-02-24T07:54:42.926910+0000","last_unstale":"2026-02-24T07:57:10.809056+0000","last_undegraded":"2026-02-24T07:57:10.809056+0000","last_fullsized":"2026-02-24T07:57:10.809056+0000","mapping_epoch":90,"log_start":"104'67805","ondisk_log_start":"104'67805","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"103'66056","last_scrub_stamp":"2026-02-24T07:57:07.321288+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_clean_scrub_stamp":"2026-02-24T07:57:07.321288+0000","objects_scrubbed":12515,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:58:17.856526+0000","scrub_duration":1042,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":687865856,"num_objects":10496,"num_object_clones":0,"num_object_copies":31488,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10496,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":67808,"num_write_kb":2505728,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.7","version":"104'68777","reported_seq":69168,"reported_epoch":104,"state":"active+clean","last_fresh":"2026-02-24T07:57:10.795408+0000","last_change":"2026-02-24T07:56:01.088786+0000","last_active":"2026-02-24T07:57:10.795408+0000","last_peered":"2026-02-24T07:57:10.795408+0000","last_clean":"2026-02-24T07:57:10.795408+0000","last_became_active":"2026-02-24T07:54:42.930142+0000","last_became_peered":"2026-02-24T07:54:42.930142+0000","last_unstale":"2026-02-24T07:57:10.795408+0000","last_undegraded":"2026-02-24T07:57:10.795408+0000","last_fullsized":"2026-02-24T07:57:10.795408+0000","mapping_epoch":90,"log_start":"104'68774","ondisk_log_start":"104'68774","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"96'34838","last_scrub_stamp":"2026-02-24T07:56:01.088744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_clean_scrub_stamp":"2026-02-24T07:56:01.088744+0000","objects_scrubbed":34284,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:57:24.696242+0000","scrub_duration":2973,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":710475776,"num_objects":10841,"num_object_clones":0,"num_object_copies":32523,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10841,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":68777,"num_write_kb":2547776,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"29'99","reported_seq":423,"reported_epoch":104,"state":"active+clean","last_fresh":"2026-02-24T07:57:08.930015+0000","last_change":"2026-02-24T07:56:33.056711+0000","last_active":"2026-02-24T07:57:08.930015+0000","last_peered":"2026-02-24T07:57:08.930015+0000","last_clean":"2026-02-24T07:57:08.930015+0000","last_became_active":"2026-02-24T07:42:54.186719+0000","last_became_peered":"2026-02-24T07:42:54.186719+0000","last_unstale":"2026-02-24T07:57:08.930015+0000","last_undegraded":"2026-02-24T07:57:08.930015+0000","last_fullsized":"2026-02-24T07:57:08.930015+0000","mapping_epoch":27,"log_start":"29'96","ondisk_log_start":"29'96","created":20,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'99","last_scrub_stamp":"2026-02-24T07:56:33.056678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:42:43.640776+0000","last_clean_scrub_stamp":"2026-02-24T07:56:33.056678+0000","objects_scrubbed":2,"log_size":3,"log_dups_size":96,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:57:56.381501+0000","scrub_duration":21,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":240,"num_write":135,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":16,"stat_sum":{"num_bytes":11191975960,"num_objects":170777,"num_object_clones":0,"num_object_copies":512331,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":170777,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1094575,"num_write_kb":40491201,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":33581379584,"data_stored":33581367368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":558,"internal_metadata":0},"log_size":48,"ondisk_log_size":48,"up":48,"acting":48,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":240,"num_write":135,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1682,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":3,"up_from":25,"seq":107374182574,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":8696436,"kb_used_data":7002144,"kb_used_omap":25,"kb_used_meta":1694246,"kb_avail":725306764,"statfs":{"total":751619276800,"available":742714126336,"internally_reserved":0,"allocated":7170195456,"data_stored":7169629774,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":26408,"internal_metadata":1734908120},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":19,"seq":81604378800,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":9556472,"kb_used_data":8393828,"kb_used_omap":26,"kb_used_meta":1162597,"kb_avail":724446728,"statfs":{"total":751619276800,"available":741833449472,"internally_reserved":0,"allocated":8595279872,"data_stored":8594710118,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":26884,"internal_metadata":1190500092},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[10,2],"upper_bound":4},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542321,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":11140276,"kb_used_data":9776864,"kb_used_omap":26,"kb_used_meta":1363301,"kb_avail":722862924,"statfs":{"total":751619276800,"available":740211634176,"internally_reserved":0,"allocated":10011508736,"data_stored":10010942534,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":26900,"internal_metadata":1396020972},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[5],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705844,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":9392760,"kb_used_data":8362660,"kb_used_omap":23,"kb_used_meta":1030056,"kb_avail":724610440,"statfs":{"total":751619276800,"available":742001090560,"internally_reserved":0,"allocated":8563363840,"data_stored":8562794086,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":23833,"internal_metadata":1054778087},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1682,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8390709248,"data_stored":8390705176,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":186,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":9770307584,"data_stored":9770303512,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":93,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8392478720,"data_stored":8392474648,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":279,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":7027884032,"data_stored":7027884032,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-24T07:57:13.667 INFO:tasks.cephadm.ceph_manager.ceph:increase pool size by 10 2026-02-24T07:57:13.667 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph osd pool set unique_pool_4 pg_num 96 2026-02-24T07:57:13.681 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:57:13.813 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:57:13.816 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:57:13.813+0000 7fb22e5ab640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:57:13.816 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:57:13.813+0000 7fb22e5ab640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:57:13.833 INFO:teuthology.orchestra.run.trial035.stdout:osd.0: {} 2026-02-24T07:57:13.833 INFO:teuthology.orchestra.run.trial035.stderr:osd.0: osd_enable_op_tracker = '' 2026-02-24T07:57:13.848 INFO:teuthology.orchestra.run.trial035.stdout:osd.1: {} 2026-02-24T07:57:13.848 INFO:teuthology.orchestra.run.trial035.stderr:osd.1: osd_enable_op_tracker = '' 2026-02-24T07:57:13.861 INFO:teuthology.orchestra.run.trial035.stdout:osd.2: {} 2026-02-24T07:57:13.861 INFO:teuthology.orchestra.run.trial035.stderr:osd.2: osd_enable_op_tracker = '' 2026-02-24T07:57:13.874 INFO:teuthology.orchestra.run.trial035.stdout:osd.3: {} 2026-02-24T07:57:13.875 INFO:teuthology.orchestra.run.trial035.stderr:osd.3: osd_enable_op_tracker = '' 2026-02-24T07:57:13.941 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:57:13.939+0000 7fb3f788d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:57:13.941 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:57:13.939+0000 7fb3f788d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:57:14.082 INFO:teuthology.orchestra.run.trial035.stderr:Error EPERM: crimson_allow_pg_split is false; pg_num increase denied for crimson pool 2026-02-24T07:57:14.082 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:57:13 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/2550503402' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2026-02-24T07:57:14.421 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-24T07:57:15.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:57:14 trial035 ceph-mon[39284]: from='client.15637 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2026-02-24T07:57:15.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:57:14 trial035 ceph-mon[39284]: pgmap v505: 17 pgs: 1 active+clean+scrubbing, 16 active+clean; 10 GiB data, 37 GiB used, 2.7 TiB / 2.7 TiB avail; 0 B/s wr, 7.92k op/s 2026-02-24T07:57:15.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:57:14 trial035 ceph-mon[39284]: osdmap e105: 4 total, 4 up, 4 in 2026-02-24T07:57:15.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:57:14 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/2684866460' entity='client.admin' cmd={"prefix": "osd pool set", "pool": "unique_pool_4", "var": "pg_num", "val": "96"} : dispatch 2026-02-24T07:57:15.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:57:14 trial035 ceph-mon[39284]: from='mgr.14284 10.20.193.35:0/3681373222' entity='mgr.x' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2026-02-24T07:57:15.918 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2026-02-24T07:57:16.064 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:57:16.193 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:57:16.191+0000 7f2ba88c0640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:57:16.193 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:57:16.192+0000 7f2ba88c0640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:57:16.208 INFO:teuthology.orchestra.run.trial035.stdout:osd.0: {} 2026-02-24T07:57:16.209 INFO:teuthology.orchestra.run.trial035.stderr:osd.0: osd_enable_op_tracker = '' 2026-02-24T07:57:16.226 INFO:teuthology.orchestra.run.trial035.stdout:osd.1: {} 2026-02-24T07:57:16.226 INFO:teuthology.orchestra.run.trial035.stderr:osd.1: osd_enable_op_tracker = '' 2026-02-24T07:57:16.239 INFO:teuthology.orchestra.run.trial035.stdout:osd.2: {} 2026-02-24T07:57:16.239 INFO:teuthology.orchestra.run.trial035.stderr:osd.2: osd_enable_op_tracker = '' 2026-02-24T07:57:16.255 INFO:teuthology.orchestra.run.trial035.stdout:osd.3: {} 2026-02-24T07:57:16.255 INFO:teuthology.orchestra.run.trial035.stderr:osd.3: osd_enable_op_tracker = '' 2026-02-24T07:57:17.204 INFO:journalctl@ceph.mgr.x.trial035.stdout:Feb 24 07:57:16 trial035 ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-mgr-x[39621]: ::ffff:10.20.193.91 - - [24/Feb/2026:07:57:16] "GET /metrics HTTP/1.1" 200 4235 "" "Prometheus/3.6.0" 2026-02-24T07:57:17.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:57:16 trial035 ceph-mon[39284]: pgmap v507: 17 pgs: 1 active+clean+scrubbing, 16 active+clean; 10 GiB data, 37 GiB used, 2.7 TiB / 2.7 TiB avail; 0 B/s wr, 4.89k op/s 2026-02-24T07:57:17.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:57:16 trial035 ceph-mon[39284]: 6.3 scrub starts 2026-02-24T07:57:17.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:57:16 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/1673905120' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2026-02-24T07:57:17.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:57:16 trial035 ceph-mon[39284]: 6.8 scrub starts 2026-02-24T07:57:17.265 INFO:journalctl@ceph.prometheus.a.trial091.stdout:Feb 24 07:57:16 trial091 ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-prometheus-a[66007]: time=2026-02-24T07:57:16.973Z level=ERROR source=scrape.go:1630 msg="Failed to determine correct type of scrape target." component="scrape manager" scrape_pool=ceph target=http://trial035:9283/metrics content_type="text/html;charset=utf-8" fallback_media_type="" err="received unsupported Content-Type \"text/html;charset=utf-8\" and no fallback_scrape_protocol specified for target" 2026-02-24T07:57:18.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:57:17 trial035 ceph-mon[39284]: purged_snaps scrub starts 2026-02-24T07:57:18.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:57:17 trial035 ceph-mon[39284]: purged_snaps scrub ok 2026-02-24T07:57:18.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:57:17 trial035 ceph-mon[39284]: 6.3 scrub ok 2026-02-24T07:57:18.205 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:57:17 trial035 ceph-mon[39284]: 6.8 scrub ok 2026-02-24T07:57:18.205 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:57:17 trial035 ceph-mon[39284]: 6.e scrub starts 2026-02-24T07:57:18.298 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2026-02-24T07:57:18.437 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:57:18.562 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:57:18.561+0000 7f0967e48640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:57:18.563 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:57:18.562+0000 7f0967e48640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:57:18.580 INFO:teuthology.orchestra.run.trial035.stdout:osd.0: {} 2026-02-24T07:57:18.580 INFO:teuthology.orchestra.run.trial035.stderr:osd.0: osd_enable_op_tracker = '' 2026-02-24T07:57:18.597 INFO:teuthology.orchestra.run.trial035.stdout:osd.1: {} 2026-02-24T07:57:18.597 INFO:teuthology.orchestra.run.trial035.stderr:osd.1: osd_enable_op_tracker = '' 2026-02-24T07:57:18.614 INFO:teuthology.orchestra.run.trial035.stdout:osd.2: {} 2026-02-24T07:57:18.614 INFO:teuthology.orchestra.run.trial035.stderr:osd.2: osd_enable_op_tracker = '' 2026-02-24T07:57:18.628 INFO:teuthology.orchestra.run.trial035.stdout:osd.3: {} 2026-02-24T07:57:18.628 INFO:teuthology.orchestra.run.trial035.stderr:osd.3: osd_enable_op_tracker = '' 2026-02-24T07:57:19.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:57:18 trial035 ceph-mon[39284]: pgmap v508: 17 pgs: 17 active+clean; 7.8 GiB data, 30 GiB used, 2.7 TiB / 2.7 TiB avail; 0 B/s wr, 9.19k op/s 2026-02-24T07:57:19.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:57:18 trial035 ceph-mon[39284]: 6.e scrub ok 2026-02-24T07:57:19.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:57:18 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/1634337539' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2026-02-24T07:57:19.421 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2026-02-24T07:57:19.422 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2026-02-24T07:57:19.422 INFO:tasks.thrashosds.thrasher:Growing pool unique_pool_4 2026-02-24T07:57:19.422 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph pg dump --format=json 2026-02-24T07:57:19.573 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:57:19.707 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:57:19.706+0000 7fa0b90b2640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:57:19.708 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:57:19.706+0000 7fa0b90b2640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:57:19.860 INFO:teuthology.orchestra.run.trial035.stdout: 2026-02-24T07:57:19.860 INFO:teuthology.orchestra.run.trial035.stderr:dumped all 2026-02-24T07:57:20.109 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:57:19 trial035 ceph-mon[39284]: pgmap v509: 17 pgs: 17 active+clean; 7.8 GiB data, 30 GiB used, 2.7 TiB / 2.7 TiB avail; 0 B/s wr, 8.13k op/s 2026-02-24T07:57:20.109 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:57:19 trial035 ceph-mon[39284]: from='client.15647 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2026-02-24T07:57:20.543 INFO:teuthology.orchestra.run.trial035.stdout:{"pg_ready":true,"pg_map":{"version":509,"stamp":"2026-02-24T07:57:19.774568+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":8425701944,"num_objects":128560,"num_object_clones":0,"num_object_copies":385680,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":128560,"num_whiteouts":0,"num_read":137,"num_read_kb":241,"num_write":1136929,"num_write_kb":40493351,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":51,"ondisk_log_size":51,"up":51,"acting":51,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":51,"num_osds":4,"num_per_pool_osds":4,"num_per_pool_omap_osds":4,"kb":2936012800,"kb_used":31184136,"kb_used_data":25716280,"kb_used_omap":103,"kb_used_meta":5467352,"kb_avail":2904828664,"statfs":{"total":3006477107200,"available":2974544551936,"internally_reserved":0,"allocated":26333470720,"data_stored":26331167780,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":106078,"internal_metadata":5598568866},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,0,0,0,5],"upper_bound":64},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-5330239488,"num_objects":-81333,"num_object_clones":0,"num_object_copies":-243999,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-81333,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":81333,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.001403"},"pg_stats":[{"pgid":"6.f","version":"105'71054","reported_seq":71482,"reported_epoch":105,"state":"active+clean","last_fresh":"2026-02-24T07:57:15.925297+0000","last_change":"2026-02-24T07:56:12.247365+0000","last_active":"2026-02-24T07:57:15.925297+0000","last_peered":"2026-02-24T07:57:15.925297+0000","last_clean":"2026-02-24T07:57:15.925297+0000","last_became_active":"2026-02-24T07:54:42.927080+0000","last_became_peered":"2026-02-24T07:54:42.927080+0000","last_unstale":"2026-02-24T07:57:15.925297+0000","last_undegraded":"2026-02-24T07:57:15.925297+0000","last_fullsized":"2026-02-24T07:57:15.925297+0000","mapping_epoch":90,"log_start":"105'71051","ondisk_log_start":"105'71051","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"96'38780","last_scrub_stamp":"2026-02-24T07:56:12.247318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_clean_scrub_stamp":"2026-02-24T07:56:12.247318+0000","objects_scrubbed":37979,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:57:38.474117+0000","scrub_duration":4670,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":521666560,"num_objects":7960,"num_object_clones":0,"num_object_copies":23880,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7960,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":71054,"num_write_kb":2528448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.e","version":"105'71135","reported_seq":71539,"reported_epoch":105,"state":"active+clean","last_fresh":"2026-02-24T07:57:15.784308+0000","last_change":"2026-02-24T07:56:05.374565+0000","last_active":"2026-02-24T07:57:15.784308+0000","last_peered":"2026-02-24T07:57:15.784308+0000","last_clean":"2026-02-24T07:57:15.784308+0000","last_became_active":"2026-02-24T07:54:42.928748+0000","last_became_peered":"2026-02-24T07:54:42.928748+0000","last_unstale":"2026-02-24T07:57:15.784308+0000","last_undegraded":"2026-02-24T07:57:15.784308+0000","last_fullsized":"2026-02-24T07:57:15.784308+0000","mapping_epoch":90,"log_start":"105'71132","ondisk_log_start":"105'71132","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"96'36307","last_scrub_stamp":"2026-02-24T07:56:05.374510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_clean_scrub_stamp":"2026-02-24T07:56:05.374510+0000","objects_scrubbed":35491,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:57:16.427813+0000","scrub_duration":4036,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":523173888,"num_objects":7983,"num_object_clones":0,"num_object_copies":23949,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7983,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":71135,"num_write_kb":2531776,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.d","version":"105'70976","reported_seq":71452,"reported_epoch":105,"state":"active+clean","last_fresh":"2026-02-24T07:57:15.784940+0000","last_change":"2026-02-24T07:57:06.430234+0000","last_active":"2026-02-24T07:57:15.784940+0000","last_peered":"2026-02-24T07:57:15.784940+0000","last_clean":"2026-02-24T07:57:15.784940+0000","last_became_active":"2026-02-24T07:54:42.926886+0000","last_became_peered":"2026-02-24T07:54:42.926886+0000","last_unstale":"2026-02-24T07:57:15.784940+0000","last_undegraded":"2026-02-24T07:57:15.784940+0000","last_fullsized":"2026-02-24T07:57:15.784940+0000","mapping_epoch":90,"log_start":"105'70973","ondisk_log_start":"105'70973","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"103'66256","last_scrub_stamp":"2026-02-24T07:57:06.430163+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_clean_scrub_stamp":"2026-02-24T07:57:06.430163+0000","objects_scrubbed":13086,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:58:07.786827+0000","scrub_duration":1340,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":533397528,"num_objects":8140,"num_object_clones":0,"num_object_copies":24420,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8140,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":70976,"num_write_kb":2531649,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.c","version":"105'71243","reported_seq":71647,"reported_epoch":105,"state":"active+clean","last_fresh":"2026-02-24T07:57:15.921365+0000","last_change":"2026-02-24T07:56:25.008652+0000","last_active":"2026-02-24T07:57:15.921365+0000","last_peered":"2026-02-24T07:57:15.921365+0000","last_clean":"2026-02-24T07:57:15.921365+0000","last_became_active":"2026-02-24T07:54:42.928238+0000","last_became_peered":"2026-02-24T07:54:42.928238+0000","last_unstale":"2026-02-24T07:57:15.921365+0000","last_undegraded":"2026-02-24T07:57:15.921365+0000","last_fullsized":"2026-02-24T07:57:15.921365+0000","mapping_epoch":90,"log_start":"105'71240","ondisk_log_start":"105'71240","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"96'44918","last_scrub_stamp":"2026-02-24T07:56:25.008567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_clean_scrub_stamp":"2026-02-24T07:56:25.008567+0000","objects_scrubbed":35276,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:57:40.096151+0000","scrub_duration":4395,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":519372800,"num_objects":7925,"num_object_clones":0,"num_object_copies":23775,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7925,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":71243,"num_write_kb":2533376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.b","version":"105'70461","reported_seq":70897,"reported_epoch":105,"state":"active+clean","last_fresh":"2026-02-24T07:57:15.921315+0000","last_change":"2026-02-24T07:56:16.120475+0000","last_active":"2026-02-24T07:57:15.921315+0000","last_peered":"2026-02-24T07:57:15.921315+0000","last_clean":"2026-02-24T07:57:15.921315+0000","last_became_active":"2026-02-24T07:54:42.926965+0000","last_became_peered":"2026-02-24T07:54:42.926965+0000","last_unstale":"2026-02-24T07:57:15.921315+0000","last_undegraded":"2026-02-24T07:57:15.921315+0000","last_fullsized":"2026-02-24T07:57:15.921315+0000","mapping_epoch":90,"log_start":"105'70458","ondisk_log_start":"105'70458","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"96'39969","last_scrub_stamp":"2026-02-24T07:56:16.120290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_clean_scrub_stamp":"2026-02-24T07:56:16.120290+0000","objects_scrubbed":38739,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:57:36.957012+0000","scrub_duration":4491,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":518062080,"num_objects":7905,"num_object_clones":0,"num_object_copies":23715,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7905,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":70461,"num_write_kb":2507712,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.a","version":"105'70864","reported_seq":71366,"reported_epoch":105,"state":"active+clean","last_fresh":"2026-02-24T07:57:15.785713+0000","last_change":"2026-02-24T07:57:02.035437+0000","last_active":"2026-02-24T07:57:15.785713+0000","last_peered":"2026-02-24T07:57:15.785713+0000","last_clean":"2026-02-24T07:57:15.785713+0000","last_became_active":"2026-02-24T07:54:42.926477+0000","last_became_peered":"2026-02-24T07:54:42.926477+0000","last_unstale":"2026-02-24T07:57:15.785713+0000","last_undegraded":"2026-02-24T07:57:15.785713+0000","last_fullsized":"2026-02-24T07:57:15.785713+0000","mapping_epoch":90,"log_start":"105'70861","ondisk_log_start":"105'70861","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"102'63835","last_scrub_stamp":"2026-02-24T07:57:02.035399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_clean_scrub_stamp":"2026-02-24T07:57:02.035399+0000","objects_scrubbed":15563,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:58:02.340758+0000","scrub_duration":1912,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":536084480,"num_objects":8180,"num_object_clones":0,"num_object_copies":24540,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8180,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":70864,"num_write_kb":2529408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"105'71227","reported_seq":71749,"reported_epoch":105,"state":"active+clean","last_fresh":"2026-02-24T07:57:15.840350+0000","last_change":"2026-02-24T07:56:57.764926+0000","last_active":"2026-02-24T07:57:15.840350+0000","last_peered":"2026-02-24T07:57:15.840350+0000","last_clean":"2026-02-24T07:57:15.840350+0000","last_became_active":"2026-02-24T07:54:42.926136+0000","last_became_peered":"2026-02-24T07:54:42.926136+0000","last_unstale":"2026-02-24T07:57:15.840350+0000","last_undegraded":"2026-02-24T07:57:15.840350+0000","last_fullsized":"2026-02-24T07:57:15.840350+0000","mapping_epoch":90,"log_start":"105'71224","ondisk_log_start":"105'71224","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"101'62265","last_scrub_stamp":"2026-02-24T07:56:57.764859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_clean_scrub_stamp":"2026-02-24T07:56:57.764859+0000","objects_scrubbed":17443,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:58:20.952507+0000","scrub_duration":1544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":533528576,"num_objects":8141,"num_object_clones":0,"num_object_copies":24423,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8141,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":71227,"num_write_kb":2539776,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,1],"acting":[0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.8","version":"105'71358","reported_seq":71736,"reported_epoch":105,"state":"active+clean","last_fresh":"2026-02-24T07:57:15.800004+0000","last_change":"2026-02-24T07:55:58.178460+0000","last_active":"2026-02-24T07:57:15.800004+0000","last_peered":"2026-02-24T07:57:15.800004+0000","last_clean":"2026-02-24T07:57:15.800004+0000","last_became_active":"2026-02-24T07:54:42.929078+0000","last_became_peered":"2026-02-24T07:54:42.929078+0000","last_unstale":"2026-02-24T07:57:15.800004+0000","last_undegraded":"2026-02-24T07:57:15.800004+0000","last_fullsized":"2026-02-24T07:57:15.800004+0000","mapping_epoch":90,"log_start":"105'71355","ondisk_log_start":"105'71355","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"96'33550","last_scrub_stamp":"2026-02-24T07:55:58.178395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_clean_scrub_stamp":"2026-02-24T07:55:58.178395+0000","objects_scrubbed":32948,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"queued for scrub","scrub_duration":2981,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":532152320,"num_objects":8120,"num_object_clones":0,"num_object_copies":24360,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8120,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":71358,"num_write_kb":2543296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.6","version":"105'70904","reported_seq":71335,"reported_epoch":105,"state":"active+clean","last_fresh":"2026-02-24T07:57:15.920690+0000","last_change":"2026-02-24T07:56:18.901177+0000","last_active":"2026-02-24T07:57:15.920690+0000","last_peered":"2026-02-24T07:57:15.920690+0000","last_clean":"2026-02-24T07:57:15.920690+0000","last_became_active":"2026-02-24T07:54:42.926045+0000","last_became_peered":"2026-02-24T07:54:42.926045+0000","last_unstale":"2026-02-24T07:57:15.920690+0000","last_undegraded":"2026-02-24T07:57:15.920690+0000","last_fullsized":"2026-02-24T07:57:15.920690+0000","mapping_epoch":90,"log_start":"105'70901","ondisk_log_start":"105'70901","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"96'41769","last_scrub_stamp":"2026-02-24T07:56:18.901111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_clean_scrub_stamp":"2026-02-24T07:56:18.901111+0000","objects_scrubbed":38122,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:57:27.205697+0000","scrub_duration":4228,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":524681216,"num_objects":8006,"num_object_clones":0,"num_object_copies":24018,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8006,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":70904,"num_write_kb":2525120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.5","version":"105'70930","reported_seq":71430,"reported_epoch":105,"state":"active+clean","last_fresh":"2026-02-24T07:57:15.782108+0000","last_change":"2026-02-24T07:57:11.626730+0000","last_active":"2026-02-24T07:57:15.782108+0000","last_peered":"2026-02-24T07:57:15.782108+0000","last_clean":"2026-02-24T07:57:15.782108+0000","last_became_active":"2026-02-24T07:54:42.927535+0000","last_became_peered":"2026-02-24T07:54:42.927535+0000","last_unstale":"2026-02-24T07:57:15.782108+0000","last_undegraded":"2026-02-24T07:57:15.782108+0000","last_fullsized":"2026-02-24T07:57:15.782108+0000","mapping_epoch":90,"log_start":"105'70927","ondisk_log_start":"105'70927","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"104'68672","last_scrub_stamp":"2026-02-24T07:57:11.626709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_clean_scrub_stamp":"2026-02-24T07:57:11.626709+0000","objects_scrubbed":10605,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:58:21.653444+0000","scrub_duration":1515,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":528482304,"num_objects":8064,"num_object_clones":0,"num_object_copies":24192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8064,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":70930,"num_write_kb":2527808,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.4","version":"105'71102","reported_seq":71576,"reported_epoch":105,"state":"active+clean","last_fresh":"2026-02-24T07:57:15.919669+0000","last_change":"2026-02-24T07:57:06.901398+0000","last_active":"2026-02-24T07:57:15.919669+0000","last_peered":"2026-02-24T07:57:15.919669+0000","last_clean":"2026-02-24T07:57:15.919669+0000","last_became_active":"2026-02-24T07:54:42.928011+0000","last_became_peered":"2026-02-24T07:54:42.928011+0000","last_unstale":"2026-02-24T07:57:15.919669+0000","last_undegraded":"2026-02-24T07:57:15.919669+0000","last_fullsized":"2026-02-24T07:57:15.919669+0000","mapping_epoch":90,"log_start":"105'71099","ondisk_log_start":"105'71099","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"103'66457","last_scrub_stamp":"2026-02-24T07:57:06.901316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_clean_scrub_stamp":"2026-02-24T07:57:06.901316+0000","objects_scrubbed":12783,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:58:16.895385+0000","scrub_duration":1738,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":510656512,"num_objects":7792,"num_object_clones":0,"num_object_copies":23376,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7792,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":71102,"num_write_kb":2524608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.3","version":"105'71357","reported_seq":71767,"reported_epoch":105,"state":"active+clean","last_fresh":"2026-02-24T07:57:15.784297+0000","last_change":"2026-02-24T07:56:06.184345+0000","last_active":"2026-02-24T07:57:15.784297+0000","last_peered":"2026-02-24T07:57:15.784297+0000","last_clean":"2026-02-24T07:57:15.784297+0000","last_became_active":"2026-02-24T07:54:42.927730+0000","last_became_peered":"2026-02-24T07:54:42.927730+0000","last_unstale":"2026-02-24T07:57:15.784297+0000","last_undegraded":"2026-02-24T07:57:15.784297+0000","last_fullsized":"2026-02-24T07:57:15.784297+0000","mapping_epoch":90,"log_start":"105'71354","ondisk_log_start":"105'71354","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"96'36664","last_scrub_stamp":"2026-02-24T07:56:06.184309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_clean_scrub_stamp":"2026-02-24T07:56:06.184309+0000","objects_scrubbed":35946,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"queued for scrub","scrub_duration":3811,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":530513920,"num_objects":8095,"num_object_clones":0,"num_object_copies":24285,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8095,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":71357,"num_write_kb":2542464,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.2","version":"105'70971","reported_seq":71359,"reported_epoch":105,"state":"active+clean","last_fresh":"2026-02-24T07:57:15.798284+0000","last_change":"2026-02-24T07:56:00.120637+0000","last_active":"2026-02-24T07:57:15.798284+0000","last_peered":"2026-02-24T07:57:15.798284+0000","last_clean":"2026-02-24T07:57:15.798284+0000","last_became_active":"2026-02-24T07:54:42.930247+0000","last_became_peered":"2026-02-24T07:54:42.930247+0000","last_unstale":"2026-02-24T07:57:15.798284+0000","last_undegraded":"2026-02-24T07:57:15.798284+0000","last_fullsized":"2026-02-24T07:57:15.798284+0000","mapping_epoch":90,"log_start":"105'70968","ondisk_log_start":"105'70968","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"96'34210","last_scrub_stamp":"2026-02-24T07:56:00.120605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_clean_scrub_stamp":"2026-02-24T07:56:00.120605+0000","objects_scrubbed":33656,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:57:29.329949+0000","scrub_duration":2957,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":526712832,"num_objects":8037,"num_object_clones":0,"num_object_copies":24111,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8037,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":70971,"num_write_kb":2528256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1","version":"105'71383","reported_seq":71816,"reported_epoch":105,"state":"active+clean","last_fresh":"2026-02-24T07:57:15.924840+0000","last_change":"2026-02-24T07:56:12.897479+0000","last_active":"2026-02-24T07:57:15.924840+0000","last_peered":"2026-02-24T07:57:15.924840+0000","last_clean":"2026-02-24T07:57:15.924840+0000","last_became_active":"2026-02-24T07:54:42.926442+0000","last_became_peered":"2026-02-24T07:54:42.926442+0000","last_unstale":"2026-02-24T07:57:15.924840+0000","last_undegraded":"2026-02-24T07:57:15.924840+0000","last_fullsized":"2026-02-24T07:57:15.924840+0000","mapping_epoch":90,"log_start":"105'71380","ondisk_log_start":"105'71380","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"96'39245","last_scrub_stamp":"2026-02-24T07:56:12.897436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_clean_scrub_stamp":"2026-02-24T07:56:12.897436+0000","objects_scrubbed":38506,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:57:42.057233+0000","scrub_duration":4279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":531955712,"num_objects":8117,"num_object_clones":0,"num_object_copies":24351,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8117,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":71383,"num_write_kb":2544000,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.0","version":"105'70393","reported_seq":70921,"reported_epoch":105,"state":"active+clean","last_fresh":"2026-02-24T07:57:15.841801+0000","last_change":"2026-02-24T07:57:07.321353+0000","last_active":"2026-02-24T07:57:15.841801+0000","last_peered":"2026-02-24T07:57:15.841801+0000","last_clean":"2026-02-24T07:57:15.841801+0000","last_became_active":"2026-02-24T07:54:42.926910+0000","last_became_peered":"2026-02-24T07:54:42.926910+0000","last_unstale":"2026-02-24T07:57:15.841801+0000","last_undegraded":"2026-02-24T07:57:15.841801+0000","last_fullsized":"2026-02-24T07:57:15.841801+0000","mapping_epoch":90,"log_start":"105'70390","ondisk_log_start":"105'70390","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"103'66056","last_scrub_stamp":"2026-02-24T07:57:07.321288+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_clean_scrub_stamp":"2026-02-24T07:57:07.321288+0000","objects_scrubbed":12515,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:58:17.856526+0000","scrub_duration":1042,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":518455296,"num_objects":7911,"num_object_clones":0,"num_object_copies":23733,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7911,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":70393,"num_write_kb":2505728,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.7","version":"105'71436","reported_seq":71829,"reported_epoch":105,"state":"active+clean","last_fresh":"2026-02-24T07:57:15.800103+0000","last_change":"2026-02-24T07:56:01.088786+0000","last_active":"2026-02-24T07:57:15.800103+0000","last_peered":"2026-02-24T07:57:15.800103+0000","last_clean":"2026-02-24T07:57:15.800103+0000","last_became_active":"2026-02-24T07:54:42.930142+0000","last_became_peered":"2026-02-24T07:54:42.930142+0000","last_unstale":"2026-02-24T07:57:15.800103+0000","last_undegraded":"2026-02-24T07:57:15.800103+0000","last_fullsized":"2026-02-24T07:57:15.800103+0000","mapping_epoch":90,"log_start":"105'71433","ondisk_log_start":"105'71433","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"96'34838","last_scrub_stamp":"2026-02-24T07:56:01.088744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_clean_scrub_stamp":"2026-02-24T07:56:01.088744+0000","objects_scrubbed":34284,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:57:24.696242+0000","scrub_duration":2973,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":536215552,"num_objects":8182,"num_object_clones":0,"num_object_copies":24546,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8182,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":71436,"num_write_kb":2547776,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"29'99","reported_seq":425,"reported_epoch":105,"state":"active+clean","last_fresh":"2026-02-24T07:57:13.930053+0000","last_change":"2026-02-24T07:56:33.056711+0000","last_active":"2026-02-24T07:57:13.930053+0000","last_peered":"2026-02-24T07:57:13.930053+0000","last_clean":"2026-02-24T07:57:13.930053+0000","last_became_active":"2026-02-24T07:42:54.186719+0000","last_became_peered":"2026-02-24T07:42:54.186719+0000","last_unstale":"2026-02-24T07:57:13.930053+0000","last_undegraded":"2026-02-24T07:57:13.930053+0000","last_fullsized":"2026-02-24T07:57:13.930053+0000","mapping_epoch":27,"log_start":"29'96","ondisk_log_start":"29'96","created":20,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'99","last_scrub_stamp":"2026-02-24T07:56:33.056678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:42:43.640776+0000","last_clean_scrub_stamp":"2026-02-24T07:56:33.056678+0000","objects_scrubbed":2,"log_size":3,"log_dups_size":96,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:57:56.381501+0000","scrub_duration":21,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":240,"num_write":135,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":16,"stat_sum":{"num_bytes":8425111576,"num_objects":128558,"num_object_clones":0,"num_object_copies":385674,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":128558,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1136794,"num_write_kb":40491201,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":25282424832,"data_stored":25282412616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":558,"internal_metadata":0},"log_size":48,"ondisk_log_size":48,"up":48,"acting":48,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":240,"num_write":135,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1682,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":3,"up_from":25,"seq":107374182575,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":7115392,"kb_used_data":5366956,"kb_used_omap":26,"kb_used_meta":1748389,"kb_avail":726887808,"statfs":{"total":751619276800,"available":744333115392,"internally_reserved":0,"allocated":5495762944,"data_stored":5495189375,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":27064,"internal_metadata":1790350920},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":19,"seq":81604378801,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":7680836,"kb_used_data":6463984,"kb_used_omap":26,"kb_used_meta":1216741,"kb_avail":726322364,"statfs":{"total":751619276800,"available":743754100736,"internally_reserved":0,"allocated":6619119616,"data_stored":6618541975,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":27629,"internal_metadata":1245942803},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542322,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":8914880,"kb_used_data":7497196,"kb_used_omap":26,"kb_used_meta":1417445,"kb_avail":725088320,"statfs":{"total":751619276800,"available":742490439680,"internally_reserved":0,"allocated":7677128704,"data_stored":7676554615,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":26900,"internal_metadata":1451464428},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,5],"upper_bound":64},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705845,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":7473028,"kb_used_data":6388144,"kb_used_omap":23,"kb_used_meta":1084776,"kb_avail":726530172,"statfs":{"total":751619276800,"available":743966896128,"internally_reserved":0,"allocated":6541459456,"data_stored":6540881815,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":24485,"internal_metadata":1110810715},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1682,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":6316101632,"data_stored":6316097560,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":186,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":7342788608,"data_stored":7342784536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":93,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":6322655232,"data_stored":6322651160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":279,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":5300879360,"data_stored":5300879360,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-24T07:57:20.545 INFO:tasks.cephadm.ceph_manager.ceph:increase pool size by 10 2026-02-24T07:57:20.546 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph osd pool set unique_pool_4 pg_num 106 2026-02-24T07:57:20.673 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2026-02-24T07:57:20.687 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:57:21.168 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:57:21.154+0000 7f3ce361d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:57:21.169 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:57:21.154+0000 7f3ce361d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:57:21.245 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:57:21.307 INFO:teuthology.orchestra.run.trial035.stderr:Error EPERM: crimson_allow_pg_split is false; pg_num increase denied for crimson pool 2026-02-24T07:57:21.700 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:57:21 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/2366242766' entity='client.admin' cmd={"prefix": "osd pool set", "pool": "unique_pool_4", "var": "pg_num", "val": "106"} : dispatch 2026-02-24T07:57:22.063 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-24T07:57:22.241 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:57:22.219+0000 7f32dcdd6640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:57:22.241 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:57:22.219+0000 7f32dcdd6640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:57:22.257 INFO:teuthology.orchestra.run.trial035.stdout:osd.0: {} 2026-02-24T07:57:22.257 INFO:teuthology.orchestra.run.trial035.stderr:osd.0: osd_enable_op_tracker = '' 2026-02-24T07:57:22.272 INFO:teuthology.orchestra.run.trial035.stdout:osd.1: {} 2026-02-24T07:57:22.272 INFO:teuthology.orchestra.run.trial035.stderr:osd.1: osd_enable_op_tracker = '' 2026-02-24T07:57:22.286 INFO:teuthology.orchestra.run.trial035.stdout:osd.2: {} 2026-02-24T07:57:22.286 INFO:teuthology.orchestra.run.trial035.stderr:osd.2: osd_enable_op_tracker = '' 2026-02-24T07:57:22.299 INFO:teuthology.orchestra.run.trial035.stdout:osd.3: {} 2026-02-24T07:57:22.299 INFO:teuthology.orchestra.run.trial035.stderr:osd.3: osd_enable_op_tracker = '' 2026-02-24T07:57:22.645 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:57:22 trial035 ceph-mon[39284]: pgmap v510: 17 pgs: 17 active+clean; 5.4 GiB data, 22 GiB used, 2.7 TiB / 2.7 TiB avail; 0 B/s wr, 8.23k op/s 2026-02-24T07:57:22.645 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:57:22 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/1596348265' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2026-02-24T07:57:23.704 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:57:23 trial035 ceph-mon[39284]: purged_snaps scrub starts 2026-02-24T07:57:23.705 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:57:23 trial035 ceph-mon[39284]: purged_snaps scrub ok 2026-02-24T07:57:24.985 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2026-02-24T07:57:25.121 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:57:25.184 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:57:24 trial035 ceph-mon[39284]: pgmap v511: 17 pgs: 17 active+clean; 5.4 GiB data, 22 GiB used, 2.7 TiB / 2.7 TiB avail; 0 B/s wr, 8.23k op/s 2026-02-24T07:57:25.246 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:57:25.244+0000 7f90d0e25640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:57:25.246 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:57:25.245+0000 7f90d0e25640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:57:25.264 INFO:teuthology.orchestra.run.trial035.stdout:osd.0: {} 2026-02-24T07:57:25.265 INFO:teuthology.orchestra.run.trial035.stderr:osd.0: osd_enable_op_tracker = '' 2026-02-24T07:57:25.278 INFO:teuthology.orchestra.run.trial035.stdout:osd.1: {} 2026-02-24T07:57:25.278 INFO:teuthology.orchestra.run.trial035.stderr:osd.1: osd_enable_op_tracker = '' 2026-02-24T07:57:25.292 INFO:teuthology.orchestra.run.trial035.stdout:osd.2: {} 2026-02-24T07:57:25.292 INFO:teuthology.orchestra.run.trial035.stderr:osd.2: osd_enable_op_tracker = '' 2026-02-24T07:57:25.308 INFO:teuthology.orchestra.run.trial035.stdout:osd.3: {} 2026-02-24T07:57:25.308 INFO:teuthology.orchestra.run.trial035.stderr:osd.3: osd_enable_op_tracker = '' 2026-02-24T07:57:26.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:57:25 trial035 ceph-mon[39284]: 6.7 scrub starts 2026-02-24T07:57:26.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:57:25 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/3520317350' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2026-02-24T07:57:27.064 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2026-02-24T07:57:27.065 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2026-02-24T07:57:27.065 INFO:tasks.thrashosds.thrasher:fixing pg num pool unique_pool_4 2026-02-24T07:57:27.065 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph pg dump --format=json 2026-02-24T07:57:27.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:57:26 trial035 ceph-mon[39284]: 6.7 scrub ok 2026-02-24T07:57:27.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:57:26 trial035 ceph-mon[39284]: pgmap v512: 17 pgs: 17 active+clean; 5.4 GiB data, 22 GiB used, 2.7 TiB / 2.7 TiB avail; 0 B/s wr, 6.95k op/s 2026-02-24T07:57:27.205 INFO:journalctl@ceph.mgr.x.trial035.stdout:Feb 24 07:57:26 trial035 ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-mgr-x[39621]: ::ffff:10.20.193.91 - - [24/Feb/2026:07:57:26] "GET /metrics HTTP/1.1" 200 4235 "" "Prometheus/3.6.0" 2026-02-24T07:57:27.205 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:57:27.349 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2026-02-24T07:57:27.356 INFO:journalctl@ceph.prometheus.a.trial091.stdout:Feb 24 07:57:26 trial091 ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-prometheus-a[66007]: time=2026-02-24T07:57:26.973Z level=ERROR source=scrape.go:1630 msg="Failed to determine correct type of scrape target." component="scrape manager" scrape_pool=ceph target=http://trial035:9283/metrics content_type="text/html;charset=utf-8" fallback_media_type="" err="received unsupported Content-Type \"text/html;charset=utf-8\" and no fallback_scrape_protocol specified for target" 2026-02-24T07:57:27.482 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:57:27.569 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:57:27.529+0000 7f67d3d50640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:57:27.570 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:57:27.529+0000 7f67d3d50640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:57:27.668 INFO:teuthology.orchestra.run.trial035.stdout: 2026-02-24T07:57:27.668 INFO:teuthology.orchestra.run.trial035.stderr:dumped all 2026-02-24T07:57:27.972 INFO:teuthology.orchestra.run.trial035.stdout:{"pg_ready":true,"pg_map":{"version":512,"stamp":"2026-02-24T07:57:25.775346+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":5796987448,"num_objects":88449,"num_object_clones":0,"num_object_copies":265347,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":88449,"num_whiteouts":0,"num_read":137,"num_read_kb":241,"num_write":1177040,"num_write_kb":40493351,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":51,"ondisk_log_size":51,"up":51,"acting":51,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":51,"num_osds":4,"num_per_pool_osds":4,"num_per_pool_omap_osds":4,"kb":2936012800,"kb_used":23470216,"kb_used_data":17786616,"kb_used_omap":104,"kb_used_meta":5683415,"kb_avail":2912542584,"statfs":{"total":3006477107200,"available":2982443606016,"internally_reserved":0,"allocated":18213494784,"data_stored":18211191844,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":106996,"internal_metadata":5819817484},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,0,0,10],"upper_bound":32},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-5395578880,"num_objects":-82330,"num_object_clones":0,"num_object_copies":-246990,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-82330,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":82330,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"11.845809"},"pg_stats":[{"pgid":"6.f","version":"105'73536","reported_seq":73965,"reported_epoch":105,"state":"active+clean","last_fresh":"2026-02-24T07:57:20.907517+0000","last_change":"2026-02-24T07:56:12.247365+0000","last_active":"2026-02-24T07:57:20.907517+0000","last_peered":"2026-02-24T07:57:20.907517+0000","last_clean":"2026-02-24T07:57:20.907517+0000","last_became_active":"2026-02-24T07:54:42.927080+0000","last_became_peered":"2026-02-24T07:54:42.927080+0000","last_unstale":"2026-02-24T07:57:20.907517+0000","last_undegraded":"2026-02-24T07:57:20.907517+0000","last_fullsized":"2026-02-24T07:57:20.907517+0000","mapping_epoch":90,"log_start":"105'73533","ondisk_log_start":"105'73533","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"96'38780","last_scrub_stamp":"2026-02-24T07:56:12.247318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_clean_scrub_stamp":"2026-02-24T07:56:12.247318+0000","objects_scrubbed":37979,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:57:38.474117+0000","scrub_duration":4670,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":359006208,"num_objects":5478,"num_object_clones":0,"num_object_copies":16434,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5478,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":73536,"num_write_kb":2528448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.e","version":"105'73628","reported_seq":74113,"reported_epoch":105,"state":"active+clean","last_fresh":"2026-02-24T07:57:20.783043+0000","last_change":"2026-02-24T07:57:17.985582+0000","last_active":"2026-02-24T07:57:20.783043+0000","last_peered":"2026-02-24T07:57:20.783043+0000","last_clean":"2026-02-24T07:57:20.783043+0000","last_became_active":"2026-02-24T07:54:42.928748+0000","last_became_peered":"2026-02-24T07:54:42.928748+0000","last_unstale":"2026-02-24T07:57:20.783043+0000","last_undegraded":"2026-02-24T07:57:20.783043+0000","last_fullsized":"2026-02-24T07:57:20.783043+0000","mapping_epoch":90,"log_start":"105'73625","ondisk_log_start":"105'73625","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"105'72242","last_scrub_stamp":"2026-02-24T07:57:17.985529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_clean_scrub_stamp":"2026-02-24T07:57:17.985529+0000","objects_scrubbed":7103,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:58:29.501859+0000","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":359792640,"num_objects":5490,"num_object_clones":0,"num_object_copies":16470,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5490,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":73628,"num_write_kb":2531776,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.d","version":"105'73534","reported_seq":74010,"reported_epoch":105,"state":"active+clean","last_fresh":"2026-02-24T07:57:20.784529+0000","last_change":"2026-02-24T07:57:06.430234+0000","last_active":"2026-02-24T07:57:20.784529+0000","last_peered":"2026-02-24T07:57:20.784529+0000","last_clean":"2026-02-24T07:57:20.784529+0000","last_became_active":"2026-02-24T07:54:42.926886+0000","last_became_peered":"2026-02-24T07:54:42.926886+0000","last_unstale":"2026-02-24T07:57:20.784529+0000","last_undegraded":"2026-02-24T07:57:20.784529+0000","last_fullsized":"2026-02-24T07:57:20.784529+0000","mapping_epoch":90,"log_start":"105'73531","ondisk_log_start":"105'73531","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"103'66256","last_scrub_stamp":"2026-02-24T07:57:06.430163+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_clean_scrub_stamp":"2026-02-24T07:57:06.430163+0000","objects_scrubbed":13086,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:58:07.786827+0000","scrub_duration":1340,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":365756440,"num_objects":5582,"num_object_clones":0,"num_object_copies":16746,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5582,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":73534,"num_write_kb":2531649,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.c","version":"105'73686","reported_seq":74090,"reported_epoch":105,"state":"active+clean","last_fresh":"2026-02-24T07:57:20.911290+0000","last_change":"2026-02-24T07:56:25.008652+0000","last_active":"2026-02-24T07:57:20.911290+0000","last_peered":"2026-02-24T07:57:20.911290+0000","last_clean":"2026-02-24T07:57:20.911290+0000","last_became_active":"2026-02-24T07:54:42.928238+0000","last_became_peered":"2026-02-24T07:54:42.928238+0000","last_unstale":"2026-02-24T07:57:20.911290+0000","last_undegraded":"2026-02-24T07:57:20.911290+0000","last_fullsized":"2026-02-24T07:57:20.911290+0000","mapping_epoch":90,"log_start":"105'73683","ondisk_log_start":"105'73683","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"96'44918","last_scrub_stamp":"2026-02-24T07:56:25.008567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_clean_scrub_stamp":"2026-02-24T07:56:25.008567+0000","objects_scrubbed":35276,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:57:40.096151+0000","scrub_duration":4395,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":359268352,"num_objects":5482,"num_object_clones":0,"num_object_copies":16446,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5482,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":73686,"num_write_kb":2533376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.b","version":"105'72871","reported_seq":73308,"reported_epoch":105,"state":"active+clean","last_fresh":"2026-02-24T07:57:20.923181+0000","last_change":"2026-02-24T07:56:16.120475+0000","last_active":"2026-02-24T07:57:20.923181+0000","last_peered":"2026-02-24T07:57:20.923181+0000","last_clean":"2026-02-24T07:57:20.923181+0000","last_became_active":"2026-02-24T07:54:42.926965+0000","last_became_peered":"2026-02-24T07:54:42.926965+0000","last_unstale":"2026-02-24T07:57:20.923181+0000","last_undegraded":"2026-02-24T07:57:20.923181+0000","last_fullsized":"2026-02-24T07:57:20.923181+0000","mapping_epoch":90,"log_start":"105'72868","ondisk_log_start":"105'72868","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"96'39969","last_scrub_stamp":"2026-02-24T07:56:16.120290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_clean_scrub_stamp":"2026-02-24T07:56:16.120290+0000","objects_scrubbed":38739,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:57:36.957012+0000","scrub_duration":4491,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":360120320,"num_objects":5495,"num_object_clones":0,"num_object_copies":16485,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5495,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":72871,"num_write_kb":2507712,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.a","version":"105'73408","reported_seq":73910,"reported_epoch":105,"state":"active+clean","last_fresh":"2026-02-24T07:57:20.784209+0000","last_change":"2026-02-24T07:57:02.035437+0000","last_active":"2026-02-24T07:57:20.784209+0000","last_peered":"2026-02-24T07:57:20.784209+0000","last_clean":"2026-02-24T07:57:20.784209+0000","last_became_active":"2026-02-24T07:54:42.926477+0000","last_became_peered":"2026-02-24T07:54:42.926477+0000","last_unstale":"2026-02-24T07:57:20.784209+0000","last_undegraded":"2026-02-24T07:57:20.784209+0000","last_fullsized":"2026-02-24T07:57:20.784209+0000","mapping_epoch":90,"log_start":"105'73405","ondisk_log_start":"105'73405","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"102'63835","last_scrub_stamp":"2026-02-24T07:57:02.035399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_clean_scrub_stamp":"2026-02-24T07:57:02.035399+0000","objects_scrubbed":15563,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:58:02.340758+0000","scrub_duration":1912,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":369360896,"num_objects":5636,"num_object_clones":0,"num_object_copies":16908,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5636,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":73408,"num_write_kb":2529408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"105'73762","reported_seq":74284,"reported_epoch":105,"state":"active+clean","last_fresh":"2026-02-24T07:57:20.841015+0000","last_change":"2026-02-24T07:56:57.764926+0000","last_active":"2026-02-24T07:57:20.841015+0000","last_peered":"2026-02-24T07:57:20.841015+0000","last_clean":"2026-02-24T07:57:20.841015+0000","last_became_active":"2026-02-24T07:54:42.926136+0000","last_became_peered":"2026-02-24T07:54:42.926136+0000","last_unstale":"2026-02-24T07:57:20.841015+0000","last_undegraded":"2026-02-24T07:57:20.841015+0000","last_fullsized":"2026-02-24T07:57:20.841015+0000","mapping_epoch":90,"log_start":"105'73759","ondisk_log_start":"105'73759","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"101'62265","last_scrub_stamp":"2026-02-24T07:56:57.764859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_clean_scrub_stamp":"2026-02-24T07:56:57.764859+0000","objects_scrubbed":17443,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:58:20.952507+0000","scrub_duration":1544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":367394816,"num_objects":5606,"num_object_clones":0,"num_object_copies":16818,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5606,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":73762,"num_write_kb":2539776,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,1],"acting":[0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.8","version":"105'73816","reported_seq":74282,"reported_epoch":105,"state":"active+clean","last_fresh":"2026-02-24T07:57:20.791939+0000","last_change":"2026-02-24T07:57:17.037734+0000","last_active":"2026-02-24T07:57:20.791939+0000","last_peered":"2026-02-24T07:57:20.791939+0000","last_clean":"2026-02-24T07:57:20.791939+0000","last_became_active":"2026-02-24T07:54:42.929078+0000","last_became_peered":"2026-02-24T07:54:42.929078+0000","last_unstale":"2026-02-24T07:57:20.791939+0000","last_undegraded":"2026-02-24T07:57:20.791939+0000","last_fullsized":"2026-02-24T07:57:20.791939+0000","mapping_epoch":90,"log_start":"105'73813","ondisk_log_start":"105'73813","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"105'71923","last_scrub_stamp":"2026-02-24T07:57:17.037659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_clean_scrub_stamp":"2026-02-24T07:57:17.037659+0000","objects_scrubbed":7693,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:58:23.557171+0000","scrub_duration":728,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":371064832,"num_objects":5662,"num_object_clones":0,"num_object_copies":16986,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5662,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":73816,"num_write_kb":2543296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.6","version":"105'73374","reported_seq":73804,"reported_epoch":105,"state":"active+clean","last_fresh":"2026-02-24T07:57:20.914014+0000","last_change":"2026-02-24T07:56:18.901177+0000","last_active":"2026-02-24T07:57:20.914014+0000","last_peered":"2026-02-24T07:57:20.914014+0000","last_clean":"2026-02-24T07:57:20.914014+0000","last_became_active":"2026-02-24T07:54:42.926045+0000","last_became_peered":"2026-02-24T07:54:42.926045+0000","last_unstale":"2026-02-24T07:57:20.914014+0000","last_undegraded":"2026-02-24T07:57:20.914014+0000","last_fullsized":"2026-02-24T07:57:20.914014+0000","mapping_epoch":90,"log_start":"105'73371","ondisk_log_start":"105'73371","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"96'41769","last_scrub_stamp":"2026-02-24T07:56:18.901111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_clean_scrub_stamp":"2026-02-24T07:56:18.901111+0000","objects_scrubbed":38122,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:57:27.205697+0000","scrub_duration":4228,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":362807296,"num_objects":5536,"num_object_clones":0,"num_object_copies":16608,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5536,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":73374,"num_write_kb":2525120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.5","version":"105'73472","reported_seq":73971,"reported_epoch":105,"state":"active+clean","last_fresh":"2026-02-24T07:57:20.785106+0000","last_change":"2026-02-24T07:57:11.626730+0000","last_active":"2026-02-24T07:57:20.785106+0000","last_peered":"2026-02-24T07:57:20.785106+0000","last_clean":"2026-02-24T07:57:20.785106+0000","last_became_active":"2026-02-24T07:54:42.927535+0000","last_became_peered":"2026-02-24T07:54:42.927535+0000","last_unstale":"2026-02-24T07:57:20.785106+0000","last_undegraded":"2026-02-24T07:57:20.785106+0000","last_fullsized":"2026-02-24T07:57:20.785106+0000","mapping_epoch":90,"log_start":"105'73469","ondisk_log_start":"105'73469","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"104'68672","last_scrub_stamp":"2026-02-24T07:57:11.626709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_clean_scrub_stamp":"2026-02-24T07:57:11.626709+0000","objects_scrubbed":10605,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:58:21.653444+0000","scrub_duration":1515,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":361889792,"num_objects":5522,"num_object_clones":0,"num_object_copies":16566,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5522,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":73472,"num_write_kb":2527808,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.4","version":"105'73523","reported_seq":73995,"reported_epoch":105,"state":"active+clean","last_fresh":"2026-02-24T07:57:20.923230+0000","last_change":"2026-02-24T07:57:06.901398+0000","last_active":"2026-02-24T07:57:20.923230+0000","last_peered":"2026-02-24T07:57:20.923230+0000","last_clean":"2026-02-24T07:57:20.923230+0000","last_became_active":"2026-02-24T07:54:42.928011+0000","last_became_peered":"2026-02-24T07:54:42.928011+0000","last_unstale":"2026-02-24T07:57:20.923230+0000","last_undegraded":"2026-02-24T07:57:20.923230+0000","last_fullsized":"2026-02-24T07:57:20.923230+0000","mapping_epoch":90,"log_start":"105'73520","ondisk_log_start":"105'73520","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"103'66457","last_scrub_stamp":"2026-02-24T07:57:06.901316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_clean_scrub_stamp":"2026-02-24T07:57:06.901316+0000","objects_scrubbed":12783,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:58:16.895385+0000","scrub_duration":1738,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":351993856,"num_objects":5371,"num_object_clones":0,"num_object_copies":16113,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5371,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":73523,"num_write_kb":2524608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.3","version":"105'73915","reported_seq":74411,"reported_epoch":105,"state":"active+clean","last_fresh":"2026-02-24T07:57:20.784268+0000","last_change":"2026-02-24T07:57:17.018259+0000","last_active":"2026-02-24T07:57:20.784268+0000","last_peered":"2026-02-24T07:57:20.784268+0000","last_clean":"2026-02-24T07:57:20.784268+0000","last_became_active":"2026-02-24T07:54:42.927730+0000","last_became_peered":"2026-02-24T07:54:42.927730+0000","last_unstale":"2026-02-24T07:57:20.784268+0000","last_undegraded":"2026-02-24T07:57:20.784268+0000","last_fullsized":"2026-02-24T07:57:20.784268+0000","mapping_epoch":90,"log_start":"105'73912","ondisk_log_start":"105'73912","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"105'71967","last_scrub_stamp":"2026-02-24T07:57:17.018183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_clean_scrub_stamp":"2026-02-24T07:57:17.018183+0000","objects_scrubbed":7680,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:58:37.123701+0000","scrub_duration":878,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":362872832,"num_objects":5537,"num_object_clones":0,"num_object_copies":16611,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5537,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":73915,"num_write_kb":2542464,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.2","version":"105'73496","reported_seq":73884,"reported_epoch":105,"state":"active+clean","last_fresh":"2026-02-24T07:57:20.792733+0000","last_change":"2026-02-24T07:56:00.120637+0000","last_active":"2026-02-24T07:57:20.792733+0000","last_peered":"2026-02-24T07:57:20.792733+0000","last_clean":"2026-02-24T07:57:20.792733+0000","last_became_active":"2026-02-24T07:54:42.930247+0000","last_became_peered":"2026-02-24T07:54:42.930247+0000","last_unstale":"2026-02-24T07:57:20.792733+0000","last_undegraded":"2026-02-24T07:57:20.792733+0000","last_fullsized":"2026-02-24T07:57:20.792733+0000","mapping_epoch":90,"log_start":"105'73493","ondisk_log_start":"105'73493","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"96'34210","last_scrub_stamp":"2026-02-24T07:56:00.120605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_clean_scrub_stamp":"2026-02-24T07:56:00.120605+0000","objects_scrubbed":33656,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:57:29.329949+0000","scrub_duration":2957,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":361234432,"num_objects":5512,"num_object_clones":0,"num_object_copies":16536,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5512,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":73496,"num_write_kb":2528256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1","version":"105'73964","reported_seq":74398,"reported_epoch":105,"state":"active+clean","last_fresh":"2026-02-24T07:57:20.914684+0000","last_change":"2026-02-24T07:56:12.897479+0000","last_active":"2026-02-24T07:57:20.914684+0000","last_peered":"2026-02-24T07:57:20.914684+0000","last_clean":"2026-02-24T07:57:20.914684+0000","last_became_active":"2026-02-24T07:54:42.926442+0000","last_became_peered":"2026-02-24T07:54:42.926442+0000","last_unstale":"2026-02-24T07:57:20.914684+0000","last_undegraded":"2026-02-24T07:57:20.914684+0000","last_fullsized":"2026-02-24T07:57:20.914684+0000","mapping_epoch":90,"log_start":"105'73961","ondisk_log_start":"105'73961","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"96'39245","last_scrub_stamp":"2026-02-24T07:56:12.897436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_clean_scrub_stamp":"2026-02-24T07:56:12.897436+0000","objects_scrubbed":38506,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:57:42.057233+0000","scrub_duration":4279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":362807296,"num_objects":5536,"num_object_clones":0,"num_object_copies":16608,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5536,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":73964,"num_write_kb":2544000,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.0","version":"105'72846","reported_seq":73374,"reported_epoch":105,"state":"active+clean","last_fresh":"2026-02-24T07:57:20.841802+0000","last_change":"2026-02-24T07:57:07.321353+0000","last_active":"2026-02-24T07:57:20.841802+0000","last_peered":"2026-02-24T07:57:20.841802+0000","last_clean":"2026-02-24T07:57:20.841802+0000","last_became_active":"2026-02-24T07:54:42.926910+0000","last_became_peered":"2026-02-24T07:54:42.926910+0000","last_unstale":"2026-02-24T07:57:20.841802+0000","last_undegraded":"2026-02-24T07:57:20.841802+0000","last_fullsized":"2026-02-24T07:57:20.841802+0000","mapping_epoch":90,"log_start":"105'72843","ondisk_log_start":"105'72843","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"103'66056","last_scrub_stamp":"2026-02-24T07:57:07.321288+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_clean_scrub_stamp":"2026-02-24T07:57:07.321288+0000","objects_scrubbed":12515,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:58:17.856526+0000","scrub_duration":1042,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":357695488,"num_objects":5458,"num_object_clones":0,"num_object_copies":16374,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5458,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":72846,"num_write_kb":2505728,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.7","version":"105'74074","reported_seq":74467,"reported_epoch":105,"state":"active+clean","last_fresh":"2026-02-24T07:57:20.798607+0000","last_change":"2026-02-24T07:56:01.088786+0000","last_active":"2026-02-24T07:57:20.798607+0000","last_peered":"2026-02-24T07:57:20.798607+0000","last_clean":"2026-02-24T07:57:20.798607+0000","last_became_active":"2026-02-24T07:54:42.930142+0000","last_became_peered":"2026-02-24T07:54:42.930142+0000","last_unstale":"2026-02-24T07:57:20.798607+0000","last_undegraded":"2026-02-24T07:57:20.798607+0000","last_fullsized":"2026-02-24T07:57:20.798607+0000","mapping_epoch":90,"log_start":"105'74071","ondisk_log_start":"105'74071","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"96'34838","last_scrub_stamp":"2026-02-24T07:56:01.088744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:54:41.918128+0000","last_clean_scrub_stamp":"2026-02-24T07:56:01.088744+0000","objects_scrubbed":34284,"log_size":3,"log_dups_size":3000,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:57:24.696242+0000","scrub_duration":2973,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":363331584,"num_objects":5544,"num_object_clones":0,"num_object_copies":16632,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5544,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":74074,"num_write_kb":2547776,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"29'99","reported_seq":425,"reported_epoch":105,"state":"active+clean","last_fresh":"2026-02-24T07:57:13.930053+0000","last_change":"2026-02-24T07:56:33.056711+0000","last_active":"2026-02-24T07:57:13.930053+0000","last_peered":"2026-02-24T07:57:13.930053+0000","last_clean":"2026-02-24T07:57:13.930053+0000","last_became_active":"2026-02-24T07:42:54.186719+0000","last_became_peered":"2026-02-24T07:42:54.186719+0000","last_unstale":"2026-02-24T07:57:13.930053+0000","last_undegraded":"2026-02-24T07:57:13.930053+0000","last_fullsized":"2026-02-24T07:57:13.930053+0000","mapping_epoch":27,"log_start":"29'96","ondisk_log_start":"29'96","created":20,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'99","last_scrub_stamp":"2026-02-24T07:56:33.056678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:42:43.640776+0000","last_clean_scrub_stamp":"2026-02-24T07:56:33.056678+0000","objects_scrubbed":2,"log_size":3,"log_dups_size":96,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:57:56.381501+0000","scrub_duration":21,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":240,"num_write":135,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":16,"stat_sum":{"num_bytes":5796397080,"num_objects":88447,"num_object_clones":0,"num_object_copies":265341,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":88447,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1176905,"num_write_kb":40491201,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":17397854208,"data_stored":17397841992,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":2298,"internal_metadata":0},"log_size":48,"ondisk_log_size":48,"up":48,"acting":48,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":240,"num_write":135,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1682,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":3,"up_from":25,"seq":107374182576,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":5496512,"kb_used_data":3711980,"kb_used_omap":26,"kb_used_meta":1784485,"kb_avail":728506688,"statfs":{"total":751619276800,"available":745990848512,"internally_reserved":0,"allocated":3801067520,"data_stored":3800493951,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":27157,"internal_metadata":1827313131},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":19,"seq":81604378802,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":5741508,"kb_used_data":4470576,"kb_used_omap":27,"kb_used_meta":1270884,"kb_avail":728261692,"statfs":{"total":751619276800,"available":745739972608,"internally_reserved":0,"allocated":4577869824,"data_stored":4577292183,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":27802,"internal_metadata":1301386086},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542323,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":6677632,"kb_used_data":5187948,"kb_used_omap":26,"kb_used_meta":1489637,"kb_avail":727325568,"statfs":{"total":751619276800,"available":744781381632,"internally_reserved":0,"allocated":5312458752,"data_stored":5311884663,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":26900,"internal_metadata":1525389036},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,10],"upper_bound":32},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705846,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":5554564,"kb_used_data":4416112,"kb_used_omap":24,"kb_used_meta":1138407,"kb_avail":728448636,"statfs":{"total":751619276800,"available":745931403264,"internally_reserved":0,"allocated":4522098688,"data_stored":4521521047,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":25137,"internal_metadata":1165729231},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1682,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4357754880,"data_stored":4357750808,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":186,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":5047914496,"data_stored":5047910424,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":93,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4348448768,"data_stored":4348444696,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1926,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":3643736064,"data_stored":3643736064,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":93,"internal_metadata":0}]}} 2026-02-24T07:57:27.974 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph osd pool set unique_pool_4 pgp_num 106 2026-02-24T07:57:28.012 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:57:28.010+0000 7f8ee1c2f640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:57:28.012 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:57:28.010+0000 7f8ee1c2f640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:57:28.029 INFO:teuthology.orchestra.run.trial035.stdout:osd.0: {} 2026-02-24T07:57:28.029 INFO:teuthology.orchestra.run.trial035.stderr:osd.0: osd_enable_op_tracker = '' 2026-02-24T07:57:28.044 INFO:teuthology.orchestra.run.trial035.stdout:osd.1: {} 2026-02-24T07:57:28.044 INFO:teuthology.orchestra.run.trial035.stderr:osd.1: osd_enable_op_tracker = '' 2026-02-24T07:57:28.056 INFO:teuthology.orchestra.run.trial035.stdout:osd.2: {} 2026-02-24T07:57:28.057 INFO:teuthology.orchestra.run.trial035.stderr:osd.2: osd_enable_op_tracker = '' 2026-02-24T07:57:28.069 INFO:teuthology.orchestra.run.trial035.stdout:osd.3: {} 2026-02-24T07:57:28.069 INFO:teuthology.orchestra.run.trial035.stderr:osd.3: osd_enable_op_tracker = '' 2026-02-24T07:57:28.077 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:57:27 trial035 ceph-mon[39284]: from='client.15655 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2026-02-24T07:57:28.077 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:57:27 trial035 ceph-mon[39284]: pgmap v513: 17 pgs: 17 active+clean; 3.0 GiB data, 15 GiB used, 2.7 TiB / 2.7 TiB avail; 0 B/s wr, 10.08k op/s 2026-02-24T07:57:28.115 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:57:28.251 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:57:28.249+0000 7fa92f1a9640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:57:28.252 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:57:28.249+0000 7fa92f1a9640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:57:28.394 INFO:teuthology.orchestra.run.trial035.stderr:Error EPERM: crimson_allow_pg_split is false; pgp_num increase denied 2026-02-24T07:57:28.433 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-24T07:57:29.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:57:28 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/1210123386' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2026-02-24T07:57:29.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:57:28 trial035 ceph-mon[39284]: 6.6 scrub starts 2026-02-24T07:57:29.205 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:57:28 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/3054694659' entity='client.admin' cmd={"prefix": "osd pool set", "pool": "unique_pool_4", "var": "pgp_num", "val": "106"} : dispatch 2026-02-24T07:57:30.112 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2026-02-24T07:57:30.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:57:29 trial035 ceph-mon[39284]: 6.6 scrub ok 2026-02-24T07:57:30.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:57:29 trial035 ceph-mon[39284]: pgmap v514: 17 pgs: 17 active+clean; 3.0 GiB data, 15 GiB used, 2.7 TiB / 2.7 TiB avail; 0 B/s wr, 6.56k op/s 2026-02-24T07:57:30.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:57:29 trial035 ceph-mon[39284]: from='mgr.14284 10.20.193.35:0/3681373222' entity='mgr.x' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2026-02-24T07:57:30.255 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:57:30.371 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:57:30.369+0000 7f055721f640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:57:30.372 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:57:30.369+0000 7f055721f640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:57:30.386 INFO:teuthology.orchestra.run.trial035.stdout:osd.0: {} 2026-02-24T07:57:30.386 INFO:teuthology.orchestra.run.trial035.stderr:osd.0: osd_enable_op_tracker = '' 2026-02-24T07:57:30.401 INFO:teuthology.orchestra.run.trial035.stdout:osd.1: {} 2026-02-24T07:57:30.401 INFO:teuthology.orchestra.run.trial035.stderr:osd.1: osd_enable_op_tracker = '' 2026-02-24T07:57:30.414 INFO:teuthology.orchestra.run.trial035.stdout:osd.2: {} 2026-02-24T07:57:30.414 INFO:teuthology.orchestra.run.trial035.stderr:osd.2: osd_enable_op_tracker = '' 2026-02-24T07:57:30.427 INFO:teuthology.orchestra.run.trial035.stdout:osd.3: {} 2026-02-24T07:57:30.427 INFO:teuthology.orchestra.run.trial035.stderr:osd.3: osd_enable_op_tracker = '' 2026-02-24T07:57:31.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:57:30 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/2370818495' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2026-02-24T07:57:31.832 INFO:tasks.radosbench.radosbench.0.trial035.stdout:Removed 632675 objects 2026-02-24T07:57:31.832 INFO:tasks.radosbench.radosbench.0.trial035.stdout:Clean up completed and total clean up time :77.8088 2026-02-24T07:57:32.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:57:31 trial035 ceph-mon[39284]: osdmap e106: 4 total, 4 up, 4 in 2026-02-24T07:57:32.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:57:31 trial035 ceph-mon[39284]: 6.2 scrub starts 2026-02-24T07:57:32.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:57:31 trial035 ceph-mon[39284]: 6.2 scrub ok 2026-02-24T07:57:32.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:57:31 trial035 ceph-mon[39284]: pgmap v516: 17 pgs: 17 active+clean; 524 MiB data, 7.3 GiB used, 2.7 TiB / 2.7 TiB avail; 0 B/s wr, 8.01k op/s 2026-02-24T07:57:32.470 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2026-02-24T07:57:32.588 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:57:32.696 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:57:32.694+0000 7fb71d524640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:57:32.697 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:57:32.694+0000 7fb71d524640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:57:32.709 INFO:teuthology.orchestra.run.trial035.stdout:osd.0: {} 2026-02-24T07:57:32.709 INFO:teuthology.orchestra.run.trial035.stderr:osd.0: osd_enable_op_tracker = '' 2026-02-24T07:57:32.719 INFO:teuthology.orchestra.run.trial035.stdout:osd.1: {} 2026-02-24T07:57:32.720 INFO:teuthology.orchestra.run.trial035.stderr:osd.1: osd_enable_op_tracker = '' 2026-02-24T07:57:32.730 INFO:teuthology.orchestra.run.trial035.stdout:osd.2: {} 2026-02-24T07:57:32.730 INFO:teuthology.orchestra.run.trial035.stderr:osd.2: osd_enable_op_tracker = '' 2026-02-24T07:57:32.740 INFO:teuthology.orchestra.run.trial035.stdout:osd.3: {} 2026-02-24T07:57:32.740 INFO:teuthology.orchestra.run.trial035.stderr:osd.3: osd_enable_op_tracker = '' 2026-02-24T07:57:33.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:57:32 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/2692181799' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2026-02-24T07:57:33.434 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2026-02-24T07:57:33.434 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph osd reweight 0 1 2026-02-24T07:57:33.550 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:57:33.652 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:57:33.649+0000 7f5b2315c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:57:33.652 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:57:33.650+0000 7f5b2315c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:57:33.933 INFO:teuthology.orchestra.run.trial035.stderr:reweighted osd.0 to 1 (10000) 2026-02-24T07:57:33.976 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph osd reweight 1 1 2026-02-24T07:57:34.094 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:57:34.193 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:57:34.191+0000 7f9397c4b640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:57:34.193 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:57:33 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/4168020493' entity='client.admin' cmd={"prefix": "osd reweight", "id": 0, "weight": 1.0} : dispatch 2026-02-24T07:57:34.193 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:57:33 trial035 ceph-mon[39284]: pgmap v517: 17 pgs: 17 active+clean; 524 MiB data, 7.3 GiB used, 2.7 TiB / 2.7 TiB avail; 0 B/s wr, 8.01k op/s 2026-02-24T07:57:34.193 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:57:33 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/4168020493' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 0, "weight": 1.0}]': finished 2026-02-24T07:57:34.193 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:57:33 trial035 ceph-mon[39284]: osdmap e107: 4 total, 4 up, 4 in 2026-02-24T07:57:34.194 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:57:34.191+0000 7f9397c4b640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:57:34.778 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2026-02-24T07:57:34.806 INFO:tasks.cephadm.ceph_manager.ceph:removing pool_name unique_pool_4 2026-02-24T07:57:34.806 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph osd pool rm unique_pool_4 unique_pool_4 --yes-i-really-really-mean-it 2026-02-24T07:57:34.895 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:57:34.936 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:57:34.951 INFO:teuthology.orchestra.run.trial035.stderr:reweighted osd.1 to 1 (10000) 2026-02-24T07:57:34.994 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph osd reweight 2 1 2026-02-24T07:57:34.997 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:57:34.995+0000 7f640bac4640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:57:34.997 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:57:34.996+0000 7f640bac4640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:57:35.010 INFO:teuthology.orchestra.run.trial035.stdout:osd.0: {} 2026-02-24T07:57:35.010 INFO:teuthology.orchestra.run.trial035.stderr:osd.0: osd_enable_op_tracker = '' 2026-02-24T07:57:35.020 INFO:teuthology.orchestra.run.trial035.stdout:osd.1: {} 2026-02-24T07:57:35.021 INFO:teuthology.orchestra.run.trial035.stderr:osd.1: osd_enable_op_tracker = '' 2026-02-24T07:57:35.031 INFO:teuthology.orchestra.run.trial035.stdout:osd.2: {} 2026-02-24T07:57:35.031 INFO:teuthology.orchestra.run.trial035.stderr:osd.2: osd_enable_op_tracker = '' 2026-02-24T07:57:35.042 INFO:teuthology.orchestra.run.trial035.stdout:osd.3: {} 2026-02-24T07:57:35.043 INFO:teuthology.orchestra.run.trial035.stderr:osd.3: osd_enable_op_tracker = '' 2026-02-24T07:57:35.049 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:57:35.047+0000 7fe399052640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:57:35.049 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:57:35.047+0000 7fe399052640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:57:35.056 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:57:34 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/1843045453' entity='client.admin' cmd={"prefix": "osd reweight", "id": 1, "weight": 1.0} : dispatch 2026-02-24T07:57:35.114 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:57:35.225 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:57:35.223+0000 7fb30c9bb640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:57:35.225 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:57:35.223+0000 7fb30c9bb640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:57:35.954 INFO:teuthology.orchestra.run.trial035.stderr:reweighted osd.2 to 1 (10000) 2026-02-24T07:57:35.955 INFO:teuthology.orchestra.run.trial035.stderr:pool 'unique_pool_4' removed 2026-02-24T07:57:35.998 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph osd reweight 3 1 2026-02-24T07:57:36.000 DEBUG:teuthology.run_tasks:Unwinding manager thrashosds 2026-02-24T07:57:36.017 INFO:tasks.thrashosds:joining thrashosds 2026-02-24T07:57:36.152 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:57:36.205 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:57:35 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/1843045453' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 1, "weight": 1.0}]': finished 2026-02-24T07:57:36.206 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:57:35 trial035 ceph-mon[39284]: osdmap e108: 4 total, 4 up, 4 in 2026-02-24T07:57:36.206 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:57:35 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/3878340208' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2026-02-24T07:57:36.206 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:57:35 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/900856314' entity='client.admin' cmd={"prefix": "osd pool rm", "pool": "unique_pool_4", "pool2": "unique_pool_4", "yes_i_really_really_mean_it": true} : dispatch 2026-02-24T07:57:36.206 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:57:35 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/1191498174' entity='client.admin' cmd={"prefix": "osd reweight", "id": 2, "weight": 1.0} : dispatch 2026-02-24T07:57:36.206 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:57:35 trial035 ceph-mon[39284]: pgmap v520: 17 pgs: 17 active+clean; 524 MiB data, 7.3 GiB used, 2.7 TiB / 2.7 TiB avail; 0 B/s wr, 6.91k op/s 2026-02-24T07:57:36.251 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:57:36.248+0000 7f4aa8c1b640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:57:36.251 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:57:36.249+0000 7f4aa8c1b640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:57:36.955 INFO:teuthology.orchestra.run.trial035.stderr:reweighted osd.3 to 1 (10000) 2026-02-24T07:57:36.994 INFO:tasks.cephadm.ceph_manager.ceph:waiting for recovery to complete 2026-02-24T07:57:36.994 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph pg dump --format=json 2026-02-24T07:57:37.110 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:57:37.208 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:57:37.206+0000 7ff30926e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:57:37.208 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:57:37.206+0000 7ff30926e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:57:37.209 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:57:36 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/900856314' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "unique_pool_4", "pool2": "unique_pool_4", "yes_i_really_really_mean_it": true}]': finished 2026-02-24T07:57:37.209 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:57:36 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/1191498174' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 2, "weight": 1.0}]': finished 2026-02-24T07:57:37.209 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:57:36 trial035 ceph-mon[39284]: osdmap e109: 4 total, 4 up, 4 in 2026-02-24T07:57:37.209 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:57:36 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/548403953' entity='client.admin' cmd={"prefix": "osd reweight", "id": 3, "weight": 1.0} : dispatch 2026-02-24T07:57:37.209 INFO:journalctl@ceph.mgr.x.trial035.stdout:Feb 24 07:57:36 trial035 ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-mgr-x[39621]: ::ffff:10.20.193.91 - - [24/Feb/2026:07:57:36] "GET /metrics HTTP/1.1" 200 4225 "" "Prometheus/3.6.0" 2026-02-24T07:57:37.317 INFO:teuthology.orchestra.run.trial035.stdout: 2026-02-24T07:57:37.318 INFO:teuthology.orchestra.run.trial035.stderr:dumped all 2026-02-24T07:57:37.353 INFO:teuthology.orchestra.run.trial035.stdout:{"pg_ready":true,"pg_map":{"version":522,"stamp":"2026-02-24T07:57:36.955017+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":240,"num_write":135,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":3,"acting":3,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":51,"num_osds":4,"num_per_pool_osds":4,"num_per_pool_omap_osds":4,"kb":2936012800,"kb_used":4945140,"kb_used_data":5348,"kb_used_omap":95,"kb_used_meta":4939616,"kb_avail":2931067660,"statfs":{"total":3006477107200,"available":3001413283840,"internally_reserved":0,"allocated":5476352,"data_stored":3107006,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":98190,"internal_metadata":5058166898},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-549191704,"num_objects":-8381,"num_object_clones":0,"num_object_copies":-25143,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-8381,"num_whiteouts":0,"num_read":-1,"num_read_kb":-1,"num_write":-1256971,"num_write_kb":-40491201,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.178885"},"pg_stats":[{"pgid":"1.0","version":"29'99","reported_seq":431,"reported_epoch":108,"state":"active+clean","last_fresh":"2026-02-24T07:57:34.951274+0000","last_change":"2026-02-24T07:56:33.056711+0000","last_active":"2026-02-24T07:57:34.951274+0000","last_peered":"2026-02-24T07:57:34.951274+0000","last_clean":"2026-02-24T07:57:34.951274+0000","last_became_active":"2026-02-24T07:42:54.186719+0000","last_became_peered":"2026-02-24T07:42:54.186719+0000","last_unstale":"2026-02-24T07:57:34.951274+0000","last_undegraded":"2026-02-24T07:57:34.951274+0000","last_fullsized":"2026-02-24T07:57:34.951274+0000","mapping_epoch":27,"log_start":"29'96","ondisk_log_start":"29'96","created":20,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'99","last_scrub_stamp":"2026-02-24T07:56:33.056678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:42:43.640776+0000","last_clean_scrub_stamp":"2026-02-24T07:56:33.056678+0000","objects_scrubbed":2,"log_size":3,"log_dups_size":96,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:57:56.381501+0000","scrub_duration":21,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":240,"num_write":135,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":240,"num_write":135,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1682,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":3,"up_from":25,"seq":107374182579,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":1153956,"kb_used_data":1488,"kb_used_omap":12,"kb_used_meta":1152435,"kb_avail":732849244,"statfs":{"total":751619276800,"available":750437625856,"internally_reserved":0,"allocated":1523712,"data_stored":926548,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12398,"internal_metadata":1180094354},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":19,"seq":81604378805,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":1452888,"kb_used_data":1476,"kb_used_omap":28,"kb_used_meta":1451363,"kb_avail":732550312,"statfs":{"total":751619276800,"available":750131519488,"internally_reserved":0,"allocated":1511424,"data_stored":922139,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":29110,"internal_metadata":1486196298},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542326,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":1090068,"kb_used_data":896,"kb_used_omap":28,"kb_used_meta":1089123,"kb_avail":732913132,"statfs":{"total":751619276800,"available":750503047168,"internally_reserved":0,"allocated":917504,"data_stored":331771,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":29589,"internal_metadata":1115262059},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705849,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":1248228,"kb_used_data":1488,"kb_used_omap":26,"kb_used_meta":1246693,"kb_avail":732754972,"statfs":{"total":751619276800,"available":750341091328,"internally_reserved":0,"allocated":1523712,"data_stored":926548,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":27093,"internal_metadata":1276614187},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1682,"internal_metadata":0}]}} 2026-02-24T07:57:37.354 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph pg dump --format=json 2026-02-24T07:57:37.356 INFO:journalctl@ceph.prometheus.a.trial091.stdout:Feb 24 07:57:36 trial091 ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-prometheus-a[66007]: time=2026-02-24T07:57:36.973Z level=ERROR source=scrape.go:1630 msg="Failed to determine correct type of scrape target." component="scrape manager" scrape_pool=ceph target=http://trial035:9283/metrics content_type="text/html;charset=utf-8" fallback_media_type="" err="received unsupported Content-Type \"text/html;charset=utf-8\" and no fallback_scrape_protocol specified for target" 2026-02-24T07:57:37.469 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:57:37.584 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:57:37.581+0000 7f04e09f4640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:57:37.584 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:57:37.582+0000 7f04e09f4640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:57:37.699 INFO:teuthology.orchestra.run.trial035.stdout: 2026-02-24T07:57:37.699 INFO:teuthology.orchestra.run.trial035.stderr:dumped all 2026-02-24T07:57:37.741 INFO:teuthology.orchestra.run.trial035.stdout:{"pg_ready":true,"pg_map":{"version":522,"stamp":"2026-02-24T07:57:36.955017+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":240,"num_write":135,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":3,"acting":3,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":51,"num_osds":4,"num_per_pool_osds":4,"num_per_pool_omap_osds":4,"kb":2936012800,"kb_used":4945140,"kb_used_data":5348,"kb_used_omap":95,"kb_used_meta":4939616,"kb_avail":2931067660,"statfs":{"total":3006477107200,"available":3001413283840,"internally_reserved":0,"allocated":5476352,"data_stored":3107006,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":98190,"internal_metadata":5058166898},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-549191704,"num_objects":-8381,"num_object_clones":0,"num_object_copies":-25143,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-8381,"num_whiteouts":0,"num_read":-1,"num_read_kb":-1,"num_write":-1256971,"num_write_kb":-40491201,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.178885"},"pg_stats":[{"pgid":"1.0","version":"29'99","reported_seq":431,"reported_epoch":108,"state":"active+clean","last_fresh":"2026-02-24T07:57:34.951274+0000","last_change":"2026-02-24T07:56:33.056711+0000","last_active":"2026-02-24T07:57:34.951274+0000","last_peered":"2026-02-24T07:57:34.951274+0000","last_clean":"2026-02-24T07:57:34.951274+0000","last_became_active":"2026-02-24T07:42:54.186719+0000","last_became_peered":"2026-02-24T07:42:54.186719+0000","last_unstale":"2026-02-24T07:57:34.951274+0000","last_undegraded":"2026-02-24T07:57:34.951274+0000","last_fullsized":"2026-02-24T07:57:34.951274+0000","mapping_epoch":27,"log_start":"29'96","ondisk_log_start":"29'96","created":20,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'99","last_scrub_stamp":"2026-02-24T07:56:33.056678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:42:43.640776+0000","last_clean_scrub_stamp":"2026-02-24T07:56:33.056678+0000","objects_scrubbed":2,"log_size":3,"log_dups_size":96,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:57:56.381501+0000","scrub_duration":21,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":240,"num_write":135,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":240,"num_write":135,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1682,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":3,"up_from":25,"seq":107374182579,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":1153956,"kb_used_data":1488,"kb_used_omap":12,"kb_used_meta":1152435,"kb_avail":732849244,"statfs":{"total":751619276800,"available":750437625856,"internally_reserved":0,"allocated":1523712,"data_stored":926548,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12398,"internal_metadata":1180094354},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":19,"seq":81604378805,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":1452888,"kb_used_data":1476,"kb_used_omap":28,"kb_used_meta":1451363,"kb_avail":732550312,"statfs":{"total":751619276800,"available":750131519488,"internally_reserved":0,"allocated":1511424,"data_stored":922139,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":29110,"internal_metadata":1486196298},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542326,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":1090068,"kb_used_data":896,"kb_used_omap":28,"kb_used_meta":1089123,"kb_avail":732913132,"statfs":{"total":751619276800,"available":750503047168,"internally_reserved":0,"allocated":917504,"data_stored":331771,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":29589,"internal_metadata":1115262059},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705849,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":1248228,"kb_used_data":1488,"kb_used_omap":26,"kb_used_meta":1246693,"kb_avail":732754972,"statfs":{"total":751619276800,"available":750341091328,"internally_reserved":0,"allocated":1523712,"data_stored":926548,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":27093,"internal_metadata":1276614187},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1682,"internal_metadata":0}]}} 2026-02-24T07:57:37.742 INFO:tasks.cephadm.ceph_manager.ceph:recovered! 2026-02-24T07:57:37.742 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2026-02-24T07:57:37.742 INFO:tasks.thrashosds.thrasher:Growing pool .mgr 2026-02-24T07:57:37.742 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph pg dump --format=json 2026-02-24T07:57:37.860 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:57:37.967 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:57:37.965+0000 7ff52e230640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:57:37.967 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:57:37.965+0000 7ff52e230640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:57:38.075 INFO:teuthology.orchestra.run.trial035.stdout: 2026-02-24T07:57:38.076 INFO:teuthology.orchestra.run.trial035.stderr:dumped all 2026-02-24T07:57:38.076 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:57:37 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/548403953' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 3, "weight": 1.0}]': finished 2026-02-24T07:57:38.076 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:57:37 trial035 ceph-mon[39284]: osdmap e110: 4 total, 4 up, 4 in 2026-02-24T07:57:38.077 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:57:37 trial035 ceph-mon[39284]: from='client.15677 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2026-02-24T07:57:38.077 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:57:37 trial035 ceph-mon[39284]: from='client.15679 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2026-02-24T07:57:38.077 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:57:37 trial035 ceph-mon[39284]: pgmap v523: 1 pgs: 1 active+clean; 577 KiB data, 4.7 GiB used, 2.7 TiB / 2.7 TiB avail 2026-02-24T07:57:38.116 INFO:teuthology.orchestra.run.trial035.stdout:{"pg_ready":true,"pg_map":{"version":523,"stamp":"2026-02-24T07:57:37.776878+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":240,"num_write":135,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":3,"acting":3,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":51,"num_osds":4,"num_per_pool_osds":4,"num_per_pool_omap_osds":4,"kb":2936012800,"kb_used":4945140,"kb_used_data":5348,"kb_used_omap":95,"kb_used_meta":4939616,"kb_avail":2931067660,"statfs":{"total":3006477107200,"available":3001413283840,"internally_reserved":0,"allocated":5476352,"data_stored":3107006,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":98190,"internal_metadata":5058166898},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-549191704,"num_objects":-8381,"num_object_clones":0,"num_object_copies":-25143,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-8381,"num_whiteouts":0,"num_read":-1,"num_read_kb":-1,"num_write":-1256971,"num_write_kb":-40491201,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000483"},"pg_stats":[{"pgid":"1.0","version":"29'99","reported_seq":431,"reported_epoch":108,"state":"active+clean","last_fresh":"2026-02-24T07:57:34.951274+0000","last_change":"2026-02-24T07:56:33.056711+0000","last_active":"2026-02-24T07:57:34.951274+0000","last_peered":"2026-02-24T07:57:34.951274+0000","last_clean":"2026-02-24T07:57:34.951274+0000","last_became_active":"2026-02-24T07:42:54.186719+0000","last_became_peered":"2026-02-24T07:42:54.186719+0000","last_unstale":"2026-02-24T07:57:34.951274+0000","last_undegraded":"2026-02-24T07:57:34.951274+0000","last_fullsized":"2026-02-24T07:57:34.951274+0000","mapping_epoch":27,"log_start":"29'96","ondisk_log_start":"29'96","created":20,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'99","last_scrub_stamp":"2026-02-24T07:56:33.056678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:42:43.640776+0000","last_clean_scrub_stamp":"2026-02-24T07:56:33.056678+0000","objects_scrubbed":2,"log_size":3,"log_dups_size":96,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:57:56.381501+0000","scrub_duration":21,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":240,"num_write":135,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":240,"num_write":135,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1682,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":3,"up_from":25,"seq":107374182579,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":1153956,"kb_used_data":1488,"kb_used_omap":12,"kb_used_meta":1152435,"kb_avail":732849244,"statfs":{"total":751619276800,"available":750437625856,"internally_reserved":0,"allocated":1523712,"data_stored":926548,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12398,"internal_metadata":1180094354},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":19,"seq":81604378805,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":1452888,"kb_used_data":1476,"kb_used_omap":28,"kb_used_meta":1451363,"kb_avail":732550312,"statfs":{"total":751619276800,"available":750131519488,"internally_reserved":0,"allocated":1511424,"data_stored":922139,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":29110,"internal_metadata":1486196298},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542326,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":1090068,"kb_used_data":896,"kb_used_omap":28,"kb_used_meta":1089123,"kb_avail":732913132,"statfs":{"total":751619276800,"available":750503047168,"internally_reserved":0,"allocated":917504,"data_stored":331771,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":29589,"internal_metadata":1115262059},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705849,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":1248228,"kb_used_data":1488,"kb_used_omap":26,"kb_used_meta":1246693,"kb_avail":732754972,"statfs":{"total":751619276800,"available":750341091328,"internally_reserved":0,"allocated":1523712,"data_stored":926548,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":27093,"internal_metadata":1276614187},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1682,"internal_metadata":0}]}} 2026-02-24T07:57:38.117 INFO:tasks.cephadm.ceph_manager.ceph:increase pool size by 10 2026-02-24T07:57:38.118 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph osd pool set .mgr pg_num 471 2026-02-24T07:57:38.238 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:57:38.641 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:57:38.639+0000 7f2369a48640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:57:38.642 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:57:38.640+0000 7f2369a48640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:57:38.753 INFO:teuthology.orchestra.run.trial035.stderr:Error EPERM: crimson_allow_pg_split is false; pg_num increase denied for crimson pool 2026-02-24T07:57:39.164 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-24T07:57:39.454 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:57:38 trial035 ceph-mon[39284]: from='client.15681 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2026-02-24T07:57:39.454 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:57:38 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/3272004920' entity='client.admin' cmd={"prefix": "osd pool set", "pool": ".mgr", "var": "pg_num", "val": "471"} : dispatch 2026-02-24T07:57:39.454 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:57:38 trial035 ceph-mon[39284]: osdmap e111: 4 total, 4 up, 4 in 2026-02-24T07:57:40.454 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:57:39 trial035 ceph-mon[39284]: pgmap v525: 1 pgs: 1 active+clean; 577 KiB data, 4.7 GiB used, 2.7 TiB / 2.7 TiB avail 2026-02-24T07:57:41.454 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:57:40 trial035 ceph-mon[39284]: osdmap e112: 4 total, 4 up, 4 in 2026-02-24T07:57:42.454 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:57:41 trial035 ceph-mon[39284]: pgmap v527: 1 pgs: 1 active+clean; 577 KiB data, 4.7 GiB used, 2.7 TiB / 2.7 TiB avail 2026-02-24T07:57:44.165 INFO:tasks.thrashosds.thrasher:fixing pg num pool .mgr 2026-02-24T07:57:44.165 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph osd pool set .mgr pgp_num 471 2026-02-24T07:57:44.283 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:57:44.399 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:57:44.396+0000 7ff152b4b640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:57:44.399 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:57:44.397+0000 7ff152b4b640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:57:44.510 INFO:teuthology.orchestra.run.trial035.stderr:Error EPERM: crimson_allow_pg_split is false; pgp_num increase denied 2026-02-24T07:57:44.542 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-24T07:57:44.542 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph -- tell 'mon.*' injectargs --mon_osd_down_out_interval=600 2026-02-24T07:57:44.658 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:57:44.759 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:57:44.757+0000 7f56cae57640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:57:44.759 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:57:44.757+0000 7f56cae57640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:57:44.765 INFO:teuthology.orchestra.run.trial035.stdout:{} 2026-02-24T07:57:44.766 INFO:teuthology.orchestra.run.trial035.stderr:mon_osd_down_out_interval = '' (not observed, change may require restart) 2026-02-24T07:57:44.801 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph osd reweight 0 1 2026-02-24T07:57:44.918 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:57:45.017 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:57:45.015+0000 7f1e95881640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:57:45.017 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:57:45.015+0000 7f1e95881640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:57:45.017 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:57:44 trial035 ceph-mon[39284]: pgmap v528: 1 pgs: 1 active+clean; 577 KiB data, 4.7 GiB used, 2.7 TiB / 2.7 TiB avail 2026-02-24T07:57:45.017 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:57:44 trial035 ceph-mon[39284]: osdmap e113: 4 total, 4 up, 4 in 2026-02-24T07:57:45.017 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:57:44 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/2788365055' entity='client.admin' cmd={"prefix": "osd pool set", "pool": ".mgr", "var": "pgp_num", "val": "471"} : dispatch 2026-02-24T07:57:45.017 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:57:44 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/136099630' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2026-02-24T07:57:45.832 INFO:teuthology.orchestra.run.trial035.stderr:reweighted osd.0 to 1 (10000) 2026-02-24T07:57:45.892 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph osd primary-affinity 0 1 2026-02-24T07:57:46.009 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:57:46.110 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:57:46.107+0000 7f5b045a5640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:57:46.110 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:57:46.108+0000 7f5b045a5640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:57:46.110 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:57:45 trial035 ceph-mon[39284]: from='mgr.14284 10.20.193.35:0/3681373222' entity='mgr.x' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2026-02-24T07:57:46.111 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:57:45 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/1720584461' entity='client.admin' cmd={"prefix": "osd reweight", "id": 0, "weight": 1.0} : dispatch 2026-02-24T07:57:46.833 INFO:teuthology.orchestra.run.trial035.stderr:set osd.0 primary-affinity to 1 (8655362) 2026-02-24T07:57:46.873 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph osd reweight 1 1 2026-02-24T07:57:46.990 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:57:47.091 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:57:47.090+0000 7f96785d6640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:57:47.091 INFO:journalctl@ceph.mgr.x.trial035.stdout:Feb 24 07:57:46 trial035 ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-mgr-x[39621]: ::ffff:10.20.193.91 - - [24/Feb/2026:07:57:46] "GET /metrics HTTP/1.1" 200 3893 "" "Prometheus/3.6.0" 2026-02-24T07:57:47.091 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:57:46 trial035 ceph-mon[39284]: pgmap v530: 1 pgs: 1 active+clean; 577 KiB data, 4.7 GiB used, 2.7 TiB / 2.7 TiB avail 2026-02-24T07:57:47.092 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:57:46 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/1720584461' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 0, "weight": 1.0}]': finished 2026-02-24T07:57:47.092 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:57:46 trial035 ceph-mon[39284]: osdmap e114: 4 total, 4 up, 4 in 2026-02-24T07:57:47.092 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:57:46 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/4142818232' entity='client.admin' cmd={"prefix": "osd primary-affinity", "id": 0, "weight": 1.0} : dispatch 2026-02-24T07:57:47.092 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:57:47.090+0000 7f96785d6640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:57:47.266 INFO:journalctl@ceph.prometheus.a.trial091.stdout:Feb 24 07:57:46 trial091 ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-prometheus-a[66007]: time=2026-02-24T07:57:46.973Z level=ERROR source=scrape.go:1630 msg="Failed to determine correct type of scrape target." component="scrape manager" scrape_pool=ceph target=http://trial035:9283/metrics content_type="text/html;charset=utf-8" fallback_media_type="" err="received unsupported Content-Type \"text/html;charset=utf-8\" and no fallback_scrape_protocol specified for target" 2026-02-24T07:57:47.837 INFO:teuthology.orchestra.run.trial035.stderr:reweighted osd.1 to 1 (10000) 2026-02-24T07:57:47.877 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph osd primary-affinity 1 1 2026-02-24T07:57:47.991 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:57:48.093 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:57:48.092+0000 7f6a58145640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:57:48.093 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:57:48.092+0000 7f6a58145640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:57:48.093 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:57:47 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/4142818232' entity='client.admin' cmd='[{"prefix": "osd primary-affinity", "id": 0, "weight": 1.0}]': finished 2026-02-24T07:57:48.094 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:57:47 trial035 ceph-mon[39284]: osdmap e115: 4 total, 4 up, 4 in 2026-02-24T07:57:48.094 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:57:47 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/2662207406' entity='client.admin' cmd={"prefix": "osd reweight", "id": 1, "weight": 1.0} : dispatch 2026-02-24T07:57:48.840 INFO:teuthology.orchestra.run.trial035.stderr:set osd.1 primary-affinity to 1 (8655362) 2026-02-24T07:57:49.059 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph osd reweight 2 1 2026-02-24T07:57:49.173 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:57:49.196 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:57:48 trial035 ceph-mon[39284]: purged_snaps scrub starts 2026-02-24T07:57:49.196 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:57:48 trial035 ceph-mon[39284]: purged_snaps scrub ok 2026-02-24T07:57:49.196 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:57:48 trial035 ceph-mon[39284]: pgmap v533: 1 pgs: 1 active+clean; 577 KiB data, 4.7 GiB used, 2.7 TiB / 2.7 TiB avail 2026-02-24T07:57:49.196 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:57:48 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/2662207406' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 1, "weight": 1.0}]': finished 2026-02-24T07:57:49.196 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:57:48 trial035 ceph-mon[39284]: osdmap e116: 4 total, 4 up, 4 in 2026-02-24T07:57:49.196 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:57:48 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/3383304888' entity='client.admin' cmd={"prefix": "osd primary-affinity", "id": 1, "weight": 1.0} : dispatch 2026-02-24T07:57:49.281 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:57:49.280+0000 7f56d9027640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:57:49.281 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:57:49.280+0000 7f56d9027640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:57:49.937 INFO:teuthology.orchestra.run.trial035.stderr:reweighted osd.2 to 1 (10000) 2026-02-24T07:57:49.977 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph osd primary-affinity 2 1 2026-02-24T07:57:50.093 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:57:50.115 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:57:49 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/3383304888' entity='client.admin' cmd='[{"prefix": "osd primary-affinity", "id": 1, "weight": 1.0}]': finished 2026-02-24T07:57:50.115 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:57:49 trial035 ceph-mon[39284]: osdmap e117: 4 total, 4 up, 4 in 2026-02-24T07:57:50.115 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:57:49 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/947237589' entity='client.admin' cmd={"prefix": "osd reweight", "id": 2, "weight": 1.0} : dispatch 2026-02-24T07:57:50.194 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:57:50.193+0000 7fb8de9e7640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:57:50.194 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:57:50.194+0000 7fb8de9e7640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:57:50.938 INFO:teuthology.orchestra.run.trial035.stderr:set osd.2 primary-affinity to 1 (8655362) 2026-02-24T07:57:50.978 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph osd reweight 3 1 2026-02-24T07:57:51.097 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:57:51.119 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:57:50 trial035 ceph-mon[39284]: pgmap v536: 1 pgs: 1 active+clean; 577 KiB data, 4.7 GiB used, 2.7 TiB / 2.7 TiB avail 2026-02-24T07:57:51.120 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:57:50 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/947237589' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 2, "weight": 1.0}]': finished 2026-02-24T07:57:51.120 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:57:50 trial035 ceph-mon[39284]: osdmap e118: 4 total, 4 up, 4 in 2026-02-24T07:57:51.120 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:57:50 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/3495730282' entity='client.admin' cmd={"prefix": "osd primary-affinity", "id": 2, "weight": 1.0} : dispatch 2026-02-24T07:57:51.198 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:57:51.197+0000 7f6d66f3f640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:57:51.198 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:57:51.197+0000 7f6d66f3f640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:57:51.942 INFO:teuthology.orchestra.run.trial035.stderr:reweighted osd.3 to 1 (10000) 2026-02-24T07:57:51.981 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph osd primary-affinity 3 1 2026-02-24T07:57:52.098 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:57:52.201 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:57:52.200+0000 7f36480a9640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:57:52.201 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:57:52.200+0000 7f36480a9640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:57:52.201 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:57:51 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/3495730282' entity='client.admin' cmd='[{"prefix": "osd primary-affinity", "id": 2, "weight": 1.0}]': finished 2026-02-24T07:57:52.201 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:57:51 trial035 ceph-mon[39284]: osdmap e119: 4 total, 4 up, 4 in 2026-02-24T07:57:52.201 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:57:51 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/935163609' entity='client.admin' cmd={"prefix": "osd reweight", "id": 3, "weight": 1.0} : dispatch 2026-02-24T07:57:52.201 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:57:51 trial035 ceph-mon[39284]: pgmap v539: 1 pgs: 1 active+clean; 577 KiB data, 4.7 GiB used, 2.7 TiB / 2.7 TiB avail 2026-02-24T07:57:52.942 INFO:teuthology.orchestra.run.trial035.stderr:set osd.3 primary-affinity to 1 (8655362) 2026-02-24T07:57:52.983 INFO:tasks.thrashosds.thrasher:joining the do_optrack_toggle greenlet 2026-02-24T07:57:52.983 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2026-02-24T07:57:52.984 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph osd dump --format=json 2026-02-24T07:57:53.101 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:57:53.202 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:57:53.200+0000 7f8afdfb4640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:57:53.202 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:57:53.201+0000 7f8afdfb4640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:57:53.202 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:57:52 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/935163609' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 3, "weight": 1.0}]': finished 2026-02-24T07:57:53.203 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:57:52 trial035 ceph-mon[39284]: osdmap e120: 4 total, 4 up, 4 in 2026-02-24T07:57:53.203 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:57:52 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/2396032516' entity='client.admin' cmd={"prefix": "osd primary-affinity", "id": 3, "weight": 1.0} : dispatch 2026-02-24T07:57:53.312 INFO:teuthology.orchestra.run.trial035.stdout: 2026-02-24T07:57:53.312 INFO:teuthology.orchestra.run.trial035.stdout:{"epoch":121,"fsid":"3a5983d2-1154-11f1-9c05-d404e6e7d460","created":"2026-02-24T07:41:42.805703+0000","modified":"2026-02-24T07:57:52.941233+0000","last_up_change":"2026-02-24T07:42:51.180844+0000","last_in_change":"2026-02-24T07:42:42.005822+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":10,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":4,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"tentacle","allow_crimson":true,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2026-02-24T07:42:42.898873+0000","flags":262145,"flags_names":"hashpspool,crimson","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"nonprimary_shards":"{}","options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":4,"score_stable":4,"optimal_score":0.75,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"8d5b0adf-0e02-40d4-8780-e64988c10e92","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.35:6802","nonce":2333513103},{"type":"v1","addr":"10.20.193.35:6803","nonce":2333513103}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.35:6804","nonce":2333513103},{"type":"v1","addr":"10.20.193.35:6805","nonce":2333513103}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.35:6808","nonce":2333513103},{"type":"v1","addr":"10.20.193.35:6809","nonce":2333513103}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.35:6806","nonce":2333513103},{"type":"v1","addr":"10.20.193.35:6807","nonce":2333513103}]},"public_addr":"10.20.193.35:6803/2333513103","cluster_addr":"10.20.193.35:6805/2333513103","heartbeat_back_addr":"10.20.193.35:6809/2333513103","heartbeat_front_addr":"10.20.193.35:6807/2333513103","state":["exists","up"]},{"osd":1,"uuid":"b9b77957-7aa8-49b2-95a7-b50dcfc6bc3e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.35:6810","nonce":4095255721},{"type":"v1","addr":"10.20.193.35:6811","nonce":4095255721}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.35:6812","nonce":4095255721},{"type":"v1","addr":"10.20.193.35:6813","nonce":4095255721}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.35:6816","nonce":4095255721},{"type":"v1","addr":"10.20.193.35:6817","nonce":4095255721}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.35:6814","nonce":4095255721},{"type":"v1","addr":"10.20.193.35:6815","nonce":4095255721}]},"public_addr":"10.20.193.35:6811/4095255721","cluster_addr":"10.20.193.35:6813/4095255721","heartbeat_back_addr":"10.20.193.35:6817/4095255721","heartbeat_front_addr":"10.20.193.35:6815/4095255721","state":["exists","up"]},{"osd":2,"uuid":"883dcd38-0b52-4889-90fe-44a624ba23cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.91:6800","nonce":1390012919},{"type":"v1","addr":"10.20.193.91:6801","nonce":1390012919}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.91:6802","nonce":1390012919},{"type":"v1","addr":"10.20.193.91:6803","nonce":1390012919}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.91:6806","nonce":1390012919},{"type":"v1","addr":"10.20.193.91:6807","nonce":1390012919}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.91:6804","nonce":1390012919},{"type":"v1","addr":"10.20.193.91:6805","nonce":1390012919}]},"public_addr":"10.20.193.91:6801/1390012919","cluster_addr":"10.20.193.91:6803/1390012919","heartbeat_back_addr":"10.20.193.91:6807/1390012919","heartbeat_front_addr":"10.20.193.91:6805/1390012919","state":["exists","up"]},{"osd":3,"uuid":"2582e7bb-0265-44e6-bf18-c68070be0dc7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.91:6808","nonce":940596907},{"type":"v1","addr":"10.20.193.91:6809","nonce":940596907}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.91:6810","nonce":940596907},{"type":"v1","addr":"10.20.193.91:6811","nonce":940596907}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.91:6814","nonce":940596907},{"type":"v1","addr":"10.20.193.91:6815","nonce":940596907}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.91:6812","nonce":940596907},{"type":"v1","addr":"10.20.193.91:6813","nonce":940596907}]},"public_addr":"10.20.193.91:6809/940596907","cluster_addr":"10.20.193.91:6811/940596907","heartbeat_back_addr":"10.20.193.91:6815/940596907","heartbeat_front_addr":"10.20.193.91:6813/940596907","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-24T07:57:16.635651+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-24T07:56:22.184982+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-24T07:56:57.057608+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-24T07:52:15.298413+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"10.20.193.35:0/1250927865":"2026-02-25T07:42:59.654301+0000","10.20.193.35:0/4081776025":"2026-02-25T07:42:59.654301+0000","10.20.193.35:6800/1865836941":"2026-02-25T07:41:53.007692+0000","10.20.193.35:0/1753799146":"2026-02-25T07:42:59.654301+0000","10.20.193.35:6800/3632488220":"2026-02-25T07:42:59.654301+0000","10.20.193.35:0/1760415898":"2026-02-25T07:42:02.689708+0000","10.20.193.35:0/1697559475":"2026-02-25T07:41:53.007692+0000","10.20.193.35:0/2628287835":"2026-02-25T07:42:02.689708+0000","10.20.193.35:0/2848450259":"2026-02-25T07:41:53.007692+0000","10.20.193.35:6801/1865836941":"2026-02-25T07:41:53.007692+0000","10.20.193.35:0/3517194207":"2026-02-25T07:41:53.007692+0000","10.20.193.35:0/2958395575":"2026-02-25T07:42:02.689708+0000","10.20.193.35:6800/432269861":"2026-02-25T07:42:02.689708+0000","10.20.193.35:6801/432269861":"2026-02-25T07:42:02.689708+0000","10.20.193.35:0/794704272":"2026-02-25T07:42:59.654301+0000","10.20.193.35:6801/3632488220":"2026-02-25T07:42:59.654301+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"isa","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2026-02-24T07:57:53.348 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2026-02-24T07:57:53.348 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph osd dump --format=json 2026-02-24T07:57:53.463 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:57:53.563 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:57:53.561+0000 7f98dc930640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:57:53.563 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:57:53.561+0000 7f98dc930640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:57:53.674 INFO:teuthology.orchestra.run.trial035.stdout: 2026-02-24T07:57:53.675 INFO:teuthology.orchestra.run.trial035.stdout:{"epoch":121,"fsid":"3a5983d2-1154-11f1-9c05-d404e6e7d460","created":"2026-02-24T07:41:42.805703+0000","modified":"2026-02-24T07:57:52.941233+0000","last_up_change":"2026-02-24T07:42:51.180844+0000","last_in_change":"2026-02-24T07:42:42.005822+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":10,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":4,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"tentacle","allow_crimson":true,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2026-02-24T07:42:42.898873+0000","flags":262145,"flags_names":"hashpspool,crimson","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"nonprimary_shards":"{}","options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":4,"score_stable":4,"optimal_score":0.75,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"8d5b0adf-0e02-40d4-8780-e64988c10e92","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.35:6802","nonce":2333513103},{"type":"v1","addr":"10.20.193.35:6803","nonce":2333513103}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.35:6804","nonce":2333513103},{"type":"v1","addr":"10.20.193.35:6805","nonce":2333513103}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.35:6808","nonce":2333513103},{"type":"v1","addr":"10.20.193.35:6809","nonce":2333513103}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.35:6806","nonce":2333513103},{"type":"v1","addr":"10.20.193.35:6807","nonce":2333513103}]},"public_addr":"10.20.193.35:6803/2333513103","cluster_addr":"10.20.193.35:6805/2333513103","heartbeat_back_addr":"10.20.193.35:6809/2333513103","heartbeat_front_addr":"10.20.193.35:6807/2333513103","state":["exists","up"]},{"osd":1,"uuid":"b9b77957-7aa8-49b2-95a7-b50dcfc6bc3e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.35:6810","nonce":4095255721},{"type":"v1","addr":"10.20.193.35:6811","nonce":4095255721}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.35:6812","nonce":4095255721},{"type":"v1","addr":"10.20.193.35:6813","nonce":4095255721}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.35:6816","nonce":4095255721},{"type":"v1","addr":"10.20.193.35:6817","nonce":4095255721}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.35:6814","nonce":4095255721},{"type":"v1","addr":"10.20.193.35:6815","nonce":4095255721}]},"public_addr":"10.20.193.35:6811/4095255721","cluster_addr":"10.20.193.35:6813/4095255721","heartbeat_back_addr":"10.20.193.35:6817/4095255721","heartbeat_front_addr":"10.20.193.35:6815/4095255721","state":["exists","up"]},{"osd":2,"uuid":"883dcd38-0b52-4889-90fe-44a624ba23cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.91:6800","nonce":1390012919},{"type":"v1","addr":"10.20.193.91:6801","nonce":1390012919}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.91:6802","nonce":1390012919},{"type":"v1","addr":"10.20.193.91:6803","nonce":1390012919}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.91:6806","nonce":1390012919},{"type":"v1","addr":"10.20.193.91:6807","nonce":1390012919}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.91:6804","nonce":1390012919},{"type":"v1","addr":"10.20.193.91:6805","nonce":1390012919}]},"public_addr":"10.20.193.91:6801/1390012919","cluster_addr":"10.20.193.91:6803/1390012919","heartbeat_back_addr":"10.20.193.91:6807/1390012919","heartbeat_front_addr":"10.20.193.91:6805/1390012919","state":["exists","up"]},{"osd":3,"uuid":"2582e7bb-0265-44e6-bf18-c68070be0dc7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.91:6808","nonce":940596907},{"type":"v1","addr":"10.20.193.91:6809","nonce":940596907}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.91:6810","nonce":940596907},{"type":"v1","addr":"10.20.193.91:6811","nonce":940596907}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.91:6814","nonce":940596907},{"type":"v1","addr":"10.20.193.91:6815","nonce":940596907}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.91:6812","nonce":940596907},{"type":"v1","addr":"10.20.193.91:6813","nonce":940596907}]},"public_addr":"10.20.193.91:6809/940596907","cluster_addr":"10.20.193.91:6811/940596907","heartbeat_back_addr":"10.20.193.91:6815/940596907","heartbeat_front_addr":"10.20.193.91:6813/940596907","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-24T07:57:16.635651+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-24T07:56:22.184982+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-24T07:56:57.057608+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-24T07:52:15.298413+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"10.20.193.35:0/1250927865":"2026-02-25T07:42:59.654301+0000","10.20.193.35:0/4081776025":"2026-02-25T07:42:59.654301+0000","10.20.193.35:6800/1865836941":"2026-02-25T07:41:53.007692+0000","10.20.193.35:0/1753799146":"2026-02-25T07:42:59.654301+0000","10.20.193.35:6800/3632488220":"2026-02-25T07:42:59.654301+0000","10.20.193.35:0/1760415898":"2026-02-25T07:42:02.689708+0000","10.20.193.35:0/1697559475":"2026-02-25T07:41:53.007692+0000","10.20.193.35:0/2628287835":"2026-02-25T07:42:02.689708+0000","10.20.193.35:0/2848450259":"2026-02-25T07:41:53.007692+0000","10.20.193.35:6801/1865836941":"2026-02-25T07:41:53.007692+0000","10.20.193.35:0/3517194207":"2026-02-25T07:41:53.007692+0000","10.20.193.35:0/2958395575":"2026-02-25T07:42:02.689708+0000","10.20.193.35:6800/432269861":"2026-02-25T07:42:02.689708+0000","10.20.193.35:6801/432269861":"2026-02-25T07:42:02.689708+0000","10.20.193.35:0/794704272":"2026-02-25T07:42:59.654301+0000","10.20.193.35:6801/3632488220":"2026-02-25T07:42:59.654301+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"isa","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2026-02-24T07:57:53.712 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph tell osd.0 flush_pg_stats 2026-02-24T07:57:53.713 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph tell osd.1 flush_pg_stats 2026-02-24T07:57:53.713 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph tell osd.2 flush_pg_stats 2026-02-24T07:57:53.714 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph tell osd.3 flush_pg_stats 2026-02-24T07:57:53.833 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:57:53.834 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:57:53.834 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:57:53.835 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:57:53.946 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:57:53.944+0000 7fa169b3b640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:57:53.946 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:57:53.944+0000 7fe86ab5b640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:57:53.946 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:57:53.945+0000 7fa169b3b640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:57:53.947 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:57:53.945+0000 7fe86ab5b640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:57:53.947 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:57:53.946+0000 7fe453eae640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:57:53.948 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:57:53.947+0000 7fe453eae640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:57:53.949 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:57:53.948+0000 7f876a77f640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:57:53.950 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:57:53.949+0000 7f876a77f640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:57:53.966 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:57:53 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/2396032516' entity='client.admin' cmd='[{"prefix": "osd primary-affinity", "id": 3, "weight": 1.0}]': finished 2026-02-24T07:57:53.966 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:57:53 trial035 ceph-mon[39284]: osdmap e121: 4 total, 4 up, 4 in 2026-02-24T07:57:53.966 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:57:53 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/3710042961' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2026-02-24T07:57:53.966 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:57:53 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/3185937339' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2026-02-24T07:57:53.966 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:57:53 trial035 ceph-mon[39284]: pgmap v542: 1 pgs: 1 active+clean; 577 KiB data, 4.7 GiB used, 2.7 TiB / 2.7 TiB avail 2026-02-24T07:57:54.008 INFO:teuthology.orchestra.run.trial035.stdout:81604378809 2026-02-24T07:57:54.009 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph osd last-stat-seq osd.2 2026-02-24T07:57:54.010 INFO:teuthology.orchestra.run.trial035.stdout:38654705853 2026-02-24T07:57:54.011 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph osd last-stat-seq osd.0 2026-02-24T07:57:54.013 INFO:teuthology.orchestra.run.trial035.stdout:107374182583 2026-02-24T07:57:54.013 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph osd last-stat-seq osd.3 2026-02-24T07:57:54.019 INFO:teuthology.orchestra.run.trial035.stdout:60129542330 2026-02-24T07:57:54.019 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph osd last-stat-seq osd.1 2026-02-24T07:57:54.126 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:57:54.175 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:57:54.176 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:57:54.224 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:57:54.230 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:57:54.229+0000 7f26cd6db640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:57:54.230 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:57:54.230+0000 7f26cd6db640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:57:54.283 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:57:54.282+0000 7f54ffeaa640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:57:54.283 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:57:54.282+0000 7f54ffeaa640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:57:54.284 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:57:54.282+0000 7ff4b0db4640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:57:54.284 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:57:54.282+0000 7ff4b0db4640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:57:54.330 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:57:54.329+0000 7efebfe1d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:57:54.331 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:57:54.329+0000 7efebfe1d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:57:54.345 INFO:teuthology.orchestra.run.trial035.stdout:81604378808 2026-02-24T07:57:54.388 INFO:tasks.cephadm.ceph_manager.ceph:need seq 81604378809 got 81604378808 for osd.2 2026-02-24T07:57:54.400 INFO:teuthology.orchestra.run.trial035.stdout:107374182582 2026-02-24T07:57:54.401 INFO:teuthology.orchestra.run.trial035.stdout:38654705852 2026-02-24T07:57:54.442 INFO:tasks.cephadm.ceph_manager.ceph:need seq 107374182583 got 107374182582 for osd.3 2026-02-24T07:57:54.445 INFO:tasks.cephadm.ceph_manager.ceph:need seq 38654705853 got 38654705852 for osd.0 2026-02-24T07:57:54.448 INFO:teuthology.orchestra.run.trial035.stdout:60129542329 2026-02-24T07:57:54.489 INFO:tasks.cephadm.ceph_manager.ceph:need seq 60129542330 got 60129542329 for osd.1 2026-02-24T07:57:55.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:57:54 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/1369120624' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 2} : dispatch 2026-02-24T07:57:55.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:57:54 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/3575677743' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 0} : dispatch 2026-02-24T07:57:55.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:57:54 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/3875971856' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 3} : dispatch 2026-02-24T07:57:55.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:57:54 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/3034662732' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 1} : dispatch 2026-02-24T07:57:55.389 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph osd last-stat-seq osd.2 2026-02-24T07:57:55.444 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph osd last-stat-seq osd.3 2026-02-24T07:57:55.446 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph osd last-stat-seq osd.0 2026-02-24T07:57:55.490 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph osd last-stat-seq osd.1 2026-02-24T07:57:55.509 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:57:55.562 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:57:55.565 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:57:55.608 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:57:55.626 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:57:55.626+0000 7fd7013c6640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:57:55.627 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:57:55.626+0000 7fd7013c6640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:57:55.675 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:57:55.674+0000 7f56f9d0b640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:57:55.676 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:57:55.674+0000 7f56f9d0b640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:57:55.680 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:57:55.678+0000 7f3fb53cb640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:57:55.680 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:57:55.679+0000 7f3fb53cb640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:57:55.701 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:57:55.700+0000 7f84cac39640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:57:55.701 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:57:55.700+0000 7f84cac39640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:57:55.740 INFO:teuthology.orchestra.run.trial035.stdout:81604378808 2026-02-24T07:57:55.780 INFO:tasks.cephadm.ceph_manager.ceph:need seq 81604378809 got 81604378808 for osd.2 2026-02-24T07:57:55.788 INFO:teuthology.orchestra.run.trial035.stdout:107374182582 2026-02-24T07:57:55.796 INFO:teuthology.orchestra.run.trial035.stdout:38654705852 2026-02-24T07:57:55.815 INFO:teuthology.orchestra.run.trial035.stdout:60129542329 2026-02-24T07:57:55.832 INFO:tasks.cephadm.ceph_manager.ceph:need seq 107374182583 got 107374182582 for osd.3 2026-02-24T07:57:55.839 INFO:tasks.cephadm.ceph_manager.ceph:need seq 38654705853 got 38654705852 for osd.0 2026-02-24T07:57:55.854 INFO:tasks.cephadm.ceph_manager.ceph:need seq 60129542330 got 60129542329 for osd.1 2026-02-24T07:57:56.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:57:55 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/1942841163' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 2} : dispatch 2026-02-24T07:57:56.204 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:57:55 trial035 ceph-mon[39284]: pgmap v543: 1 pgs: 1 active+clean; 577 KiB data, 4.7 GiB used, 2.7 TiB / 2.7 TiB avail 2026-02-24T07:57:56.205 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:57:55 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/3048443945' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 3} : dispatch 2026-02-24T07:57:56.205 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:57:55 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/3464069326' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 0} : dispatch 2026-02-24T07:57:56.205 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:57:55 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/1324680903' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 1} : dispatch 2026-02-24T07:57:56.781 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph osd last-stat-seq osd.2 2026-02-24T07:57:56.833 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph osd last-stat-seq osd.3 2026-02-24T07:57:56.839 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph osd last-stat-seq osd.0 2026-02-24T07:57:56.855 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph osd last-stat-seq osd.1 2026-02-24T07:57:56.900 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:57:56.949 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:57:56.998 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:57:56.998 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:57:57.015 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:57:57.014+0000 7f204ad1f640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:57:57.016 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:57:57.014+0000 7f204ad1f640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:57:57.058 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:57:56 trial035 ceph-mon[39284]: osdmap e122: 4 total, 4 up, 4 in 2026-02-24T07:57:57.058 INFO:journalctl@ceph.mgr.x.trial035.stdout:Feb 24 07:57:56 trial035 ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-mgr-x[39621]: ::ffff:10.20.193.91 - - [24/Feb/2026:07:57:56] "GET /metrics HTTP/1.1" 200 3893 "" "Prometheus/3.6.0" 2026-02-24T07:57:57.065 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:57:57.063+0000 7fd0cbfce640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:57:57.065 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:57:57.064+0000 7fd0cbfce640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:57:57.101 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:57:57.100+0000 7f048bd40640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:57:57.101 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:57:57.100+0000 7f048bd40640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:57:57.102 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:57:57.100+0000 7f211cccb640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:57:57.102 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:57:57.100+0000 7f211cccb640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:57:57.131 INFO:teuthology.orchestra.run.trial035.stdout:81604378809 2026-02-24T07:57:57.172 INFO:tasks.cephadm.ceph_manager.ceph:need seq 81604378809 got 81604378809 for osd.2 2026-02-24T07:57:57.172 DEBUG:teuthology.parallel:result is None 2026-02-24T07:57:57.179 INFO:teuthology.orchestra.run.trial035.stdout:107374182583 2026-02-24T07:57:57.216 INFO:teuthology.orchestra.run.trial035.stdout:38654705853 2026-02-24T07:57:57.220 INFO:tasks.cephadm.ceph_manager.ceph:need seq 107374182583 got 107374182583 for osd.3 2026-02-24T07:57:57.220 DEBUG:teuthology.parallel:result is None 2026-02-24T07:57:57.221 INFO:teuthology.orchestra.run.trial035.stdout:60129542330 2026-02-24T07:57:57.255 INFO:tasks.cephadm.ceph_manager.ceph:need seq 38654705853 got 38654705853 for osd.0 2026-02-24T07:57:57.256 DEBUG:teuthology.parallel:result is None 2026-02-24T07:57:57.282 INFO:tasks.cephadm.ceph_manager.ceph:need seq 60129542330 got 60129542330 for osd.1 2026-02-24T07:57:57.282 DEBUG:teuthology.parallel:result is None 2026-02-24T07:57:57.282 INFO:tasks.cephadm.ceph_manager.ceph:waiting for recovery to complete 2026-02-24T07:57:57.283 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph pg dump --format=json 2026-02-24T07:57:57.356 INFO:journalctl@ceph.prometheus.a.trial091.stdout:Feb 24 07:57:56 trial091 ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-prometheus-a[66007]: time=2026-02-24T07:57:56.973Z level=ERROR source=scrape.go:1630 msg="Failed to determine correct type of scrape target." component="scrape manager" scrape_pool=ceph target=http://trial035:9283/metrics content_type="text/html;charset=utf-8" fallback_media_type="" err="received unsupported Content-Type \"text/html;charset=utf-8\" and no fallback_scrape_protocol specified for target" 2026-02-24T07:57:57.438 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:57:57.538 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:57:57.537+0000 7f6c42408640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:57:57.538 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:57:57.537+0000 7f6c42408640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:57:57.650 INFO:teuthology.orchestra.run.trial035.stdout: 2026-02-24T07:57:57.650 INFO:teuthology.orchestra.run.trial035.stderr:dumped all 2026-02-24T07:57:57.686 INFO:teuthology.orchestra.run.trial035.stdout:{"pg_ready":true,"pg_map":{"version":544,"stamp":"2026-02-24T07:57:55.949659+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":240,"num_write":135,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":3,"acting":3,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":3,"num_osds":4,"num_per_pool_osds":4,"num_per_pool_omap_osds":3,"kb":2936012800,"kb_used":4962092,"kb_used_data":4252,"kb_used_omap":133,"kb_used_meta":4957626,"kb_avail":2931050708,"statfs":{"total":3006477107200,"available":3001395924992,"internally_reserved":0,"allocated":4354048,"data_stored":2738612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":136577,"internal_metadata":5076609663},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.011513"},"pg_stats":[{"pgid":"1.0","version":"29'99","reported_seq":455,"reported_epoch":121,"state":"active+clean","last_fresh":"2026-02-24T07:57:55.171446+0000","last_change":"2026-02-24T07:56:33.056711+0000","last_active":"2026-02-24T07:57:55.171446+0000","last_peered":"2026-02-24T07:57:55.171446+0000","last_clean":"2026-02-24T07:57:55.171446+0000","last_became_active":"2026-02-24T07:42:54.186719+0000","last_became_peered":"2026-02-24T07:42:54.186719+0000","last_unstale":"2026-02-24T07:57:55.171446+0000","last_undegraded":"2026-02-24T07:57:55.171446+0000","last_fullsized":"2026-02-24T07:57:55.171446+0000","mapping_epoch":27,"log_start":"29'96","ondisk_log_start":"29'96","created":20,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'99","last_scrub_stamp":"2026-02-24T07:56:33.056678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:42:43.640776+0000","last_clean_scrub_stamp":"2026-02-24T07:56:33.056678+0000","objects_scrubbed":2,"log_size":3,"log_dups_size":96,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:57:56.381501+0000","scrub_duration":21,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":240,"num_write":135,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":240,"num_write":135,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1682,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":3,"up_from":25,"seq":107374182584,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":1153676,"kb_used_data":1208,"kb_used_omap":21,"kb_used_meta":1152426,"kb_avail":732849524,"statfs":{"total":751619276800,"available":750437912576,"internally_reserved":0,"allocated":1236992,"data_stored":832245,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":21619,"internal_metadata":1180085133},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":19,"seq":81604378810,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":1452620,"kb_used_data":1208,"kb_used_omap":38,"kb_used_meta":1451353,"kb_avail":732550580,"statfs":{"total":751619276800,"available":750131793920,"internally_reserved":0,"allocated":1236992,"data_stored":832245,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":39042,"internal_metadata":1486186366},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542331,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":734003200,"kb_used":1107848,"kb_used_data":628,"kb_used_omap":38,"kb_used_meta":1107161,"kb_avail":732895352,"statfs":{"total":751619276800,"available":750484840448,"internally_reserved":0,"allocated":643072,"data_stored":241877,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":39633,"internal_metadata":1133733167},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705854,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":1247948,"kb_used_data":1208,"kb_used_omap":35,"kb_used_meta":1246684,"kb_avail":732755252,"statfs":{"total":751619276800,"available":750341378048,"internally_reserved":0,"allocated":1236992,"data_stored":832245,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":36283,"internal_metadata":1276604997},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1682,"internal_metadata":0}]}} 2026-02-24T07:57:57.687 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph pg dump --format=json 2026-02-24T07:57:57.803 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:57:57.904 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:57:57.903+0000 7f869f06a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:57:57.904 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:57:57.903+0000 7f869f06a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:57:58.015 INFO:teuthology.orchestra.run.trial035.stdout: 2026-02-24T07:57:58.015 INFO:teuthology.orchestra.run.trial035.stderr:dumped all 2026-02-24T07:57:58.053 INFO:teuthology.orchestra.run.trial035.stdout:{"pg_ready":true,"pg_map":{"version":545,"stamp":"2026-02-24T07:57:57.779435+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":240,"num_write":135,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":3,"acting":3,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":3,"num_osds":4,"num_per_pool_osds":4,"num_per_pool_omap_osds":3,"kb":2936012800,"kb_used":4962092,"kb_used_data":4252,"kb_used_omap":133,"kb_used_meta":4957626,"kb_avail":2931050708,"statfs":{"total":3006477107200,"available":3001395924992,"internally_reserved":0,"allocated":4354048,"data_stored":2738612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":136577,"internal_metadata":5076609663},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.000805"},"pg_stats":[{"pgid":"1.0","version":"29'99","reported_seq":455,"reported_epoch":121,"state":"active+clean","last_fresh":"2026-02-24T07:57:55.171446+0000","last_change":"2026-02-24T07:56:33.056711+0000","last_active":"2026-02-24T07:57:55.171446+0000","last_peered":"2026-02-24T07:57:55.171446+0000","last_clean":"2026-02-24T07:57:55.171446+0000","last_became_active":"2026-02-24T07:42:54.186719+0000","last_became_peered":"2026-02-24T07:42:54.186719+0000","last_unstale":"2026-02-24T07:57:55.171446+0000","last_undegraded":"2026-02-24T07:57:55.171446+0000","last_fullsized":"2026-02-24T07:57:55.171446+0000","mapping_epoch":27,"log_start":"29'96","ondisk_log_start":"29'96","created":20,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'99","last_scrub_stamp":"2026-02-24T07:56:33.056678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T07:42:43.640776+0000","last_clean_scrub_stamp":"2026-02-24T07:56:33.056678+0000","objects_scrubbed":2,"log_size":3,"log_dups_size":96,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:57:56.381501+0000","scrub_duration":21,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":240,"num_write":135,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":240,"num_write":135,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1682,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":3,"up_from":25,"seq":107374182584,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":1153676,"kb_used_data":1208,"kb_used_omap":21,"kb_used_meta":1152426,"kb_avail":732849524,"statfs":{"total":751619276800,"available":750437912576,"internally_reserved":0,"allocated":1236992,"data_stored":832245,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":21619,"internal_metadata":1180085133},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":19,"seq":81604378810,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":1452620,"kb_used_data":1208,"kb_used_omap":38,"kb_used_meta":1451353,"kb_avail":732550580,"statfs":{"total":751619276800,"available":750131793920,"internally_reserved":0,"allocated":1236992,"data_stored":832245,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":39042,"internal_metadata":1486186366},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542331,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":734003200,"kb_used":1107848,"kb_used_data":628,"kb_used_omap":38,"kb_used_meta":1107161,"kb_avail":732895352,"statfs":{"total":751619276800,"available":750484840448,"internally_reserved":0,"allocated":643072,"data_stored":241877,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":39633,"internal_metadata":1133733167},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705854,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":1247948,"kb_used_data":1208,"kb_used_omap":35,"kb_used_meta":1246684,"kb_avail":732755252,"statfs":{"total":751619276800,"available":750341378048,"internally_reserved":0,"allocated":1236992,"data_stored":832245,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":36283,"internal_metadata":1276604997},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1682,"internal_metadata":0}]}} 2026-02-24T07:57:58.054 INFO:tasks.cephadm.ceph_manager.ceph:recovered! 2026-02-24T07:57:58.054 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph config rm mgr mgr_debug_aggressive_pg_num_changes 2026-02-24T07:57:58.168 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:57:58.190 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:57:57 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/172545611' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 2} : dispatch 2026-02-24T07:57:58.190 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:57:57 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/1880777554' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 3} : dispatch 2026-02-24T07:57:58.190 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:57:57 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/276813362' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 0} : dispatch 2026-02-24T07:57:58.190 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:57:57 trial035 ceph-mon[39284]: from='client.? 10.20.193.35:0/3436801941' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 1} : dispatch 2026-02-24T07:57:58.190 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:57:57 trial035 ceph-mon[39284]: from='client.15743 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2026-02-24T07:57:58.190 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:57:57 trial035 ceph-mon[39284]: pgmap v545: 1 pgs: 1 active+clean; 577 KiB data, 4.7 GiB used, 2.7 TiB / 2.7 TiB avail 2026-02-24T07:57:58.276 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:57:58.275+0000 7f9486443640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:57:58.277 INFO:teuthology.orchestra.run.trial035.stderr:2026-02-24T07:57:58.276+0000 7f9486443640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-24T07:57:58.422 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2026-02-24T07:57:58.436 INFO:tasks.cephadm:Teardown begin 2026-02-24T07:57:58.436 DEBUG:teuthology.orchestra.run.trial035:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2026-02-24T07:57:58.461 DEBUG:teuthology.orchestra.run.trial091:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2026-02-24T07:57:58.489 INFO:tasks.cephadm:Disabling cephadm mgr module 2026-02-24T07:57:58.489 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:ff63f3d997e38f69382fd23ac0206f07f7b05017 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 -- ceph mgr module disable cephadm 2026-02-24T07:57:58.614 INFO:teuthology.orchestra.run.trial035.stderr:Inferring config /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/mon.a/config 2026-02-24T07:57:58.633 INFO:teuthology.orchestra.run.trial035.stderr:Error: statfs /etc/ceph/ceph.client.admin.keyring: no such file or directory 2026-02-24T07:57:58.649 DEBUG:teuthology.orchestra.run:got remote process result: 125 2026-02-24T07:57:58.650 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2026-02-24T07:57:58.650 DEBUG:teuthology.orchestra.run.trial035:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2026-02-24T07:57:58.668 DEBUG:teuthology.orchestra.run.trial091:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2026-02-24T07:57:58.686 INFO:tasks.cephadm:Stopping all daemons... 2026-02-24T07:57:58.687 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2026-02-24T07:57:58.687 DEBUG:teuthology.orchestra.run.trial035:> sudo systemctl stop ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460@mon.a 2026-02-24T07:57:58.729 INFO:journalctl@ceph.mgr.x.trial035.stdout:Feb 24 07:57:58 trial035 ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-mgr-x[39621]: [24/Feb/2026:07:57:58] ENGINE Bus STOPPING 2026-02-24T07:57:58.983 INFO:journalctl@ceph.mgr.x.trial035.stdout:Feb 24 07:57:58 trial035 ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-mgr-x[39621]: [24/Feb/2026:07:57:58] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283)) shut down 2026-02-24T07:57:58.983 INFO:journalctl@ceph.mgr.x.trial035.stdout:Feb 24 07:57:58 trial035 ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-mgr-x[39621]: [24/Feb/2026:07:57:58] ENGINE Bus STOPPED 2026-02-24T07:57:58.983 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:57:58 trial035 systemd[1]: Stopping Ceph mon.a for 3a5983d2-1154-11f1-9c05-d404e6e7d460... 2026-02-24T07:57:58.983 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:57:58 trial035 ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-mon-a[39280]: 2026-02-24T07:57:58.806+0000 7f4604e8c640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2026-02-24T07:57:58.983 INFO:journalctl@ceph.mon.a.trial035.stdout:Feb 24 07:57:58 trial035 ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-mon-a[39280]: 2026-02-24T07:57:58.806+0000 7f4604e8c640 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2026-02-24T07:57:59.092 DEBUG:teuthology.orchestra.run.trial035:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460@mon.a.service' 2026-02-24T07:57:59.170 DEBUG:teuthology.orchestra.run:got remote process result: None 2026-02-24T07:57:59.170 INFO:tasks.cephadm.mon.a:Stopped mon.a 2026-02-24T07:57:59.171 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2026-02-24T07:57:59.171 DEBUG:teuthology.orchestra.run.trial035:> sudo systemctl stop ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460@mgr.x 2026-02-24T07:57:59.442 DEBUG:teuthology.orchestra.run.trial035:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460@mgr.x.service' 2026-02-24T07:57:59.477 DEBUG:teuthology.orchestra.run:got remote process result: None 2026-02-24T07:57:59.478 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2026-02-24T07:57:59.478 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2026-02-24T07:57:59.478 DEBUG:teuthology.orchestra.run.trial035:> sudo systemctl stop ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460@osd.0 2026-02-24T07:57:59.954 INFO:journalctl@ceph.osd.0.trial035.stdout:Feb 24 07:57:59 trial035 systemd[1]: Stopping Ceph osd.0 for 3a5983d2-1154-11f1-9c05-d404e6e7d460... 2026-02-24T07:57:59.954 INFO:journalctl@ceph.osd.0.trial035.stdout:Feb 24 07:57:59 trial035 ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-osd-0[55839]: 2026-02-24T07:57:59.613+0000 7fd8d69c5640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --osd-objectstore=bluestore (PID: 1) UID: 0 2026-02-24T07:57:59.954 INFO:journalctl@ceph.osd.0.trial035.stdout:Feb 24 07:57:59 trial035 ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-osd-0[55839]: 2026-02-24T07:57:59.613+0000 7fd8d69c5640 -1 osd.0 122 *** Got signal Terminated *** 2026-02-24T07:57:59.954 INFO:journalctl@ceph.osd.0.trial035.stdout:Feb 24 07:57:59 trial035 ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-osd-0[55839]: 2026-02-24T07:57:59.613+0000 7fd8d69c5640 -1 osd.0 122 *** Immediate shutdown (osd_fast_shutdown=true) *** 2026-02-24T07:58:05.106 INFO:journalctl@ceph.prometheus.a.trial091.stdout:Feb 24 07:58:04 trial091 ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-prometheus-a[66007]: time=2026-02-24T07:58:04.844Z level=ERROR source=refresh.go:90 msg="Unable to refresh target groups" component="discovery manager scrape" discovery=http config=node-exporter err="Get \"http://10.20.193.35:8765/sd/prometheus/sd-config?service=node-exporter\": dial tcp 10.20.193.35:8765: connect: connection refused" 2026-02-24T07:58:05.106 INFO:journalctl@ceph.prometheus.a.trial091.stdout:Feb 24 07:58:04 trial091 ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-prometheus-a[66007]: time=2026-02-24T07:58:04.844Z level=ERROR source=refresh.go:90 msg="Unable to refresh target groups" component="discovery manager scrape" discovery=http config=ceph err="Get \"http://10.20.193.35:8765/sd/prometheus/sd-config?service=ceph\": dial tcp 10.20.193.35:8765: connect: connection refused" 2026-02-24T07:58:05.204 INFO:journalctl@ceph.osd.0.trial035.stdout:Feb 24 07:58:04 trial035 podman[153614]: 2026-02-24 07:58:04.946899752 +0000 UTC m=+5.345084014 container died 1a909b31430d8e9c3ba44c47bae1cf87d67860fc995b33cbb34efa9746e5a53c (image=quay.ceph.io/ceph-ci/ceph@sha256:9a5db51bca6d139691bb8c7054be8834f3ba065d1c0aaaecbafb190e389fb148, name=ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-osd-0, org.label-schema.license=GPLv2, CEPH_REF=wip-shraddhaag-enable-cephadm-tests, ceph=True, CEPH_SHA1=ff63f3d997e38f69382fd23ac0206f07f7b05017, OSD_FLAVOR=default, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_GIT_REPO=https://github.com/ceph/ceph-ci.git, org.opencontainers.image.documentation=https://docs.ceph.com/, GANESHA_REPO_BASEURL=https://buildlogs.centos.org/centos/$releasever-stream/storage/$basearch/nfsganesha-5/, io.buildah.version=1.33.7, org.label-schema.schema-version=1.0, org.label-schema.build-date=20260223, org.label-schema.vendor=CentOS, org.opencontainers.image.authors=Ceph Release Team , FROM_IMAGE=quay.io/centos/centos:stream9) 2026-02-24T07:58:05.204 INFO:journalctl@ceph.osd.0.trial035.stdout:Feb 24 07:58:04 trial035 podman[153614]: 2026-02-24 07:58:04.958742881 +0000 UTC m=+5.356927143 container remove 1a909b31430d8e9c3ba44c47bae1cf87d67860fc995b33cbb34efa9746e5a53c (image=quay.ceph.io/ceph-ci/ceph@sha256:9a5db51bca6d139691bb8c7054be8834f3ba065d1c0aaaecbafb190e389fb148, name=ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-osd-0, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.opencontainers.image.authors=Ceph Release Team , org.label-schema.vendor=CentOS, org.label-schema.build-date=20260223, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_SHA1=ff63f3d997e38f69382fd23ac0206f07f7b05017, org.opencontainers.image.documentation=https://docs.ceph.com/, io.buildah.version=1.33.7, CEPH_GIT_REPO=https://github.com/ceph/ceph-ci.git, GANESHA_REPO_BASEURL=https://buildlogs.centos.org/centos/$releasever-stream/storage/$basearch/nfsganesha-5/, OSD_FLAVOR=default, CEPH_REF=wip-shraddhaag-enable-cephadm-tests, FROM_IMAGE=quay.io/centos/centos:stream9) 2026-02-24T07:58:05.204 INFO:journalctl@ceph.osd.0.trial035.stdout:Feb 24 07:58:04 trial035 bash[153614]: ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-osd-0 2026-02-24T07:58:05.204 INFO:journalctl@ceph.osd.0.trial035.stdout:Feb 24 07:58:05 trial035 podman[153747]: 2026-02-24 07:58:05.085777023 +0000 UTC m=+0.015983628 container create a998cbce3942039e3f6a3c1f542620c6eda8338bcdd83f65c520560005425903 (image=quay.ceph.io/ceph-ci/ceph@sha256:9a5db51bca6d139691bb8c7054be8834f3ba065d1c0aaaecbafb190e389fb148, name=ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-osd-0-deactivate, org.opencontainers.image.authors=Ceph Release Team , CEPH_REF=wip-shraddhaag-enable-cephadm-tests, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_SHA1=ff63f3d997e38f69382fd23ac0206f07f7b05017, org.label-schema.build-date=20260223, org.label-schema.name=CentOS Stream 9 Base Image, org.opencontainers.image.documentation=https://docs.ceph.com/, CEPH_GIT_REPO=https://github.com/ceph/ceph-ci.git, org.label-schema.vendor=CentOS, ceph=True, FROM_IMAGE=quay.io/centos/centos:stream9, GANESHA_REPO_BASEURL=https://buildlogs.centos.org/centos/$releasever-stream/storage/$basearch/nfsganesha-5/, OSD_FLAVOR=default, io.buildah.version=1.33.7) 2026-02-24T07:58:05.204 INFO:journalctl@ceph.osd.0.trial035.stdout:Feb 24 07:58:05 trial035 podman[153747]: 2026-02-24 07:58:05.110675614 +0000 UTC m=+0.040882239 container init a998cbce3942039e3f6a3c1f542620c6eda8338bcdd83f65c520560005425903 (image=quay.ceph.io/ceph-ci/ceph@sha256:9a5db51bca6d139691bb8c7054be8834f3ba065d1c0aaaecbafb190e389fb148, name=ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-osd-0-deactivate, ceph=True, org.opencontainers.image.documentation=https://docs.ceph.com/, CEPH_GIT_REPO=https://github.com/ceph/ceph-ci.git, org.label-schema.build-date=20260223, CEPH_REF=wip-shraddhaag-enable-cephadm-tests, org.label-schema.license=GPLv2, FROM_IMAGE=quay.io/centos/centos:stream9, org.label-schema.vendor=CentOS, GANESHA_REPO_BASEURL=https://buildlogs.centos.org/centos/$releasever-stream/storage/$basearch/nfsganesha-5/, CEPH_SHA1=ff63f3d997e38f69382fd23ac0206f07f7b05017, org.opencontainers.image.authors=Ceph Release Team , OSD_FLAVOR=default, io.buildah.version=1.33.7, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0) 2026-02-24T07:58:05.204 INFO:journalctl@ceph.osd.0.trial035.stdout:Feb 24 07:58:05 trial035 podman[153747]: 2026-02-24 07:58:05.113191302 +0000 UTC m=+0.043397917 container start a998cbce3942039e3f6a3c1f542620c6eda8338bcdd83f65c520560005425903 (image=quay.ceph.io/ceph-ci/ceph@sha256:9a5db51bca6d139691bb8c7054be8834f3ba065d1c0aaaecbafb190e389fb148, name=ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-osd-0-deactivate, org.opencontainers.image.documentation=https://docs.ceph.com/, org.label-schema.license=GPLv2, FROM_IMAGE=quay.io/centos/centos:stream9, org.label-schema.schema-version=1.0, CEPH_REF=wip-shraddhaag-enable-cephadm-tests, ceph=True, org.label-schema.build-date=20260223, OSD_FLAVOR=default, org.opencontainers.image.authors=Ceph Release Team , io.buildah.version=1.33.7, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_GIT_REPO=https://github.com/ceph/ceph-ci.git, GANESHA_REPO_BASEURL=https://buildlogs.centos.org/centos/$releasever-stream/storage/$basearch/nfsganesha-5/, CEPH_SHA1=ff63f3d997e38f69382fd23ac0206f07f7b05017, org.label-schema.vendor=CentOS) 2026-02-24T07:58:05.204 INFO:journalctl@ceph.osd.0.trial035.stdout:Feb 24 07:58:05 trial035 podman[153747]: 2026-02-24 07:58:05.113711208 +0000 UTC m=+0.043917823 container attach a998cbce3942039e3f6a3c1f542620c6eda8338bcdd83f65c520560005425903 (image=quay.ceph.io/ceph-ci/ceph@sha256:9a5db51bca6d139691bb8c7054be8834f3ba065d1c0aaaecbafb190e389fb148, name=ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-osd-0-deactivate, org.opencontainers.image.documentation=https://docs.ceph.com/, FROM_IMAGE=quay.io/centos/centos:stream9, org.label-schema.schema-version=1.0, GANESHA_REPO_BASEURL=https://buildlogs.centos.org/centos/$releasever-stream/storage/$basearch/nfsganesha-5/, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2, org.opencontainers.image.authors=Ceph Release Team , org.label-schema.build-date=20260223, OSD_FLAVOR=default, CEPH_GIT_REPO=https://github.com/ceph/ceph-ci.git, CEPH_SHA1=ff63f3d997e38f69382fd23ac0206f07f7b05017, CEPH_REF=wip-shraddhaag-enable-cephadm-tests, io.buildah.version=1.33.7, org.label-schema.name=CentOS Stream 9 Base Image) 2026-02-24T07:58:05.204 INFO:journalctl@ceph.osd.0.trial035.stdout:Feb 24 07:58:05 trial035 podman[153747]: 2026-02-24 07:58:05.079221685 +0000 UTC m=+0.009428300 image pull 3992f70354a3fdc3bbd8e07cbd698e424b71669298ffe35e8f0565a46973aa85 quay.ceph.io/ceph-ci/ceph@sha256:9a5db51bca6d139691bb8c7054be8834f3ba065d1c0aaaecbafb190e389fb148 2026-02-24T07:58:05.275 DEBUG:teuthology.orchestra.run.trial035:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460@osd.0.service' 2026-02-24T07:58:05.311 DEBUG:teuthology.orchestra.run:got remote process result: None 2026-02-24T07:58:05.311 INFO:tasks.cephadm.osd.0:Stopped osd.0 2026-02-24T07:58:05.312 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2026-02-24T07:58:05.312 DEBUG:teuthology.orchestra.run.trial035:> sudo systemctl stop ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460@osd.1 2026-02-24T07:58:05.704 INFO:journalctl@ceph.osd.1.trial035.stdout:Feb 24 07:58:05 trial035 systemd[1]: Stopping Ceph osd.1 for 3a5983d2-1154-11f1-9c05-d404e6e7d460... 2026-02-24T07:58:05.705 INFO:journalctl@ceph.osd.1.trial035.stdout:Feb 24 07:58:05 trial035 ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-osd-1[62992]: 2026-02-24T07:58:05.447+0000 7f9e1d092640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --osd-objectstore=bluestore (PID: 1) UID: 0 2026-02-24T07:58:05.705 INFO:journalctl@ceph.osd.1.trial035.stdout:Feb 24 07:58:05 trial035 ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-osd-1[62992]: 2026-02-24T07:58:05.447+0000 7f9e1d092640 -1 osd.1 122 *** Got signal Terminated *** 2026-02-24T07:58:05.705 INFO:journalctl@ceph.osd.1.trial035.stdout:Feb 24 07:58:05 trial035 ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-osd-1[62992]: 2026-02-24T07:58:05.447+0000 7f9e1d092640 -1 osd.1 122 *** Immediate shutdown (osd_fast_shutdown=true) *** 2026-02-24T07:58:11.121 INFO:journalctl@ceph.osd.1.trial035.stdout:Feb 24 07:58:10 trial035 podman[153882]: 2026-02-24 07:58:10.821058964 +0000 UTC m=+5.384544180 container died 24045a3a2d0a7043e319958d698b70dcb650699e2617668c192f61877d77386b (image=quay.ceph.io/ceph-ci/ceph@sha256:9a5db51bca6d139691bb8c7054be8834f3ba065d1c0aaaecbafb190e389fb148, name=ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-osd-1, GANESHA_REPO_BASEURL=https://buildlogs.centos.org/centos/$releasever-stream/storage/$basearch/nfsganesha-5/, org.opencontainers.image.authors=Ceph Release Team , ceph=True, io.buildah.version=1.33.7, org.opencontainers.image.documentation=https://docs.ceph.com/, CEPH_GIT_REPO=https://github.com/ceph/ceph-ci.git, FROM_IMAGE=quay.io/centos/centos:stream9, org.label-schema.build-date=20260223, OSD_FLAVOR=default, org.label-schema.schema-version=1.0, CEPH_SHA1=ff63f3d997e38f69382fd23ac0206f07f7b05017, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_REF=wip-shraddhaag-enable-cephadm-tests) 2026-02-24T07:58:11.121 INFO:journalctl@ceph.osd.1.trial035.stdout:Feb 24 07:58:10 trial035 podman[153882]: 2026-02-24 07:58:10.832170533 +0000 UTC m=+5.395655749 container remove 24045a3a2d0a7043e319958d698b70dcb650699e2617668c192f61877d77386b (image=quay.ceph.io/ceph-ci/ceph@sha256:9a5db51bca6d139691bb8c7054be8834f3ba065d1c0aaaecbafb190e389fb148, name=ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-osd-1, io.buildah.version=1.33.7, ceph=True, org.opencontainers.image.authors=Ceph Release Team , CEPH_REF=wip-shraddhaag-enable-cephadm-tests, org.label-schema.schema-version=1.0, CEPH_SHA1=ff63f3d997e38f69382fd23ac0206f07f7b05017, GANESHA_REPO_BASEURL=https://buildlogs.centos.org/centos/$releasever-stream/storage/$basearch/nfsganesha-5/, OSD_FLAVOR=default, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.opencontainers.image.documentation=https://docs.ceph.com/, FROM_IMAGE=quay.io/centos/centos:stream9, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20260223, CEPH_GIT_REPO=https://github.com/ceph/ceph-ci.git) 2026-02-24T07:58:11.122 INFO:journalctl@ceph.osd.1.trial035.stdout:Feb 24 07:58:10 trial035 bash[153882]: ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-osd-1 2026-02-24T07:58:11.122 INFO:journalctl@ceph.osd.1.trial035.stdout:Feb 24 07:58:10 trial035 podman[154013]: 2026-02-24 07:58:10.963774461 +0000 UTC m=+0.015762386 container create 20268ca60014d53009c7cf937917b05e305f519641c43667e90eb8b18950e144 (image=quay.ceph.io/ceph-ci/ceph@sha256:9a5db51bca6d139691bb8c7054be8834f3ba065d1c0aaaecbafb190e389fb148, name=ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-osd-1-deactivate, org.opencontainers.image.authors=Ceph Release Team , org.label-schema.vendor=CentOS, CEPH_GIT_REPO=https://github.com/ceph/ceph-ci.git, CEPH_SHA1=ff63f3d997e38f69382fd23ac0206f07f7b05017, org.label-schema.build-date=20260223, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, io.buildah.version=1.33.7, GANESHA_REPO_BASEURL=https://buildlogs.centos.org/centos/$releasever-stream/storage/$basearch/nfsganesha-5/, FROM_IMAGE=quay.io/centos/centos:stream9, org.label-schema.schema-version=1.0, OSD_FLAVOR=default, CEPH_REF=wip-shraddhaag-enable-cephadm-tests, org.label-schema.license=GPLv2, org.opencontainers.image.documentation=https://docs.ceph.com/) 2026-02-24T07:58:11.122 INFO:journalctl@ceph.osd.1.trial035.stdout:Feb 24 07:58:10 trial035 podman[154013]: 2026-02-24 07:58:10.996698413 +0000 UTC m=+0.048686338 container init 20268ca60014d53009c7cf937917b05e305f519641c43667e90eb8b18950e144 (image=quay.ceph.io/ceph-ci/ceph@sha256:9a5db51bca6d139691bb8c7054be8834f3ba065d1c0aaaecbafb190e389fb148, name=ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-osd-1-deactivate, org.label-schema.schema-version=1.0, OSD_FLAVOR=default, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, org.opencontainers.image.documentation=https://docs.ceph.com/, FROM_IMAGE=quay.io/centos/centos:stream9, CEPH_SHA1=ff63f3d997e38f69382fd23ac0206f07f7b05017, CEPH_REF=wip-shraddhaag-enable-cephadm-tests, org.label-schema.license=GPLv2, io.buildah.version=1.33.7, org.opencontainers.image.authors=Ceph Release Team , CEPH_GIT_REPO=https://github.com/ceph/ceph-ci.git, org.label-schema.build-date=20260223, GANESHA_REPO_BASEURL=https://buildlogs.centos.org/centos/$releasever-stream/storage/$basearch/nfsganesha-5/, ceph=True) 2026-02-24T07:58:11.122 INFO:journalctl@ceph.osd.1.trial035.stdout:Feb 24 07:58:10 trial035 podman[154013]: 2026-02-24 07:58:10.999052089 +0000 UTC m=+0.051040014 container start 20268ca60014d53009c7cf937917b05e305f519641c43667e90eb8b18950e144 (image=quay.ceph.io/ceph-ci/ceph@sha256:9a5db51bca6d139691bb8c7054be8834f3ba065d1c0aaaecbafb190e389fb148, name=ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-osd-1-deactivate, io.buildah.version=1.33.7, org.opencontainers.image.authors=Ceph Release Team , org.label-schema.vendor=CentOS, CEPH_REF=wip-shraddhaag-enable-cephadm-tests, GANESHA_REPO_BASEURL=https://buildlogs.centos.org/centos/$releasever-stream/storage/$basearch/nfsganesha-5/, org.label-schema.license=GPLv2, org.opencontainers.image.documentation=https://docs.ceph.com/, FROM_IMAGE=quay.io/centos/centos:stream9, org.label-schema.schema-version=1.0, OSD_FLAVOR=default, CEPH_GIT_REPO=https://github.com/ceph/ceph-ci.git, CEPH_SHA1=ff63f3d997e38f69382fd23ac0206f07f7b05017, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20260223, ceph=True) 2026-02-24T07:58:11.122 INFO:journalctl@ceph.osd.1.trial035.stdout:Feb 24 07:58:10 trial035 podman[154013]: 2026-02-24 07:58:10.99947179 +0000 UTC m=+0.051459715 container attach 20268ca60014d53009c7cf937917b05e305f519641c43667e90eb8b18950e144 (image=quay.ceph.io/ceph-ci/ceph@sha256:9a5db51bca6d139691bb8c7054be8834f3ba065d1c0aaaecbafb190e389fb148, name=ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-osd-1-deactivate, org.label-schema.vendor=CentOS, GANESHA_REPO_BASEURL=https://buildlogs.centos.org/centos/$releasever-stream/storage/$basearch/nfsganesha-5/, ceph=True, OSD_FLAVOR=default, org.label-schema.build-date=20260223, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, org.opencontainers.image.documentation=https://docs.ceph.com/, org.label-schema.schema-version=1.0, org.opencontainers.image.authors=Ceph Release Team , CEPH_REF=wip-shraddhaag-enable-cephadm-tests, FROM_IMAGE=quay.io/centos/centos:stream9, CEPH_GIT_REPO=https://github.com/ceph/ceph-ci.git, CEPH_SHA1=ff63f3d997e38f69382fd23ac0206f07f7b05017, io.buildah.version=1.33.7) 2026-02-24T07:58:11.122 INFO:journalctl@ceph.osd.1.trial035.stdout:Feb 24 07:58:11 trial035 podman[154013]: 2026-02-24 07:58:10.956886625 +0000 UTC m=+0.008874560 image pull 3992f70354a3fdc3bbd8e07cbd698e424b71669298ffe35e8f0565a46973aa85 quay.ceph.io/ceph-ci/ceph@sha256:9a5db51bca6d139691bb8c7054be8834f3ba065d1c0aaaecbafb190e389fb148 2026-02-24T07:58:11.122 INFO:journalctl@ceph.osd.1.trial035.stdout:Feb 24 07:58:11 trial035 podman[154013]: 2026-02-24 07:58:11.120104139 +0000 UTC m=+0.172092064 container died 20268ca60014d53009c7cf937917b05e305f519641c43667e90eb8b18950e144 (image=quay.ceph.io/ceph-ci/ceph@sha256:9a5db51bca6d139691bb8c7054be8834f3ba065d1c0aaaecbafb190e389fb148, name=ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-osd-1-deactivate, org.label-schema.license=GPLv2, io.buildah.version=1.33.7, FROM_IMAGE=quay.io/centos/centos:stream9, org.label-schema.schema-version=1.0, org.opencontainers.image.authors=Ceph Release Team , org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, CEPH_REF=wip-shraddhaag-enable-cephadm-tests, org.opencontainers.image.documentation=https://docs.ceph.com/, CEPH_GIT_REPO=https://github.com/ceph/ceph-ci.git, OSD_FLAVOR=default, CEPH_SHA1=ff63f3d997e38f69382fd23ac0206f07f7b05017, org.label-schema.build-date=20260223, GANESHA_REPO_BASEURL=https://buildlogs.centos.org/centos/$releasever-stream/storage/$basearch/nfsganesha-5/) 2026-02-24T07:58:11.148 DEBUG:teuthology.orchestra.run.trial035:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460@osd.1.service' 2026-02-24T07:58:11.185 DEBUG:teuthology.orchestra.run:got remote process result: None 2026-02-24T07:58:11.186 INFO:tasks.cephadm.osd.1:Stopped osd.1 2026-02-24T07:58:11.186 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2026-02-24T07:58:11.186 DEBUG:teuthology.orchestra.run.trial091:> sudo systemctl stop ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460@osd.2 2026-02-24T07:58:11.606 INFO:journalctl@ceph.osd.2.trial091.stdout:Feb 24 07:58:11 trial091 systemd[1]: Stopping Ceph osd.2 for 3a5983d2-1154-11f1-9c05-d404e6e7d460... 2026-02-24T07:58:11.606 INFO:journalctl@ceph.osd.2.trial091.stdout:Feb 24 07:58:11 trial091 ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-osd-2[46763]: 2026-02-24T07:58:11.284+0000 7f698b584640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --osd-objectstore=bluestore (PID: 1) UID: 0 2026-02-24T07:58:11.606 INFO:journalctl@ceph.osd.2.trial091.stdout:Feb 24 07:58:11 trial091 ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-osd-2[46763]: 2026-02-24T07:58:11.284+0000 7f698b584640 -1 osd.2 122 *** Got signal Terminated *** 2026-02-24T07:58:11.606 INFO:journalctl@ceph.osd.2.trial091.stdout:Feb 24 07:58:11 trial091 ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-osd-2[46763]: 2026-02-24T07:58:11.284+0000 7f698b584640 -1 osd.2 122 *** Immediate shutdown (osd_fast_shutdown=true) *** 2026-02-24T07:58:17.012 INFO:journalctl@ceph.osd.2.trial091.stdout:Feb 24 07:58:16 trial091 podman[68549]: 2026-02-24 07:58:16.717594459 +0000 UTC m=+5.445136514 container died 8a0910f7b6fa429fc58b033c58536dc86f3630e8e148c78617c852c69d470a87 (image=quay.ceph.io/ceph-ci/ceph@sha256:9a5db51bca6d139691bb8c7054be8834f3ba065d1c0aaaecbafb190e389fb148, name=ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-osd-2, org.label-schema.name=CentOS Stream 9 Base Image, org.opencontainers.image.authors=Ceph Release Team , CEPH_REF=wip-shraddhaag-enable-cephadm-tests, GANESHA_REPO_BASEURL=https://buildlogs.centos.org/centos/$releasever-stream/storage/$basearch/nfsganesha-5/, org.opencontainers.image.documentation=https://docs.ceph.com/, ceph=True, org.label-schema.schema-version=1.0, CEPH_SHA1=ff63f3d997e38f69382fd23ac0206f07f7b05017, OSD_FLAVOR=default, org.label-schema.vendor=CentOS, FROM_IMAGE=quay.io/centos/centos:stream9, org.label-schema.license=GPLv2, io.buildah.version=1.33.7, CEPH_GIT_REPO=https://github.com/ceph/ceph-ci.git, org.label-schema.build-date=20260223) 2026-02-24T07:58:17.013 INFO:journalctl@ceph.osd.2.trial091.stdout:Feb 24 07:58:16 trial091 podman[68549]: 2026-02-24 07:58:16.73049431 +0000 UTC m=+5.458036375 container remove 8a0910f7b6fa429fc58b033c58536dc86f3630e8e148c78617c852c69d470a87 (image=quay.ceph.io/ceph-ci/ceph@sha256:9a5db51bca6d139691bb8c7054be8834f3ba065d1c0aaaecbafb190e389fb148, name=ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-osd-2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, org.opencontainers.image.authors=Ceph Release Team , ceph=True, org.opencontainers.image.documentation=https://docs.ceph.com/, CEPH_GIT_REPO=https://github.com/ceph/ceph-ci.git, CEPH_REF=wip-shraddhaag-enable-cephadm-tests, GANESHA_REPO_BASEURL=https://buildlogs.centos.org/centos/$releasever-stream/storage/$basearch/nfsganesha-5/, org.label-schema.build-date=20260223, org.label-schema.vendor=CentOS, OSD_FLAVOR=default, io.buildah.version=1.33.7, FROM_IMAGE=quay.io/centos/centos:stream9, CEPH_SHA1=ff63f3d997e38f69382fd23ac0206f07f7b05017, org.label-schema.license=GPLv2) 2026-02-24T07:58:17.013 INFO:journalctl@ceph.osd.2.trial091.stdout:Feb 24 07:58:16 trial091 bash[68549]: ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-osd-2 2026-02-24T07:58:17.013 INFO:journalctl@ceph.osd.2.trial091.stdout:Feb 24 07:58:16 trial091 podman[68676]: 2026-02-24 07:58:16.85599764 +0000 UTC m=+0.015947362 container create e3dd2e1a056187fbb8a5cf9ceeac8385158830e3b1440e7affc31348456d7877 (image=quay.ceph.io/ceph-ci/ceph@sha256:9a5db51bca6d139691bb8c7054be8834f3ba065d1c0aaaecbafb190e389fb148, name=ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-osd-2-deactivate, org.opencontainers.image.documentation=https://docs.ceph.com/, org.label-schema.build-date=20260223, ceph=True, org.label-schema.schema-version=1.0, OSD_FLAVOR=default, io.buildah.version=1.33.7, CEPH_REF=wip-shraddhaag-enable-cephadm-tests, org.label-schema.name=CentOS Stream 9 Base Image, org.opencontainers.image.authors=Ceph Release Team , FROM_IMAGE=quay.io/centos/centos:stream9, org.label-schema.vendor=CentOS, GANESHA_REPO_BASEURL=https://buildlogs.centos.org/centos/$releasever-stream/storage/$basearch/nfsganesha-5/, CEPH_GIT_REPO=https://github.com/ceph/ceph-ci.git, org.label-schema.license=GPLv2, CEPH_SHA1=ff63f3d997e38f69382fd23ac0206f07f7b05017) 2026-02-24T07:58:17.013 INFO:journalctl@ceph.osd.2.trial091.stdout:Feb 24 07:58:16 trial091 podman[68676]: 2026-02-24 07:58:16.886833915 +0000 UTC m=+0.046783627 container init e3dd2e1a056187fbb8a5cf9ceeac8385158830e3b1440e7affc31348456d7877 (image=quay.ceph.io/ceph-ci/ceph@sha256:9a5db51bca6d139691bb8c7054be8834f3ba065d1c0aaaecbafb190e389fb148, name=ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-osd-2-deactivate, io.buildah.version=1.33.7, CEPH_REF=wip-shraddhaag-enable-cephadm-tests, org.label-schema.build-date=20260223, CEPH_GIT_REPO=https://github.com/ceph/ceph-ci.git, FROM_IMAGE=quay.io/centos/centos:stream9, org.opencontainers.image.authors=Ceph Release Team , org.label-schema.vendor=CentOS, org.opencontainers.image.documentation=https://docs.ceph.com/, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.schema-version=1.0, OSD_FLAVOR=default, GANESHA_REPO_BASEURL=https://buildlogs.centos.org/centos/$releasever-stream/storage/$basearch/nfsganesha-5/, org.label-schema.license=GPLv2, CEPH_SHA1=ff63f3d997e38f69382fd23ac0206f07f7b05017) 2026-02-24T07:58:17.013 INFO:journalctl@ceph.osd.2.trial091.stdout:Feb 24 07:58:16 trial091 podman[68676]: 2026-02-24 07:58:16.890010556 +0000 UTC m=+0.049960278 container start e3dd2e1a056187fbb8a5cf9ceeac8385158830e3b1440e7affc31348456d7877 (image=quay.ceph.io/ceph-ci/ceph@sha256:9a5db51bca6d139691bb8c7054be8834f3ba065d1c0aaaecbafb190e389fb148, name=ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-osd-2-deactivate, OSD_FLAVOR=default, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20260223, CEPH_GIT_REPO=https://github.com/ceph/ceph-ci.git, org.opencontainers.image.documentation=https://docs.ceph.com/, FROM_IMAGE=quay.io/centos/centos:stream9, ceph=True, org.label-schema.schema-version=1.0, org.opencontainers.image.authors=Ceph Release Team , org.label-schema.license=GPLv2, CEPH_SHA1=ff63f3d997e38f69382fd23ac0206f07f7b05017, GANESHA_REPO_BASEURL=https://buildlogs.centos.org/centos/$releasever-stream/storage/$basearch/nfsganesha-5/, io.buildah.version=1.33.7, CEPH_REF=wip-shraddhaag-enable-cephadm-tests) 2026-02-24T07:58:17.013 INFO:journalctl@ceph.osd.2.trial091.stdout:Feb 24 07:58:16 trial091 podman[68676]: 2026-02-24 07:58:16.890410298 +0000 UTC m=+0.050360020 container attach e3dd2e1a056187fbb8a5cf9ceeac8385158830e3b1440e7affc31348456d7877 (image=quay.ceph.io/ceph-ci/ceph@sha256:9a5db51bca6d139691bb8c7054be8834f3ba065d1c0aaaecbafb190e389fb148, name=ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-osd-2-deactivate, ceph=True, org.label-schema.schema-version=1.0, GANESHA_REPO_BASEURL=https://buildlogs.centos.org/centos/$releasever-stream/storage/$basearch/nfsganesha-5/, CEPH_REF=wip-shraddhaag-enable-cephadm-tests, CEPH_GIT_REPO=https://github.com/ceph/ceph-ci.git, org.opencontainers.image.authors=Ceph Release Team , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, OSD_FLAVOR=default, org.label-schema.vendor=CentOS, CEPH_SHA1=ff63f3d997e38f69382fd23ac0206f07f7b05017, org.opencontainers.image.documentation=https://docs.ceph.com/, FROM_IMAGE=quay.io/centos/centos:stream9, io.buildah.version=1.33.7, org.label-schema.build-date=20260223) 2026-02-24T07:58:17.013 INFO:journalctl@ceph.osd.2.trial091.stdout:Feb 24 07:58:16 trial091 podman[68676]: 2026-02-24 07:58:16.848972291 +0000 UTC m=+0.008922023 image pull 3992f70354a3fdc3bbd8e07cbd698e424b71669298ffe35e8f0565a46973aa85 quay.ceph.io/ceph-ci/ceph@sha256:9a5db51bca6d139691bb8c7054be8834f3ba065d1c0aaaecbafb190e389fb148 2026-02-24T07:58:17.081 DEBUG:teuthology.orchestra.run.trial091:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460@osd.2.service' 2026-02-24T07:58:17.120 DEBUG:teuthology.orchestra.run:got remote process result: None 2026-02-24T07:58:17.120 INFO:tasks.cephadm.osd.2:Stopped osd.2 2026-02-24T07:58:17.120 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2026-02-24T07:58:17.120 DEBUG:teuthology.orchestra.run.trial091:> sudo systemctl stop ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460@osd.3 2026-02-24T07:58:17.357 INFO:journalctl@ceph.osd.3.trial091.stdout:Feb 24 07:58:17 trial091 systemd[1]: Stopping Ceph osd.3 for 3a5983d2-1154-11f1-9c05-d404e6e7d460... 2026-02-24T07:58:17.357 INFO:journalctl@ceph.osd.3.trial091.stdout:Feb 24 07:58:17 trial091 ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-osd-3[54462]: 2026-02-24T07:58:17.248+0000 7f4e00748640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --osd-objectstore=bluestore (PID: 1) UID: 0 2026-02-24T07:58:17.357 INFO:journalctl@ceph.osd.3.trial091.stdout:Feb 24 07:58:17 trial091 ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-osd-3[54462]: 2026-02-24T07:58:17.248+0000 7f4e00748640 -1 osd.3 122 *** Got signal Terminated *** 2026-02-24T07:58:17.357 INFO:journalctl@ceph.osd.3.trial091.stdout:Feb 24 07:58:17 trial091 ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-osd-3[54462]: 2026-02-24T07:58:17.248+0000 7f4e00748640 -1 osd.3 122 *** Immediate shutdown (osd_fast_shutdown=true) *** 2026-02-24T07:58:22.856 INFO:journalctl@ceph.osd.3.trial091.stdout:Feb 24 07:58:22 trial091 podman[68809]: 2026-02-24 07:58:22.549867481 +0000 UTC m=+5.313376784 container died 4150509624bf369b68854c9d5aba1dffd5cda9c0f2aeec6fc2291150c654fcda (image=quay.ceph.io/ceph-ci/ceph@sha256:9a5db51bca6d139691bb8c7054be8834f3ba065d1c0aaaecbafb190e389fb148, name=ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-osd-3, org.label-schema.license=GPLv2, org.opencontainers.image.authors=Ceph Release Team , OSD_FLAVOR=default, CEPH_REF=wip-shraddhaag-enable-cephadm-tests, org.opencontainers.image.documentation=https://docs.ceph.com/, org.label-schema.schema-version=1.0, ceph=True, CEPH_GIT_REPO=https://github.com/ceph/ceph-ci.git, org.label-schema.vendor=CentOS, CEPH_SHA1=ff63f3d997e38f69382fd23ac0206f07f7b05017, GANESHA_REPO_BASEURL=https://buildlogs.centos.org/centos/$releasever-stream/storage/$basearch/nfsganesha-5/, io.buildah.version=1.33.7, FROM_IMAGE=quay.io/centos/centos:stream9, org.label-schema.build-date=20260223, org.label-schema.name=CentOS Stream 9 Base Image) 2026-02-24T07:58:22.856 INFO:journalctl@ceph.osd.3.trial091.stdout:Feb 24 07:58:22 trial091 podman[68809]: 2026-02-24 07:58:22.560135696 +0000 UTC m=+5.323644999 container remove 4150509624bf369b68854c9d5aba1dffd5cda9c0f2aeec6fc2291150c654fcda (image=quay.ceph.io/ceph-ci/ceph@sha256:9a5db51bca6d139691bb8c7054be8834f3ba065d1c0aaaecbafb190e389fb148, name=ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-osd-3, io.buildah.version=1.33.7, org.opencontainers.image.documentation=https://docs.ceph.com/, org.opencontainers.image.authors=Ceph Release Team , CEPH_SHA1=ff63f3d997e38f69382fd23ac0206f07f7b05017, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.build-date=20260223, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, CEPH_GIT_REPO=https://github.com/ceph/ceph-ci.git, CEPH_REF=wip-shraddhaag-enable-cephadm-tests, org.label-schema.license=GPLv2, GANESHA_REPO_BASEURL=https://buildlogs.centos.org/centos/$releasever-stream/storage/$basearch/nfsganesha-5/, FROM_IMAGE=quay.io/centos/centos:stream9, OSD_FLAVOR=default) 2026-02-24T07:58:22.856 INFO:journalctl@ceph.osd.3.trial091.stdout:Feb 24 07:58:22 trial091 bash[68809]: ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-osd-3 2026-02-24T07:58:22.856 INFO:journalctl@ceph.osd.3.trial091.stdout:Feb 24 07:58:22 trial091 podman[68942]: 2026-02-24 07:58:22.685667506 +0000 UTC m=+0.016463435 container create fc7d77da306bdad72eab9b72a5d0ccc07f1c6f95b986e581965f3823334a7cf0 (image=quay.ceph.io/ceph-ci/ceph@sha256:9a5db51bca6d139691bb8c7054be8834f3ba065d1c0aaaecbafb190e389fb148, name=ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-osd-3-deactivate, CEPH_SHA1=ff63f3d997e38f69382fd23ac0206f07f7b05017, CEPH_REF=wip-shraddhaag-enable-cephadm-tests, org.opencontainers.image.authors=Ceph Release Team , ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20260223, org.opencontainers.image.documentation=https://docs.ceph.com/, OSD_FLAVOR=default, CEPH_GIT_REPO=https://github.com/ceph/ceph-ci.git, FROM_IMAGE=quay.io/centos/centos:stream9, GANESHA_REPO_BASEURL=https://buildlogs.centos.org/centos/$releasever-stream/storage/$basearch/nfsganesha-5/, org.label-schema.license=GPLv2, io.buildah.version=1.33.7) 2026-02-24T07:58:22.856 INFO:journalctl@ceph.osd.3.trial091.stdout:Feb 24 07:58:22 trial091 podman[68942]: 2026-02-24 07:58:22.726645531 +0000 UTC m=+0.057441460 container init fc7d77da306bdad72eab9b72a5d0ccc07f1c6f95b986e581965f3823334a7cf0 (image=quay.ceph.io/ceph-ci/ceph@sha256:9a5db51bca6d139691bb8c7054be8834f3ba065d1c0aaaecbafb190e389fb148, name=ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-osd-3-deactivate, GANESHA_REPO_BASEURL=https://buildlogs.centos.org/centos/$releasever-stream/storage/$basearch/nfsganesha-5/, FROM_IMAGE=quay.io/centos/centos:stream9, org.opencontainers.image.documentation=https://docs.ceph.com/, org.label-schema.schema-version=1.0, CEPH_GIT_REPO=https://github.com/ceph/ceph-ci.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20260223, CEPH_REF=wip-shraddhaag-enable-cephadm-tests, org.opencontainers.image.authors=Ceph Release Team , CEPH_SHA1=ff63f3d997e38f69382fd23ac0206f07f7b05017, OSD_FLAVOR=default, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.33.7) 2026-02-24T07:58:22.856 INFO:journalctl@ceph.osd.3.trial091.stdout:Feb 24 07:58:22 trial091 podman[68942]: 2026-02-24 07:58:22.729023098 +0000 UTC m=+0.059819027 container start fc7d77da306bdad72eab9b72a5d0ccc07f1c6f95b986e581965f3823334a7cf0 (image=quay.ceph.io/ceph-ci/ceph@sha256:9a5db51bca6d139691bb8c7054be8834f3ba065d1c0aaaecbafb190e389fb148, name=ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-osd-3-deactivate, ceph=True, org.label-schema.build-date=20260223, CEPH_REF=wip-shraddhaag-enable-cephadm-tests, CEPH_GIT_REPO=https://github.com/ceph/ceph-ci.git, org.label-schema.name=CentOS Stream 9 Base Image, GANESHA_REPO_BASEURL=https://buildlogs.centos.org/centos/$releasever-stream/storage/$basearch/nfsganesha-5/, OSD_FLAVOR=default, org.label-schema.vendor=CentOS, io.buildah.version=1.33.7, CEPH_SHA1=ff63f3d997e38f69382fd23ac0206f07f7b05017, org.label-schema.license=GPLv2, org.opencontainers.image.documentation=https://docs.ceph.com/, org.label-schema.schema-version=1.0, FROM_IMAGE=quay.io/centos/centos:stream9, org.opencontainers.image.authors=Ceph Release Team ) 2026-02-24T07:58:22.857 INFO:journalctl@ceph.osd.3.trial091.stdout:Feb 24 07:58:22 trial091 podman[68942]: 2026-02-24 07:58:22.729720514 +0000 UTC m=+0.060516473 container attach fc7d77da306bdad72eab9b72a5d0ccc07f1c6f95b986e581965f3823334a7cf0 (image=quay.ceph.io/ceph-ci/ceph@sha256:9a5db51bca6d139691bb8c7054be8834f3ba065d1c0aaaecbafb190e389fb148, name=ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-osd-3-deactivate, org.opencontainers.image.documentation=https://docs.ceph.com/, org.label-schema.vendor=CentOS, FROM_IMAGE=quay.io/centos/centos:stream9, OSD_FLAVOR=default, org.label-schema.schema-version=1.0, CEPH_REF=wip-shraddhaag-enable-cephadm-tests, CEPH_GIT_REPO=https://github.com/ceph/ceph-ci.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.opencontainers.image.authors=Ceph Release Team , GANESHA_REPO_BASEURL=https://buildlogs.centos.org/centos/$releasever-stream/storage/$basearch/nfsganesha-5/, ceph=True, org.label-schema.build-date=20260223, CEPH_SHA1=ff63f3d997e38f69382fd23ac0206f07f7b05017, io.buildah.version=1.33.7) 2026-02-24T07:58:22.857 INFO:journalctl@ceph.osd.3.trial091.stdout:Feb 24 07:58:22 trial091 podman[68942]: 2026-02-24 07:58:22.678935441 +0000 UTC m=+0.009731360 image pull 3992f70354a3fdc3bbd8e07cbd698e424b71669298ffe35e8f0565a46973aa85 quay.ceph.io/ceph-ci/ceph@sha256:9a5db51bca6d139691bb8c7054be8834f3ba065d1c0aaaecbafb190e389fb148 2026-02-24T07:58:22.902 DEBUG:teuthology.orchestra.run.trial091:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460@osd.3.service' 2026-02-24T07:58:22.939 DEBUG:teuthology.orchestra.run:got remote process result: None 2026-02-24T07:58:22.940 INFO:tasks.cephadm.osd.3:Stopped osd.3 2026-02-24T07:58:22.940 INFO:tasks.cephadm.prometheus.a:Stopping prometheus.a... 2026-02-24T07:58:22.940 DEBUG:teuthology.orchestra.run.trial091:> sudo systemctl stop ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460@prometheus.a 2026-02-24T07:58:23.121 INFO:journalctl@ceph.prometheus.a.trial091.stdout:Feb 24 07:58:23 trial091 systemd[1]: Stopping Ceph prometheus.a for 3a5983d2-1154-11f1-9c05-d404e6e7d460... 2026-02-24T07:58:23.121 INFO:journalctl@ceph.prometheus.a.trial091.stdout:Feb 24 07:58:23 trial091 ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-prometheus-a[66007]: time=2026-02-24T07:58:23.079Z level=WARN source=main.go:1076 msg="Received an OS signal, exiting gracefully..." signal=terminated 2026-02-24T07:58:23.121 INFO:journalctl@ceph.prometheus.a.trial091.stdout:Feb 24 07:58:23 trial091 ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-prometheus-a[66007]: time=2026-02-24T07:58:23.080Z level=INFO source=main.go:1101 msg="Stopping scrape discovery manager..." 2026-02-24T07:58:23.121 INFO:journalctl@ceph.prometheus.a.trial091.stdout:Feb 24 07:58:23 trial091 ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-prometheus-a[66007]: time=2026-02-24T07:58:23.080Z level=INFO source=main.go:1115 msg="Stopping notify discovery manager..." 2026-02-24T07:58:23.121 INFO:journalctl@ceph.prometheus.a.trial091.stdout:Feb 24 07:58:23 trial091 ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-prometheus-a[66007]: time=2026-02-24T07:58:23.080Z level=INFO source=manager.go:190 msg="Stopping rule manager..." component="rule manager" 2026-02-24T07:58:23.121 INFO:journalctl@ceph.prometheus.a.trial091.stdout:Feb 24 07:58:23 trial091 ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-prometheus-a[66007]: time=2026-02-24T07:58:23.080Z level=INFO source=main.go:1097 msg="Scrape discovery manager stopped" 2026-02-24T07:58:23.122 INFO:journalctl@ceph.prometheus.a.trial091.stdout:Feb 24 07:58:23 trial091 ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-prometheus-a[66007]: time=2026-02-24T07:58:23.080Z level=INFO source=manager.go:206 msg="Rule manager stopped" component="rule manager" 2026-02-24T07:58:23.122 INFO:journalctl@ceph.prometheus.a.trial091.stdout:Feb 24 07:58:23 trial091 ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-prometheus-a[66007]: time=2026-02-24T07:58:23.080Z level=INFO source=main.go:1152 msg="Stopping scrape manager..." 2026-02-24T07:58:23.122 INFO:journalctl@ceph.prometheus.a.trial091.stdout:Feb 24 07:58:23 trial091 ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-prometheus-a[66007]: time=2026-02-24T07:58:23.080Z level=INFO source=main.go:1111 msg="Notify discovery manager stopped" 2026-02-24T07:58:23.122 INFO:journalctl@ceph.prometheus.a.trial091.stdout:Feb 24 07:58:23 trial091 ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-prometheus-a[66007]: time=2026-02-24T07:58:23.080Z level=INFO source=main.go:1144 msg="Scrape manager stopped" 2026-02-24T07:58:23.122 INFO:journalctl@ceph.prometheus.a.trial091.stdout:Feb 24 07:58:23 trial091 ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-prometheus-a[66007]: time=2026-02-24T07:58:23.081Z level=INFO source=manager.go:543 msg="Stopping notification manager..." component=notifier 2026-02-24T07:58:23.122 INFO:journalctl@ceph.prometheus.a.trial091.stdout:Feb 24 07:58:23 trial091 ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-prometheus-a[66007]: time=2026-02-24T07:58:23.081Z level=INFO source=manager.go:285 msg="Draining any remaining notifications..." component=notifier 2026-02-24T07:58:23.122 INFO:journalctl@ceph.prometheus.a.trial091.stdout:Feb 24 07:58:23 trial091 ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-prometheus-a[66007]: time=2026-02-24T07:58:23.081Z level=INFO source=manager.go:291 msg="Remaining notifications drained" component=notifier 2026-02-24T07:58:23.122 INFO:journalctl@ceph.prometheus.a.trial091.stdout:Feb 24 07:58:23 trial091 ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-prometheus-a[66007]: time=2026-02-24T07:58:23.081Z level=INFO source=manager.go:218 msg="Notification manager stopped" component=notifier 2026-02-24T07:58:23.122 INFO:journalctl@ceph.prometheus.a.trial091.stdout:Feb 24 07:58:23 trial091 ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-prometheus-a[66007]: time=2026-02-24T07:58:23.081Z level=INFO source=main.go:1422 msg="Notifier manager stopped" 2026-02-24T07:58:23.122 INFO:journalctl@ceph.prometheus.a.trial091.stdout:Feb 24 07:58:23 trial091 ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-prometheus-a[66007]: time=2026-02-24T07:58:23.081Z level=INFO source=main.go:1436 msg="See you next time!" 2026-02-24T07:58:23.122 INFO:journalctl@ceph.prometheus.a.trial091.stdout:Feb 24 07:58:23 trial091 podman[69079]: 2026-02-24 07:58:23.090064463 +0000 UTC m=+0.023686323 container died bbd51718b6d93b57c3fef297bdf0f64ac87c522c6f61881faddb3d62aab49563 (image=quay.io/prometheus/prometheus:v3.6.0, name=ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-prometheus-a, maintainer=The Prometheus Authors , org.opencontainers.image.source=https://github.com/prometheus/prometheus) 2026-02-24T07:58:23.123 INFO:journalctl@ceph.prometheus.a.trial091.stdout:Feb 24 07:58:23 trial091 podman[69079]: 2026-02-24 07:58:23.100397188 +0000 UTC m=+0.034019028 container remove bbd51718b6d93b57c3fef297bdf0f64ac87c522c6f61881faddb3d62aab49563 (image=quay.io/prometheus/prometheus:v3.6.0, name=ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-prometheus-a, maintainer=The Prometheus Authors , org.opencontainers.image.source=https://github.com/prometheus/prometheus) 2026-02-24T07:58:23.123 INFO:journalctl@ceph.prometheus.a.trial091.stdout:Feb 24 07:58:23 trial091 bash[69079]: ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-prometheus-a 2026-02-24T07:58:23.163 DEBUG:teuthology.orchestra.run.trial091:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460@prometheus.a.service' 2026-02-24T07:58:23.197 DEBUG:teuthology.orchestra.run:got remote process result: None 2026-02-24T07:58:23.197 INFO:tasks.cephadm.prometheus.a:Stopped prometheus.a 2026-02-24T07:58:23.198 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 --force --keep-logs 2026-02-24T07:58:23.319 INFO:teuthology.orchestra.run.trial035.stdout:Deleting cluster with fsid: 3a5983d2-1154-11f1-9c05-d404e6e7d460 2026-02-24T07:58:24.583 DEBUG:teuthology.orchestra.run.trial091:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 --force --keep-logs 2026-02-24T07:58:24.700 INFO:teuthology.orchestra.run.trial091.stdout:Deleting cluster with fsid: 3a5983d2-1154-11f1-9c05-d404e6e7d460 2026-02-24T07:58:25.541 INFO:journalctl@ceph.node-exporter.b.trial091.stdout:Feb 24 07:58:25 trial091 systemd[1]: Stopping Ceph node-exporter.b for 3a5983d2-1154-11f1-9c05-d404e6e7d460... 2026-02-24T07:58:25.541 INFO:journalctl@ceph.node-exporter.b.trial091.stdout:Feb 24 07:58:25 trial091 podman[69398]: 2026-02-24 07:58:25.23777644 +0000 UTC m=+0.021726432 container died 42595c8ca7ba5feb96b0769ff36383d32706295000e421cf541c9d6afb4eefaa (image=quay.io/prometheus/node-exporter:v1.9.1, name=ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-node-exporter-b, maintainer=The Prometheus Authors ) 2026-02-24T07:58:25.541 INFO:journalctl@ceph.node-exporter.b.trial091.stdout:Feb 24 07:58:25 trial091 podman[69398]: 2026-02-24 07:58:25.245644447 +0000 UTC m=+0.029594439 container remove 42595c8ca7ba5feb96b0769ff36383d32706295000e421cf541c9d6afb4eefaa (image=quay.io/prometheus/node-exporter:v1.9.1, name=ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-node-exporter-b, maintainer=The Prometheus Authors ) 2026-02-24T07:58:25.541 INFO:journalctl@ceph.node-exporter.b.trial091.stdout:Feb 24 07:58:25 trial091 bash[69398]: ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460-node-exporter-b 2026-02-24T07:58:25.541 INFO:journalctl@ceph.node-exporter.b.trial091.stdout:Feb 24 07:58:25 trial091 systemd[1]: ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460@node-exporter.b.service: Main process exited, code=exited, status=143/n/a 2026-02-24T07:58:25.541 INFO:journalctl@ceph.node-exporter.b.trial091.stdout:Feb 24 07:58:25 trial091 systemd[1]: ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460@node-exporter.b.service: Failed with result 'exit-code'. 2026-02-24T07:58:25.541 INFO:journalctl@ceph.node-exporter.b.trial091.stdout:Feb 24 07:58:25 trial091 systemd[1]: Stopped Ceph node-exporter.b for 3a5983d2-1154-11f1-9c05-d404e6e7d460. 2026-02-24T07:58:25.541 INFO:journalctl@ceph.node-exporter.b.trial091.stdout:Feb 24 07:58:25 trial091 systemd[1]: ceph-3a5983d2-1154-11f1-9c05-d404e6e7d460@node-exporter.b.service: Consumed 2.714s CPU time. 2026-02-24T07:58:25.815 DEBUG:teuthology.orchestra.run.trial035:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2026-02-24T07:58:25.844 DEBUG:teuthology.orchestra.run.trial091:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2026-02-24T07:58:25.871 INFO:tasks.cephadm:Archiving crash dumps... 2026-02-24T07:58:25.875 DEBUG:teuthology.misc:Transferring archived files from trial035:/var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/crash to /home/teuthworker/mnt/teuthology/shraddhaag-2026-02-24_06:51:01-crimson-rados-main-distro-debug-trial/67566/remote/trial035/crash 2026-02-24T07:58:25.876 DEBUG:teuthology.orchestra.run.trial035:> sudo tar c -f - -C /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/crash -- . 2026-02-24T07:58:25.909 INFO:teuthology.orchestra.run.trial035.stderr:tar: /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/crash: Cannot open: No such file or directory 2026-02-24T07:58:25.909 INFO:teuthology.orchestra.run.trial035.stderr:tar: Error is not recoverable: exiting now 2026-02-24T07:58:25.912 DEBUG:teuthology.misc:Transferring archived files from trial091:/var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/crash to /home/teuthworker/mnt/teuthology/shraddhaag-2026-02-24_06:51:01-crimson-rados-main-distro-debug-trial/67566/remote/trial091/crash 2026-02-24T07:58:25.913 DEBUG:teuthology.orchestra.run.trial091:> sudo tar c -f - -C /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/crash -- . 2026-02-24T07:58:25.938 INFO:teuthology.orchestra.run.trial091.stderr:tar: /var/lib/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/crash: Cannot open: No such file or directory 2026-02-24T07:58:25.939 INFO:teuthology.orchestra.run.trial091.stderr:tar: Error is not recoverable: exiting now 2026-02-24T07:58:25.940 INFO:tasks.cephadm:Checking cluster log for badness... 2026-02-24T07:58:25.940 DEBUG:teuthology.orchestra.run.trial035:> sudo grep -E '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/ceph.log | grep -E -v '\(MDS_ALL_DOWN\)' | grep -E -v '\(MDS_UP_LESS_THAN_MAX\)' | grep -E -v '\(POOL_APP_NOT_ENABLED\)' | grep -E -v 'but it is still running' | grep -E -v 'objects unfound and apparently lost' | grep -E -v 'overall HEALTH_' | grep -E -v '\(OSDMAP_FLAGS\)' | grep -E -v '\(OSD_' | grep -E -v '\(PG_' | grep -E -v '\(POOL_' | grep -E -v '\(CACHE_POOL_' | grep -E -v '\(SMALLER_PGP_NUM\)' | grep -E -v '\(OBJECT_' | grep -E -v SLOW_OPS | grep -E -v '\(REQUEST_SLOW\)' | grep -E -v '\(TOO_FEW_PGS\)' | grep -E -v 'slow request' | grep -E -v 'timeout on replica' | grep -E -v 'late reservation from' | grep -E -v MON_DOWN | grep -E -v OSDMAP_FLAGS | grep -E -v OSD_DOWN | grep -E -v PG_DEGRADED | grep -E -v PG_AVAILABILITY | grep -E -v POOL_APP_NOT_ENABLED | grep -E -v 'mons down' | grep -E -v 'mon down' | grep -E -v 'out of quorum' | grep -E -v noscrub | grep -E -v nodeep-scrub | grep -E -v 'Degraded data redundancy' | grep -E -v 'is down' | grep -E -v 'osds down' | grep -E -v 'pg .*? is .*?degraded.*?, acting' | grep -E -v 'pg .*? is stuck' | grep -E -v 'pg degraded' | grep -E -v PG_BACKFILL_FULL | grep -E -v 'Low space hindering backfill .*? backfill_toofull' | grep -E -v OSD_ROOT_DOWN | grep -E -v 'pgs degraded' | grep -E -v 'pgs undersized' | grep -E -v 'is active.*backfill_toofull.*' | head -n 1 2026-02-24T07:58:25.976 INFO:tasks.cephadm:Compressing logs... 2026-02-24T07:58:25.977 DEBUG:teuthology.orchestra.run.trial035:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2026-02-24T07:58:26.018 DEBUG:teuthology.orchestra.run.trial091:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2026-02-24T07:58:26.040 INFO:teuthology.orchestra.run.trial035.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2026-02-24T07:58:26.040 INFO:teuthology.orchestra.run.trial035.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2026-02-24T07:58:26.040 INFO:teuthology.orchestra.run.trial035.stderr:gzip -5 --verbose -- /var/log/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/ceph-mon.a.log 2026-02-24T07:58:26.040 INFO:teuthology.orchestra.run.trial035.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/ceph-osd.0.log 2026-02-24T07:58:26.041 INFO:teuthology.orchestra.run.trial035.stderr:/var/log/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/ceph.cephadm.log 2026-02-24T07:58:26.041 INFO:teuthology.orchestra.run.trial035.stderr:/var/log/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/ceph.audit.log 2026-02-24T07:58:26.041 INFO:teuthology.orchestra.run.trial091.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/cephadm.log 2026-02-24T07:58:26.041 INFO:teuthology.orchestra.run.trial091.stderr:: No such file or directory 2026-02-24T07:58:26.042 INFO:teuthology.orchestra.run.trial091.stderr:gzip -5 --verbose -- /var/log/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/ceph-osd.2.log 2026-02-24T07:58:26.042 INFO:teuthology.orchestra.run.trial091.stderr:gzip -5 --verbose -- /var/log/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/ceph-osd.3.log 2026-02-24T07:58:26.042 INFO:teuthology.orchestra.run.trial035.stderr:gzip -5 --verbose -- /var/log/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/ceph-volume.log 2026-02-24T07:58:26.042 INFO:teuthology.orchestra.run.trial035.stderr:/var/log/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/ceph.cephadm.log: /var/log/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/ceph-osd.1.log 2026-02-24T07:58:26.042 INFO:teuthology.orchestra.run.trial035.stderr: 82.0% -- replaced with /var/log/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/ceph.cephadm.log.gz 2026-02-24T07:58:26.042 INFO:teuthology.orchestra.run.trial035.stderr:/var/log/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/ceph-mgr.x.log 2026-02-24T07:58:26.042 INFO:teuthology.orchestra.run.trial035.stderr:/var/log/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/ceph.log 2026-02-24T07:58:26.043 INFO:teuthology.orchestra.run.trial091.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/ceph-volume.log 2026-02-24T07:58:26.043 INFO:teuthology.orchestra.run.trial035.stderr:/var/log/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/ceph-mgr.x.log: /var/log/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/ceph.log: 89.6% -- replaced with /var/log/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/ceph.audit.log.gz 2026-02-24T07:58:26.044 INFO:teuthology.orchestra.run.trial091.stderr:/var/log/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/ceph-osd.2.log: /var/log/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/ceph-osd.3.log: /var/log/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/ceph-volume.log: 83.8% -- replaced with /var/log/ceph/cephadm.log.gz 2026-02-24T07:58:26.044 INFO:teuthology.orchestra.run.trial035.stderr: 86.0% -- replaced with /var/log/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/ceph.log.gz 2026-02-24T07:58:26.044 INFO:teuthology.orchestra.run.trial035.stderr: 94.2% -- replaced with /var/log/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/ceph-volume.log.gz 2026-02-24T07:58:26.045 INFO:teuthology.orchestra.run.trial091.stderr: 94.2% -- replaced with /var/log/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/ceph-volume.log.gz 2026-02-24T07:58:26.046 INFO:teuthology.orchestra.run.trial035.stderr: 95.7% -- replaced with /var/log/ceph/cephadm.log.gz 2026-02-24T07:58:26.136 INFO:teuthology.orchestra.run.trial035.stderr: 90.0% -- replaced with /var/log/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/ceph-mgr.x.log.gz 2026-02-24T07:58:26.342 INFO:teuthology.orchestra.run.trial035.stderr: 89.7% -- replaced with /var/log/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/ceph-mon.a.log.gz 2026-02-24T08:13:15.492 INFO:teuthology.orchestra.run.trial035.stderr: 94.1% -- replaced with /var/log/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/ceph-osd.0.log.gz 2026-02-24T08:14:22.198 INFO:teuthology.orchestra.run.trial091.stderr: 93.8% -- replaced with /var/log/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/ceph-osd.2.log.gz 2026-02-24T08:15:49.280 INFO:teuthology.orchestra.run.trial091.stderr: 93.9% -- replaced with /var/log/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/ceph-osd.3.log.gz 2026-02-24T08:15:49.284 INFO:teuthology.orchestra.run.trial091.stderr: 2026-02-24T08:15:49.284 INFO:teuthology.orchestra.run.trial091.stderr:real 17m23.250s 2026-02-24T08:15:49.285 INFO:teuthology.orchestra.run.trial091.stderr:user 30m41.570s 2026-02-24T08:15:49.285 INFO:teuthology.orchestra.run.trial091.stderr:sys 2m25.342s 2026-02-24T08:16:54.949 INFO:teuthology.orchestra.run.trial035.stderr: 93.8% -- replaced with /var/log/ceph/3a5983d2-1154-11f1-9c05-d404e6e7d460/ceph-osd.1.log.gz 2026-02-24T08:16:54.952 INFO:teuthology.orchestra.run.trial035.stderr: 2026-02-24T08:16:54.952 INFO:teuthology.orchestra.run.trial035.stderr:real 18m28.920s 2026-02-24T08:16:54.952 INFO:teuthology.orchestra.run.trial035.stderr:user 30m37.089s 2026-02-24T08:16:54.952 INFO:teuthology.orchestra.run.trial035.stderr:sys 2m28.682s 2026-02-24T08:16:54.952 INFO:tasks.cephadm:Archiving logs... 2026-02-24T08:16:54.953 DEBUG:teuthology.misc:Transferring archived files from trial035:/var/log/ceph to /home/teuthworker/mnt/teuthology/shraddhaag-2026-02-24_06:51:01-crimson-rados-main-distro-debug-trial/67566/remote/trial035/log 2026-02-24T08:16:54.954 DEBUG:teuthology.orchestra.run.trial035:> sudo tar c -f - -C /var/log/ceph -- . 2026-02-24T08:19:10.221 DEBUG:teuthology.misc:Transferring archived files from trial091:/var/log/ceph to /home/teuthworker/mnt/teuthology/shraddhaag-2026-02-24_06:51:01-crimson-rados-main-distro-debug-trial/67566/remote/trial091/log 2026-02-24T08:19:10.222 DEBUG:teuthology.orchestra.run.trial091:> sudo tar c -f - -C /var/log/ceph -- . 2026-02-24T08:21:27.896 INFO:tasks.cephadm:Removing cluster... 2026-02-24T08:21:27.896 DEBUG:teuthology.orchestra.run.trial035:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 --force 2026-02-24T08:21:28.029 INFO:teuthology.orchestra.run.trial035.stdout:Deleting cluster with fsid: 3a5983d2-1154-11f1-9c05-d404e6e7d460 2026-02-24T08:21:30.542 DEBUG:teuthology.orchestra.run.trial091:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 3a5983d2-1154-11f1-9c05-d404e6e7d460 --force 2026-02-24T08:21:30.696 INFO:teuthology.orchestra.run.trial091.stdout:Deleting cluster with fsid: 3a5983d2-1154-11f1-9c05-d404e6e7d460 2026-02-24T08:21:33.122 INFO:tasks.cephadm:Removing cephadm ... 2026-02-24T08:21:33.122 DEBUG:teuthology.orchestra.run.trial035:> rm -rf /home/ubuntu/cephtest/cephadm 2026-02-24T08:21:33.141 DEBUG:teuthology.orchestra.run.trial091:> rm -rf /home/ubuntu/cephtest/cephadm 2026-02-24T08:21:33.158 INFO:tasks.cephadm:Teardown complete 2026-02-24T08:21:33.158 DEBUG:teuthology.run_tasks:Unwinding manager install 2026-02-24T08:21:33.174 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2026-02-24T08:21:33.175 DEBUG:teuthology.orchestra.run.trial035:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2026-02-24T08:21:33.183 DEBUG:teuthology.orchestra.run.trial091:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2026-02-24T08:21:33.391 DEBUG:teuthology.run_tasks:Unwinding manager clock 2026-02-24T08:21:33.402 INFO:teuthology.task.clock:Checking final clock skew... 2026-02-24T08:21:33.402 DEBUG:teuthology.orchestra.run.trial035:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2026-02-24T08:21:33.404 DEBUG:teuthology.orchestra.run.trial091:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2026-02-24T08:21:33.419 INFO:teuthology.orchestra.run.trial035.stderr:bash: line 1: ntpq: command not found 2026-02-24T08:21:33.421 INFO:teuthology.orchestra.run.trial091.stderr:bash: line 1: ntpq: command not found 2026-02-24T08:21:33.452 INFO:teuthology.orchestra.run.trial091.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2026-02-24T08:21:33.452 INFO:teuthology.orchestra.run.trial091.stdout:=============================================================================== 2026-02-24T08:21:33.452 INFO:teuthology.orchestra.run.trial091.stdout:^- 172-104-209-204.ip.linod> 4 6 377 36 +1662us[+1662us] +/- 32ms 2026-02-24T08:21:33.452 INFO:teuthology.orchestra.run.trial091.stdout:^- ntp.maxhost.io 2 6 377 33 +277us[ +277us] +/- 55ms 2026-02-24T08:21:33.452 INFO:teuthology.orchestra.run.trial091.stdout:^- 144.34.193.110.16clouds.> 2 8 377 489 -583us[ -304us] +/- 86ms 2026-02-24T08:21:33.453 INFO:teuthology.orchestra.run.trial091.stdout:^* chi3.us.ntp.li 2 7 377 38 +84us[ +357us] +/- 11ms 2026-02-24T08:21:33.453 INFO:teuthology.orchestra.run.trial035.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2026-02-24T08:21:33.453 INFO:teuthology.orchestra.run.trial035.stdout:=============================================================================== 2026-02-24T08:21:33.453 INFO:teuthology.orchestra.run.trial035.stdout:^- 204.2.134.172 3 6 377 36 +308us[ +374us] +/- 81ms 2026-02-24T08:21:33.454 INFO:teuthology.orchestra.run.trial035.stdout:^- rn-03.koehn.com 2 7 377 39 +112us[ +178us] +/- 37ms 2026-02-24T08:21:33.454 INFO:teuthology.orchestra.run.trial035.stdout:^- eterna.binary.net 2 6 377 34 +142us[ +142us] +/- 61ms 2026-02-24T08:21:33.454 INFO:teuthology.orchestra.run.trial035.stdout:^* chi3.us.ntp.li 2 6 377 35 +19us[ +86us] +/- 11ms 2026-02-24T08:21:33.454 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2026-02-24T08:21:33.465 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2026-02-24T08:21:33.466 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2026-02-24T08:21:33.479 DEBUG:teuthology.orchestra.run.trial035:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2026-02-24T08:21:33.570 DEBUG:teuthology.orchestra.run.trial091:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2026-02-24T08:21:33.629 DEBUG:teuthology.orchestra.run.trial035:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 2026-02-24T08:21:33.657 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-24T08:21:33.658 DEBUG:teuthology.orchestra.run.trial091:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 2026-02-24T08:21:33.684 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-24T08:21:33.685 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2026-02-24T08:21:33.700 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2026-02-24T08:21:33.716 INFO:teuthology.task.internal:Duration was 3349.034855 seconds 2026-02-24T08:21:33.716 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2026-02-24T08:21:33.730 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2026-02-24T08:21:33.731 DEBUG:teuthology.orchestra.run.trial035:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2026-02-24T08:21:33.734 DEBUG:teuthology.orchestra.run.trial091:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2026-02-24T08:21:33.769 INFO:teuthology.orchestra.run.trial035.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2026-02-24T08:21:33.771 INFO:teuthology.orchestra.run.trial091.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2026-02-24T08:21:33.884 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2026-02-24T08:21:33.884 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@trial035.front.sepia.ceph.com 2026-02-24T08:21:33.885 DEBUG:teuthology.orchestra.run.trial035:> grep -E --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | grep -E -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | grep -E -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2026-02-24T08:21:33.904 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@trial091.front.sepia.ceph.com 2026-02-24T08:21:33.904 DEBUG:teuthology.orchestra.run.trial091:> grep -E --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | grep -E -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | grep -E -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2026-02-24T08:21:33.944 INFO:teuthology.task.internal.syslog:Gathering journactl... 2026-02-24T08:21:33.944 DEBUG:teuthology.orchestra.run.trial035:> sudo journalctl > /home/ubuntu/cephtest/archive/syslog/journalctl.log 2026-02-24T08:21:33.947 DEBUG:teuthology.orchestra.run.trial091:> sudo journalctl > /home/ubuntu/cephtest/archive/syslog/journalctl.log 2026-02-24T08:21:34.304 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2026-02-24T08:21:34.304 DEBUG:teuthology.orchestra.run.trial035:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --max-args=1 --max-procs=0 --verbose --no-run-if-empty -- gzip -5 --verbose -- 2026-02-24T08:21:34.307 DEBUG:teuthology.orchestra.run.trial091:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --max-args=1 --max-procs=0 --verbose --no-run-if-empty -- gzip -5 --verbose -- 2026-02-24T08:21:34.328 INFO:teuthology.orchestra.run.trial035.stderr:gzip -5 --verbose -- /home/ubuntu/cephtest/archive/syslog/misc.log 2026-02-24T08:21:34.329 INFO:teuthology.orchestra.run.trial035.stderr:gzip -5 --verbose -- /home/ubuntu/cephtest/archive/syslog/kern.log 2026-02-24T08:21:34.329 INFO:teuthology.orchestra.run.trial035.stderr:/home/ubuntu/cephtest/archive/syslog/misc.log: gzip -5 --verbose -- /home/ubuntu/cephtest/archive/syslog/journalctl.log 2026-02-24T08:21:34.329 INFO:teuthology.orchestra.run.trial035.stderr: 0.0% -- replaced with /home/ubuntu/cephtest/archive/syslog/misc.log.gz 2026-02-24T08:21:34.330 INFO:teuthology.orchestra.run.trial035.stderr:/home/ubuntu/cephtest/archive/syslog/kern.log: 0.0% -- replaced with /home/ubuntu/cephtest/archive/syslog/kern.log.gz 2026-02-24T08:21:34.334 INFO:teuthology.orchestra.run.trial091.stderr:gzip -5 --verbose -- /home/ubuntu/cephtest/archive/syslog/misc.log 2026-02-24T08:21:34.334 INFO:teuthology.orchestra.run.trial091.stderr:gzip -5 --verbose -- /home/ubuntu/cephtest/archive/syslog/kern.log 2026-02-24T08:21:34.334 INFO:teuthology.orchestra.run.trial091.stderr:/home/ubuntu/cephtest/archive/syslog/misc.log: 0.0% -- replaced with /home/ubuntu/cephtest/archive/syslog/misc.log.gz 2026-02-24T08:21:34.334 INFO:teuthology.orchestra.run.trial091.stderr:gzip -5 --verbose -- /home/ubuntu/cephtest/archive/syslog/journalctl.log 2026-02-24T08:21:34.335 INFO:teuthology.orchestra.run.trial091.stderr:/home/ubuntu/cephtest/archive/syslog/kern.log: 0.0% -- replaced with /home/ubuntu/cephtest/archive/syslog/kern.log.gz 2026-02-24T08:21:34.349 INFO:teuthology.orchestra.run.trial091.stderr:/home/ubuntu/cephtest/archive/syslog/journalctl.log: 86.3% -- replaced with /home/ubuntu/cephtest/archive/syslog/journalctl.log.gz 2026-02-24T08:21:34.429 INFO:teuthology.orchestra.run.trial035.stderr:/home/ubuntu/cephtest/archive/syslog/journalctl.log: 89.6% -- replaced with /home/ubuntu/cephtest/archive/syslog/journalctl.log.gz 2026-02-24T08:21:34.431 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2026-02-24T08:21:34.448 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2026-02-24T08:21:34.448 DEBUG:teuthology.orchestra.run.trial035:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2026-02-24T08:21:34.497 DEBUG:teuthology.orchestra.run.trial091:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2026-02-24T08:21:34.524 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2026-02-24T08:21:34.540 DEBUG:teuthology.orchestra.run.trial035:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2026-02-24T08:21:34.543 DEBUG:teuthology.orchestra.run.trial091:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2026-02-24T08:21:34.565 INFO:teuthology.orchestra.run.trial035.stdout:kernel.core_pattern = core 2026-02-24T08:21:34.589 INFO:teuthology.orchestra.run.trial091.stdout:kernel.core_pattern = core 2026-02-24T08:21:34.606 DEBUG:teuthology.orchestra.run.trial035:> test -e /home/ubuntu/cephtest/archive/coredump 2026-02-24T08:21:34.639 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-24T08:21:34.640 DEBUG:teuthology.orchestra.run.trial091:> test -e /home/ubuntu/cephtest/archive/coredump 2026-02-24T08:21:34.662 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-24T08:21:34.662 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2026-02-24T08:21:34.679 INFO:teuthology.task.internal:Transferring archived files... 2026-02-24T08:21:34.680 DEBUG:teuthology.misc:Transferring archived files from trial035:/home/ubuntu/cephtest/archive to /home/teuthworker/mnt/teuthology/shraddhaag-2026-02-24_06:51:01-crimson-rados-main-distro-debug-trial/67566/remote/trial035 2026-02-24T08:21:34.681 DEBUG:teuthology.orchestra.run.trial035:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2026-02-24T08:21:34.755 DEBUG:teuthology.misc:Transferring archived files from trial091:/home/ubuntu/cephtest/archive to /home/teuthworker/mnt/teuthology/shraddhaag-2026-02-24_06:51:01-crimson-rados-main-distro-debug-trial/67566/remote/trial091 2026-02-24T08:21:34.755 DEBUG:teuthology.orchestra.run.trial091:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2026-02-24T08:21:34.793 INFO:teuthology.task.internal:Removing archive directory... 2026-02-24T08:21:34.794 DEBUG:teuthology.orchestra.run.trial035:> rm -rf -- /home/ubuntu/cephtest/archive 2026-02-24T08:21:34.797 DEBUG:teuthology.orchestra.run.trial091:> rm -rf -- /home/ubuntu/cephtest/archive 2026-02-24T08:21:34.841 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2026-02-24T08:21:34.859 INFO:teuthology.task.internal:Not uploading archives. 2026-02-24T08:21:34.860 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2026-02-24T08:21:34.876 INFO:teuthology.task.internal:Tidying up after the test... 2026-02-24T08:21:34.876 DEBUG:teuthology.orchestra.run.trial035:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2026-02-24T08:21:34.879 DEBUG:teuthology.orchestra.run.trial091:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2026-02-24T08:21:34.894 INFO:teuthology.orchestra.run.trial035.stdout: 917525 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Feb 24 08:21 /home/ubuntu/cephtest 2026-02-24T08:21:34.896 INFO:teuthology.orchestra.run.trial091.stdout: 920834 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Feb 24 08:21 /home/ubuntu/cephtest 2026-02-24T08:21:34.898 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2026-02-24T08:21:34.915 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2026-02-24T08:21:34.953 INFO:teuthology.run:Summary data: description: crimson-rados/thrash/{0-crimson_install 0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{default} clusters/{crimson-fixed-2} crimson-supported-all-distro/centos_latest crimson_qa_overrides deploy/cephadm objectstore/seastore/rbm$/{crimson_seastore_rbm} thrashers/{pggrow} thrashosds-health workloads/radosbench} duration: 3349.0348551273346 flavor: debug owner: scheduled_shraddhaag@soko04.front.sepia.ceph.com success: true 2026-02-24T08:21:34.953 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com 2026-02-24T08:21:35.083 INFO:teuthology.run:pass